ceos/opencode
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(simulation): add named drive instances (#35648)

Browse source
This commit is contained in:
James Long 2026-07-06 22:52:10 -04:00 • committed by GitHub
commit fcb1d4b418
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 199 additions and 173 deletions
Download patch file Download diff file Expand all files Collapse all files

8
packages/opencode/specs/simulation/simulated-network-llm.md
View file

@ -65,7 +65,7 @@ Exchange = { id, body, queue: Queue<Item | Error | Done>, deferred lifecycle }
### 4. Backend control WebSocket (simulation-gated)
Started when the simulation module loads (lazy import, `OPENCODE_SIMULATION` only): a loopback JSON-RPC 2.0 WebSocket on `127.0.0.1:40950+`, hosted by the backend process. Drivers connect to it directly — the standalone topology has exactly one backend per TUI, so there is no proxying through the frontend. This socket is also the headless-simulation interface: it works with no TUI at all.
Started when `OPENCODE_DRIVE` names a registry manifest: a loopback JSON-RPC 2.0 WebSocket at that manifest's exact backend endpoint, hosted by the backend process. Drivers connect to it directly — the standalone topology has exactly one backend per TUI, so there is no proxying through the frontend. This socket is also the headless-simulation interface: it works with no TUI at all.
Server -> driver notification (after `llm.attach`; pending exchanges are replayed on attach so late-attaching drivers miss nothing):
@ -101,8 +101,8 @@ Failure injection (`llm.fail`: HTTP status instead of SSE) is specced but not ye
A driver manages two loopback WebSocket connections:
- TUI control server (`127.0.0.1:40900+`) — UI state, actions, render, trace.
- Backend control server (`127.0.0.1:40950+`) — LLM exchanges, network log.
- TUI control server (manifest `endpoints.ui`) — UI state, actions, render, trace.
- Backend control server (manifest `endpoints.backend`) — LLM exchanges, network log.
Both speak the same JSON-RPC shape. Headless drivers use only the backend socket plus the normal HTTP API. Multiple drivers are out of scope; last attach wins.
@ -117,7 +117,7 @@ The driver-facing model must be selectable in the TUI. Simulation seeds config (
## End-to-end flow
```
driver TUI sim server (40900+) backend + control WS (40950+)
driver TUI drive server backend + drive WS
| | |
|-- ui.action (submit) ----->| |
| |-- (normal app HTTP) ---->| session runner starts

26
packages/opencode/specs/simulation/simulation-phases.md
View file

@ -12,11 +12,11 @@ This phase proves the core shape without swapping every foundational layer yet.
Implementation checklist:
- [x] Add `OPENCODE_SIMULATION=1` activation in V1/full-TUI startup.
- [x] Add `OPENCODE_DRIVE=<name>` activation in V1/full-TUI startup.
- [x] Add simulation trace service with in-memory append-only records.
- [x] Add OpenTUI UI state extraction for screen, focus, elements, and generated actions.
- [x] Add OpenTUI UI action execution for typing, keys, enter, arrows, focus, and click.
- [x] Add reusable JSON-RPC WebSocket server on `127.0.0.1:40900+`.
- [x] Add reusable JSON-RPC WebSocket server at the manifest's UI endpoint.
- [x] Expose `ui.state`, `ui.action`, `ui.render`.
- [x] Expose `trace.list`, `trace.clear`, `trace.export`.
- [x] Wire visible V1/full-TUI renderer path through the same action protocol.
@ -24,8 +24,8 @@ Implementation checklist:
Scope:
- Add `OPENCODE_SIMULATION=1` activation.
- Start a TUI-owned JSON-RPC WebSocket server on `127.0.0.1:40900+`.
- Add `OPENCODE_DRIVE=<name>` activation.
- Start a TUI-owned JSON-RPC WebSocket server at the manifest's UI endpoint.
- Expose `ui.state`, `ui.action`, `ui.render`.
- Use the old simulation action model: type text, press keys, press enter, arrows, focus, click.
- Support fake OpenTUI renderer and visible renderer through the same action protocol.
@ -34,7 +34,7 @@ Scope:
Done when:
- `OPENCODE_SIMULATION=1 bun run dev` starts the normal app.
- `OPENCODE_DRIVE=<name> bun run dev` starts the normal app and UI drive server.
- A local driver can connect to the WebSocket.
- The driver can inspect current screen/elements/actions.
- The driver can execute real TUI inputs.
@ -54,19 +54,19 @@ Goal: make the app safe and controlled by swapping the lowest layers, not app lo
Implementation checklist:
- [x] Add `packages/simulation/src/backend` as the home for backend simulation layer replacements, exported from `backend/index.ts` as `simulationReplacements`; `@opencode-ai/simulation` is private/non-published and depends on logic/framework packages (`core`, `llm`, `effect`, OpenTUI), while `server` and `tui` consume it.
- [x] Wire simulation replacements through the server's `makeRoutes` via `Layer.unwrap` + dynamic `import("@opencode-ai/simulation/backend")` gated on `OPENCODE_SIMULATION`, so the simulation module is never loaded eagerly and `makeRoutes` stays synchronous.
- [x] Wire simulation replacements through the server's `makeRoutes` via `Layer.unwrap` + dynamic `import("@opencode-ai/simulation/backend")` gated on `OPENCODE_SIMULATE`, so the simulation module is never loaded eagerly and `makeRoutes` stays synchronous.
- [x] Implement in-memory `FileSystem.FileSystem` (`simulation/filesystem.ts`) replacing the `NodeFileSystem` platform node. Backed by a flat path map; implements the operations the app uses (stat, access, chmod, realPath, read/write file, make/read directory, remove, rename, copy, copyFile, temp dirs, read-only open handles); unused operations die with a clear defect; `watch` fails as unsupported.
- [x] Root the fake filesystem at `OPENCODE_SIMULATION_ROOT` (falling back to `process.cwd()` at layer-build time). The anchor is a real, empty host directory the runner creates and cds into.
- [x] Root the fake filesystem at `process.cwd()` at layer-build time. The anchor is a real, empty host directory the runner creates and cds into.
- [x] Deny host filesystem escapes loudly: content/mutation operations outside the root fail with `PermissionDenied` simulation errors. Probe operations (`stat`/`access`/`exists`) report `NotFound` outside the root so walk-up loops (project discovery, `findUp`, `globUp`) terminate naturally.
- [x] Add `SimulationFSUtil` replacement (`simulation/fs-util.ts`): wraps the real `FSUtil` layer and reroutes `readDirectoryEntries`, `glob`, and `globUp` — which bypass the injected `FileSystem` via node `fs/promises` and the `glob` package — through the simulated filesystem.
- [x] Fix `LayerNode.hoist` conflict detection to compare node implementations instead of object identity; replacement rewriting produces dependency-rewritten copies of the same node, which previously false-positived as "conflicting implementations".
- [x] Add snapshot seeding from `OPENCODE_SIMULATION_STATE`: `project/` contents of the snapshot directory are read from the host once at layer-build time and seeded into the in-memory tree joined onto the anchor root.
- [x] Verify end to end: `opencode serve` boots with `OPENCODE_SIMULATION=1` + `OPENCODE_SIMULATION_ROOT` + path/DB env seams (`OPENCODE_CONFIG_DIR`, `OPENCODE_TEST_HOME`, `OPENCODE_DB=:memory:`); `fs.list`/`fs.read` observe only seeded in-memory files; the anchor directory on the host remains empty after the run.
- [ ] Create the anchor directory + `chdir` + env seam setup automatically in CLI startup when simulation mode is enabled (currently set manually by the runner; a full run needs `OPENCODE_SIMULATION_ROOT/STATE`, `OPENCODE_CONFIG_DIR`, `OPENCODE_TEST_HOME`, `OPENCODE_DB=:memory:`, and `XDG_*_HOME` pointed into the anchor, plus Bun's `--preload=@opentui/solid/preload` when launched outside `packages/cli`).
- [x] Add snapshot seeding from `OPENCODE_SIMULATE_STATE`: `files/` contents of the snapshot directory are read from the host once at layer-build time and seeded into the in-memory tree joined onto the anchor root.
- [x] Verify end to end: `opencode serve` boots with `OPENCODE_SIMULATE=1` + `OPENCODE_SIMULATE_STATE` + path/DB env seams (`OPENCODE_CONFIG_DIR`, `OPENCODE_TEST_HOME`, `OPENCODE_DB=:memory:`); `fs.list`/`fs.read` observe only seeded in-memory files; the anchor directory on the host remains empty after the run.
- [ ] Create the anchor directory + `chdir` + env seam setup automatically in CLI startup when simulation mode is enabled (currently set manually by the runner; a full run needs `OPENCODE_SIMULATE_STATE`, `OPENCODE_CONFIG_DIR`, `OPENCODE_TEST_HOME`, `OPENCODE_DB=:memory:`, and `XDG_*_HOME` pointed into the anchor, plus Bun's `--preload=@opentui/solid/preload` when launched outside `packages/cli`).
- [ ] Assert the anchor directory is still empty at the end of the run (KV/log/flock still write through real XDG paths; they are contained in the anchor by the env seams but not yet in-memory).
- [x] Add simulated network registry (`packages/simulation/src/backend/network.ts`): replaces the `httpClient` platform node, resolves all outbound HTTP against an in-memory route table, denies unknown destinations loudly, and keeps a bounded request log (design: `simulated-network-llm.md`).
- [x] Add driver-answered LLM as an OpenAI route in the simulated network (`openai.ts` + `llm-exchange.ts`): provider requests open exchanges; the driver streams chunks back which are encoded as real OpenAI Chat SSE (schema-checked against `OpenAIChatEvent`) and consumed by the real protocol pipeline. No enqueue store — the driver is the model.
- [x] Add backend-hosted simulation control WebSocket (`control.ts`): JSON-RPC on `127.0.0.1:40950+`, started when the simulation module loads. Drivers connect directly (standalone topology — no frontend proxy): `llm.attach` (replays pending exchanges), `llm.chunk`, `llm.finish`, `llm.pending`, `network.log`; `llm.request` notifications push opened exchanges. This is also the headless-simulation interface. Drivers manage two sockets: TUI control (40900+) for UI, backend control (40950+) for LLM/network.
- [x] Add backend-hosted drive control WebSocket (`control.ts`): JSON-RPC at the named manifest's backend endpoint, started when `OPENCODE_DRIVE` is set. Drivers connect directly (standalone topology — no frontend proxy): `llm.attach` (replays pending exchanges), `llm.chunk`, `llm.finish`, `llm.pending`, `network.log`; `llm.request` notifications push opened exchanges. This is also the headless-simulation interface. Drivers manage the manifest's UI endpoint for UI control and backend endpoint for LLM/network control.
- [x] Answer `https://models.dev/api.json` with an empty catalog in the simulated network; providers come from seeded config (`opencode.json` in the snapshot defines an openai-compatible provider with a dummy `apiKey`, which passes the catalog availability gate and resolves onto the real openai-chat route).
- [x] Fix `buildLocationServiceMap` to apply replacements when compiling hoisted global nodes; platform-node replacements (filesystem, httpClient) were silently ignored inside hoisted globals.
- [x] Verify end to end headless (real route stack in-process + backend control WS: prompt -> `llm.request` -> driver chunks -> assistant message contains driver text; script: `packages/server/script/e2e-sim.ts`) and through the TUI (fake renderer, both sockets: type + submit via TUI WS, answer `llm.request` via backend WS, assistant reply rendered on screen; script: `packages/tui/script/sim-llm-driver.ts`).
@ -78,7 +78,7 @@ Scope:
- Wire simulation replacements through `AppNodeBuilder.build(...)` and `AppNodeBuilderV1.build(...)`.
- Create a real, empty anchor directory (`mkdtemp`) and `process.chdir` into it before any command resolves its working directory; skip creation when the runner already spawned the app inside an anchor.
- Root the in-memory filesystem at `process.cwd()` (the anchor). No cwd monkey-patching: cwd, `$PWD`, and `path.resolve()` stay truthful.
- Add snapshot loading from `OPENCODE_SIMULATION_STATE`: read the snapshot directory once at startup and seed the in-memory filesystem (snapshot `project/` paths joined onto the anchor root), config, env, and optional LLM/network state from it.
- Add snapshot loading from `OPENCODE_SIMULATE_STATE`: read the snapshot directory once at startup and seed the in-memory filesystem (snapshot `files/` paths joined onto the anchor root), config, env, and optional LLM/network state from it.
- Route config/data/state/cache/temp paths into the simulated space using existing env seams (`OPENCODE_CONFIG_DIR`, `OPENCODE_TEST_HOME`, `OPENCODE_DB=:memory:`), set before `packages/core/src/global.ts` import-time path setup runs.
- Deny host filesystem escapes loudly (paths outside the anchor root fail with typed simulation errors).
- Assert the anchor directory on the host is still empty at the end of the run; anything written there means a code path bypassed the simulated filesystem.
@ -97,7 +97,7 @@ Done when:
- Unknown network fails with a simulation error.
- Host filesystem escape fails with a simulation error.
- The anchor directory on the host is empty after a run.
- The app boots from a snapshot directory via `OPENCODE_SIMULATION_STATE` and observes the seeded project files, config, and env through normal app paths.
- The app boots from a snapshot directory via `OPENCODE_SIMULATE_STATE` and observes the seeded project files, config, and env through normal app paths.
- A driver can seed a project filesystem.
- A driver can enqueue an LLM script and submit a prompt through the TUI.
- The real session/tool path consumes the scripted LLM behavior.

40
packages/opencode/specs/simulation/simulation.md
View file

@ -11,7 +11,7 @@ The first milestone is an interactive exploration and model-based testing enviro
This is not intended to be a custom simulated app or a separate `simulate` command. The normal app should run, with simulation enabled by one required flag:
```sh
OPENCODE_SIMULATION=1 bun run dev
OPENCODE_SIMULATE=1 bun run dev
```
## Non-Goals
@ -21,7 +21,7 @@ OPENCODE_SIMULATION=1 bun run dev
- Do not build shrinking in the first milestone.
- Do not make generated randomized runs part of CI yet.
- Do not build differential testing in the first milestone.
- Do not expose simulation controls when `OPENCODE_SIMULATION` is not set.
- Do not expose drive controls when `OPENCODE_DRIVE` is not set.
## Design Principles
@ -37,12 +37,12 @@ OPENCODE_SIMULATION=1 bun run dev
## Activation
`OPENCODE_SIMULATION=1` is the only required flag.
`OPENCODE_SIMULATE=1` swaps the backend's foundational layers for simulated implementations. `OPENCODE_DRIVE=<name>` independently starts the frontend and backend control WebSockets using the exact endpoints from the named opencode-drive registry manifest. `OPENCODE_DRIVE=1` starts an unnamed instance at `ws://127.0.0.1:40900` for the UI and `ws://127.0.0.1:40950` for the backend.
Initial state is provided through an optional snapshot directory:
```sh
OPENCODE_SIMULATION=1 OPENCODE_SIMULATION_STATE=/path/to/snapshot bun run dev
OPENCODE_SIMULATE=1 OPENCODE_DRIVE=demo OPENCODE_SIMULATE_STATE=/path/to/snapshot bun run dev
```
Optional flags can be added later, but should stay minimal. Reasonable optional parameters later include renderer mode, trace output path, seed, or port override.
@ -54,15 +54,15 @@ When enabled:
- The app creates and changes into a real, empty anchor directory (see Filesystem).
- The app reads the snapshot directory, if provided, and seeds all simulated state from it.
- The app builds with simulation layer replacements.
- The TUI process starts a loopback WebSocket control server.
- The TUI and backend processes start loopback WebSocket control servers when `OPENCODE_DRIVE` is set.
- Simulation-gated backend control routes become available only to the frontend/control path.
- In-memory trace recording starts automatically.
Path seams reuse existing environment variables where they already exist: `OPENCODE_CONFIG_DIR` for global config, `OPENCODE_TEST_HOME` for home, and `OPENCODE_DB=:memory:` for the database. Simulation mode should set these before foundational modules load rather than inventing parallel mechanisms.
## Control Server
## Control Servers
The external control surface lives in the TUI/frontend process, not the backend API server.
The UI control surface lives in the TUI/frontend process. A separate backend control surface handles simulated LLM and network operations.
This is important because the frontend has direct access to the renderer, screen state, focus state, interactable elements, and user input APIs. The backend remains the normal backend, with only simulation-gated control routes used internally by the frontend when needed.
@ -70,9 +70,10 @@ Protocol:
- JSON-RPC 2.0 over WebSocket.
- Loopback only.
- Start at `127.0.0.1:40900`.
- If occupied, scan upward and report the actual URL.
- External drivers connect only to this frontend WebSocket.
- `OPENCODE_DRIVE` names a manifest in the opencode-drive registry, or is `1` for the unnamed default endpoints.
- The manifest supplies exact loopback `ui` and `backend` WebSocket endpoints.
- Startup fails rather than scanning when either manifest endpoint is unavailable.
- External drivers connect to both WebSockets when they need UI and backend controls.
The app should not send JSON-RPC requests back to the driver in the first milestone. The driver sends requests; the app responds and emits notifications/events as useful.
@ -129,7 +130,7 @@ Both fake OpenTUI renderer and visible terminal renderer should share this proto
The backend server should be exactly the normal backend server.
Simulation-only backend routes may exist, but only when `OPENCODE_SIMULATION=1`. They are private implementation details for the frontend simulation server to proxy commands like filesystem seeding, LLM scripting, network registration, and snapshots.
Simulation-only backend routes may exist, but only when `OPENCODE_SIMULATE=1`. They are private implementation details for commands like filesystem seeding, LLM scripting, network registration, and snapshots.
External drivers should not use backend simulation routes directly.
@ -197,13 +198,13 @@ The anchor may be created by the app itself at activation, or by an external run
## Initial State Snapshot
`OPENCODE_SIMULATION_STATE` points at a directory containing one complete initial state. On startup the app slurps this directory once and constructs all simulated state from it. The snapshot is never written back to; it is a pure input.
`OPENCODE_SIMULATE_STATE` points at a directory containing one complete initial state. On startup the app slurps this directory once and constructs all simulated state from it. The snapshot is never written back to; it is a pure input.
Proposed layout:
```text
snapshot/
project/... # workspace files, seeded into the in-memory FS under the anchor root
files/... # workspace files, seeded into the in-memory FS under the anchor root
config/opencode.json # global config; the directory backs OPENCODE_CONFIG_DIR
env.json # extra environment values to apply
llm/... # scripted LLM behavior to pre-enqueue (optional)
@ -212,8 +213,8 @@ snapshot/
Rules:
- Paths inside `project/` are snapshot-relative. The loader joins them onto the anchor root, so absolute virtual paths look like real host paths under the anchor.
- Anything the config references (skills, instructions, reference paths) must exist inside `project/`. A snapshot that references missing files is invalid.
- Paths inside `files/` are snapshot-relative. The loader joins them onto the anchor root, so absolute virtual paths look like real host paths under the anchor.
- Anything the config references (skills, instructions, reference paths) must exist inside `files/`. A snapshot that references missing files is invalid.
- The snapshot directory format is the contract between external state generators and the app. Generators (such as the opencode-probe project) produce snapshot directories plus a derived expected model; the app consumes only the snapshot.
- Seeding through the control server (`backend.filesystem.seed` and friends) remains available for incremental changes during a run; the snapshot covers initial state.
@ -443,8 +444,8 @@ More advanced model/refinement, metamorphic, and differential properties are fut
The first major demo should show this system as a real environment for exploring the app in controlled states:
1. Start opencode normally with `OPENCODE_SIMULATION=1`.
2. TUI starts and exposes the simulation WebSocket on `127.0.0.1:40900+`.
1. Start opencode normally with `OPENCODE_SIMULATE=1` and `OPENCODE_DRIVE=<name>`.
2. TUI and backend start their drive WebSockets at the named manifest endpoints.
3. External runner connects.
4. Runner provides a snapshot directory (or seeds the in-memory project filesystem through the control server).
5. Runner generates and enables plugin-provided config state.
@ -459,10 +460,11 @@ The first major demo should show this system as a real environment for exploring
## Done-When Checklist
- `OPENCODE_SIMULATION=1` starts the normal app with simulation wiring.
- `OPENCODE_SIMULATE=1` starts the normal app with simulation wiring.
- `OPENCODE_DRIVE=<name>` starts both drive WebSockets at the manifest endpoints.
- Simulation code is isolated under a dedicated simulation/testing area.
- App changes outside simulation are limited to activation hooks, builder replacements, TUI startup, and gated backend routes.
- TUI exposes JSON-RPC WebSocket on `127.0.0.1:40900+`.
- TUI and backend expose JSON-RPC WebSockets at the manifest endpoints.
- Driver can call `ui.state`.
- Driver can execute generated UI actions.
- Fake and visible renderer paths use the same action protocol.