diff --git a/specs/v2/remote-workspace-execution.md b/specs/v2/remote-workspace-execution.md new file mode 100644 index 0000000000..880833b05b --- /dev/null +++ b/specs/v2/remote-workspace-execution.md @@ -0,0 +1,492 @@ +# Remote Workspace Execution + +Status: implementation plan + +Tracking: Organizer item `2ff92129` + +## Goal + +Prove that one OpenCode V2 Session can operate on a real hosted development +environment whose checkout does not exist on the OpenCode server host. + +The first slice must support: + +- one stable Workspace identity referenced by a Session Location; +- one provider-backed sandbox for that Workspace; +- reconnecting to the existing provider sandbox; +- reading, writing, editing, globbing, and grepping Workspace files; +- running and interrupting foreground shell commands; +- retaining Workspace files across provider suspension and reconnection; +- local Locations continuing to behave exactly as they do today. + +The design must permit Modal, Daytona, and E2B implementations without placing +provider concepts in Session, Location, or tool APIs. + +## Non-Goals + +The first slice does not include: + +- Slack integration or other product surfaces; +- background or detached commands; +- PTY sessions; +- filesystem watches; +- user-visible snapshots, Session revert, or Workspace forks; +- concurrent mutating Sessions in one Workspace; +- provider migration; +- arbitrary plugin code transparently using the remote filesystem; +- post-crash retry of ambiguous provider or tool work; +- running the Session runner inside the sandbox; +- clustering Session execution. + +## Domain Model + +### Session + +A Session owns conversation and agent execution history. The Session runner, +LLM calls, tool registry, permissions, durable events, and tool settlement stay +in the central OpenCode server. + +### Workspace + +A Workspace is the stable project environment referenced by +`Location.workspaceID`. It may be the primary local checkout, a local git +worktree, or a provider-backed hosted environment. + +Workspace identity is orthogonal to Session identity. The first product slice +may create one primary Session per Workspace, but this is policy rather than an +identity invariant. + +The Workspace record stores placement metadata. It does not mirror or +synchronize files, dependencies, or Git state. The environment filesystem is +authoritative. + +### Provider Sandbox + +A provider sandbox is implementation machinery backing a hosted Workspace. Its +provider identifier and lifecycle state belong to the Workspace placement, not +to Session or Location. + +The Workspace ID remains stable when the provider reconnects, resumes, or +recreates its compute from retained state. + +## API Shape + +Keep provider lifecycle separate from environment capabilities. + +```ts +interface SandboxProvider { + readonly create: (input: CreateInput) => Effect.Effect + readonly connect: (binding: SandboxBinding) => Effect.Effect + readonly suspend: (connection: SandboxConnection) => Effect.Effect + readonly reconcile: (binding: SandboxBinding) => Effect.Effect + readonly delete: (binding: SandboxBinding) => Effect.Effect +} + +interface SandboxConnection { + readonly binding: SandboxBinding + readonly environment: SandboxEnvironment +} +``` + +`SandboxBinding` is opaque provider state persisted with the Workspace. Core +does not inspect it beyond selecting the provider that owns it. Persisted +binding data uses provider-specific Effect Schemas whose encoded form is +`Schema.Json`, not `unknown`. Both `connect` and `suspend` may replace the +binding. This is necessary for providers such as Modal where restoring a +filesystem snapshot creates a new provider Sandbox ID. + +A replacement binding must remain reconnectable and include enough state to +identify any retired provider resources. Workspace persists it before exposing +a new connection or destroying the previous durable resource, then calls the +provider's idempotent `reconcile` and persists the simplified binding it +returns. A crash at either boundary therefore leaves a valid binding whose +cleanup can resume on the next lifecycle operation rather than an +undiscoverable sandbox or snapshot. Creation compensates by deleting provider +state if the initial Workspace write fails. `CreateInput` carries a +caller-generated stable provider identity derived from the Workspace ID, and +provider creation is idempotent or discoverable by that identity. Workspace +uses interruption-masked acquire/use/release around allocation and initial +persistence, so interruption after the binding becomes known either commits +the Workspace or deletes the allocation. Automatic retry of ambiguous command +work remains out of scope. + +```ts +interface SandboxEnvironment { + readonly platform: "linux" + readonly directory: string + readonly files: WorkspaceFileBackend + readonly process: ChildProcessSpawner["Service"] + readonly shell: WorkspaceShell + readonly ripgrep: Effect.Effect +} + +interface WorkspaceShell { + readonly executable: string + readonly args: (command: string) => readonly string[] + readonly environmentOverrides: Readonly> + readonly detached: boolean +} +``` + +Core exposes these capabilities through a Location-scoped +`WorkspaceEnvironment.Service`. The process-global Workspace service owns one +connection manager per Workspace ID. Its Effect `RcRef` acquires a scoped +`SandboxConnection` lazily, so constructing or inspecting a Location does not +wake provider compute. Each Workspace operation borrows the connection in a +scope; concurrent operations share it, and an idle TTL releases the provider +client after the final borrower without suspending compute or discarding the +durable binding. + +The manager also owns a lifecycle gate and active-lease count. Suspension +blocks new leases, waits for current leases to close, performs and persists the +provider transition, invalidates the `RcRef`, then reopens the gate. Thus a +suspend cannot run under an active command or file operation, and an old +connection cannot be handed to a new operation after a binding transition. + +The provider filesystem boundary contains only primitives needed beneath the +existing Core policy services: + +```ts +interface WorkspaceFileBackend { + readonly inspect: (path: string) => Effect.Effect + readonly realPath: (path: string) => Effect.Effect + readonly read: (path: string) => Effect.Effect + readonly list: (path: string) => Effect.Effect + readonly ensureDirectory: (path: string) => Effect.Effect + readonly createExclusive: (path: string, content: Uint8Array) => Effect.Effect + readonly write: (path: string, content: Uint8Array) => Effect.Effect + readonly writeIfUnchanged: ( + path: string, + expected: Uint8Array, + content: Uint8Array, + ) => Effect.Effect + readonly remove: (path: string) => Effect.Effect +} +``` + +`writeIfUnchanged` is one provider-side operation. A remote implementation must +not emulate it with an unlocked client-side read followed by write. +`FileMutation` remains the owner of create/write/remove semantics, stale-edit +errors, parent-directory creation, result metadata, BOM handling, and +OpenCode-side mutation ordering. `LocationMutation` retains symlink-safe path +containment by composing `realPath` and `inspect`, including its nearest +existing ancestor resolution for new targets. + +Provider process transports adapt native command handles to Effect's scoped +`ChildProcessSpawner` contract. Scope finalization interrupts an unfinished +provider process. Existing `AppProcess` and `Shell` continue to own timeout, +abort, bounded output, streaming, command IDs, and settlement, but `Shell` must +take executable, arguments, environment, and detached policy from +`WorkspaceShell` instead of `process.env` or `process.platform`. Hosted policy +preserves the provider's baseline environment and adds only explicit safe +OpenCode overrides; it never copies the server environment. + +Foreground `Shell` acquisition is scoped. It uses `Effect.acquireRelease` +around process creation so the interrupt finalizer is installed atomically +before acquisition completes, closing the current create-to-`onInterrupt` +window. The first hosted slice rejects background requests. Existing `Ripgrep` +continues to own command construction, bounded JSON parsing, result schemas, +and invalid-pattern behavior; the environment supplies process transport and +the remote ripgrep executable path. + +OpenCode services expose three distinct lifecycle operations: + +```ts +Workspace.get(id) // Read metadata without waking compute. +Workspace.connect(id) // Ensure usable compute through a short-lived lease. +Workspace.remove(id) // Delete provider state and the Workspace record. +``` + +Core also has an internal +`Workspace.borrow(id): Effect` +operation used by the Location environment. File operations scope the borrow +to one operation. A spawned process holds its borrow in the caller's process +scope until the handle exits or is finalized. Raw provider connections and +bindings are not public API. + +## Placement + +`Location.workspaceID` remains the only Session-to-Workspace reference. + +```text +Location without workspaceID + -> current implicit-local Location graph + +Location with workspaceID + -> load Workspace + -> lexically normalize its directory against Workspace root metadata + -> build a lazy Location graph + -> borrow the Workspace environment only when an operation runs +``` + +The provider sandbox ID never appears in a Session row or durable Session +event. + +The existing experimental Workspace adapter API is not preserved. It mixes +configuration, provisioning, discovery, lifecycle, and request routing. The +replacement should be derived from the lifecycle and environment interfaces +above. + +## Host And Workspace Authority + +Remote execution forces a split that is currently implicit in `FSUtil`: + +- host authority owns OpenCode global configuration, caches, credentials, + durable events, and managed tool output; +- Workspace authority owns repository files, project instructions, project + configuration, project skills, Git state, search, and commands. + +Do not implement the permanent remote boundary by emulating Effect's complete +low-level `FileSystem.FileSystem` interface. Migrate Location-owned consumers +to the Workspace file backend, or compose existing policy services over that +backend and the environment's `ChildProcessSpawner`. Host-global consumers +continue using the host filesystem. + +## Provider Strategy + +Build one provider-neutral contract and one real reference provider. + +Vercel is the recommended first adapter because the tracer has verified the +complete lifecycle against an available account. Its stable persistent name, +automatic snapshot-on-stop, direct filesystem API, and reconnectable command +handles map closely to the proposed contract. + +Daytona is the next comparison because its public API directly supports +filesystem operations, command sessions, interruption, and durable stop/pause +semantics. Modal follows with an intentionally different binding transition: +suspension snapshots the filesystem and reconnection creates a new Sandbox. +E2B can provide a fourth validation through pause/resume if the first three +leave meaningful ambiguity. + +No in-sandbox OpenCode worker is required for the first Daytona/E2B/Modal +slice. Their SDKs can implement the small environment contract directly. Add a +provider-neutral worker later only when supporting providers such as exe.dev, +or when detached process reconnection and richer runtime semantics justify it. + +## Implementation Sequence + +The first integration PR is a narrow vertical slice across stages 1 through 4: + +- add the provider-neutral file, process, environment, and lifecycle contracts; +- add the local environment and a test-only fake hosted provider; +- replace the experimental Workspace storage shape with public metadata plus + private placement state; +- add the Workspace connection manager and Location-scoped + `WorkspaceEnvironment.Service`; +- make `Location.boundNode` resolve hosted project/root metadata without + `Project.resolve` or any host-path access; and +- prove lazy connection and scoped release using a hosted directory that does + not exist on the test host. + +It does not add Vercel or migrate every tool. Its purpose is to establish the +placement seam and prevent cloud-provider details from shaping later Core +changes. + +### 1. Define Behavioral Contracts + +- Add internal Sandbox provider, binding, file-backend, process-transport, and + environment services in Core. +- Include provider-side real-path and directory primitives plus explicit + Workspace shell/environment policy in the contract. +- Keep provider registration process-global and provider implementations out + of tool modules. +- Define typed errors for unavailable Workspace, connection failure, file + operations, stale content, command failure, and unsupported capability. +- Add a contract test suite that can run against any environment + implementation. +- Cover create, reconnect, file retention, conditional write, command output, + timeout, caller interruption, and cleanup when a caller fails before wait. +- Observe a long command running before interruption, then assert promptly that + its provider process no longer exists. +- Inject failures around replacement-binding persistence and verify that the + durable binding can reconnect and reconcile retired resources. +- Interrupt creation around allocation and initial persistence, then verify + that the stable provider identity resolves to either one committed Workspace + or no retained provider resource. + +Result: contracts and reusable tests are ready for the local and fake +implementations in the same first integration PR. + +### 2. Establish Local Parity + +- Implement the file backend against the current local filesystem and use the + current `ChildProcessSpawner` plus ripgrep resolver. +- Compose the existing `FileMutation`, `AppProcess`, `Shell`, and `Ripgrep` + services over those local primitives. +- Run the contract suite against the local implementation. +- Do not route Sessions through it yet. + +Result: the contract describes existing semantics rather than only one cloud +provider's API. + +### 3. Replace The Experimental Workspace Adapter Core + +- Define a safe Workspace public metadata model in Schema instead of exposing + only its ID and events. +- Replace arbitrary adapter `type` plus `extra: unknown` with an explicit local + or sandbox placement in Core-owned storage. Sandbox placement carries a + provider key and provider-schema-validated JSON binding; it is not included + in browser-facing Workspace metadata. +- Add a Core Workspace service for create, get, connect, suspend, and remove. +- Keep provider lookup behind a process-global provider registry. +- Key one scoped connection manager by Workspace ID. Fence suspend/remove + against new leases and wait for active leases before changing placement. +- Persist replacement bindings before destructive cleanup and reconcile + interrupted transitions on the next lifecycle operation. +- Preserve `Workspace.ID` and existing Session `workspace_id` references. +- Migrate or reset only experimental Workspace rows; do not add compatibility + machinery without a concrete persisted-data requirement. + +Result: Workspace owns stable identity and placement without owning files. + +### 4. Introduce The Location Environment Seam + +- Add a Location-scoped `WorkspaceEnvironment.Service` whose local + implementation wraps the current filesystem and process services. +- Give hosted Workspaces an implementation whose operations borrow the + process-global Workspace connection manager. +- Change `Location.boundNode` so a Location with `workspaceID` gets project and + root metadata from the Workspace record instead of calling host-local + `Project.resolve(directory)`. +- Treat Workspace metadata as the canonical root. Interpret Location directory + as a working directory within that root and lexically reject paths outside + it before looking up the Location graph. Resolve symlinks only when an + operation borrows the environment. +- Prove this with a fake hosted Workspace whose directory does not exist on the + test host. Do not add a cloud SDK in this change. + +Result: Core has one placement seam and remote Location identity no longer +requires a host checkout. + +### 5. Separate Host Files From Workspace Files + +- Inventory every Location-scoped `FSUtil`, `Ripgrep`, `AppProcess`, and direct + Node filesystem use. +- Adapt `FileMutation` to the Workspace file backend while preserving its + current policy and result surface. +- Adapt `LocationMutation` to provider-side `realPath` and `inspect` so existing + targets, missing-target ancestors, and symlink escapes retain current + containment behavior. +- Adapt hosted process transport once to `ChildProcessSpawner`, then reuse + `AppProcess`, `Shell`, and `Ripgrep` rather than adding provider command or + search state machines. +- Change `Shell` to consume Workspace shell/environment policy and add scoped + foreground acquisition with an atomic interrupt finalizer. Reject background + shell requests for hosted Workspaces in this slice. +- Move read, edit, patch, path resolution, project instruction discovery, and + project configuration onto those environment-aware services. +- Keep global config, global skills, caches, credentials, and managed tool + output on host services. +- Report the environment platform in built-in instructions instead of + `process.platform`. +- Make unsupported remote snapshot, watcher, VCS, and PTY services explicit for + the first slice rather than accidentally acting on the server host. + +Result: no supported Workspace operation can silently touch the wrong host. + +### 6. Make The Full Location Graph Environment-Aware + +- Migrate each supported Location service to `WorkspaceEnvironment.Service` + rather than selecting services through dynamic `LayerNode` replacements. +- Preserve the current implicit-local path when `workspaceID` is absent. +- Cache the connection manager by Workspace ID. Cache Location graphs by the + canonical `{ workspaceID, directoryWithinWorkspace }` identity. +- Keep Location graphs valid across binding replacement: their environment + service borrows the current Workspace connection for each operation rather + than retaining provider state. +- Add an end-to-end fake-provider test that materializes the full Location graph + without reading, searching, spawning, or watching the host path. + +Result: a remote Location can boot when its directory does not exist locally. + +### 7. Implement One Real Provider + +- Add the Vercel provider behind a narrow optional package or lazy import so + its SDK does not affect local startup. +- Provision a named persistent Sandbox from provider configuration. +- Persist only the binding required to reconnect. +- Use the Vercel SDK directly for lifecycle and files. Adapt detached SDK + command handles to scoped `ChildProcessSpawner` handles; do not shell through + the Vercel CLI or expose detached commands to tools. +- Implement atomic conditional write inside the sandbox, plus suspend, + reconnect, and delete semantics. +- Never inject OpenCode model-provider credentials into the sandbox. +- Run the generic environment contract suite against a real provider when + credentials are available. + +Result: a provider-backed Workspace can be created and reconnected directly. + +### 8. Prove The Hosted Coding Slice + +- Create a provider-backed Workspace containing a fixture repository. +- Create a V2 Session whose Location references that Workspace. +- Verify that the central server has no copy of the checkout. +- Prompt the Session to read a file, edit it, search it, and run `git status`. +- Interrupt a long-running foreground command. +- Suspend the Workspace, reconnect it, and verify that edited files remain. +- Confirm that a normal local Session still passes its existing test suite. + +Result: the milestone goal is complete. + +### 9. Add Additional Providers + +- Run Daytona and Modal implementations through the same contract suite, then + add E2B if it reveals another required lifecycle shape. +- Document where each lifecycle maps differently while preserving OpenCode's + observable contract. +- Add capabilities only when a required operation cannot be represented by the + common contract. + +Result: provider genericity is demonstrated rather than assumed. + +## Acceptance Tests + +The first milestone is accepted when all of the following hold: + +- A Session with no `workspaceID` behaves exactly as before. +- A Session with a hosted Workspace boots without its directory on the server. +- `read`, `write`, `edit`, `patch`, `glob`, `grep`, and foreground `shell` + operate only in the hosted environment. +- Permission checks and tool settlement remain central. +- Shell interruption reaches the provider process. +- Interrupting a waiting tool fiber, timing it out, or failing before wait does + not leave its provider process running. +- A stale conditional edit fails rather than overwriting changed content. +- Workspace metadata can be read without resuming compute. +- Reconnection targets the existing provider resource when available. +- A crash after persisting a replacement binding leaves placement reconnectable + and its retired resource eligible for reconciliation. +- Suspension followed by reconnection retains files. +- Suspension waits for active Workspace leases and admits no new operations + until its binding transition is durable. +- Deletion removes retained provider state. +- No model-provider credential is present in sandbox environment variables. +- Hosted commands do not inherit the OpenCode server's environment, shell, or + platform decisions. +- Existing and newly created paths cannot escape the hosted Workspace through + symlinks or missing parent directories. +- The same environment contract tests pass against local and hosted + implementations. + +## Deferred Design Pressure + +The following concerns are real but do not expand the first milestone: + +- Mutating remote requests need idempotency before automatic network retries. +- Background jobs need durable ownership and reconnection semantics. +- Workspace sharing needs a mutation lease or another concurrency policy. +- Provider migration needs an explicit file-transfer or snapshot contract. +- Portable plugins need explicit Workspace capabilities; ambient `fs` and + process imports remain local-only. +- A resident OpenCode worker becomes useful for SSH-only providers, richer Git + operations, PTY, LSP, watches, and detached commands. +- Clustered Session execution and stale-runner fencing remain separate from + sandbox placement. + +## Research + +Disposable Daytona, Modal, and Vercel tracer experiments informed this plan and +were removed after review. Vercel was exercised live through create, reconnect, +file I/O, foreground command interruption, stop, resume with retained files, +and deletion. Daytona and Modal adapters were type-checked but not run because +credentials were unavailable.