From b28546a6a5b8670671a746ca0f72bec1a296d408 Mon Sep 17 00:00:00 2001 From: Kit Langton Date: Thu, 4 Jun 2026 20:28:48 -0400 Subject: [PATCH] feat(core): register v2 system context sources --- CONTEXT.md | 11 +- packages/core/src/instruction-context.ts | 71 +++++++ packages/core/src/location-layer.ts | 7 +- packages/core/src/plugin/boot.ts | 4 + packages/core/src/session/context-epoch.ts | 4 +- packages/core/src/session/runner/llm.ts | 8 +- ...-context.ts => system-context-builtins.ts} | 22 +-- packages/core/src/system-context-registry.ts | 48 +++++ .../core/test/instruction-context.test.ts | 185 ++++++++++++++++++ .../core/test/session-runner-recorded.test.ts | 37 ++-- packages/core/test/session-runner.test.ts | 47 ++--- ...est.ts => system-context-builtins.test.ts} | 72 +++++-- .../core/test/system-context-registry.test.ts | 113 +++++++++++ specs/v2/refreshable-context-sources.md | 66 ++++--- 14 files changed, 588 insertions(+), 107 deletions(-) create mode 100644 packages/core/src/instruction-context.ts rename packages/core/src/{session-system-context.ts => system-context-builtins.ts} (70%) create mode 100644 packages/core/src/system-context-registry.ts create mode 100644 packages/core/test/instruction-context.test.ts rename packages/core/test/{session-system-context.test.ts => system-context-builtins.test.ts} (51%) create mode 100644 packages/core/test/system-context-registry.test.ts diff --git a/CONTEXT.md b/CONTEXT.md index 2758a944cf..562cd43dcf 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -12,6 +12,9 @@ _Avoid_: System prompt One independently observed typed value within the **System Context**, represented by a stable key, JSON codec, infallible loader, pure baseline/update renderers, and an optional removal renderer for dynamic sources. _Avoid_: Prompt fragment +**System Context Registry**: +The Location-scoped registry of ordered, scoped producers that contribute to the current **System Context**. + **Mid-Conversation System Message**: A durable chronological instruction that tells the model the newly effective state of a changed **Context Source**. _Avoid_: System update, system notification, raw text diff @@ -35,6 +38,7 @@ The point immediately before a provider call, after durable input promotion and ## Relationships - A **System Context** is an opaque carrier composed from zero or more **Context Sources**. +- The **System Context Registry** uses stable-keyed scoped contributions to assemble the current **System Context**; contributor removal naturally removes its sources at the next **Safe Provider-Turn Boundary**. - A changed **Context Source** may produce one **Mid-Conversation System Message** containing its newly effective state. - A **Mid-Conversation System Message** persists the exact combined rendered text sent to the model. - The current **Context Snapshot** advances atomically with the corresponding durable **Mid-Conversation System Message**. @@ -45,7 +49,7 @@ The point immediately before a provider call, after durable input promotion and - The first provider turn renders the latest **Baseline System Context** and initializes its **Context Snapshot** without emitting a redundant **Mid-Conversation System Message**. - Compaction starts a new **Context Epoch** with a freshly rendered **Baseline System Context** and **Context Snapshot**; prior **Mid-Conversation System Messages** remain durable audit history but leave projected model history. - A newly registered core or plugin-defined **Context Source** absent from the current snapshot emits its baseline rendering once at the next **Safe Provider-Turn Boundary**. -- **Context Source** keys are stable and namespaced; duplicate keys fail composition. `SystemContext.combine(...)` preserves caller order; future plugin-source assembly must append plugin-defined sources in lexicographic key order so rendered context remains deterministic. +- **Context Source** keys are stable and namespaced; duplicate keys fail composition. `SystemContext.combine(...)` preserves caller order; the **System Context Registry** evaluates producers concurrently and combines them in stable contribution-key order so rendered context remains deterministic. - Each **Context Source** loader returns one coherent typed value. `SystemContext.make(...)` hides that value type so differently typed sources compose uniformly. Its codec compares and stores that value; its pure renderers produce model-visible baseline, update, and removal text only when needed. - `SystemContext.initialize(...)` observes a composed **System Context** once and produces a fresh **Baseline System Context** with its **Context Snapshot**. - `SystemContext.reconcile(...)` observes a composed **System Context** once and returns exactly one next action: unchanged, updated, replaced, or replacement blocked. @@ -56,7 +60,8 @@ The point immediately before a provider call, after durable input promotion and - A discovered nested project instruction remains active for the session while it stays in the same location and is folded into later **Baseline System Contexts** after compaction. - Location-scoped services naturally re-resolve effective context when a moved session next runs in its destination location. - Instruction discovery, source identity, persistence, and file loading belong to the instruction service; the **System Context** abstraction only composes effectful producers and renders loaded values. -- Plugin-defined **Context Sources** register through a scoped replayable registry so plugin hot reload adds and removes sources predictably. +- The first instruction-service slice observes global and upward project `AGENTS.md` files as one ordered aggregate **Context Source** at each **Safe Provider-Turn Boundary**. +- Built-in, instruction, and plugin-defined context producers register through the **System Context Registry** with stable contribution keys so plugin hot reload and Location-scope cleanup add and remove sources predictably. - Context source changes never wake idle sessions; the next naturally scheduled **Safe Provider-Turn Boundary** loads and compares current values lazily. - Once admitted, a **Mid-Conversation System Message** remains durable even if the following provider attempt fails and is replayed unchanged on retry. - **Mid-Conversation System Messages** remain durable Session-message history; normal user-facing transcript surfaces may hide them. @@ -67,7 +72,7 @@ The point immediately before a provider call, after durable input promotion and - Compaction or a model/provider switch starts a new **Context Epoch** because the baseline can be replaced without preserving the prior provider cache. - A model/provider switch always starts a new **Context Epoch** while preserving chronological conversation history. - A **Mid-Conversation System Message** lowers to the provider's native chronological instruction role when supported and to a wrapped chronological fallback otherwise. -- When an effective instruction file changes, its **Mid-Conversation System Message** includes the complete current contents and supersedes the prior version from that source; when it is removed, the message states that it no longer applies. +- When the effective aggregate instruction set changes, its **Mid-Conversation System Message** includes the complete current ordered set and supersedes the prior aggregate value; when no ambient instructions remain, the message states that previously loaded instructions no longer apply. ## Example dialogue diff --git a/packages/core/src/instruction-context.ts b/packages/core/src/instruction-context.ts new file mode 100644 index 0000000000..c764c13067 --- /dev/null +++ b/packages/core/src/instruction-context.ts @@ -0,0 +1,71 @@ +export * as InstructionContext from "./instruction-context" + +import { Array, Effect, Layer, Schema } from "effect" +import { join } from "path" +import { FSUtil } from "./fs-util" +import { Global } from "./global" +import { Location } from "./location" +import { AbsolutePath } from "./schema" +import { SystemContext } from "./system-context" +import { SystemContextRegistry } from "./system-context-registry" + +class File extends Schema.Class("InstructionContext.File")({ + path: AbsolutePath, + content: Schema.String, +}) {} + +const Files = Schema.Array(File) + +export const layer = Layer.effectDiscard( + Effect.gen(function* () { + const fs = yield* FSUtil.Service + const global = yield* Global.Service + const location = yield* Location.Service + const registry = yield* SystemContextRegistry.Service + + const source = (value: ReadonlyArray | SystemContext.Unavailable) => + SystemContext.make({ + key: SystemContext.Key.make("core/instructions"), + codec: Schema.toCodecJson(Files), + load: Effect.succeed(value), + baseline: render, + update: (_previous, current) => render(current), + removed: () => "Previously loaded instructions no longer apply.", + }) + + const observe = Effect.fn("InstructionContext.observe")(function* () { + const discovered = new Set( + (yield* fs.up({ targets: ["AGENTS.md"], start: location.directory, stop: location.project.directory })).map( + FSUtil.resolve, + ), + ) + const paths = Array.dedupe([FSUtil.resolve(join(global.config, "AGENTS.md")), ...discovered]) + const files = yield* Effect.forEach( + paths, + (path) => + fs.readFileStringSafe(path).pipe( + Effect.map((content) => (content === undefined ? undefined : new File({ path: AbsolutePath.make(path), content }))), + ), + { concurrency: "unbounded" }, + ) + if (files.some((file, index) => file === undefined && discovered.has(paths[index]))) return SystemContext.unavailable + return files.filter((file): file is File => file !== undefined) + }) + + yield* registry.contribute({ + key: "core/instructions", + load: observe().pipe( + Effect.map((files) => + files === SystemContext.unavailable ? source(files) : files.length === 0 ? SystemContext.empty : source(files), + ), + Effect.catch(() => Effect.succeed(source(SystemContext.unavailable))), + ), + }) + }), +) + +export const locationLayer = layer + +function render(files: ReadonlyArray) { + return files.map((file) => `Instructions from: ${file.path}\n${file.content}`).join("\n\n") +} diff --git a/packages/core/src/location-layer.ts b/packages/core/src/location-layer.ts index 194afd9013..d6c01e1c86 100644 --- a/packages/core/src/location-layer.ts +++ b/packages/core/src/location-layer.ts @@ -40,13 +40,14 @@ import { RequestExecutor } from "@opencode-ai/llm/route" import * as SessionRunnerLLM from "./session/runner/llm" import { SessionRunnerModel } from "./session/runner/model" import { SessionRunCoordinator } from "./session/run-coordinator" -import { SessionSystemContext } from "./session-system-context" +import { SystemContextBuiltIns } from "./system-context-builtins" import { FetchHttpClient } from "effect/unstable/http" export class LocationServiceMap extends LayerMap.Service()("@opencode/example/LocationServiceMap", { lookup: (ref: Location.Ref) => { const location = Location.layer(ref) const permissionsAndTools = ToolRegistry.layer.pipe(Layer.provideMerge(PermissionV2.locationLayer)) + const systemContext = SystemContextBuiltIns.locationLayer const services = Layer.mergeAll( location, Policy.locationLayer, @@ -56,12 +57,12 @@ export class LocationServiceMap extends LayerMap.Service()(" Catalog.locationLayer, CommandV2.locationLayer, AgentV2.locationLayer, - PluginBoot.locationLayer, + PluginBoot.locationLayer.pipe(Layer.provide(systemContext)), FileSystem.locationLayer, Watcher.locationLayer, Pty.locationLayer, SkillV2.locationLayer, - SessionSystemContext.locationLayer, + systemContext, permissionsAndTools, LocationMutation.locationLayer.pipe(Layer.orDie), ).pipe(Layer.provideMerge(location)) diff --git a/packages/core/src/plugin/boot.ts b/packages/core/src/plugin/boot.ts index 6a589a1ef0..7cdc54fcf2 100644 --- a/packages/core/src/plugin/boot.ts +++ b/packages/core/src/plugin/boot.ts @@ -25,6 +25,7 @@ import { EnvPlugin } from "./env" import { ModelsDevPlugin } from "./models-dev" import { ProviderPlugins } from "./provider" import { SkillV2 } from "../skill" +import { SystemContextRegistry } from "../system-context-registry" type Plugin = { id: PluginV2.ID @@ -42,6 +43,7 @@ type Plugin = { | Config.Service | ModelsDev.Service | SkillV2.Service + | SystemContextRegistry.Service > } @@ -67,6 +69,7 @@ export const layer = Layer.effect( const fs = yield* FSUtil.Service const global = yield* Global.Service const skill = yield* SkillV2.Service + const systemContext = yield* SystemContextRegistry.Service const done = yield* Deferred.make() const add = Effect.fn("PluginBoot.add")(function* (input: Plugin) { @@ -86,6 +89,7 @@ export const layer = Layer.effect( Effect.provideService(Global.Service, global), Effect.provideService(SkillV2.Service, skill), Effect.provideService(PluginV2.Service, plugin), + Effect.provideService(SystemContextRegistry.Service, systemContext), ), }) }) diff --git a/packages/core/src/session/context-epoch.ts b/packages/core/src/session/context-epoch.ts index 2b9fdfb9f3..7374b58eff 100644 --- a/packages/core/src/session/context-epoch.ts +++ b/packages/core/src/session/context-epoch.ts @@ -4,8 +4,8 @@ import { and, eq, isNull, lt, or, sql } from "drizzle-orm" import { DateTime, Effect, Schema } from "effect" import type { Database } from "../database/database" import { EventV2 } from "../event" -import { SessionSystemContext } from "../session-system-context" import { SystemContext } from "../system-context" +import { SystemContextRegistry } from "../system-context-registry" import { SessionEvent } from "./event" import { SessionMessageID } from "./message-id" import { SessionSchema } from "./schema" @@ -16,7 +16,7 @@ type DatabaseService = Database.Interface["db"] export const prepare = Effect.fn("SessionContextEpoch.prepare")(function* ( db: DatabaseService, events: EventV2.Interface, - context: SessionSystemContext.Interface, + context: SystemContextRegistry.Interface, sessionID: SessionSchema.ID, ) { const [value, stored] = yield* Effect.all([context.load(), find(db, sessionID)], { concurrency: "unbounded" }) diff --git a/packages/core/src/session/runner/llm.ts b/packages/core/src/session/runner/llm.ts index 417bf8fd15..f67fe18f8e 100644 --- a/packages/core/src/session/runner/llm.ts +++ b/packages/core/src/session/runner/llm.ts @@ -14,7 +14,7 @@ import { SessionRunnerModel } from "./model" import { Database } from "../../database/database" import { SessionInput } from "../input" import { QuestionV2 } from "../../question" -import { SessionSystemContext } from "../../session-system-context" +import { SystemContextRegistry } from "../../system-context-registry" import { SessionContextEpoch } from "../context-epoch" /** @@ -36,8 +36,8 @@ import { SessionContextEpoch } from "../context-epoch" * - [x] Resolve the selected model through the location-scoped runner environment. * - [ ] Load the selected agent and effective permissions. * - [ ] Build provider/model-specific base instructions and environment facts. - * - [ ] Load configured project instructions such as `AGENTS.md`, remote instructions, and - * nearby nested instructions discovered while files are read. + * - [x] Load global and upward project `AGENTS.md` instructions. + * - [ ] Load configured and remote instructions plus nearby nested instructions discovered while files are read. * - [ ] List available skills in the system prompt and expose a tool for loading skill bodies. * - [ ] Resolve referenced files, directories, agents, repositories, MCP resources, and media. * - [ ] Apply steering reminders, plugin transforms, and structured-output policy. @@ -87,7 +87,7 @@ export const layer = Layer.effect( const tools = yield* ToolRegistry.Service const models = yield* SessionRunnerModel.Service const store = yield* SessionStore.Service - const systemContext = yield* SessionSystemContext.Service + const systemContext = yield* SystemContextRegistry.Service const db = (yield* Database.Service).db const getSession = Effect.fn("SessionRunner.getSession")(function* (sessionID: SessionSchema.ID) { const session = yield* store.get(sessionID) diff --git a/packages/core/src/session-system-context.ts b/packages/core/src/system-context-builtins.ts similarity index 70% rename from packages/core/src/session-system-context.ts rename to packages/core/src/system-context-builtins.ts index 1b66da721e..f1ff09bc91 100644 --- a/packages/core/src/session-system-context.ts +++ b/packages/core/src/system-context-builtins.ts @@ -1,19 +1,15 @@ -export * as SessionSystemContext from "./session-system-context" +export * as SystemContextBuiltIns from "./system-context-builtins" -import { Context, DateTime, Effect, Layer, Schema } from "effect" +import { DateTime, Effect, Layer, Schema } from "effect" +import { InstructionContext } from "./instruction-context" import { Location } from "./location" import { SystemContext } from "./system-context" +import { SystemContextRegistry } from "./system-context-registry" -export interface Interface { - readonly load: () => Effect.Effect -} - -export class Service extends Context.Service()("@opencode/v2/SessionSystemContext") {} - -export const layer = Layer.effect( - Service, +const builtIns = Layer.effectDiscard( Effect.gen(function* () { const location = yield* Location.Service + const registry = yield* SystemContextRegistry.Service const environment = [ "", ` Working directory: ${location.directory}`, @@ -40,8 +36,12 @@ export const layer = Layer.effect( }), ]) - return Service.of({ load: () => Effect.succeed(context) }) + yield* registry.contribute({ key: "core/builtins", load: Effect.succeed(context) }) }), ) +export const layer = Layer.mergeAll(builtIns, InstructionContext.layer).pipe( + Layer.provideMerge(SystemContextRegistry.layer), +) + export const locationLayer = layer diff --git a/packages/core/src/system-context-registry.ts b/packages/core/src/system-context-registry.ts new file mode 100644 index 0000000000..65c9b11a19 --- /dev/null +++ b/packages/core/src/system-context-registry.ts @@ -0,0 +1,48 @@ +export * as SystemContextRegistry from "./system-context-registry" + +import { Context, Effect, Layer, Ref, Scope } from "effect" +import { SystemContext } from "./system-context" + +export interface Contribution { + readonly key: string + readonly load: Effect.Effect +} + +export interface Interface { + readonly contribute: (contribution: Contribution) => Effect.Effect + readonly load: () => Effect.Effect +} + +export class Service extends Context.Service()("@opencode/v2/SystemContextRegistry") {} + +export const layer = Layer.effect( + Service, + Effect.gen(function* () { + const contributions = yield* Ref.make>([]) + + return Service.of({ + contribute: Effect.fn("SystemContextRegistry.contribute")(function* (contribution) { + yield* Effect.acquireRelease( + Ref.modify(contributions, (current) => { + if (current.some((item) => item.key === contribution.key)) return [false, current] + return [true, [...current, contribution]] + }).pipe( + Effect.flatMap((added) => + added ? Effect.void : Effect.die(`Duplicate system context contribution key: ${contribution.key}`), + ), + Effect.as(contribution), + ), + (entry) => Ref.update(contributions, (current) => current.filter((item) => item !== entry)), + ) + }), + load: Effect.fn("SystemContextRegistry.load")(function* () { + const current = (yield* Ref.get(contributions)).toSorted((a, b) => a.key.localeCompare(b.key)) + return SystemContext.combine( + yield* Effect.forEach(current, (contribution) => contribution.load, { concurrency: "unbounded" }), + ) + }), + }) + }), +) + +export const locationLayer = layer diff --git a/packages/core/test/instruction-context.test.ts b/packages/core/test/instruction-context.test.ts new file mode 100644 index 0000000000..2eea3586fb --- /dev/null +++ b/packages/core/test/instruction-context.test.ts @@ -0,0 +1,185 @@ +import { describe, expect } from "bun:test" +import { Effect, Layer } from "effect" +import fs from "fs/promises" +import path from "path" +import { FSUtil } from "@opencode-ai/core/fs-util" +import { Global } from "@opencode-ai/core/global" +import { InstructionContext } from "@opencode-ai/core/instruction-context" +import { Location } from "@opencode-ai/core/location" +import { AbsolutePath } from "@opencode-ai/core/schema" +import { SystemContext } from "@opencode-ai/core/system-context" +import { SystemContextRegistry } from "@opencode-ai/core/system-context-registry" +import { location } from "./fixture/location" +import { tmpdir } from "./fixture/tmpdir" +import { testEffect } from "./lib/effect" + +const it = testEffect(Layer.empty) + +describe("InstructionContext", () => { + it.live("loads global and upward project AGENTS.md files as one aggregate context", () => + Effect.acquireRelease( + Effect.promise(() => tmpdir()), + (tmp) => Effect.promise(() => tmp[Symbol.asyncDispose]()), + ).pipe( + Effect.flatMap((tmp) => + Effect.gen(function* () { + const global = path.join(tmp.path, "global") + const project = path.join(tmp.path, "project") + const directory = path.join(project, "packages", "core") + const outside = path.join(tmp.path, "AGENTS.md") + const globalFile = path.join(global, "AGENTS.md") + const projectFile = path.join(project, "AGENTS.md") + const packageFile = path.join(directory, "AGENTS.md") + yield* Effect.promise(async () => { + await fs.mkdir(global, { recursive: true }) + await fs.mkdir(directory, { recursive: true }) + await fs.writeFile(outside, "outside") + await fs.writeFile(globalFile, "global") + await fs.writeFile(projectFile, "project") + await fs.writeFile(packageFile, "package") + }) + + const load = SystemContextRegistry.Service.pipe( + Effect.flatMap((service) => service.load()), + Effect.provide(InstructionContext.layer.pipe(Layer.provideMerge(SystemContextRegistry.layer))), + Effect.provide(FSUtil.defaultLayer), + Effect.provide(Global.layerWith({ config: global })), + Effect.provide( + Layer.succeed( + Location.Service, + Location.Service.of( + location( + { directory: AbsolutePath.make(directory) }, + { projectDirectory: AbsolutePath.make(project) }, + ), + ), + ), + ), + ) + + const initialized = yield* SystemContext.initialize(yield* load) + expect(initialized.baseline).toBe( + [ + `Instructions from: ${globalFile}\nglobal`, + `Instructions from: ${packageFile}\npackage`, + `Instructions from: ${projectFile}\nproject`, + ].join("\n\n"), + ) + expect(initialized.baseline).not.toContain("outside") + + yield* Effect.promise(() => fs.writeFile(packageFile, "changed")) + expect(yield* SystemContext.reconcile(yield* load, initialized.snapshot)).toMatchObject({ + _tag: "Updated", + text: expect.stringContaining(`Instructions from: ${packageFile}\nchanged`), + }) + + yield* Effect.promise(() => Promise.all([fs.rm(globalFile), fs.rm(packageFile), fs.rm(projectFile)])) + expect(yield* SystemContext.reconcile(yield* load, initialized.snapshot)).toEqual({ + _tag: "Updated", + text: "Previously loaded instructions no longer apply.", + snapshot: {}, + }) + }), + ), + ), + ) + + it.live("keeps an empty AGENTS.md as available context", () => + Effect.acquireRelease( + Effect.promise(() => tmpdir()), + (tmp) => Effect.promise(() => tmp[Symbol.asyncDispose]()), + ).pipe( + Effect.flatMap((tmp) => + Effect.gen(function* () { + const file = path.join(tmp.path, "AGENTS.md") + yield* Effect.promise(() => fs.writeFile(file, "")) + const context = yield* SystemContextRegistry.Service.pipe( + Effect.flatMap((service) => service.load()), + Effect.provide(InstructionContext.layer.pipe(Layer.provideMerge(SystemContextRegistry.layer))), + Effect.provide(FSUtil.defaultLayer), + Effect.provide(Global.layerWith({ config: path.join(tmp.path, "global") })), + Effect.provide( + Layer.succeed( + Location.Service, + Location.Service.of(location({ directory: AbsolutePath.make(tmp.path) })), + ), + ), + ) + + expect((yield* SystemContext.initialize(context)).baseline).toBe(`Instructions from: ${file}\n`) + }), + ), + ), + ) + + it.effect("preserves admitted instructions while observation is unavailable", () => + Effect.gen(function* () { + const failingFS = Layer.effect( + FSUtil.Service, + FSUtil.Service.pipe( + Effect.map((fs) => FSUtil.Service.of({ ...fs, up: () => Effect.fail(new FSUtil.FileSystemError({ method: "up" })) })), + ), + ).pipe(Layer.provide(FSUtil.defaultLayer)) + const context = yield* SystemContextRegistry.Service.pipe( + Effect.flatMap((service) => service.load()), + Effect.provide(InstructionContext.layer.pipe(Layer.provideMerge(SystemContextRegistry.layer))), + Effect.provide(failingFS), + Effect.provide(Global.layerWith({ config: "/global" })), + Effect.provide( + Layer.succeed( + Location.Service, + Location.Service.of(location({ directory: AbsolutePath.make("/repo") })), + ), + ), + ) + + expect( + yield* SystemContext.reconcile(context, { + "core/instructions": { + value: [{ path: "/repo/AGENTS.md", content: "old" }], + removed: "Previously loaded instructions no longer apply.", + }, + }), + ).toEqual({ _tag: "Unchanged" }) + }), + ) + + it.effect("preserves admitted instructions when a discovered file disappears before read", () => + Effect.gen(function* () { + const file = AbsolutePath.make("/repo/AGENTS.md") + const racingFS = Layer.effect( + FSUtil.Service, + FSUtil.Service.pipe( + Effect.map((fs) => + FSUtil.Service.of({ + ...fs, + up: () => Effect.succeed([file]), + readFileStringSafe: () => Effect.succeed(undefined), + }), + ), + ), + ).pipe(Layer.provide(FSUtil.defaultLayer)) + const context = yield* SystemContextRegistry.Service.pipe( + Effect.flatMap((service) => service.load()), + Effect.provide(InstructionContext.layer.pipe(Layer.provideMerge(SystemContextRegistry.layer))), + Effect.provide(racingFS), + Effect.provide(Global.layerWith({ config: "/global" })), + Effect.provide( + Layer.succeed( + Location.Service, + Location.Service.of(location({ directory: AbsolutePath.make("/repo") })), + ), + ), + ) + + expect( + yield* SystemContext.reconcile(context, { + "core/instructions": { + value: [{ path: file, content: "old" }], + removed: "Previously loaded instructions no longer apply.", + }, + }), + ).toEqual({ _tag: "Unchanged" }) + }), + ) +}) diff --git a/packages/core/test/session-runner-recorded.test.ts b/packages/core/test/session-runner-recorded.test.ts index c9252b86bb..edd244e432 100644 --- a/packages/core/test/session-runner-recorded.test.ts +++ b/packages/core/test/session-runner-recorded.test.ts @@ -19,8 +19,8 @@ import { SessionRunnerModel } from "@opencode-ai/core/session/runner/model" import { ToolRegistry } from "@opencode-ai/core/tool/registry" import { SessionTable } from "@opencode-ai/core/session/sql" import { SessionStore } from "@opencode-ai/core/session/store" -import { SessionSystemContext } from "@opencode-ai/core/session-system-context" import { SystemContext } from "@opencode-ai/core/system-context" +import { SystemContextRegistry } from "@opencode-ai/core/system-context-registry" import { describe, expect } from "bun:test" import { eq } from "drizzle-orm" import { Effect, Layer, Schema } from "effect" @@ -57,22 +57,25 @@ const model = OpenAIChat.route }) .model({ id: "gpt-4o-mini" }) const models = SessionRunnerModel.layerWith(() => Effect.succeed(model)) -const systemContext = Layer.succeed( - SessionSystemContext.Service, - SessionSystemContext.Service.of({ - load: () => - Effect.succeed( - SystemContext.make({ - key: SystemContext.Key.make("test/context"), - codec: Schema.toCodecJson(Schema.String), - load: Effect.succeed("Recorded context"), - baseline: String, - update: (_previous, current) => current, - removed: () => "Recorded context removed", - }), - ), - }), -) +const systemContext = Layer.effectDiscard( + SystemContextRegistry.Service.pipe( + Effect.flatMap((registry) => + registry.contribute({ + key: "test/context", + load: Effect.succeed( + SystemContext.make({ + key: SystemContext.Key.make("test/context"), + codec: Schema.toCodecJson(Schema.String), + load: Effect.succeed("Recorded context"), + baseline: String, + update: (_previous, current) => current, + removed: () => "Recorded context removed", + }), + ), + }), + ), + ), +).pipe(Layer.provideMerge(SystemContextRegistry.layer)) const runner = SessionRunnerLLM.defaultLayer.pipe( Layer.provide(database), Layer.provide(store), diff --git a/packages/core/test/session-runner.test.ts b/packages/core/test/session-runner.test.ts index a960061f45..ef8b8617c8 100644 --- a/packages/core/test/session-runner.test.ts +++ b/packages/core/test/session-runner.test.ts @@ -34,8 +34,8 @@ import { ApplicationTools } from "@opencode-ai/core/tool/application-tools" import { NativeTool } from "@opencode-ai/core/tool/native" import { SessionContextEpochTable, SessionInputTable, SessionMessageTable, SessionTable } from "@opencode-ai/core/session/sql" import { SessionStore } from "@opencode-ai/core/session/store" -import { SessionSystemContext } from "@opencode-ai/core/session-system-context" import { SystemContext } from "@opencode-ai/core/system-context" +import { SystemContextRegistry } from "@opencode-ai/core/system-context-registry" import { ModelV2 } from "@opencode-ai/core/model" import { ProviderV2 } from "@opencode-ai/core/provider" import { Cause, DateTime, Deferred, Effect, Fiber, Layer, Schema, Stream } from "effect" @@ -145,28 +145,31 @@ const systemContextKey = SystemContext.Key.make("test/context") let systemBaseline = "Initial context" let systemRemoved = false let systemUnavailable = false -const systemContext = Layer.succeed( - SessionSystemContext.Service, - SessionSystemContext.Service.of({ - load: () => - Effect.succeed( - SystemContext.combine( - systemRemoved - ? [] - : [ - SystemContext.make({ - key: systemContextKey, - codec: Schema.toCodecJson(Schema.String), - load: Effect.sync(() => (systemUnavailable ? SystemContext.unavailable : systemBaseline)), - baseline: String, - update: (_previous, current) => current, - removed: () => "System context source removed: test/context", - }), - ], +const systemContext = Layer.effectDiscard( + SystemContextRegistry.Service.pipe( + Effect.flatMap((registry) => + registry.contribute({ + key: "test/context", + load: Effect.sync(() => + SystemContext.combine( + systemRemoved + ? [] + : [ + SystemContext.make({ + key: systemContextKey, + codec: Schema.toCodecJson(Schema.String), + load: Effect.sync(() => (systemUnavailable ? SystemContext.unavailable : systemBaseline)), + baseline: String, + update: (_previous, current) => current, + removed: () => "System context source removed: test/context", + }), + ], + ), ), - ), - }), -) + }), + ), + ), +).pipe(Layer.provideMerge(SystemContextRegistry.layer)) const runner = SessionRunnerLLM.layer.pipe( Layer.provide(database), Layer.provide(store), diff --git a/packages/core/test/session-system-context.test.ts b/packages/core/test/system-context-builtins.test.ts similarity index 51% rename from packages/core/test/session-system-context.test.ts rename to packages/core/test/system-context-builtins.test.ts index d0fb03afa7..ed854abcd8 100644 --- a/packages/core/test/session-system-context.test.ts +++ b/packages/core/test/system-context-builtins.test.ts @@ -2,9 +2,12 @@ import { describe, expect } from "bun:test" import { Effect, Layer } from "effect" import * as TestClock from "effect/testing/TestClock" import { Location } from "@opencode-ai/core/location" +import { FSUtil } from "@opencode-ai/core/fs-util" +import { Global } from "@opencode-ai/core/global" import { AbsolutePath } from "@opencode-ai/core/schema" -import { SessionSystemContext } from "@opencode-ai/core/session-system-context" import { SystemContext } from "@opencode-ai/core/system-context" +import { SystemContextBuiltIns } from "@opencode-ai/core/system-context-builtins" +import { SystemContextRegistry } from "@opencode-ai/core/system-context-registry" import { location } from "./fixture/location" import { testEffect } from "./lib/effect" @@ -12,24 +15,44 @@ const directory = AbsolutePath.make("/repo/packages/core") const projectDirectory = AbsolutePath.make("/repo") const timestamp = Date.parse("2026-06-03T12:00:00.000Z") const localDate = (time: number) => new Date(time).toDateString() +const locationLayer = Layer.succeed( + Location.Service, + Location.Service.of( + location({ directory }, { projectDirectory, vcs: { type: "git", store: AbsolutePath.make("/repo/.git") } }), + ), +) const it = testEffect( - SessionSystemContext.locationLayer.pipe( - Layer.provide( - Layer.succeed( - Location.Service, - Location.Service.of( - location({ directory }, { projectDirectory, vcs: { type: "git", store: AbsolutePath.make("/repo/.git") } }), - ), - ), + SystemContextBuiltIns.locationLayer.pipe( + Layer.provide(FSUtil.defaultLayer), + Layer.provide(Global.layerWith({ config: "/global" })), + Layer.provide(locationLayer), + ), +) +const instructionFS = Layer.effect( + FSUtil.Service, + FSUtil.Service.pipe( + Effect.map((fs) => + FSUtil.Service.of({ + ...fs, + up: () => Effect.succeed(["/repo/AGENTS.md"]), + readFileStringSafe: (path) => Effect.succeed(path === "/repo/AGENTS.md" ? "Be precise." : undefined), + }), ), ), +).pipe(Layer.provide(FSUtil.defaultLayer)) +const itWithInstructions = testEffect( + SystemContextBuiltIns.locationLayer.pipe( + Layer.provide(instructionFS), + Layer.provide(Global.layerWith({ config: "/global" })), + Layer.provide(locationLayer), + ), ) -describe("SessionSystemContext", () => { +describe("SystemContextBuiltIns", () => { it.effect("loads location-scoped environment and host-local date context", () => Effect.gen(function* () { yield* TestClock.setTime(timestamp) - const context = yield* SessionSystemContext.Service + const context = yield* SystemContextRegistry.Service const initialized = yield* SystemContext.initialize(yield* context.load()) expect(initialized.baseline).toBe( @@ -51,7 +74,7 @@ describe("SessionSystemContext", () => { it.effect("reconciles the date without repeating unchanged environment context", () => Effect.gen(function* () { yield* TestClock.setTime(timestamp) - const context = yield* SessionSystemContext.Service + const context = yield* SystemContextRegistry.Service const initialized = yield* SystemContext.initialize(yield* context.load()) yield* TestClock.setTime(timestamp + 24 * 60 * 60 * 1000) @@ -67,11 +90,34 @@ describe("SessionSystemContext", () => { it.effect("does not update again within the same local calendar day", () => Effect.gen(function* () { yield* TestClock.setTime(timestamp) - const context = yield* SessionSystemContext.Service + const context = yield* SystemContextRegistry.Service const initialized = yield* SystemContext.initialize(yield* context.load()) yield* TestClock.setTime(timestamp + 60 * 60 * 1000) expect(yield* SystemContext.reconcile(yield* context.load(), initialized.snapshot)).toEqual({ _tag: "Unchanged" }) }), ) + + itWithInstructions.effect("composes ambient instructions after built-in context", () => + Effect.gen(function* () { + yield* TestClock.setTime(timestamp) + const context = yield* SystemContextRegistry.Service + + expect((yield* SystemContext.initialize(yield* context.load())).baseline).toBe( + [ + "Here is some useful information about the environment you are running in:", + "", + ` Working directory: ${directory}`, + ` Workspace root folder: ${projectDirectory}`, + " Is directory a git repo: yes", + ` Platform: ${process.platform}`, + "", + "", + `Today's date: ${localDate(timestamp)}`, + "", + "Instructions from: /repo/AGENTS.md\nBe precise.", + ].join("\n"), + ) + }), + ) }) diff --git a/packages/core/test/system-context-registry.test.ts b/packages/core/test/system-context-registry.test.ts new file mode 100644 index 0000000000..b1191cb065 --- /dev/null +++ b/packages/core/test/system-context-registry.test.ts @@ -0,0 +1,113 @@ +import { describe, expect } from "bun:test" +import { Cause, Effect, Exit, Schema, Scope } from "effect" +import { SystemContext } from "@opencode-ai/core/system-context" +import { SystemContextRegistry } from "@opencode-ai/core/system-context-registry" +import { testEffect } from "./lib/effect" + +const contribution = (key: string, text: string, sourceKey = key) => ({ + key, + load: Effect.succeed( + SystemContext.make({ + key: SystemContext.Key.make(sourceKey), + codec: Schema.toCodecJson(Schema.String), + load: Effect.succeed(text), + baseline: String, + update: (_previous, current) => current, + }), + ), +}) + +const it = testEffect(SystemContextRegistry.layer) + +describe("SystemContextRegistry", () => { + it.effect("loads empty system context when there are no contributions", () => + Effect.gen(function* () { + const registry = yield* SystemContextRegistry.Service + + expect(yield* SystemContext.initialize(yield* registry.load())).toEqual({ baseline: "", snapshot: {} }) + }), + ) + + it.effect("loads scoped contributions in stable key order", () => + Effect.gen(function* () { + const registry = yield* SystemContextRegistry.Service + yield* registry.contribute(contribution("test/second", "second")) + yield* registry.contribute(contribution("test/first", "first")) + + expect((yield* SystemContext.initialize(yield* registry.load())).baseline).toBe("first\n\nsecond") + }), + ) + + it.effect("re-evaluates contribution producers on each load", () => + Effect.gen(function* () { + const registry = yield* SystemContextRegistry.Service + let loads = 0 + yield* registry.contribute({ + key: "test/dynamic", + load: Effect.sync(() => { + loads++ + return SystemContext.empty + }), + }) + + yield* registry.load() + yield* registry.load() + + expect(loads).toBe(2) + }), + ) + + it.effect("propagates contribution producer failures", () => + Effect.gen(function* () { + const registry = yield* SystemContextRegistry.Service + const failure = new Error("contribution failed") + yield* registry.contribute({ key: "test/failure", load: Effect.die(failure) }) + + const exit = yield* registry.load().pipe(Effect.exit) + + expect(Exit.isFailure(exit)).toBe(true) + if (Exit.isFailure(exit)) expect(Cause.squash(exit.cause)).toBe(failure) + }), + ) + + it.effect("rejects duplicate source keys from separate contributions", () => + Effect.gen(function* () { + const registry = yield* SystemContextRegistry.Service + yield* registry.contribute(contribution("test/first", "first", "test/duplicate")) + yield* registry.contribute(contribution("test/second", "second", "test/duplicate")) + + const exit = yield* registry.load().pipe(Effect.exit) + + expect(Exit.isFailure(exit)).toBe(true) + if (Exit.isFailure(exit)) { + expect(Cause.squash(exit.cause)).toBeInstanceOf(SystemContext.DuplicateKeyError) + expect(Cause.squash(exit.cause)).toMatchObject({ key: SystemContext.Key.make("test/duplicate") }) + } + }), + ) + + it.effect("rejects duplicate contribution keys", () => + Effect.gen(function* () { + const registry = yield* SystemContextRegistry.Service + yield* registry.contribute(contribution("test/duplicate", "first")) + + const exit = yield* registry.contribute(contribution("test/duplicate", "second", "test/other")).pipe(Effect.exit) + + expect(Exit.isFailure(exit)).toBe(true) + if (Exit.isFailure(exit)) expect(Cause.pretty(exit.cause)).toContain("Duplicate system context contribution key") + }), + ) + + it.effect("removes a contribution when its owning scope closes", () => + Effect.gen(function* () { + const registry = yield* SystemContextRegistry.Service + const scope = yield* Scope.make() + yield* registry.contribute(contribution("test/scoped", "scoped")).pipe(Scope.provide(scope)) + + expect((yield* SystemContext.initialize(yield* registry.load())).baseline).toBe("scoped") + + yield* Scope.close(scope, Exit.void) + expect(yield* SystemContext.initialize(yield* registry.load())).toEqual({ baseline: "", snapshot: {} }) + }), + ) +}) diff --git a/specs/v2/refreshable-context-sources.md b/specs/v2/refreshable-context-sources.md index d1d8ff70ee..29a6ce4579 100644 --- a/specs/v2/refreshable-context-sources.md +++ b/specs/v2/refreshable-context-sources.md @@ -15,7 +15,7 @@ source signal -> Context Epoch compares and admits exact changed bytes durably ``` -The first ambient `AGENTS.md` slice will not depend on filesystem watching. It will directly observe local instruction state whenever `SessionSystemContext.load()` naturally runs before a provider turn. +The first ambient `AGENTS.md` slice will not depend on filesystem watching. Its scoped contributor will directly observe local instruction state whenever `SystemContextRegistry.load()` naturally runs before a provider turn. Watcher-backed caches are a later efficiency optimization for roots with proven subscription coverage. URLs remain separate observations with an independently chosen refresh policy. @@ -28,6 +28,7 @@ Watcher-backed caches are a later efficiency optimization for roots with proven | `State.create(...)` | Rebuild replayable plugin and config contribution state from scoped transforms. | | `SynchronizedRef.modifyEffect(...)` | Serialize effectful state refresh and store the next value only after success. | | `SystemContext` | Convert coherent source samples into one immutable baseline, chronological updates, unavailable state, and removal tombstones. | +| `SystemContextRegistry` | Assemble Location-scoped built-in, instruction, and plugin context producers in stable contribution-key order. | | `LocationServiceMap` | Own and clean up Location-scoped services, watcher subscriptions, and observation caches together. | The missing reusable piece is deliberately small: retain the last successful value, mark it stale, and serialize refresh attempts. @@ -91,9 +92,9 @@ stateDiagram-v2 Stale --> Stale: invalidate or reload fails ``` -### Why Custom Instead Of Effect `Cache` +### Why Custom Instead Of Existing Effect Caches -Effect `Cache`, `ScopedCache`, and `Effect.cachedInvalidateWithTTL(...)` cache failed exits. Context-source observation requires a narrower rule: +Effect `Cache`, `ScopedCache`, and `Effect.cachedInvalidateWithTTL(...)` cache failed exits. Effect `Resource` preserves its prior value after a failed refresh, but eagerly acquires and does not reload lazily after explicit invalidation. Context-source observation requires the narrower lazy `Empty` / `Fresh` / `Stale` rule: ```text failed refresh @@ -147,23 +148,24 @@ embedded skill -> direct value, no refreshable ``` -## Ambient Instruction Service +## Ambient Instruction Contributor -Add a Location-scoped service: +Add a Location-scoped contributor to `SystemContextRegistry`: ```ts -export interface InstructionContext.Interface { - readonly loadAmbient: () => Effect.Effect -} +yield* registry.contribute({ + key: "core/instructions", + load: loadAmbientInstructions(), +}) ``` -`InstructionContext` owns instruction discovery, stable source identity, deterministic ordering, and source loading. `SystemContext` remains unaware of files and URLs. +`InstructionContext` owns instruction discovery, deterministic ordering, and source loading. `SystemContextRegistry` owns contributor composition and lifecycle. `SystemContext` remains unaware of files and URLs. -Each effective instruction becomes one independently keyed `SystemContext.Source` closed into the aggregate context with `SystemContext.make(...)`: +The first slice closes one coherent ordered instruction set into an aggregate source: ```text -core/instructions/file/ -core/instructions/url/ +core/instructions +-> [{ path, content }, ...] ``` Rendered text retains the human-readable source identity: @@ -185,46 +187,46 @@ The first implementation directly observes global and upward project `AGENTS.md` ```mermaid sequenceDiagram participant Runner as Safe Provider Boundary + participant Registry as System Context Registry participant Instructions as Instruction Context participant Files participant Epoch as Context Epoch - Runner->>Instructions: loadAmbient + Runner->>Registry: load + Registry->>Instructions: run contribution Instructions->>Files: discover and read AGENTS.md files Files-->>Instructions: coherent current observation - Instructions-->>Runner: composed SystemContext + Instructions-->>Registry: instruction SystemContext + Registry-->>Runner: composed SystemContext Runner->>Epoch: compare and durably admit changes ``` ## Source Outcomes -Discovery state and per-source byte observation are separate concerns. +Discovery and file reads form one coherent aggregate observation in the first slice. ```text -discovery --> which stable source identities currently exist? +successful discovery and reads +-> one ordered aggregate instruction value -observation --> what bytes or temporary failure does each identity currently produce? +temporary discovery or read failure +-> aggregate SystemContext.unavailable ``` | Observation | Source outcome | | -------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | -| Local scan succeeds and discovers readable file | Available source with exact contents. | -| Local scan succeeds and a previously discovered file is absent | Remove source so `SystemContext` emits a tombstone. | -| Local scan fails transiently | Preserve the domain-owned prior source graph as unavailable or fail the current turn; never emit mass removals. | -| Known local file read fails transiently | Preserve the source as `SystemContext.unavailable`. | -| Known local file read reports not-found after discovery | Invalidate discovery and report unavailable until a coherent rescan confirms removal. | -| Empty local file | Available exact content, not absence. | +| Local scan succeeds and discovers readable file | Include its exact contents in the available aggregate source. | +| Local scan succeeds and a previously discovered file is absent | Remove it from the aggregate value; remove the aggregate source when no instructions remain. | +| Local scan or file read fails transiently | Preserve the admitted aggregate source as `SystemContext.unavailable`; never emit mass removals. | +| Empty local file | Include the empty exact content in the available aggregate source. | | URL returns `2xx` body | Available source with exact contents. | | URL times out or returns transient failure | `SystemContext.unavailable`. | | URL returns `404` or `410` | Decide the explicit removal contract before URL implementation. | -Instruction removal text must be model-meaningful. If instruction source keys hash source identities, add source-specific removal rendering before unlink support is considered complete: +Aggregate instruction removal text must be model-meaningful: ```text -Instructions removed: /repo/packages/core/AGENTS.md -Do not continue applying instructions previously loaded from this source. +Previously loaded instructions no longer apply. ``` ## First Ambient Slice @@ -234,7 +236,7 @@ Implement only: ```text global config AGENTS.md + upward project AGENTS.md ancestors -+ one keyed source per file ++ one aggregate core/instructions source + direct safe-turn observation ``` @@ -362,7 +364,7 @@ Nested instructions discovered after successful read-tool activity remain a Sess ## Lifecycle -- Location scope owns instruction services, optional watcher-consumer fibers, and refreshable state. +- Location scope owns the System Context Registry, scoped context contributions, optional watcher-consumer fibers, and refreshable state. - `Effect.forkScoped(...)` interrupts watcher-consumer fibers when the cached Location runtime is disposed. - Stream finalization unsubscribes `EventV2` PubSub subscriptions. - `Watcher.locationLayer` separately finalizes native Parcel watcher subscriptions. @@ -374,8 +376,8 @@ Nested instructions discovered after successful read-tool activity remain a Sess 1. Add and unit-test `Refreshable.make(load)` with `get` and `invalidate`. 2. Add model-meaningful instruction removal rendering support before unlink lands. -3. Add Location-scoped `InstructionContext` for global and upward project `AGENTS.md` only. -4. Compose instruction sources into `SessionSystemContext.load()`. +3. Add the Location-scoped `SystemContextRegistry` backed by stable-keyed scoped contributions. +4. Register built-in and ambient instruction producers with `SystemContextRegistry`. 5. Observe local instructions directly at each safe provider boundary. 6. Test add, edit, unlink, empty file, transient scan failure, transient read failure, restart, and deterministic ordering. 7. Add configured local exact paths and globs.