chore(observability): merge v2

This commit is contained in:
starptech 2026-07-08 18:16:17 +02:00
commit a18e5de4af
435 changed files with 18249 additions and 12191 deletions

View file

@ -113,6 +113,23 @@ Keep provider facades small and explicit:
`Provider.make(...)` remains available for simple static provider definitions, but new built-in providers should prefer plain configured facades unless a helper removes real duplication without adding runtime behavior.
### Provider Package Entrypoints
Catalog-selected native providers use package-like export paths from `@opencode-ai/llm`. They are internal entrypoints in one npm package, not separately published provider packages. Every entrypoint implements `ProviderPackage.Definition` and exposes `model(modelID, settings)`, where settings are serializable provider configuration plus common `headers`, `body`, and `limits` overlays.
```ts
import { model } from "@opencode-ai/llm/providers/openai/responses"
const selected = model("gpt-5", {
apiKey,
transport: "websocket",
})
```
Keep semantic APIs as separate entrypoints, such as OpenAI `chat` and `responses`. Keep transport choices inside the semantic entrypoint settings, so OpenAI Responses HTTP and WebSocket share one entrypoint. Provider facades may still expose named selectors such as `responsesWebSocket` for direct typed call sites; the package-like contract maps its settings to those selectors before returning an executable `Model`.
Do not expose `Route` in provider package settings. Route composition stays an implementation detail behind `model(...)`.
### Folder layout
```
@ -303,7 +320,7 @@ recorded.effect("streams text", () =>
)
```
Replay is the default. `RECORD=true` records fresh cassettes and requires the listed env vars. Cassettes are written as pretty-printed JSON so multi-interaction diffs stay reviewable.
Replay is the default. `RECORD=true` records fresh cassettes locally and requires the listed env vars; unset `CI` before recording because CI always forces replay. Cassettes are written as pretty-printed JSON so multi-interaction diffs stay reviewable.
Pass `provider`, `protocol`, and optional `tags` to `recordedTests(...)` / `recorded.effect.with(...)` so cassettes carry searchable metadata. Use recorded-test filters to replay or record a narrow subset without rewriting a whole file:
@ -316,6 +333,6 @@ Filters apply in replay and record mode. Combine them with `RECORD=true` when re
**Binary response bodies.** Most providers stream text (SSE, JSON). The recorder treats known textual media types (`text/*`, JSON/XML structured types, JavaScript, forms, YAML, and SVG) as text and stores every other response as base64 with `bodyEncoding: "base64"`. This preserves binary formats such as AWS event-stream frames without a lossy UTF-8 round trip.
**Matching strategy.** Replay walks the cassette in record order via an internal cursor: the Nth runtime request is served by the Nth recorded interaction, and each one is validated by comparing method, URL, allow-listed headers, and the canonical JSON body. This handles tool loops (each round's request differs as history grows) and retry/polling scenarios (successive byte-identical requests with different responses) uniformly. If a test reorders its requests, re-record the cassette. `scriptedResponses` (in `test/lib/http.ts`) is the deterministic counterpart for tests that don't need a live provider; it scripts response bodies in order without reading from disk.
**Matching strategy.** A runtime request atomically claims the first unused recorded interaction that matches its method, URL, allow-listed headers, and canonical JSON body. Distinct requests may replay in any order or concurrently. Repeated identical requests consume their matching responses in cassette order, preserving deterministic retry and polling behavior. `scriptedResponses` (in `test/lib/http.ts`) is the deterministic counterpart for tests that don't need a live provider; it scripts response bodies in order without reading from disk.
Do not blanket re-record an entire test file when adding one cassette. `RECORD=true` rewrites every recorded case that runs, and provider streams contain volatile IDs, timestamps, fingerprints, and obfuscation fields. Prefer deleting the one cassette you intend to refresh, or run a focused test pattern that only registers the scenario you want to record. Keep stable existing cassettes unchanged unless their request shape or expected behavior changed.

View file

@ -106,6 +106,32 @@ const gateway = CloudflareAIGateway.configure({
Included providers: OpenAI, Anthropic, Google (Gemini), Amazon Bedrock, Azure OpenAI, Cloudflare AI Gateway, Cloudflare Workers AI, GitHub Copilot, OpenRouter, xAI, plus generic OpenAI-compatible helpers for DeepSeek, Cerebras, Groq, Fireworks, Together, etc.
### Package-like entrypoints
Native catalog integrations load provider behavior through package-like entrypoints. These are export paths from the same `@opencode-ai/llm` npm package, not independently published packages. Each entrypoint exports the same `model(modelID, settings)` contract, and `settings` contains serializable provider configuration plus common `headers`, `body`, and `limits` overlays.
```ts
import { model } from "@opencode-ai/llm/providers/openai/responses"
const selected = model("gpt-5", {
apiKey: process.env.OPENAI_API_KEY,
transport: "websocket",
headers: { "x-application": "opencode" },
limits: { context: 200_000, output: 64_000 },
})
```
OpenAI Chat and OpenAI Responses are separate semantic entrypoints:
- `@opencode-ai/llm/providers/openai/chat`
- `@opencode-ai/llm/providers/openai/responses`
Responses HTTP versus WebSocket is a scoped `transport` setting on the Responses entrypoint, not another entrypoint. Azure follows the same Chat/Responses split at `providers/azure/chat` and `providers/azure/responses`. Anthropic, OpenAI-compatible Chat, Google Gemini, and Amazon Bedrock expose their single native API through their existing provider paths.
Provider facades such as `OpenAI.configure(...).responses(...)` remain the direct application API. Package-like entrypoints are the self-similar loading contract used when a catalog selects behavior by export path.
Other provider exports listed above remain direct facades until they explicitly implement the package-like contract. Exporting a provider facade does not implicitly make it a catalog-loadable provider package.
## Provider options & HTTP overlays
Three escape hatches in order of stability:

View file

@ -1,6 +1,6 @@
# LLM Provider Parity Status
Last reviewed: 2026-07-02
Last reviewed: 2026-07-08
This file tracks the gap between the native `@opencode-ai/llm` package and the AI SDK provider packages that opencode still depends on for many catalog/runtime paths.
@ -64,26 +64,27 @@ Everything else currently fails with `SessionRunnerModel.UnsupportedApiError` wh
5. Azure is only a provider facade, not a full runtime replacement. Native Azure exists, but the catalog runner does not select it, and token auth/resource variants need review.
6. Provider option typing is uneven. OpenAI, Anthropic, Gemini, Bedrock, and OpenRouter each expose a small typed subset plus raw HTTP overlays; this is useful but not equivalent to AI SDK provider option coverage.
7. Structured output is not provider-native yet. `LLM.generateObject` still uses a synthetic tool strategy, while the future design expects native structured output where reliable and tool fallback where needed.
8. Package/namespace boundaries need to be made explicit in docs and exports. Protocol namespaces exist, but planned public groupings should call out OpenAI Chat, OpenAI Responses, OpenAI-compatible Chat, OpenAI-compatible Responses, Anthropic Messages, Gemini, Vertex Gemini, Vertex Anthropic Messages, Bedrock Converse, and Bedrock Mantle as separate API slices.
8. Package/namespace boundaries for the current native loading set are explicit in docs and exports. Other exported provider facades are not catalog package entrypoints until they implement the contract. Missing native API boundaries remain for OpenAI-compatible Responses, Vertex Gemini, Vertex Anthropic Messages, and Bedrock Mantle.
9. Recorded coverage is uneven. OpenAI, Anthropic, Gemini, Bedrock Converse, Cloudflare, OpenRouter, and several OpenAI-compatible Chat providers have cassettes. Azure, Vertex, and Mantle need first-class recorded scenarios before switching defaults.
## Proposed Native Namespace Shape
## Native Namespace Shape
These are implementation/API slices, not separate npm packages.
| Namespace | Purpose |
| --- | --- |
| `OpenAI.Chat` or `OpenAIChat` | OpenAI `/chat/completions` semantics. |
| `OpenAI.Responses` or `OpenAIResponses` | OpenAI `/responses` HTTP and WebSocket semantics. |
| `OpenAICompatible.Chat` or `OpenAICompatibleChat` | Generic OpenAI-compatible `/chat/completions`. |
| `OpenAICompatible.Responses` or `OpenAICompatibleResponses` | Generic OpenAI-compatible `/responses`. Missing today. |
| `Anthropic.Messages` or `AnthropicMessages` | Anthropic Messages API. |
| `Google.Gemini` or `Gemini` | Gemini Developer API. |
| `GoogleVertex.Gemini` | Vertex Gemini API. Missing today. |
| `GoogleVertex.AnthropicMessages` | Vertex-hosted Anthropic Messages API. Missing today. |
| `Bedrock.Converse` or `BedrockConverse` | AWS Bedrock Converse API. |
| `Bedrock.Mantle` | AWS Bedrock Mantle OpenAI-compatible APIs. Missing today. |
| `Azure.OpenAIChat` / `Azure.OpenAIResponses` | Azure deployment specializations over OpenAI protocols. |
| API slice | Package-like entrypoint | Purpose |
| --- | --- | --- |
| OpenAI Chat | `@opencode-ai/llm/providers/openai/chat` | OpenAI `/chat/completions` semantics. |
| OpenAI Responses | `@opencode-ai/llm/providers/openai/responses` | OpenAI `/responses` semantics with HTTP/WebSocket selected through settings. |
| OpenAI-compatible Chat | `@opencode-ai/llm/providers/openai-compatible` | Generic OpenAI-compatible `/chat/completions`. |
| OpenAI-compatible Responses | Missing | Generic OpenAI-compatible `/responses`. |
| Anthropic Messages | `@opencode-ai/llm/providers/anthropic` | Anthropic Messages API. |
| Gemini Developer API | `@opencode-ai/llm/providers/google` | Google AI Studio Gemini API. |
| Vertex Gemini | Missing | Vertex Gemini API. |
| Vertex Anthropic Messages | Missing | Vertex-hosted Anthropic Messages API. |
| Bedrock Converse | `@opencode-ai/llm/providers/amazon-bedrock` | AWS Bedrock Converse API. |
| Bedrock Mantle | Missing | AWS Bedrock Mantle OpenAI-compatible APIs. |
| Azure OpenAI Chat | `@opencode-ai/llm/providers/azure/chat` | Azure specialization of OpenAI Chat. |
| Azure OpenAI Responses | `@opencode-ai/llm/providers/azure/responses` | Azure specialization of OpenAI Responses. |
## Suggested Next Work Slices

View file

@ -342,14 +342,24 @@ const response =
)
```
HTTP versus WebSocket is represented as named route selectors, not as model or
request overrides. Same protocol, different transport, different route:
For direct provider-facade calls, HTTP versus WebSocket is represented as named
route selectors, not as model or request overrides. Same protocol, different
transport, different route:
```ts
OpenAI.responses("gpt-4o")
OpenAI.responsesWebSocket("gpt-4o")
```
The package-like OpenAI Responses entrypoint instead keeps transport scoped to
Responses settings while preserving the same `model(...)` contract:
```ts
import { model } from "@opencode-ai/llm/providers/openai/responses"
model("gpt-4o", { apiKey, transport: "websocket" })
```
The client should not require a different public layer just because a selected
route uses WebSocket. Use one `LLMClient.layer` with HTTP and WebSocket runtime
capabilities available; routes that do not need WebSocket simply never touch it.
@ -468,10 +478,10 @@ const model =
```
That boundary can branch on durable config/catalog metadata and call typed
provider APIs directly. Transport selection belongs there too: map metadata like
`endpoint.websocket` to `OpenAI.responsesWebSocket(apiModelID)`; otherwise use
the normal `OpenAI.responses(apiModelID)` route. The client runtime only executes
the route carried by the model.
provider APIs directly. A direct provider-facade boundary maps metadata like
`endpoint.websocket` to `OpenAI.responsesWebSocket(apiModelID)`. A package-loading
boundary passes `transport: "websocket"` to the OpenAI Responses entrypoint.
The client runtime only executes the route carried by the resulting model.
## Competitive Shape
@ -507,8 +517,9 @@ App boundary = explicit durable-config -> typed-provider call
id.
- No `model(id, overrides)` escape hatch. Model selection takes the model id;
endpoint/auth/deployment customization happens by configuring the route first.
- No transport override on model/request. HTTP SSE versus WebSocket is a named
route selector such as `responses` versus `responsesWebSocket`.
- No transport override on an executable model or request. Direct provider
facades use `responses` versus `responsesWebSocket`; the package-like Responses
entrypoint maps its scoped `transport` setting before constructing the model.
- No separate public `LLMClient.layerWithWebSocket`. The runtime should expose one
client layer with the available transport capabilities.
- No executable `ModelRef`. The executable handle is `Model`; durable model

View file

@ -20,6 +20,8 @@
"./providers/amazon-bedrock": "./src/providers/amazon-bedrock.ts",
"./providers/anthropic": "./src/providers/anthropic.ts",
"./providers/azure": "./src/providers/azure.ts",
"./providers/azure/responses": "./src/providers/azure/responses.ts",
"./providers/azure/chat": "./src/providers/azure/chat.ts",
"./providers/cloudflare": "./src/providers/cloudflare.ts",
"./providers/github-copilot": "./src/providers/github-copilot.ts",
"./providers/google": "./src/providers/google.ts",

View file

@ -876,6 +876,7 @@ export const protocol = Protocol.make({
export const route = Route.make({
id: ADAPTER,
provider: "anthropic",
providerMetadataKey: "anthropic",
protocol,
endpoint: Endpoint.path(PATH, { baseURL: DEFAULT_BASE_URL }),
auth: Auth.none,

View file

@ -657,6 +657,7 @@ export const protocol = Protocol.make({
export const route = Route.make({
id: ADAPTER,
provider: "bedrock",
providerMetadataKey: "bedrock",
protocol,
// Bedrock's URL embeds the region in the route endpoint host and the
// validated modelId in the path. We read the validated body so the URL

View file

@ -500,6 +500,7 @@ export const protocol = Protocol.make({
export const route = Route.make({
id: ADAPTER,
provider: "google",
providerMetadataKey: "google",
protocol,
// Gemini's path embeds the model id and pins SSE framing at the URL level.
endpoint: Endpoint.path(({ request }) => `/models/${request.model.id}:streamGenerateContent?alt=sse`, {

View file

@ -493,6 +493,7 @@ export const httpTransport = HttpTransport.sseJson.with<OpenAIChatBody>()
export const route = Route.make({
id: ADAPTER,
provider: "openai",
providerMetadataKey: "openai",
protocol,
endpoint: Endpoint.path(PATH, { baseURL: DEFAULT_BASE_URL }),
auth: Auth.none,

View file

@ -16,6 +16,7 @@ export type OpenAICompatibleChatModelInput = RouteRoutedModelInput
*/
export const route = Route.make({
id: ADAPTER,
providerMetadataKey: "openai",
protocol: OpenAIChat.protocol,
endpoint: Endpoint.path("/chat/completions"),
framing: Framing.sse,

View file

@ -990,6 +990,7 @@ export const httpTransport = HttpTransport.sseJson.with<OpenAIResponsesBody>()
export const route = Route.make({
id: ADAPTER,
provider: "openai",
providerMetadataKey: "openai",
protocol,
endpoint,
auth,
@ -1018,6 +1019,7 @@ export const webSocketTransport = WebSocketTransport.jsonTransport.with<
export const webSocketRoute = Route.make({
id: `${ADAPTER}-websocket`,
provider: "openai",
providerMetadataKey: "openai",
protocol,
endpoint,
auth,

View file

@ -6,6 +6,7 @@ const patterns = [
/input is too long for requested model/i,
/exceeds the context window/i,
/input token count.*exceeds the maximum/i,
/tokens in request more than max tokens allowed/i,
/maximum prompt length is \d+/i,
/reduce the length of the messages/i,
/maximum context length is \d+ tokens/i,

View file

@ -1,6 +1,7 @@
import { Auth } from "../route/auth"
import { type AtLeastOne, type ProviderAuthOption } from "../route/auth-options"
import type { Route as RouteDef, RouteDefaultsInput } from "../route/client"
import type { ProviderPackage } from "../provider-package"
import { ProviderID, type ModelID } from "../schema"
import * as OpenAIChat from "../protocols/openai-chat"
import * as OpenAIResponses from "../protocols/openai-responses"
@ -23,6 +24,14 @@ export type ModelOptions = AzureURL &
}
export type Config = ModelOptions
export type Settings = ProviderPackage.Settings &
AzureURL & {
readonly apiKey?: string
readonly apiVersion?: string
readonly queryParams?: Readonly<Record<string, string>>
readonly providerOptions?: OpenAIProviderOptionsInput
}
const resourceBaseURL = (resourceName: string) => `https://${resourceName.trim()}.openai.azure.com/openai/v1`
const responsesRoute = OpenAIResponses.route.with({
@ -108,3 +117,24 @@ export const provider = {
id,
configure,
}
const config = (settings: Settings): Config => {
const common = {
apiKey: settings.apiKey,
apiVersion: settings.apiVersion,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
queryParams: settings.queryParams === undefined ? undefined : { ...settings.queryParams },
}
if (settings.baseURL !== undefined) return { ...common, baseURL: settings.baseURL }
if (settings.resourceName !== undefined) return { ...common, resourceName: settings.resourceName }
throw new Error("Azure requires resourceName or baseURL")
}
export const responsesModel: ProviderPackage.Definition<Settings>["model"] = (modelID, settings) =>
configure(config(settings)).responses(modelID)
export const chatModel: ProviderPackage.Definition<Settings>["model"] = (modelID, settings) =>
configure(config(settings)).chat(modelID)
export const model = responsesModel

View file

@ -0,0 +1,2 @@
export { chatModel as model } from "../azure"
export type { Settings } from "../azure"

View file

@ -0,0 +1,2 @@
export { responsesModel as model } from "../azure"
export type { Settings } from "../azure"

View file

@ -1,7 +1,8 @@
import type { RouteDefaultsInput } from "../route/client"
import { Auth } from "../route/auth"
import type { ProviderAuthOption } from "../route/auth-options"
import { ProviderID, type ModelID } from "../schema"
import type { ProviderPackage } from "../provider-package"
import { ProviderID, type ModelID, type ProviderOptions } from "../schema"
import * as Gemini from "../protocols/gemini"
export const id = ProviderID.make("google")
@ -10,6 +11,12 @@ export const routes = [Gemini.route]
export type Config = RouteDefaultsInput & ProviderAuthOption<"optional"> & { readonly baseURL?: string }
export interface Settings extends ProviderPackage.Settings {
readonly apiKey?: string
readonly baseURL?: string
readonly providerOptions?: ProviderOptions
}
const auth = (options: ProviderAuthOption<"optional">) => {
if ("auth" in options && options.auth) return options.auth
return Auth.optional("apiKey" in options ? options.apiKey : undefined, "apiKey")
@ -32,4 +39,12 @@ export const configure = (input: Config = {}) => {
}
export const provider = configure()
export const model = provider.model
export const model: ProviderPackage.Definition<Settings>["model"] = (modelID, settings) =>
configure({
apiKey: settings.apiKey,
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
}).model(modelID)

View file

@ -36,6 +36,8 @@ export interface RouteBody<Body> {
export interface Route<Body, Prepared = unknown> {
readonly id: string
readonly provider?: ProviderID
/** ProviderMetadata namespace emitted and consumed by this route. */
readonly providerMetadataKey?: string
readonly protocol: ProtocolID
readonly endpoint: Endpoint<Body>
readonly auth: Auth
@ -184,6 +186,8 @@ export interface MakeInput<Body, Frame, Event, State> {
readonly id: string
/** Provider identity for route-owned model construction. */
readonly provider?: string | ProviderID
/** ProviderMetadata namespace emitted and consumed by this route. */
readonly providerMetadataKey?: string
/** Semantic API contract — owns body construction, body schema, and parsing. */
readonly protocol: Protocol<Body, Frame, Event, State>
/** Where the request is sent. */
@ -203,6 +207,8 @@ export interface MakeTransportInput<Body, Prepared, Frame, Event, State> {
readonly id: string
/** Provider identity for route-owned model construction. */
readonly provider?: string | ProviderID
/** ProviderMetadata namespace emitted and consumed by this route. */
readonly providerMetadataKey?: string
/** Semantic API contract — owns body construction, body schema, and parsing. */
readonly protocol: Protocol<Body, Frame, Event, State>
/** Where the request is sent. */
@ -248,6 +254,7 @@ function makeFromTransport<Body, Prepared, Frame, Event, State>(
const route: Route<Body, Prepared> = {
id: routeInput.id,
provider: routeInput.provider === undefined ? undefined : ProviderID.make(routeInput.provider),
providerMetadataKey: routeInput.providerMetadataKey,
protocol: protocol.id,
endpoint: routeInput.endpoint,
auth: routeInput.auth ?? Auth.none,
@ -329,6 +336,7 @@ export function make<Body, Prepared, Frame, Event, State>(
return makeFromTransport({
id: input.id,
provider: input.provider,
providerMetadataKey: input.providerMetadataKey,
protocol,
endpoint: input.endpoint,
auth: input.auth,

View file

@ -0,0 +1,8 @@
import { describe, expect, test } from "bun:test"
import { isContextOverflow } from "../src"
describe("provider error classification", () => {
test("classifies Z.AI GLM token limit messages as context overflow", () => {
expect(isContextOverflow("tokens in request more than max tokens allowed")).toBe(true)
})
})

View file

@ -10,10 +10,15 @@ describe("provider package entrypoints", () => {
import("@opencode-ai/llm/providers/anthropic"),
import("@opencode-ai/llm/providers/openai-compatible"),
import("@opencode-ai/llm/providers/amazon-bedrock"),
import("@opencode-ai/llm/providers/azure"),
import("@opencode-ai/llm/providers/azure/responses"),
import("@opencode-ai/llm/providers/azure/chat"),
import("@opencode-ai/llm/providers/google"),
])
for (const module of modules) expect(module.model).toBeFunction()
expect(modules[0].model).toBe(modules[1].model)
expect(modules[6].model).toBe(modules[7].model)
})
test("maps package settings onto the executable model", () => {
@ -49,4 +54,49 @@ describe("provider package entrypoints", () => {
"OpenAI-Project": "proj_123",
})
})
test("selects Azure API entrypoints with the same model contract", async () => {
const Azure = await import("@opencode-ai/llm/providers/azure")
const AzureChat = await import("@opencode-ai/llm/providers/azure/chat")
const AzureResponses = await import("@opencode-ai/llm/providers/azure/responses")
const settings = {
apiKey: "fixture",
resourceName: "opencode-test",
headers: { "x-application": "opencode" },
body: { service_tier: "priority" },
limits: { context: 200_000, output: 64_000 },
}
const responses = AzureResponses.model("deployment", settings)
const chat = AzureChat.model("deployment", settings)
expect(Azure.model("deployment", settings).route.id).toBe("azure-openai-responses")
expect(responses.route.id).toBe("azure-openai-responses")
expect(responses.route.endpoint.baseURL).toBe("https://opencode-test.openai.azure.com/openai/v1")
expect(responses.route.defaults.headers).toEqual({ "x-application": "opencode" })
expect(responses.route.defaults.http?.body).toEqual({ service_tier: "priority" })
expect(responses.route.defaults.limits).toEqual({ context: 200_000, output: 64_000 })
expect(chat.route.id).toBe("azure-openai-chat")
})
test("maps Google package settings onto the Gemini model", async () => {
const Google = await import("@opencode-ai/llm/providers/google")
const selected = Google.model("gemini-2.5-flash", {
apiKey: "fixture",
baseURL: "https://generativelanguage.test/v1beta",
headers: { "x-application": "opencode" },
body: { safetySettings: [] },
limits: { context: 1_000_000, output: 65_536 },
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: 1_024 } } },
})
expect(selected.route.id).toBe("gemini")
expect(selected.route.endpoint.baseURL).toBe("https://generativelanguage.test/v1beta")
expect(selected.route.defaults.headers).toEqual({ "x-application": "opencode" })
expect(selected.route.defaults.http?.body).toEqual({ safetySettings: [] })
expect(selected.route.defaults.limits).toEqual({ context: 1_000_000, output: 65_536 })
expect(selected.route.defaults.providerOptions).toEqual({
gemini: { thinkingConfig: { thinkingBudget: 1_024 } },
})
})
})

View file

@ -12,7 +12,6 @@ const openAI = OpenAI.configure({
})
const openAIChat = openAI.chat("gpt-4o-mini")
const openAIResponses = openAI.responses("gpt-5.5")
const openAIResponsesWebSocket = openAI.responsesWebSocket("gpt-4.1-mini")
const anthropic = Anthropic.configure({
apiKey: process.env.ANTHROPIC_API_KEY ?? "fixture",
})
@ -89,14 +88,6 @@ describeRecordedGoldenScenarios([
{ id: "image-tool-result", temperature: false, maxTokens: 40 },
],
},
{
name: "OpenAI Responses WebSocket gpt-4.1-mini",
prefix: "openai-responses-websocket",
model: openAIResponsesWebSocket,
transport: "websocket",
requires: ["OPENAI_API_KEY"],
scenarios: ["tool-loop"],
},
{
name: "Anthropic Haiku 4.5",
prefix: "anthropic-messages",

View file

@ -28,7 +28,7 @@ type TargetInput = {
readonly transport?: Transport
readonly prefix?: string
readonly tags?: ReadonlyArray<string>
readonly metadata?: Record<string, unknown>
readonly metadata?: HttpRecorder.CassetteMetadata
readonly options?: HttpRecorder.RecorderOptions
readonly scenarios: ReadonlyArray<ScenarioInput>
}
@ -43,7 +43,7 @@ const defaultPrefix = (target: TargetInput) => {
const metadata = (target: TargetInput) => ({
provider: target.model.provider,
protocol: target.protocol,
...(target.protocol ? { protocol: target.protocol } : {}),
route: target.model.route.id,
transport: target.transport ?? "http",
model: target.model.id,

View file

@ -1,3 +1,4 @@
import type { HttpRecorder } from "@opencode-ai/http-recorder"
import { test, type TestOptions } from "bun:test"
import { Effect, type Layer } from "effect"
import { testEffect } from "./lib/effect"
@ -11,7 +12,7 @@ export type RecordedGroupOptions = {
readonly protocol?: string
readonly requires?: ReadonlyArray<string>
readonly tags?: ReadonlyArray<string>
readonly metadata?: Record<string, unknown>
readonly metadata?: HttpRecorder.CassetteMetadata
}
export type RecordedCaseOptions = {
@ -21,7 +22,7 @@ export type RecordedCaseOptions = {
readonly protocol?: string
readonly requires?: ReadonlyArray<string>
readonly tags?: ReadonlyArray<string>
readonly metadata?: Record<string, unknown>
readonly metadata?: HttpRecorder.CassetteMetadata
}
export const recordedEffectGroup = <
@ -36,7 +37,7 @@ export const recordedEffectGroup = <
readonly layer: (input: {
readonly cassette: string
readonly tags: ReadonlyArray<string>
readonly metadata: Record<string, unknown>
readonly metadata: HttpRecorder.CassetteMetadata
readonly recording: boolean
readonly options: Options
readonly caseOptions: CaseOptions

View file

@ -1,11 +1,8 @@
import { NodeFileSystem } from "@effect/platform-node"
import { HttpRecorder } from "@opencode-ai/http-recorder"
import { HttpRecorderInternal } from "@opencode-ai/http-recorder/internal"
import { Layer } from "effect"
import { FetchHttpClient } from "effect/unstable/http"
import * as path from "node:path"
import { fileURLToPath } from "node:url"
import { LLMClient, RequestExecutor } from "../src/route"
import { LLMClient, RequestExecutor, WebSocketExecutor } from "../src/route"
import type { Service as LLMClientService } from "../src/route/client"
import type { Service as RequestExecutorService } from "../src/route/executor"
import type { Service as WebSocketExecutorService } from "../src/route/transport/websocket"
@ -14,7 +11,6 @@ import {
type RecordedCaseOptions as RunnerCaseOptions,
type RecordedGroupOptions,
} from "./recorded-runner"
import { webSocketCassetteLayer } from "./recorded-websocket"
const __dirname = path.dirname(fileURLToPath(import.meta.url))
const FIXTURES_DIR = path.resolve(__dirname, "fixtures", "recordings")
@ -64,31 +60,27 @@ export const recordedTests = (options: RecordedTestsOptions) =>
recordedEffectGroup<RecordedEnv, never, RecordedTestsOptions, RecordedCaseOptions>({
duplicateLabel: "recorded cassette",
options,
cassetteExists: (cassette) => HttpRecorderInternal.hasCassetteSync(cassette, { directory: FIXTURES_DIR }),
cassetteExists: (cassette) => HttpRecorder.hasCassetteSync(cassette, { directory: FIXTURES_DIR }),
layer: ({ cassette, metadata, options, caseOptions, recording }) => {
const recorderOptions = mergeOptions(options.options, caseOptions.options)
const recorderMetadata = {
...recorderOptions?.metadata,
...metadata,
}
const mode = recording ? "record" : "replay"
const cassetteService = HttpRecorderInternal.Cassette.fileSystem({ directory: FIXTURES_DIR }).pipe(
Layer.provide(NodeFileSystem.layer),
)
if (recording) {
if (process.env.CI !== undefined) throw new Error("Unset CI before recording HTTP cassettes")
HttpRecorder.removeCassetteSync(cassette, { directory: FIXTURES_DIR })
}
const requestExecutor = RequestExecutor.layer.pipe(
Layer.provide(
HttpRecorderInternal.recordingLayer(cassette, {
mode,
HttpRecorder.layerFetch(cassette, {
...recorderOptions,
directory: FIXTURES_DIR,
metadata: recorderMetadata,
redactor: HttpRecorderInternal.Redactor.make(recorderOptions?.redact),
match: recorderOptions?.match,
}).pipe(Layer.provide(FetchHttpClient.layer)),
}),
),
)
const deps = Layer.mergeAll(
requestExecutor,
webSocketCassetteLayer(cassette, { metadata: recorderMetadata, mode }),
)
return Layer.mergeAll(deps, LLMClient.layer.pipe(Layer.provide(deps))).pipe(Layer.provide(cassetteService))
const deps = Layer.mergeAll(requestExecutor, WebSocketExecutor.layer)
return Layer.mergeAll(deps, LLMClient.layer.pipe(Layer.provide(deps)))
},
})

View file

@ -1,26 +0,0 @@
import { HttpRecorderInternal } from "@opencode-ai/http-recorder/internal"
import { Effect, Layer } from "effect"
import { WebSocketExecutor } from "../src/route"
import type { Service as WebSocketExecutorService } from "../src/route/transport/websocket"
const liveWebSocket = WebSocketExecutor.open
export const webSocketCassetteLayer = (
cassette: string,
input: { readonly metadata?: Record<string, unknown>; readonly mode: HttpRecorderInternal.RecordReplayMode },
): Layer.Layer<WebSocketExecutorService, never, HttpRecorderInternal.Cassette.Service> =>
Layer.effect(
WebSocketExecutor.Service,
Effect.gen(function* () {
const cassetteService = yield* HttpRecorderInternal.Cassette.Service
const executor = yield* HttpRecorderInternal.makeWebSocketExecutor({
name: cassette,
mode: input.mode,
metadata: input.metadata,
cassette: cassetteService,
live: { open: liveWebSocket },
compareClientMessagesAsJson: true,
})
return WebSocketExecutor.Service.of(executor)
}),
)