feat(llm): complete provider package entrypoints (#35907)

This commit is contained in:
Shoubhit Dash 2026-07-08 18:41:46 +05:30 committed by GitHub
commit 3cbcd8d727
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 182 additions and 26 deletions

View file

@ -113,6 +113,23 @@ Keep provider facades small and explicit:
`Provider.make(...)` remains available for simple static provider definitions, but new built-in providers should prefer plain configured facades unless a helper removes real duplication without adding runtime behavior. `Provider.make(...)` remains available for simple static provider definitions, but new built-in providers should prefer plain configured facades unless a helper removes real duplication without adding runtime behavior.
### Provider Package Entrypoints
Catalog-selected native providers use package-like export paths from `@opencode-ai/llm`. They are internal entrypoints in one npm package, not separately published provider packages. Every entrypoint implements `ProviderPackage.Definition` and exposes `model(modelID, settings)`, where settings are serializable provider configuration plus common `headers`, `body`, and `limits` overlays.
```ts
import { model } from "@opencode-ai/llm/providers/openai/responses"
const selected = model("gpt-5", {
apiKey,
transport: "websocket",
})
```
Keep semantic APIs as separate entrypoints, such as OpenAI `chat` and `responses`. Keep transport choices inside the semantic entrypoint settings, so OpenAI Responses HTTP and WebSocket share one entrypoint. Provider facades may still expose named selectors such as `responsesWebSocket` for direct typed call sites; the package-like contract maps its settings to those selectors before returning an executable `Model`.
Do not expose `Route` in provider package settings. Route composition stays an implementation detail behind `model(...)`.
### Folder layout ### Folder layout
``` ```

View file

@ -106,6 +106,32 @@ const gateway = CloudflareAIGateway.configure({
Included providers: OpenAI, Anthropic, Google (Gemini), Amazon Bedrock, Azure OpenAI, Cloudflare AI Gateway, Cloudflare Workers AI, GitHub Copilot, OpenRouter, xAI, plus generic OpenAI-compatible helpers for DeepSeek, Cerebras, Groq, Fireworks, Together, etc. Included providers: OpenAI, Anthropic, Google (Gemini), Amazon Bedrock, Azure OpenAI, Cloudflare AI Gateway, Cloudflare Workers AI, GitHub Copilot, OpenRouter, xAI, plus generic OpenAI-compatible helpers for DeepSeek, Cerebras, Groq, Fireworks, Together, etc.
### Package-like entrypoints
Native catalog integrations load provider behavior through package-like entrypoints. These are export paths from the same `@opencode-ai/llm` npm package, not independently published packages. Each entrypoint exports the same `model(modelID, settings)` contract, and `settings` contains serializable provider configuration plus common `headers`, `body`, and `limits` overlays.
```ts
import { model } from "@opencode-ai/llm/providers/openai/responses"
const selected = model("gpt-5", {
apiKey: process.env.OPENAI_API_KEY,
transport: "websocket",
headers: { "x-application": "opencode" },
limits: { context: 200_000, output: 64_000 },
})
```
OpenAI Chat and OpenAI Responses are separate semantic entrypoints:
- `@opencode-ai/llm/providers/openai/chat`
- `@opencode-ai/llm/providers/openai/responses`
Responses HTTP versus WebSocket is a scoped `transport` setting on the Responses entrypoint, not another entrypoint. Azure follows the same Chat/Responses split at `providers/azure/chat` and `providers/azure/responses`. Anthropic, OpenAI-compatible Chat, Google Gemini, and Amazon Bedrock expose their single native API through their existing provider paths.
Provider facades such as `OpenAI.configure(...).responses(...)` remain the direct application API. Package-like entrypoints are the self-similar loading contract used when a catalog selects behavior by export path.
Other provider exports listed above remain direct facades until they explicitly implement the package-like contract. Exporting a provider facade does not implicitly make it a catalog-loadable provider package.
## Provider options & HTTP overlays ## Provider options & HTTP overlays
Three escape hatches in order of stability: Three escape hatches in order of stability:

View file

@ -1,6 +1,6 @@
# LLM Provider Parity Status # LLM Provider Parity Status
Last reviewed: 2026-07-02 Last reviewed: 2026-07-08
This file tracks the gap between the native `@opencode-ai/llm` package and the AI SDK provider packages that opencode still depends on for many catalog/runtime paths. This file tracks the gap between the native `@opencode-ai/llm` package and the AI SDK provider packages that opencode still depends on for many catalog/runtime paths.
@ -64,26 +64,27 @@ Everything else currently fails with `SessionRunnerModel.UnsupportedApiError` wh
5. Azure is only a provider facade, not a full runtime replacement. Native Azure exists, but the catalog runner does not select it, and token auth/resource variants need review. 5. Azure is only a provider facade, not a full runtime replacement. Native Azure exists, but the catalog runner does not select it, and token auth/resource variants need review.
6. Provider option typing is uneven. OpenAI, Anthropic, Gemini, Bedrock, and OpenRouter each expose a small typed subset plus raw HTTP overlays; this is useful but not equivalent to AI SDK provider option coverage. 6. Provider option typing is uneven. OpenAI, Anthropic, Gemini, Bedrock, and OpenRouter each expose a small typed subset plus raw HTTP overlays; this is useful but not equivalent to AI SDK provider option coverage.
7. Structured output is not provider-native yet. `LLM.generateObject` still uses a synthetic tool strategy, while the future design expects native structured output where reliable and tool fallback where needed. 7. Structured output is not provider-native yet. `LLM.generateObject` still uses a synthetic tool strategy, while the future design expects native structured output where reliable and tool fallback where needed.
8. Package/namespace boundaries need to be made explicit in docs and exports. Protocol namespaces exist, but planned public groupings should call out OpenAI Chat, OpenAI Responses, OpenAI-compatible Chat, OpenAI-compatible Responses, Anthropic Messages, Gemini, Vertex Gemini, Vertex Anthropic Messages, Bedrock Converse, and Bedrock Mantle as separate API slices. 8. Package/namespace boundaries for the current native loading set are explicit in docs and exports. Other exported provider facades are not catalog package entrypoints until they implement the contract. Missing native API boundaries remain for OpenAI-compatible Responses, Vertex Gemini, Vertex Anthropic Messages, and Bedrock Mantle.
9. Recorded coverage is uneven. OpenAI, Anthropic, Gemini, Bedrock Converse, Cloudflare, OpenRouter, and several OpenAI-compatible Chat providers have cassettes. Azure, Vertex, and Mantle need first-class recorded scenarios before switching defaults. 9. Recorded coverage is uneven. OpenAI, Anthropic, Gemini, Bedrock Converse, Cloudflare, OpenRouter, and several OpenAI-compatible Chat providers have cassettes. Azure, Vertex, and Mantle need first-class recorded scenarios before switching defaults.
## Proposed Native Namespace Shape ## Native Namespace Shape
These are implementation/API slices, not separate npm packages. These are implementation/API slices, not separate npm packages.
| Namespace | Purpose | | API slice | Package-like entrypoint | Purpose |
| --- | --- | | --- | --- | --- |
| `OpenAI.Chat` or `OpenAIChat` | OpenAI `/chat/completions` semantics. | | OpenAI Chat | `@opencode-ai/llm/providers/openai/chat` | OpenAI `/chat/completions` semantics. |
| `OpenAI.Responses` or `OpenAIResponses` | OpenAI `/responses` HTTP and WebSocket semantics. | | OpenAI Responses | `@opencode-ai/llm/providers/openai/responses` | OpenAI `/responses` semantics with HTTP/WebSocket selected through settings. |
| `OpenAICompatible.Chat` or `OpenAICompatibleChat` | Generic OpenAI-compatible `/chat/completions`. | | OpenAI-compatible Chat | `@opencode-ai/llm/providers/openai-compatible` | Generic OpenAI-compatible `/chat/completions`. |
| `OpenAICompatible.Responses` or `OpenAICompatibleResponses` | Generic OpenAI-compatible `/responses`. Missing today. | | OpenAI-compatible Responses | Missing | Generic OpenAI-compatible `/responses`. |
| `Anthropic.Messages` or `AnthropicMessages` | Anthropic Messages API. | | Anthropic Messages | `@opencode-ai/llm/providers/anthropic` | Anthropic Messages API. |
| `Google.Gemini` or `Gemini` | Gemini Developer API. | | Gemini Developer API | `@opencode-ai/llm/providers/google` | Google AI Studio Gemini API. |
| `GoogleVertex.Gemini` | Vertex Gemini API. Missing today. | | Vertex Gemini | Missing | Vertex Gemini API. |
| `GoogleVertex.AnthropicMessages` | Vertex-hosted Anthropic Messages API. Missing today. | | Vertex Anthropic Messages | Missing | Vertex-hosted Anthropic Messages API. |
| `Bedrock.Converse` or `BedrockConverse` | AWS Bedrock Converse API. | | Bedrock Converse | `@opencode-ai/llm/providers/amazon-bedrock` | AWS Bedrock Converse API. |
| `Bedrock.Mantle` | AWS Bedrock Mantle OpenAI-compatible APIs. Missing today. | | Bedrock Mantle | Missing | AWS Bedrock Mantle OpenAI-compatible APIs. |
| `Azure.OpenAIChat` / `Azure.OpenAIResponses` | Azure deployment specializations over OpenAI protocols. | | Azure OpenAI Chat | `@opencode-ai/llm/providers/azure/chat` | Azure specialization of OpenAI Chat. |
| Azure OpenAI Responses | `@opencode-ai/llm/providers/azure/responses` | Azure specialization of OpenAI Responses. |
## Suggested Next Work Slices ## Suggested Next Work Slices

View file

@ -342,14 +342,24 @@ const response =
) )
``` ```
HTTP versus WebSocket is represented as named route selectors, not as model or For direct provider-facade calls, HTTP versus WebSocket is represented as named
request overrides. Same protocol, different transport, different route: route selectors, not as model or request overrides. Same protocol, different
transport, different route:
```ts ```ts
OpenAI.responses("gpt-4o") OpenAI.responses("gpt-4o")
OpenAI.responsesWebSocket("gpt-4o") OpenAI.responsesWebSocket("gpt-4o")
``` ```
The package-like OpenAI Responses entrypoint instead keeps transport scoped to
Responses settings while preserving the same `model(...)` contract:
```ts
import { model } from "@opencode-ai/llm/providers/openai/responses"
model("gpt-4o", { apiKey, transport: "websocket" })
```
The client should not require a different public layer just because a selected The client should not require a different public layer just because a selected
route uses WebSocket. Use one `LLMClient.layer` with HTTP and WebSocket runtime route uses WebSocket. Use one `LLMClient.layer` with HTTP and WebSocket runtime
capabilities available; routes that do not need WebSocket simply never touch it. capabilities available; routes that do not need WebSocket simply never touch it.
@ -468,10 +478,10 @@ const model =
``` ```
That boundary can branch on durable config/catalog metadata and call typed That boundary can branch on durable config/catalog metadata and call typed
provider APIs directly. Transport selection belongs there too: map metadata like provider APIs directly. A direct provider-facade boundary maps metadata like
`endpoint.websocket` to `OpenAI.responsesWebSocket(apiModelID)`; otherwise use `endpoint.websocket` to `OpenAI.responsesWebSocket(apiModelID)`. A package-loading
the normal `OpenAI.responses(apiModelID)` route. The client runtime only executes boundary passes `transport: "websocket"` to the OpenAI Responses entrypoint.
the route carried by the model. The client runtime only executes the route carried by the resulting model.
## Competitive Shape ## Competitive Shape
@ -507,8 +517,9 @@ App boundary = explicit durable-config -> typed-provider call
id. id.
- No `model(id, overrides)` escape hatch. Model selection takes the model id; - No `model(id, overrides)` escape hatch. Model selection takes the model id;
endpoint/auth/deployment customization happens by configuring the route first. endpoint/auth/deployment customization happens by configuring the route first.
- No transport override on model/request. HTTP SSE versus WebSocket is a named - No transport override on an executable model or request. Direct provider
route selector such as `responses` versus `responsesWebSocket`. facades use `responses` versus `responsesWebSocket`; the package-like Responses
entrypoint maps its scoped `transport` setting before constructing the model.
- No separate public `LLMClient.layerWithWebSocket`. The runtime should expose one - No separate public `LLMClient.layerWithWebSocket`. The runtime should expose one
client layer with the available transport capabilities. client layer with the available transport capabilities.
- No executable `ModelRef`. The executable handle is `Model`; durable model - No executable `ModelRef`. The executable handle is `Model`; durable model

View file

@ -19,6 +19,8 @@
"./providers/amazon-bedrock": "./src/providers/amazon-bedrock.ts", "./providers/amazon-bedrock": "./src/providers/amazon-bedrock.ts",
"./providers/anthropic": "./src/providers/anthropic.ts", "./providers/anthropic": "./src/providers/anthropic.ts",
"./providers/azure": "./src/providers/azure.ts", "./providers/azure": "./src/providers/azure.ts",
"./providers/azure/responses": "./src/providers/azure/responses.ts",
"./providers/azure/chat": "./src/providers/azure/chat.ts",
"./providers/cloudflare": "./src/providers/cloudflare.ts", "./providers/cloudflare": "./src/providers/cloudflare.ts",
"./providers/github-copilot": "./src/providers/github-copilot.ts", "./providers/github-copilot": "./src/providers/github-copilot.ts",
"./providers/google": "./src/providers/google.ts", "./providers/google": "./src/providers/google.ts",

View file

@ -1,6 +1,7 @@
import { Auth } from "../route/auth" import { Auth } from "../route/auth"
import { type AtLeastOne, type ProviderAuthOption } from "../route/auth-options" import { type AtLeastOne, type ProviderAuthOption } from "../route/auth-options"
import type { Route as RouteDef, RouteDefaultsInput } from "../route/client" import type { Route as RouteDef, RouteDefaultsInput } from "../route/client"
import type { ProviderPackage } from "../provider-package"
import { ProviderID, type ModelID } from "../schema" import { ProviderID, type ModelID } from "../schema"
import * as OpenAIChat from "../protocols/openai-chat" import * as OpenAIChat from "../protocols/openai-chat"
import * as OpenAIResponses from "../protocols/openai-responses" import * as OpenAIResponses from "../protocols/openai-responses"
@ -23,6 +24,14 @@ export type ModelOptions = AzureURL &
} }
export type Config = ModelOptions export type Config = ModelOptions
export type Settings = ProviderPackage.Settings &
AzureURL & {
readonly apiKey?: string
readonly apiVersion?: string
readonly queryParams?: Readonly<Record<string, string>>
readonly providerOptions?: OpenAIProviderOptionsInput
}
const resourceBaseURL = (resourceName: string) => `https://${resourceName.trim()}.openai.azure.com/openai/v1` const resourceBaseURL = (resourceName: string) => `https://${resourceName.trim()}.openai.azure.com/openai/v1`
const responsesRoute = OpenAIResponses.route.with({ const responsesRoute = OpenAIResponses.route.with({
@ -108,3 +117,24 @@ export const provider = {
id, id,
configure, configure,
} }
const config = (settings: Settings): Config => {
const common = {
apiKey: settings.apiKey,
apiVersion: settings.apiVersion,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
queryParams: settings.queryParams === undefined ? undefined : { ...settings.queryParams },
}
if (settings.baseURL !== undefined) return { ...common, baseURL: settings.baseURL }
if (settings.resourceName !== undefined) return { ...common, resourceName: settings.resourceName }
throw new Error("Azure requires resourceName or baseURL")
}
export const responsesModel: ProviderPackage.Definition<Settings>["model"] = (modelID, settings) =>
configure(config(settings)).responses(modelID)
export const chatModel: ProviderPackage.Definition<Settings>["model"] = (modelID, settings) =>
configure(config(settings)).chat(modelID)
export const model = responsesModel

View file

@ -0,0 +1,2 @@
export { chatModel as model } from "../azure"
export type { Settings } from "../azure"

View file

@ -0,0 +1,2 @@
export { responsesModel as model } from "../azure"
export type { Settings } from "../azure"

View file

@ -1,7 +1,8 @@
import type { RouteDefaultsInput } from "../route/client" import type { RouteDefaultsInput } from "../route/client"
import { Auth } from "../route/auth" import { Auth } from "../route/auth"
import type { ProviderAuthOption } from "../route/auth-options" import type { ProviderAuthOption } from "../route/auth-options"
import { ProviderID, type ModelID } from "../schema" import type { ProviderPackage } from "../provider-package"
import { ProviderID, type ModelID, type ProviderOptions } from "../schema"
import * as Gemini from "../protocols/gemini" import * as Gemini from "../protocols/gemini"
export const id = ProviderID.make("google") export const id = ProviderID.make("google")
@ -10,6 +11,12 @@ export const routes = [Gemini.route]
export type Config = RouteDefaultsInput & ProviderAuthOption<"optional"> & { readonly baseURL?: string } export type Config = RouteDefaultsInput & ProviderAuthOption<"optional"> & { readonly baseURL?: string }
export interface Settings extends ProviderPackage.Settings {
readonly apiKey?: string
readonly baseURL?: string
readonly providerOptions?: ProviderOptions
}
const auth = (options: ProviderAuthOption<"optional">) => { const auth = (options: ProviderAuthOption<"optional">) => {
if ("auth" in options && options.auth) return options.auth if ("auth" in options && options.auth) return options.auth
return Auth.optional("apiKey" in options ? options.apiKey : undefined, "apiKey") return Auth.optional("apiKey" in options ? options.apiKey : undefined, "apiKey")
@ -32,4 +39,12 @@ export const configure = (input: Config = {}) => {
} }
export const provider = configure() export const provider = configure()
export const model = provider.model export const model: ProviderPackage.Definition<Settings>["model"] = (modelID, settings) =>
configure({
apiKey: settings.apiKey,
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
}).model(modelID)

View file

@ -10,10 +10,15 @@ describe("provider package entrypoints", () => {
import("@opencode-ai/llm/providers/anthropic"), import("@opencode-ai/llm/providers/anthropic"),
import("@opencode-ai/llm/providers/openai-compatible"), import("@opencode-ai/llm/providers/openai-compatible"),
import("@opencode-ai/llm/providers/amazon-bedrock"), import("@opencode-ai/llm/providers/amazon-bedrock"),
import("@opencode-ai/llm/providers/azure"),
import("@opencode-ai/llm/providers/azure/responses"),
import("@opencode-ai/llm/providers/azure/chat"),
import("@opencode-ai/llm/providers/google"),
]) ])
for (const module of modules) expect(module.model).toBeFunction() for (const module of modules) expect(module.model).toBeFunction()
expect(modules[0].model).toBe(modules[1].model) expect(modules[0].model).toBe(modules[1].model)
expect(modules[6].model).toBe(modules[7].model)
}) })
test("maps package settings onto the executable model", () => { test("maps package settings onto the executable model", () => {
@ -49,4 +54,49 @@ describe("provider package entrypoints", () => {
"OpenAI-Project": "proj_123", "OpenAI-Project": "proj_123",
}) })
}) })
test("selects Azure API entrypoints with the same model contract", async () => {
const Azure = await import("@opencode-ai/llm/providers/azure")
const AzureChat = await import("@opencode-ai/llm/providers/azure/chat")
const AzureResponses = await import("@opencode-ai/llm/providers/azure/responses")
const settings = {
apiKey: "fixture",
resourceName: "opencode-test",
headers: { "x-application": "opencode" },
body: { service_tier: "priority" },
limits: { context: 200_000, output: 64_000 },
}
const responses = AzureResponses.model("deployment", settings)
const chat = AzureChat.model("deployment", settings)
expect(Azure.model("deployment", settings).route.id).toBe("azure-openai-responses")
expect(responses.route.id).toBe("azure-openai-responses")
expect(responses.route.endpoint.baseURL).toBe("https://opencode-test.openai.azure.com/openai/v1")
expect(responses.route.defaults.headers).toEqual({ "x-application": "opencode" })
expect(responses.route.defaults.http?.body).toEqual({ service_tier: "priority" })
expect(responses.route.defaults.limits).toEqual({ context: 200_000, output: 64_000 })
expect(chat.route.id).toBe("azure-openai-chat")
})
test("maps Google package settings onto the Gemini model", async () => {
const Google = await import("@opencode-ai/llm/providers/google")
const selected = Google.model("gemini-2.5-flash", {
apiKey: "fixture",
baseURL: "https://generativelanguage.test/v1beta",
headers: { "x-application": "opencode" },
body: { safetySettings: [] },
limits: { context: 1_000_000, output: 65_536 },
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: 1_024 } } },
})
expect(selected.route.id).toBe("gemini")
expect(selected.route.endpoint.baseURL).toBe("https://generativelanguage.test/v1beta")
expect(selected.route.defaults.headers).toEqual({ "x-application": "opencode" })
expect(selected.route.defaults.http?.body).toEqual({ safetySettings: [] })
expect(selected.route.defaults.limits).toEqual({ context: 1_000_000, output: 65_536 })
expect(selected.route.defaults.providerOptions).toEqual({
gemini: { thinkingConfig: { thinkingBudget: 1_024 } },
})
})
}) })