* studio: wire the chat Search button to OpenAI's built-in web_search tool
When the active model is an OpenAI external provider and the user
clicks the existing Search pill in the composer, the chat-completion
request now carries the unified enable_tools shorthand:
enable_tools: true
enabled_tools: ["web_search"]
The backend's stream_chat_completion threads enabled_tools through
to _stream_openai_responses, which translates it into the Responses
API tool schema:
body["tools"] = [{"type": "web_search"}]
per the OpenAI Responses tool spec
(https://developers.openai.com/api/docs/guides/tools). OpenAI then
runs the search server-side before the model replies; the search-
informed answer streams back through the existing
response.output_text.delta path. web_search_call lifecycle events
are silently ignored for now — sources / status indicators are
follow-up scope.
Frontend:
- provider-capabilities.ts: new providerSupportsBuiltinWebSearch()
helper. Returns true only for `openai` today; Anthropic
(web_search_20250305), Gemini grounded-search, and OpenRouter
variants can be added later with matching backend translation.
- chat-page.tsx: both model-switch paths (the onChange handler and
the inferenceParams.checkpoint useEffect) set supportsTools to
match the new helper, and force toolsEnabled=false on every
external switch so the Search toggle is opt-in by default.
- chat-adapter.ts: external branch adds enable_tools +
enabled_tools=["web_search"] to the request body when the
toggle is on AND the active provider supports built-in
web-search. Local-model branch is unchanged — it continues to
route the same shorthand through our local tool runtime.
Backend:
- routes/inference.py: forwards payload.enabled_tools to
stream_chat_completion at the proxy site (line 1599).
- external_provider.py: stream_chat_completion gains an
enabled_tools parameter; _stream_openai_responses appends
{"type": "web_search"} to body["tools"] when the list contains
"web_search". Other tools (file_search, code_interpreter,
image_generation, computer_use_preview) are easy follow-ups in
the same block.
Reuses the existing pydantic ChatCompletionRequest.enabled_tools
field, so no schema migrations.
* studio/backend: surface OpenAI server-side web_search in the chat UI
When the user has the chat Search button toggled on and OpenAI's
/v1/responses invokes the built-in web_search tool, _stream_openai_responses
now translates the tool's lifecycle events and citation annotations
into the same _toolEvent shape that local-tool calls use. The result:
the chat UI shows a web_search tool-call card mid-stream, then lists
the cited sources at the end of the message — identical to how local
web_search renders.
SSE event translation:
- response.output_item.added with item.type=web_search_call ->
emit _toolEvent tool_start. Carries item.action.query as args
when OpenAI ships it on the added event.
- response.output_item.done with item.type=web_search_call ->
backfill the query if it only arrives on the done variant. The
existing reasoning branch on the same event is preserved as an
if/elif under a shared isinstance guard.
- response.output_text.annotation.added with type=url_citation ->
collect into the most-recent web_search_call.citations list.
- response.output_text.delta with inline annotations[] (older
API variant) -> same collection path, so both wire shapes work.
- response.completed -> emit _toolEvent tool_end per call with
citations formatted as
Title: <title>\nURL: <url>\nSnippet: <snippet>
blocks joined by `\n---\n`. The frontend's
parseSourcesFromResult already lifts this format into source
content parts at end-of-stream.
- response.incomplete -> close out web_search cards with whatever
citations had landed, so a truncated response does not leave a
perpetually "running" tool card in the UI.
Both reasoning and web_search work simultaneously on the same turn —
the body sends `reasoning: {effort, summary}` and `tools: [{type:
"web_search"}]` independently, and the SSE handler tracks them
through separate channels.
Diagnostic: finally-block logger now reports per stream
web_search_requested - whether the client asked for it
web_search_invocations - how many calls OpenAI actually made
citations - total URLs cited
queries - the search queries the model issued
reasoning_emitted - whether <think> content was streamed
so reports of "I clicked Search and nothing happened" can be triaged
from the backend log without browser devtools.
* studio/backend: fix empty query + per-card '(no sources cited)' on OpenAI web_search
Two display bugs on the OpenAI Responses web_search → chat-UI bridge:
1. Tool cards showed "Searching for ''" — query missing.
OpenAI's response.output_item.added for web_search_call does not
reliably populate action.query across API versions; the canonical
place is output_item.done. The previous code emitted tool_start
at added with empty args and tried to backfill at done, but the
frontend's _toolEvent: tool_start is a one-shot push (no update
mechanism), so the args stayed empty.
Fix: defer both tool_start *and* a placeholder tool_end emission
to output_item.done, where action.query is guaranteed populated.
added now just initialises tracking. Frontend then renders one
card per call with the right "Searching for: <query>" label.
2. Every card showed "(no sources cited)".
The previous code tried to attribute url_citation annotations
to individual web_search_call invocations, but OpenAI's
annotations carry no link back to a specific search call —
they're just URLs the model cited from the aggregated search
pool. With N invocations and M annotations, the previous logic
bucketed all M into the last call and stamped "(no sources
cited)" on the rest.
Fix: collect citations into a single shared all_url_citations
list, dedup by URL. At response.completed (and
response.incomplete) overwrite the *last* web_search_call's
tool_end result with the aggregated Title:/URL:/Snippet:
blocks. The frontend's parseSourcesFromResult already flatMaps
every web_search result, so one non-empty result is enough to
surface the full source-pill set at the message tail. Other
tool cards get an empty result string (no '(no sources)' text).
Diagnostic log unchanged in shape; total_citations now reads
len(all_url_citations) directly.
* studio/chat: split Code and Search pill gates so external models cannot enable Code
The previous wire-up set supportsTools=true for OpenAI external
models to light up the Search pill, but supportsTools also gates the
Code pill, so Code became clickable for OpenAI even though external
providers have no local code execution.
Separate the two gates so each pill reflects what's actually
available:
- chat-runtime-store: new `supportsBuiltinWebSearch: boolean` flag.
Distinct from supportsTools — that one still means "runtime has a
local tool sandbox" (Code, python, our DuckDuckGo web_search).
This one means "the active external provider exposes a server-side
web_search tool we can opt into" (OpenAI's /v1/responses today).
- chat-page model-switch (both code paths): for external models,
supportsTools is now forced to false (no local Code path) and
supportsBuiltinWebSearch follows providerSupportsBuiltinWebSearch.
Local-model paths are unaffected — they only set supportsTools.
- shared-composer: Search pill gates on
`searchDisabled = !modelLoaded || !(supportsTools ||
supportsBuiltinWebSearch)`. Code pill gates on
`codeDisabled = !modelLoaded || !supportsTools` — strictly the
local runtime, so external models keep Code greyed out.
A `toolsDisabled = codeDisabled` alias is left in place for any
later-touched call site that may still reference the old name.
No backend changes — chat-adapter already calls
providerSupportsBuiltinWebSearch directly, independent of the store
flags, so the request shape and the backend translation are
unchanged.
* studio/chat: default external reasoning effort to medium, not the carry-over
When switching to an external model with reasoning support, the effort
dropdown was inheriting whatever value the user had set on a prior
model — frequently "xhigh" left over from a previous Opus/gpt-5
session. That meant every fresh OpenAI/Anthropic selection started at
Extra High, burning tokens unintentionally.
Both model-switch sites in chat-page (the useEffect on
inferenceParams.checkpoint and the onChange callback) now pick
"medium" whenever the new model's level list contains it, instead of
the clamped carry-over. The clamp still fires as a fallback for the
narrow case where a model doesn't expose medium (e.g. gpt-5.3-chat-
latest which only has medium anyway — no change there). Users can
still pick another level explicitly via the Think dropdown.
* studio/chat: also light the Search pill in the welcome-screen composer
There are two composers in the chat feature. shared-composer.tsx
renders inside an active thread, and assistant-ui/thread.tsx has its
own WebSearchToggle / CodeToolsToggle that ship the welcome-screen
"Send a message…" composer (visible before the first user message).
The previous fix split supportsTools and supportsBuiltinWebSearch in
shared-composer but never touched the welcome-screen toggles in
thread.tsx — they both still gated on supportsTools alone, so the
Search pill stayed greyed on the welcome screen even for OpenAI
external models that legitimately support web_search server-side.
Mirror the shared-composer rule in WebSearchToggle:
disabled = !modelLoaded || !(supportsTools || supportsBuiltinWebSearch)
CodeToolsToggle is left as-is — its current
`disabled = !(modelLoaded && supportsTools)` is correct: external
models have no local code-execution sandbox, so Code stays greyed
when supportsTools=false (which is what chat-page now writes for
external selections).
* studio/backend: wire Anthropic server-side web_search end-to-end
Mirrors the OpenAI web_search integration for Anthropic's
web_search_20250305 tool. When the user toggles Search on with an
Anthropic model selected, the request now carries the documented
tool entry:
tools: [{type: "web_search_20250305", name: "web_search",
max_uses: 5}]
on /v1/messages, and the SSE translation surfaces tool cards +
source pills in the chat UI exactly the same way as OpenAI.
stream_chat_completion now forwards enabled_tools into the
Anthropic branch (was only doing this for the OpenAI Responses
branch). _stream_anthropic gains an enabled_tools parameter and
the web_search request-body block plus three additional event
handlers:
- content_block_start with type=server_tool_use, name=web_search:
start tracking a new call. id becomes the tool_call_id.
- content_block_delta with type=input_json_delta inside a
server_tool_use block: buffer the partial_json so we can read
out the search query when the block closes.
- content_block_start with type=web_search_tool_result: capture
the per-call result list (urls + titles) that Anthropic ships
inline.
- content_block_stop: closes whichever block we're inside —
* server_tool_use -> emit _toolEvent: tool_start with the
parsed query as args.
* web_search_tool_result -> emit _toolEvent: tool_end with
Title:/URL: blocks the frontend's parseSourcesFromResult
lifts into source pills.
* thinking block -> existing </think> close.
Unlike OpenAI we get per-call results directly, so no aggregated-
last-call fallback is needed — each tool card carries its own
citations.
Diagnostic log on stream completion now reports
web_search_requested / invocations / total_results / queries,
matching the OpenAI shape.
Frontend providerSupportsBuiltinWebSearch returns true for
'anthropic' as well, so the Search pill lights up on Claude
models the same way it does on OpenAI. The existing chat-adapter
external branch already sends enabled_tools=['web_search'] based
on this helper — no adapter changes needed.
* studio: wire OpenRouter built-in web search via :online model suffix
OpenRouter exposes a universal "add web search to any model" shortcut:
append `:online` to the model id and the gateway runs the search
server-side, streaming citations back as annotations on text deltas.
Documented at https://openrouter.ai/docs/features/web-search
Hook the existing Search toggle into that path:
Backend (external_provider.py, default OAI-compat branch):
- When provider_type == 'openrouter' and enabled_tools contains
'web_search', rewrite body['model']:
openai/gpt-4o -> openai/gpt-4o:online
anthropic/claude-sonnet-4-5:free -> anthropic/claude-sonnet-4-5:online
Any existing `:variant` (`:free`, `:nitro`, etc.) is replaced —
OpenRouter variants are mutually exclusive.
- `openrouter/free` is skipped: it's a meta-router and `:online` is
not a valid suffix on it (the gateway 400s).
- A one-line INFO log fires whenever the rewrite happens so the
diagnostic backend log shows exactly which model id the request
was promoted to.
Frontend (provider-capabilities.ts):
- providerSupportsBuiltinWebSearch now returns true for 'openrouter'
alongside 'openai' and 'anthropic'. The Search pill lights up and
the existing chat-adapter external branch already forwards
enabled_tools=['web_search'] based on this helper — no adapter
changes needed.
No new SSE event handling: OpenRouter does not emit a separate
web_search_call event the way OpenAI/Anthropic do. Citations come
back as text annotations via the existing reasoning_details path
the adapter already parses, so source data flows through without
extra translation. A per-call tool-card UX ("Searching for: …")
would require synthesizing one client-side; deferred to a follow-up
if the bare-citation flow feels too minimal.
* studio: wire Mistral built-in web search connector
Same shape as OpenAI's web_search tool, lives on
/v1/chat/completions instead of /v1/responses. When the chat
Search pill is toggled on with a Mistral model selected, the
backend now appends
{"type": "web_search"}
to body["tools"] before the request goes out. Idempotent —
won't double-append if a future call site adds it first. Models
in the registry allowlist that don't support the connector
(codestral, devstral, ministral, mistral-tiny) will surface a
400 from upstream; the existing default-path error log captures
it. Mistral's docs:
https://docs.mistral.ai/capabilities/agents/connectors/websearch
Frontend providerSupportsBuiltinWebSearch returns true for
'mistral' now, alongside openai / anthropic / openrouter. The
Search pill lights up for Mistral models and the existing
adapter branch already sends enabled_tools=['web_search'] off
this helper — no adapter changes.
No SSE translation yet — Mistral streams citations inline as
text annotations or `references` in the final assistant content,
not as a separate web_search_call event. Citations flow through
to the message body as text; a per-call tool-card UX with
"Searching for: …" indicators is a follow-up if needed.
* studio/backend: fix OpenRouter web_search to use plugins shape + synthesize tool card
Two changes against the actual OpenRouter docs at
https://openrouter.ai/docs/guides/features/plugins/web-search:
Request shape:
The previous commit appended :online to the model id, which works on
concrete model ids but rejects on meta-routers like openrouter/free —
and that's exactly the model the user was testing with, so neither
the request rewrite nor the diagnostic log fired. Switch to the
universal plugins shape:
body["plugins"] = [{"id": "web"}]
Per the docs this is "exactly equivalent" to :online but works on
every model id including openrouter/free and openrouter/auto. No
model suffix manipulation, idempotent if added twice.
Tool-card synthesis:
OpenRouter doesn't emit a structured web_search_call event the way
OpenAI/Anthropic do — citations come back only as `annotations` of
type=url_citation on delta/message objects. To match the chat-UI
tool-card UX the user expects ("Searching for: …" indicator,
source pills at message tail), synthesize the events client-side
in the default OAI-compat stream loop:
- On stream open (after the 200 status check): yield a synthetic
_toolEvent: tool_start with tool_name=web_search, fixed id
"openrouter_web_search". The chat-UI then renders the running
tool card before any text streams.
- During the SSE loop: scan every chunk's choices[].delta and
choices[].message for `annotations: [{type: "url_citation",
url_citation: {url, title, content}}]` entries. Dedup by URL
into a citations list. Handles both the nested-url_citation
shape OpenRouter documents and the flat-on-annotation shape
some upstreams ship.
- On [DONE] (or stream-close without [DONE]): emit synthetic
tool_end carrying the citations as
Title: …\nURL: …\nSnippet: …\n---\n…
blocks the existing parseSourcesFromResult lifts into source
pills at message tail.
Diagnostic log on completion now also reports
web_search_requested + citation count alongside the existing
chosen-model / event-count telemetry.
* studio: drop Mistral built-in web_search — connector lives on Agents API only
Mistral's web_search is exclusively on /v1/agents + /v1/conversations;
sending it on /v1/chat/completions returns
"WebSearchTool connector is not supported". Wiring it would require a
dedicated Agents streaming path. Remove from the frontend capability map
and revert the chat-completions tool injection.
* studio: wire Kimi $web_search builtin via two-call round-trip
Kimi's $web_search lives on /v1/chat/completions but requires a client
round-trip per https://platform.kimi.ai/docs/guide/use-web-search:
the first call returns tool_calls with function.arguments populated;
the caller echoes those arguments back as a role=tool message; the
second call streams the final answer with search results incorporated.
The docs also mandate thinking=disabled while the builtin is active.
Backend: new _stream_kimi_web_search helper dispatched from
stream_chat_completion when provider_type=='kimi' and 'web_search' in
enabled_tools. Buffers tool_calls across deltas, falls back to a plain
stream if the model declines to search, and synthesizes tool_start
(with parsed query) / tool_end (with any url_citation annotations) so
the chat UI's web-search card behaves the same as other providers.
Frontend: kimi added to providerSupportsBuiltinWebSearch so the Search
pill lights up in the composer.
* studio/chat: mutual exclusion of Think + Search on Kimi composer
Kimi's $web_search builtin requires thinking=disabled per
https://platform.kimi.ai/docs/guide/use-web-search, so the two states
cannot coexist. Make the pills mutually exclusive in both composers
(shared and welcome-screen): clicking Search turns Think off; clicking
Think back on turns Search off. Default Think to on when a Kimi model
is selected — k2.6/k2.5 ship with thinking enabled out of the box.
* studio/chat: fix wrong provider var name in onChange branch
selectedProvider, not provider — TS2304 in tsc -b.
* studio/backend: add diagnostics to Kimi $web_search round-trip
Log the actual function.arguments from the first call (so we can see
the model's search query) and the second call's usage.prompt_tokens +
any annotation type names that came through. prompt_tokens spiking
above the input message length is direct proof the server injected
search results into context. annotation_types lets us learn the shape
Kimi uses for citations if/when they emit any.
* studio: per-provider defaults — Anthropic xhigh + Search on, OpenAI high + Search on, Opus 4.7 gains max
Anthropic: Think effort defaults to the highest level the model
supports (xhigh on 4.6/4.7, high on 4.5) and Search starts on, since
the web_search_20250305 tool returns structured citations end-to-end.
OpenAI: Think effort defaults to 'high' (the gpt-5.x reasoning sweet
spot for /v1/responses + web_search) and Search starts on.
Opus 4.7: 'max' added as an effort level above 'xhigh' in both
backend (_ANTHROPIC_THINKING_SPECS) and frontend (ANTHROPIC_REASONING_MODELS).
Kimi diagnostics: emit tool_end immediately after tool_start so the
web-search card transitions to 'complete' before the second-call
answer streams, log first-call args + second-call usage/prompt_tokens
+ any annotation type names, request stream_options.include_usage so
the second call exposes usage in SSE.
* studio/backend: harden Kimi fallback path with HTTPError handler + manual aiter_lines loop
Addresses PR review feedback (#5443): the no-search fallback streaming
path was using `async for response.aiter_lines()` and had no
`httpx.HTTPError` guard around the POST. Switch to the manual
__anext__ loop pattern used elsewhere in this module (avoids the
Python 3.13 + httpcore 1.0.x GeneratorExit propagation issue) and wrap
the whole request in a try/except so network failures surface as a
proper SSE error frame instead of a raw traceback.
* studio: add external provider support for chat inference
Adds the ability to connect to OpenAI, Mistral, Google, Cohere, Together,
Fireworks, and Perplexity from the Studio chat interface.
- Provider configs stored in SQLite (no API keys persisted)
- RSA-2048 key pair generated at startup for client-side key encryption
- httpx proxy client streams SSE responses in OpenAI-compatible format
- New /api/providers routes: registry, CRUD, test, models
- /v1/chat/completions routes to external provider when provider fields present
- Integration test suite covering CRUD, connection, model listing, and inference
- Frontend spec doc with full API contract
* remove frontend spec doc from branch
* fix auth fixture: handle forced password change on fresh install
* fix tests: default port 8000, allow 400 for no-model-loaded
* fix: update Cohere models to current (command-r retired Sept 2025)
* feat: add OpenRouter as 8th provider
* feat: add native Anthropic provider with Messages API translation
* fix: correct Anthropic base URL and drop top_p (conflicts with temperature)
* feat: add DeepSeek provider (deepseek-chat, deepseek-reasoner)
* feat: rename google -> gemini, refresh model list to 2.5 series
* feat: remove together, fireworks, perplexity providers
* feat: multimodal image support for external providers
- Add _build_external_messages() that preserves image_url parts for
vision-capable providers instead of stripping them
- Update _proxy_to_external_provider() to use new helper
- Translate image_url content parts to Anthropic native image format
in _stream_anthropic()
- Add TestVisionInference pytest class (1x1 PNG smoke test)
* test: use sloth photo URL for vision test, add Anthropic remote URL support
* fix: update Mistral model to mistral-small-2506
* update mistral default model to mistral-large-2512
* fix gemini vision test: download image as base64 data URI instead of remote URL
* add gemini-3-flash-preview as default gemini model
* fix gemini truncated reply (max_tokens 16->64) and suppress GeneratorExit on client disconnect
* increase vision test max_tokens to 215
* fix GeneratorExit: aclose stream generator before closing httpx client
* fix httpcore GeneratorExit: explicitly aclose aiter_lines before response closes
* fix duplicate [DONE] and suppress httpcore RuntimeError on Python 3.13 asyncgen cleanup
* fix: call response.aclose() before lines_gen.aclose() to prevent httpcore RuntimeError on Python 3.13
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Potential fix for code scanning alert no. 36: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* review: add comments for manual iteration rationale, mask password in test print, clarify Anthropic URL/models support
* perf: use shared module-level httpx client for connection pooling across requests
* studio: add API provider UI and integrate wiring (#4737)
* feat: expose external models in selector and chat settings
* feat(chat): wire external providers to backend + RSA key flow
- Fetch registry/configs; create/update/delete saved providers
- Encrypt API keys (Web Crypto RSA-OAEP) for test/models/chat
- External model selection + chat payload (provider_id/type, external_model, encrypted key, optional base URL)
- Local storage for keys + provider list; small UX/copy and guardrails
* add missing providers-api.ts file by Imagineer99
* fix: address PR review comments — system prompt visibility, retry loop, test logging
* feat(studio): encrypt external provider API keys at rest in localStorage
API keys for external providers (OpenAI, Mistral, etc.) were stored as
plaintext in localStorage, vulnerable to browser extensions and XSS.
Add password-derived AES-256-GCM encryption: on login the user's password
is used via PBKDF2 (100k iterations, SHA-256) to derive an in-memory
encryption key. API keys are encrypted before writing to localStorage and
decrypted on read. The derived key is never persisted — cleared on logout,
re-derived on next login.
Legacy plaintext keys are transparently migrated on first access. Password
changes re-encrypt all stored keys. No backend changes required — the
existing RSA-OAEP transit encryption is unaffected.
* fix: cast PBKDF2 salt to BufferSource for strict TypeScript lib types
* fix: persist session password in sessionStorage to survive page refreshes
* feat(studio): preserve image parts in external provider chat requests
toOpenAIMessage() now returns multimodal content arrays (OpenAI vision
format) when messages contain images, instead of always flattening to
plain text. This enables vision-capable external providers (OpenAI,
Gemini, Anthropic, etc.) to receive user images. The backend already
handles image_url content parts in _build_external_messages().
* studio: fix external models selectable in chat-only mode (#4779)
* fix: external models selectable in chat-only mode
* fix: model selector tabs default to active model kind
* Studio: API external provider registry + curated catalogs (HF/OpenRouter) and chat UX (#4787)
* fix: external models selectable in chat-only mode
* fix: model selector tabs default to active model kind
* feat(studio): expand provider registry, curated catalogs, and chat UX
- Add Hugging Face, Kimi, Qwen; remove Cohere; reorder registry
- model_list_mode curated for HF/OpenRouter; lightweight /models check
- API returns default models for curated providers; expose model_list_mode
- Frontend: provider logos in model picker, providerType on external models
- Chat providers dialog: curated vs remote flows, motion polish
- Thread: LayoutGroup + composer motion alignment with app easing
* fix(studio): disable Anthropic tool-calling flag and preselect curated defaults
* feat(studio): add external provider logos and ApiProviderLogo helper
* Studio: Polish API Providers dialog (#4899)
* fix: lower verbage in API providers page
* fix: fix(studio): tune API Providers dialog width with rem-based responsive caps
* feat: add custom provider support (#4902)
* fix: replace crypto.subtle with node-forge for HTTP compatibility
crypto.subtle is only available in secure contexts (HTTPS/localhost),
which breaks provider API key encryption when Studio is accessed over
plain HTTP on remote GPU VMs. Switch to node-forge for RSA-OAEP and
AES-256-GCM operations — same algorithms, works on any origin.
* fix: store provider API keys as plaintext in localStorage
Drop AES-256-GCM at-rest encryption for provider API keys. The
session-password-derived encryption broke on auto-login via refresh
token (password never captured), causing keys to silently vanish.
API keys are still RSA-encrypted in transit via node-forge. At-rest
encryption in localStorage added no real security since the
decryption key also had to live client-side.
Removes crypto-storage.ts, session password plumbing, and
reEncryptAllKeys.
* fix: use max_completion_tokens for OpenAI provider
Newer OpenAI models (gpt-4o, gpt-5.x) reject the max_tokens param
and require max_completion_tokens instead. Other providers still use
max_tokens.
* fix: skip empty assistant messages in external provider requests
Some providers (Mistral) reject assistant messages with empty content.
Filter them out when building the message list for external providers.
* Update model-selector.tsx
* Update model-selector.tsx
* Update model-selector.tsx
* Update chat-adapter.ts
* Update chat-adapter.ts
* Update chat-page.tsx
* Update chat-settings-sheet.tsx
* Update chat-settings-sheet.tsx
* Update chat-settings-sheet.tsx
* Update chat-providers-dialog.tsx
* feat: polish providers settings form UI
* style: polish provider row icon sizing and alignment
* style: stabilize provider layout
* style: add provider API key visibility toggle
* fix: add provider render on empty list
* studio/frontend: sync package-lock.json with package.json
npm ci was failing because node-forge and @types/node-forge were
declared in package.json but missing from the lockfile. Ran
npm install to regenerate.
* studio/backend: fix backend CI failures for providers router
- test_desktop_auth: include providers_router in the routes stub so
studio.backend.main imports cleanly under the monkeypatched module
- test_providers_api: skip the whole module when STUDIO_TEST_PASSWORD
is unset (it is an integration test against a live Studio server,
same shape as the already-ignored test_studio_api.py)
* studio/chat: drive ChatSettingsPanel from a per-provider capability map
Replace the binary isExternalModel toggle in the sampling section with a
provider-aware capability map. Each external provider type advertises
which of top_k / min_p / repetition_penalty / presence_penalty its
chat-completions API actually accepts, so the panel only renders the
knobs that map onto the active provider's request body.
Anthropic now exposes top_k; DeepSeek hides presence_penalty (deprecated
in their docs); OpenRouter and custom providers continue to show every
knob (OpenRouter drops unsupported server-side, custom assumes
OpenAI-compat or a permissive vLLM/Ollama backend). Local models are
unaffected — null capabilities means 'show everything'.
chat-adapter.ts now forwards top_k / presence_penalty to the external
proxy only when the active provider's capabilities permit it, so the
request body matches what the UI shows.
* studio/backend: forward top_k to Anthropic; filter OpenAI model list
Two paired changes so the frontend capability map has matching backend
behaviour:
1. ExternalProviderClient.stream_chat_completion now accepts top_k and
forwards it to the Anthropic Messages body. OpenAI-compat providers
(which all reject unknown sampling params) still receive only the
fields they document. The proxy route in routes/inference.py passes
payload.top_k through, so a UI request with top_k actually reaches
Anthropic instead of being silently dropped at the boundary.
2. PROVIDER_REGISTRY['openai'] gains a model_id_allowlist regex that
scopes the /models picker to current-gen ids (gpt-5.5 / gpt-5.4 /
gpt-5.3 / gpt-4.5 / o3 families). The remote /v1/models listing
otherwise returns dozens of historical snapshots, fine-tunes and
non-chat models (embeddings, TTS, image, moderation) that we never
want in the chat UI. default_models is refreshed to match.
* studio/chat: relax presence_penalty to optional on OpenAIChatCompletionsRequest
Followup to 1fbf445a — chat-adapter now omits presence_penalty for
providers that do not accept it (Anthropic / DeepSeek), but the
request type still required it as a non-optional number, breaking
tsc. The backend pydantic model already defaults presence_penalty
to 0, so making it optional client-side matches reality.
* studio/backend: route OpenAI traffic through /v1/responses
OpenAI's new flagship models (gpt-5.x) return 404 'This is not a chat
model' on /v1/chat/completions and are only reachable via /v1/responses.
Add a dedicated _stream_openai_responses path in ExternalProviderClient
that:
- Translates outbound messages into the Responses shape: system messages
are folded into the top-level 'instructions' field, user/assistant
messages become {role, content} items with input_text / input_image
content parts (data URLs and https URLs both pass through).
- Drops presence_penalty / top_k / frequency_penalty, none of which the
Responses contract accepts.
- Translates inbound SSE events back into OpenAI Chat Completions
chunks so the frontend keeps a single SSE shape:
response.output_text.delta -> delta chunk with content
response.completed -> chunk with finish_reason='stop'
response.incomplete -> chunk with finish_reason='length'
response.failed / error -> propagated error SSE line
Stream terminates with data: [DONE] (Responses emits this verbatim).
stream_chat_completion dispatches all provider_type='openai' calls to
this path; other OpenAI-compatible providers (mistral, gemini, etc.)
continue to use /v1/chat/completions.
Frontend provider-capabilities map updated to hide presence_penalty for
OpenAI in the chat settings panel, matching the new request contract.
Includes unit coverage in tests/test_openai_responses_translation.py
exercising the request body translation, image-part rewriting, and
SSE-to-chat-completions translation via httpx.MockTransport.
* studio/chat: clamp external max_tokens to 32k to stay within provider caps
The chat settings slider already capped maxTokens at 32768 for external
models, but a value persisted from a prior local-model session (where
the cap can be 128k+) was sent verbatim to the provider — Claude Opus
returns 'max_tokens: 131072 > 128000' on requests like that, and other
providers have stricter limits still.
Expose EXTERNAL_MAX_OUTPUT_TOKENS from provider-capabilities (32k) and
use it both for the slider max and as the clamp inside chat-adapter's
external-request body. 32k sits below the tightest declared output
limit across the providers we ship and well above what a typical chat
reply needs; the local-model path is unaffected.
* studio: drop temperature/top_p for OpenAI reasoning models
gpt-5.x / o3 / gpt-4.5 are reasoning-class models served via
/v1/responses, and reject temperature and top_p with
'Unsupported parameter' 400s. The OpenAI registry allowlist already
scopes the picker to those families, so neither knob ever applies on
this branch.
- external_provider._stream_openai_responses no longer puts
temperature or top_p in the request body (kept on the method
signature for API symmetry with the other stream methods).
- ProviderCapabilities gains temperature/topP flags; OpenAI sets both
to false. ChatSettingsPanel hides the sliders for OpenAI so the user
does not see inert controls.
- chat-adapter omits temperature/top_p from the external request body
when the active provider does not advertise them.
- OpenAIChatCompletionsRequest type marks both as optional, matching
the new chat-adapter shape.
- test_responses_request_body_uses_input_and_instructions: assertions
flipped to confirm temperature / top_p are absent from the body.
* studio: stop forwarding top_k to Anthropic
Claude 4.x (Opus / Sonnet / Haiku 4.x) returns 400 'top_k is
deprecated for this model' on any request that includes top_k. It
was always optional on the older 3.x line, so dropping it
unconditionally for every Anthropic call is the simplest path —
no per-model gate to maintain.
- external_provider._stream_anthropic no longer adds top_k to the
Messages body (kept on the method signature for API symmetry).
- provider-capabilities sets anthropic.topK = false so the chat
settings panel hides the Top K slider for Anthropic providers
and chat-adapter does not send top_k in the external request.
* studio: gate Anthropic top_k drop to Claude 4.7 only
Previous commit (b5aa6ffd) dropped top_k for every Anthropic call,
but only Claude 4.7 (Opus/Sonnet/Haiku) actually rejects it. 4.6, 4.5,
and the 3.x line still accept top_k and use it as documented.
Backend: _stream_anthropic matches the model id against
^claude-(opus|sonnet|haiku)-4-7(-|.|$) and only strips top_k when it
hits. Every other Claude generation continues to receive the value
from the chat settings panel.
Frontend: anthropic.topK is restored to true so the Top K slider is
visible again — the backend handles the per-model drop, and the
4.7 case is silent (request still succeeds without top_k).
* chore: hide dated openai models in provider select
* studio/providers: apply model_id_denylist when listing remote models
The OpenAI registry entry gained a model_id_denylist regex matching
dated snapshot ids (-YYYY-MM-DD) in 048d73bf, but the list-models
route was never consulting it, so the snapshots still showed up
alongside their canonical ids (gpt-5.5 and gpt-5.5-2026-04-23 both
listed). Apply the denylist with .search() right after the allowlist
filter so dated entries are dropped before the response is built.
* studio/chat: seed registry default_models for remote providers in picker
The Anthropic provider runs in remote model-list mode, so the picker
started with an empty availableModels until the user clicked
'Load Models'. If that /api/providers/models call fails (e.g. the
known transient decryption error during key rotation), the user sees
no models at all — claude-haiku-4-5 in particular was missing from
the dialog even though it is seeded in the registry.
Always pre-populate availableModels with the registry's default_models
when a provider type is selected (curated and remote alike), and have
loadModels() return the union of defaults + the live /models response
so registry-seeded ids are reachable regardless of what the provider's
endpoint returns or whether the call succeeds at all.
* studio/backend: diagnostic logging on provider key decryption
Decryption failures currently log just 'Failed to decrypt API key:
Decryption failed', which leaves no way to tell whether the cause is
a stale public key in the browser, a corrupted ciphertext, an
unexpected exception class, or a server-side keypair rotation. That's
the gap the next reproduction needs to close.
- key_exchange now publishes a short SHA256 fingerprint of the public
key PEM. init_key_pair logs the fingerprint on generation and warns
if it is ever called a second time (re-init silently invalidates
every browser that cached the previous public key).
- decrypt_api_key wraps both the base64 decode and the RSA decrypt
in dedicated try/excepts that log exception type, ciphertext byte
length (RSA-2048 should be exactly 256), input string length, and
the current public-key fingerprint.
- GET /api/providers/public-key returns the fingerprint alongside the
PEM so the frontend can correlate a future encrypt-time fingerprint
against the decrypt-time fingerprint and prove or rule out a
keypair rotation as the cause.
- The /test and /models route-level decrypt warnings now include the
exception class name (alongside the existing message).
* studio/providers: hide dated Anthropic snapshots from the model picker
Anthropic's /v1/models returns dated snapshot ids (e.g.
claude-3-5-sonnet-20241022, claude-3-5-haiku-20241022) alongside
the canonical names users actually want to pick. Same intent as
the OpenAI denylist added in 048d73bf, just a different date
format — Anthropic uses -YYYYMMDD (no dashes) while OpenAI uses
-YYYY-MM-DD.
- Add model_id_denylist = re.compile(r'-\d{8}$') to the anthropic
registry entry. The /api/providers/models route already applies
any denylist after fetching, so dated ids drop out automatically.
- Strip the dated 3.5 ids from default_models so the seeded picker
no longer surfaces them; keep claude-opus-4-7 and the 4.5 family
as the curated set.
Net effect: the picker shows opus-4-7 / opus-4-5 / sonnet-4-5 /
haiku-4-5 only, regardless of whether the remote /models call
succeeds or fails.
* fix: provider dialog and mistral short list
* style: fix provider dialog curated list styling
* fix: provider dialog curated model ids placeholder reference
* style: rename Providers to Cloud and tighten dialog header spacing
* UX: rename Providers to Cloud, remove header shortcut
* studio/chat: normalize structured delta.content from reasoning providers
Mistral's magistral (and similarly-shaped reasoning models) stream
chat-completion deltas where choices[0].delta.content is an array of
structured parts rather than a plain string, e.g.
[{ type: 'text', text: '...' }, { type: 'thinking', thinking: '...' }]
The accumulator did 'cumulativeText += delta', which coerced each
part to '[object Object]' and produced output like
'[object Object][object Object]...Hey there!'.
Add extractDeltaText() to normalize delta.content before append:
- string → returned as-is
- array of parts → text/output_text parts contribute their .text or
.content; thinking/reasoning parts are re-wrapped inline as
<think>...</think> so the downstream parseAssistantContent lifts
them into a reasoning part the same way it does for providers that
emit thinking inline. magistral keeps its thinking panel; no other
provider's output shape changes.
- unknown shapes → dropped rather than stringified, so a stray field
cannot pollute the rendered chat with '[object Object]'.
* Studio: restore Cloud icon shortcut in chat header
Brings back the header chip that opens Settings -> Cloud (external
providers) directly from the chat view. Same button as before the
bf24e604 removal: single-mode only, opens useSettingsDialogStore on
the 'connections' tab, tooltip 'API providers'.
* studio/chat: strip trailing template literal from external provider streams
Mistral's magistral occasionally appends a literal '${response}' token
after its actual answer — likely a training-format artifact, since it
keeps happening with an empty system prompt and only on that model.
Apply a tight strip in the chat-adapter SSE accumulator: when the
active provider is external, drop a trailing '${...}' template literal
(with optional whitespace) from cumulativeText after each chunk. The
regex anchors to end-of-string, so mid-stream fragments ('${re')
remain untouched and only collapse once the closing brace arrives.
Local-model output is unaffected.
* studio/providers: scope Kimi picker to kimi-k2.6 / kimi-k2.5
Mirror what the live Kimi docs surface as the current models
(https://platform.kimi.ai/docs/models). Everything else the
remote /v1/models call returns — moonshot-v1-* legacy ids and
dated k2 previews like kimi-k2-0711-preview — is filtered out.
- default_models: ['kimi-k2.6', 'kimi-k2.5'] (was four
legacy moonshot-v1 ids plus the dated k2 preview)
- model_id_allowlist: ^kimi-k2\.[56]$ applied in the
/api/providers/models route after the live fetch
- doc-link comments point at platform.kimi.ai overview /
models / list-models for the next refresh
* studio: drop temperature/top_p for Kimi reasoning models
Kimi k2.5/k2.6 are reasoning-class. The API locks temperature and
top_p to fixed defaults and 400s on any other value with
'invalid temperature: only 1 is allowed for this model'.
The frontend capability map already gated these knobs out of the
external request body, but the OpenAI-compat path on the backend
unconditionally re-adds them from the pydantic ChatCompletionRequest
defaults (temperature=0.7 etc), so the gate was bypassed end-to-end.
Add a generic body_omit hook on the provider registry that
stream_chat_completion consults after building the body, and use it
to strip temperature/top_p for Kimi. Frontend provider-capabilities
flips kimi.temperature and kimi.topP to false so the sliders are
hidden in the chat settings panel as well.
* studio/providers: scope Gemini picker to current 3.x + *-latest aliases
Google's /v1beta/openai/models returns dozens of historical,
experimental, and non-chat ids that we never want in the chat UI.
Cap the picker to the current curated set:
- gemini-3.1-pro-preview
- gemini-3.1-flash-lite
- gemini-3-flash-preview
- gemini-pro-latest
- gemini-flash-latest
- gemini-flash-lite-latest
Default_models seeded with these, model_id_allowlist applied in
the /api/providers/models route to drop anything else the live
fetch returns.
* studio/providers: switch Hugging Face to remote model listing
Per the Inference Providers docs
(https://huggingface.co/docs/inference-providers/index),
GET https://router.huggingface.co/v1/models returns the full
chat-model catalog across all providers, including per-provider
metadata. The OpenAI-compatible endpoint we already use for
chat completions accepts the same Bearer token, so flipping
model_list_mode from 'curated' to 'remote' lets users discover
models via the existing list_models() path without any new
wiring.
- model_list_mode: 'remote' (was 'curated')
- default_models refreshed with current popular ids
(gpt-oss-120b, DeepSeek-V3, Llama-3.3-70B, Qwen2.5-72B) so the
picker still has a sensible seed if /v1/models fails
- notes updated to reference the docs page and clarify the
endpoint is chat-only
* UX: chat cloud icon changed to model select signifier
* studio/providers: org allowlist + count cap for HF Inference picker
The HF /v1/models response is the full cross-provider catalog (hundreds
of ids — community fine-tunes, mirrors, fp8 variants, dated snapshots).
Scope the picker to the first-party org repos worth surfacing and cap
the post-filter list.
- model_id_allowlist matches the org prefixes openai/, deepseek-ai/,
google/, meta-llama/, Qwen/, moonshotai/, mistralai/, zai-org/.
Anything outside those orgs is dropped.
- model_id_limit (new registry field) caps the post-filter list. The
list-models route now slices [:limit] after allowlist/denylist; set
to 15 for HF Inference. Other providers leave it unset and behave
exactly as before.
- default_models stays as the seed so the flagship ids users care
about (gpt-oss-120b, DeepSeek-V3, Llama-3.3-70B, Qwen2.5-72B) are
always reachable regardless of the API's response order.
Dedup is already handled in loadModels() via Set, so no additional
work needed there.
* style: adjust cloud icon right margin with rem spacing
* Studio: cloud openai reasoning level toggle (#5402)
* feat: cloud openai reasoning level toggle
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: honor enable_thinking=false
* fix: prevent local reasoning toggle regressions and align OpenAI effort levels
* fix: isolate external OpenAI reasoning toggle state
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* fix: clamp reasoning effort
* fix: align OpenAI reasoning effort
* fix: clear stale GGUF badge state
* ui: new badge on cloud setting
* fix: separate selected models from cached provider model list
* Studio: anthropic effort by model family (#5412)
* feat: external thinking control and Anthropic effort mapping
* fix: anthropic thinking constraints and 4.6 max effort mapping
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: harden Anthropic thinking params and effort mapping
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio/backend: drop top_p from Anthropic body when thinking is enabled
PR 5412 added body['top_p'] = max(0.95, min(top_p, 1.0)) inside the
thinking branch of _stream_anthropic, but Anthropic returns 400 on
extended/adaptive thinking when both temperature and top_p are set:
invalid_request_error: temperature and top_p cannot both be
specified for this model. Please use only one.
(Observed on Claude Opus 4.6.) The contract for thinking-enabled
requests is temperature=1 with neither top_p nor top_k allowed.
Replace the body['top_p'] = ... line with body.pop('top_p', None).
Defensive pop rather than a bare delete: the base body construction
above does not currently set top_p, but a future edit that adds it
would silently reintroduce the regression.
* studio/chat: force reasoningEnabled=true on local reasoning-effort models
Followup to PR 5402 / 5412. The model-status refresh path in
use-chat-model-runtime carried reasoningEnabled forward verbatim for
every reasoning-capable model. That left one observable edge case:
1. user picks an external model that supports Off (gpt-5.x, Claude
4.x), clicks Off — store sets reasoningEnabled=false
2. user switches back to a local reasoning-effort model
(gpt-oss / Harmony-style) which does NOT support Off
3. composer's effectiveReasoningEnabled override paints the UI as
'Think: <level>' (on)
4. chat-adapter sees reasoningEnabled=false on the local branch
and sends '{}', so the backend's _request_reasoning_kwargs
returns None and the Harmony template falls back to its own
default effort instead of the displayed level
Mirror the composer's override in the store on load: for local
reasoning-effort models (where supportsReasoningOff is false), force
reasoningEnabled=true so the store and the UI agree on every send.
Other reasoning styles still inherit prior state — only the
reasoning-effort family changes.
* studio/backend: align Anthropic thinking with the extended-thinking docs
Two compliance fixes against
https://platform.claude.com/docs/en/build-with-claude/extended-thinking
1. Adaptive-mode effort field shape
The docs spell adaptive thinking as:
{'thinking': {'type': 'adaptive'}, 'effort': {'type': '<level>'}}
We had been sending the legacy 'output_config: {effort: <level>}'
shape, which Anthropic appears to silently ignore — adaptive ran
at the server default effort regardless of the user's selection.
Rename to 'effort: {type: <level>}'.
2. thinking_delta event translation
The Messages-API streams reasoning content as
content_block_delta events with delta.type == 'thinking_delta',
which our SSE loop was dropping entirely. On Claude 4.5/4.6 with
display=summarized (the default), the user would see the answer
text but never the reasoning panel. Wrap thinking_delta.thinking
as inline <think>...</think> chunks (same pattern as the OpenAI
Responses path) so the frontend's parseAssistantContent lifts it
into the reasoning channel. The </think> closer fires on the
first text_delta transition, on content_block_stop for the
thinking block, on message_delta, and on message_stop —
whichever arrives first — so no model path can leak an
unclosed <think> into chat output.
signature_delta events are left as no-ops; they carry
verification metadata, not user-visible content.
Adds test_anthropic_thinking_translation.py with httpx.MockTransport
coverage of: effort shape on adaptive (Claude 4.6), budget_tokens
shape on manual (Claude 4.5), thinking_delta wrapping with signature
suppression, and thinking-only turns (display=omitted on Opus 4.7).
* studio/backend: revert Anthropic adaptive effort to output_config nesting
The previous commit (0a664df4) moved the adaptive-thinking effort
field to a top-level 'effort: {type: <level>}' based on a misread of
the docs page. The actual Messages API schema nests it under
output_config:
thinking: optional ThinkingConfigParam ({type: 'adaptive'})
output_config: optional OutputConfig
effort: optional 'low' | 'medium' | 'high' | 'xhigh' | 'max'
Sending the top-level field produced:
400 invalid_request_error: effort: Extra inputs are not permitted
Restore the body to:
body['thinking'] = {'type': 'adaptive'}
body['output_config'] = {'effort': effort}
This was the shape PR 5412 originally shipped (and the author
validated against live APIs). My 'compliance fix' was a regression.
The companion thinking_delta SSE translation added in 0a664df4 stays
— that part WAS missing from the previous shape and is unchanged
by this revert. Test pinning the body shape flipped to assert
output_config.effort, top-level effort is asserted absent.
* studio/backend: opt in to summarized thinking display on adaptive
Per the adaptive-thinking docs, the 'display' field on the thinking
config defaults to 'omitted' on Claude Opus 4.7 (and Mythos Preview).
With 'omitted' the API still emits a thinking content block, but its
'thinking' field is empty — only the signature_delta arrives.
Our SSE handler would then surface a stray '<think></think>' for the
empty block and the reasoning panel would stay blank for the entire
response. Set 'display': 'summarized' explicitly on the adaptive
thinking config so Opus 4.7 emits thinking_delta events the same way
Opus 4.6 / Sonnet 4.6 do (where 'summarized' is the default, making
the explicit setting a no-op there).
The manual-thinking branch (Claude 4.5) is unaffected — its default
is also 'summarized', and we have no reason to override it.
* studio/backend: log Anthropic SSE event counts for thinking diagnostics
Reports of 'no reasoning panel content on Anthropic' have two
distinct causes that produce the same symptom:
1. Anthropic streamed thinking_delta events but our frontend
dropped them somewhere on the rendering side.
2. Anthropic did not emit thinking_delta at all (adaptive mode
can skip thinking for simple prompts even with effort=high,
and display=summarized only re-enables the *content* — it
does not force thinking to happen).
Tally each event type for the duration of one stream and log the
counts in the finally branch, so the next 'no reasoning content'
report shows immediately whether thinking_delta was even on the
wire. Zero counts → upstream (model/effort/prompt choice).
Non-zero counts → triage moves to chat-adapter / parse-assistant
-content / the reasoning component.
* studio/backend: route external_provider logs through structlog
The studio backend wires structlog as the active logger (via
LogConfig.setup_logging at main.py:262), but external_provider.py
was using stdlib logging.getLogger(__name__) for every diagnostic.
The stdlib root logger defaults to WARNING with no handlers
attached, so plain logger.info('...') and logger.debug('...') from
this module were being silently dropped — including the
'Proxying chat completion to <url>' and the new
'Anthropic stream event counts' lines. Only WARNING/ERROR survived
(via the implicit fallthrough that the user actually observed
when an Anthropic call 400'd).
Switch the module-level logger to structlog.get_logger(__name__),
matching the routes/providers.py and routes/inference.py pattern.
All existing call sites use printf-style positional args, which
structlog accepts unchanged — no other edits needed.
* studio/backend: disable read timeout on SSE streams to external providers
Anthropic Opus 4.7 (adaptive thinking) and OpenAI gpt-5.x (/v1/responses)
can pause for tens of seconds between bytes while the model is
internally reasoning. httpx's read timeout is the *gap* between
successive reads, not a wall clock on the whole request — so the
shared 120s default was cutting streams mid-response:
log: Anthropic stream event counts (... text_delta: 11)
Read timeout from anthropic
(eleven text deltas in, no content_block_stop, no message_stop)
Add a separate _stream_timeout on ExternalProviderClient with
read = None (no gap timeout) and the same 10s / 120s connect/write/
pool bounds, then use it at the three SSE streaming call sites:
default OpenAI-compat chat completions, _stream_anthropic, and
_stream_openai_responses. Non-streaming call sites (chat_completion,
list_models, verify_models_endpoint_lightweight) keep self._timeout
because a stuck non-streaming response should still fail fast.
* studio/backend: log outbound Anthropic request shape for thinking debug
After bumping to Xhigh effort the user still saw zero thinking_delta
events and only one content_block_start, meaning Anthropic Opus 4.7
opened no thinking block at all. Per the effort docs that should be
impossible — Xhigh always thinks. Two open hypotheses:
1. Our adaptive branch is not wiring output_config.effort onto the
outbound body for this code path (regex miss, frontend never
propagated reasoning_effort, etc).
2. Anthropic is silently accepting output_config as an unknown
field and falling back to high default effort regardless.
Add a single-line structlog INFO right before the stream POST that
echoes the keys actually present on the body (thinking, output_config,
temperature, presence of top_p / top_k, max_tokens). Messages are
deliberately excluded to keep PII out of the log. With this in place
the next 'no thinking on 4.7 at Xhigh' report shows immediately
whether we sent the effort knob — separating client bug from
provider behaviour.
* studio/chat: surface delta.reasoning_content from Kimi / DeepSeek thinking
Kimi (kimi-k2.6, kimi-k2-thinking) and DeepSeek's reasoner stream
their thinking content via a separate top-level field on the
chat-completion delta — choices[0].delta.reasoning_content — rather
than as a structured part inside delta.content. Per Kimi docs:
In streaming output (stream=True), the reasoning_content field
will always appear before the content field.
Our chat-adapter SSE loop only read delta.content (via
extractDeltaText), so the entire reasoning channel from these
providers was being silently dropped — kimi-k2.6 thinks by default
yet the chat UI showed no reasoning panel.
In the adapter:
- Read both delta.content and delta.reasoning_content per chunk
- When reasoning_content arrives, open a <think> block in
cumulativeText (mirrors how the backend wraps Anthropic
thinking_delta and OpenAI Responses reasoning summaries)
- When content arrives after reasoning, close </think> first
- On stream end, force-close any still-open <think> so
parseAssistantContent can lift it into a reasoning part cleanly
Anthropic and OpenAI Responses paths are unaffected — they already
wrap as <think> on the backend and never set reasoning_content.
* studio: Kimi thinking toggle + 16k max_tokens floor
Two coordinated changes so Kimi's thinking is user-controllable and
the response budget meets the docs' floor.
Toggle (frontend + backend):
- getExternalReasoningCapabilities now handles provider=='kimi':
kimi-k2.6 -> reasoning_style=enable_thinking, reasoningOff allowed
kimi-k2-thinking -> always on (reasoningAlwaysOn=true, no off)
kimi-k2.5 (and anything else) -> no reasoning controls
- chat-adapter already forwards enable_thinking on the
enable_thinking-style branch, so the user toggle reaches the
backend without additional wiring there.
- external_provider stream_chat_completion now translates the
boolean into Kimi's wire shape on the default OAI-compat path:
enable_thinking=True -> body['thinking'] = {type: enabled, keep: all}
enable_thinking=False -> body['thinking'] = {type: disabled}
kimi-k2-thinking ignores the toggle so the API never gets a
disabled value it would reject. Other providers on the same
path are unaffected (gated on provider_type == 'kimi').
Max tokens floor:
- New EXTERNAL_MIN_OUTPUT_TOKENS_BY_PROVIDER table and
getExternalMinOutputTokens helper. Kimi entry = 16000 per docs:
'Set max_tokens >= 16,000 to ensure the full reasoning_content
and final content can be returned without truncation.'
- chat-adapter clamps the outbound max_tokens to
min(max(stored, providerMin), EXTERNAL_MAX_OUTPUT_TOKENS),
so a stored value of 4096 still becomes 16000 when sending to
Kimi (other providers unaffected, min stays effectively 64).
- chat-settings-sheet's Max Tokens slider min mirrors the same
floor when an external Kimi model is selected, so the slider
cannot show a value lower than what we'd actually send.
- chat-page threads activeExternalProviderType down to the panel.
* fix: stabilize external reasoning controls for Anthropic 4.6 and OpenAI o3
normalize Anthropic 4.6 reasoning effort handling by accepting max as an alias and mapping it to xhigh, while keeping Sonnet/Opus 4.6 in default model suggestions.
broaden reasoning effort typing across backend/frontend and migrate persisted max selections to xhigh for compatibility.
remove reasoning.summary=\"auto\" from OpenAI /v1/responses payloads to avoid o3 eligibility/gating errors.
tighten provider model filtering to hide retired gpt-5.3 IDs and add exact/prefix filtering support in provider routes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: add openrouter/free + full reasoning passthrough on OpenRouter
Four-layer wire-up so the OpenRouter free-router model (which picks
a free model at random per request, filtered by needed capabilities)
shows up in the picker and its reasoning channel surfaces in the
chat UI.
Registry:
- providers.py: openrouter/free seeded at the top of openrouter
default_models. Curated list, so picker shows it immediately.
Frontend capability map:
- provider-capabilities.ts: getExternalReasoningCapabilities now
treats openrouter as enable_thinking style with off support. The
Think dropdown appears for every OpenRouter model; the gateway
silently no-ops the parameter for models that do not reason, so
surfacing one toggle on every model is safe.
Backend reasoning passthrough:
- external_provider.py stream_chat_completion (default OAI-compat
branch): for provider_type=='openrouter', translate the request:
reasoning_effort in {low,medium,high} -> body['reasoning'] =
{'effort': <level>}
enable_thinking=True -> body['reasoning'] = {'enabled': True}
enable_thinking=False -> body['reasoning'] = {'enabled': False}
Matches the documented shape at
https://openrouter.ai/docs/guides/best-practices/reasoning-tokens
with effort and max_tokens mutually exclusive.
Frontend SSE reader:
- chat-adapter.ts: OpenRouter streams reasoning as a third shape we
did not handle yet: delta.reasoning_details is an array of parts
like {type: 'reasoning.text', text: '...'}. Pull text from every
part, merge with the existing delta.reasoning_content channel
used by Kimi/DeepSeek, and feed the combined string through the
same <think>...</think> wrap path so parseAssistantContent lifts
it into the reasoning panel. Anthropic/OpenAI Responses paths
already wrap on the backend, so they never set this field — no
cross-provider interference.
* studio/backend: surface OpenRouter SSE errors and router-chosen model in logs
The frontend showed 'Provider returned error' for some openrouter/free
requests with nothing on the backend side to triage from — the
existing 4xx error log only fires when the upstream returns a non-200
status code, but OpenRouter (and most OAI-compat providers) return
200 OK and emit the actual failure as an SSE error event mid-stream,
which our default-path stream loop forwarded verbatim without
logging.
Best-effort diagnostics on the default OpenAI-compat stream path:
- Peek at every `data:` line in the inner forward loop, parse JSON
best-effort (silently skip on failure so nothing is dropped).
- Count event types: delta / error / done.
- On any chunk containing an `error` field, emit a structlog WARNING
with the provider type and the error payload — same trail the
user would otherwise have to dig out of browser devtools.
- Latch the first non-empty `chunk.model` field. OpenRouter reports
the router-picked underlying model there per request, so the
finally-block summary log shows which free model handled the call.
In the finally block:
'openrouter stream complete (model=openrouter/free,
chosen=google/gemini-2.5-flash, events={delta: 47, done: 1})'
Zero overhead for non-error streams (a json.loads per chunk +
dict-key lookups). The structlog logger is already configured at
INFO; ERROR and WARNING surface in JSON logs without further setup.
Hoists `import json as _json` to module top so the default path can
reuse it; the existing in-function imports in _stream_anthropic and
_stream_openai_responses are now redundant but harmless.
* studio/chat: show router-picked model after 'openrouter/free:' in chip
When the user picks openrouter/free, the gateway routes each request
to a different underlying free model. Until now there was no way to
tell which one actually replied without reading the backend logs.
Surface the picked model in the active-model chip:
- chat-runtime-store gains lastOpenRouterChosenModel: string|null
plus a setter. Reset on every model switch unless the user stays
on openrouter/free.
- chat-adapter SSE loop latches chunk.model into the store on
every chunk whose top-level model differs from
openrouter/free, gated on the active checkpoint being
openrouter/free under an OpenRouter provider.
- chat-page externalModels useMemo appends :<chosen> to the display
name for the openrouter/free option when the store has a value,
so ModelSelector renders e.g.
'openrouter/free:google/gemini-2.5-flash'
in the chip. Other models unaffected.
- Model-switch callback in chat-page clears the cached value when
the user moves to any model other than openrouter/free, so the
chip never shows a stale suffix from a previous session.
* studio/chat: shorten openrouter/free chip to openrouter:<short-chosen>
The full display name in use was:
openrouter/free:inclusionai/ring-2.6-1t-20260508:free
The `:free` suffix on the underlying id already conveys 'free model',
which made the leading `/free` on the router id redundant, and the
`inclusionai/` org prefix was just noise crowding the chip.
Trim both. Now the chip renders as:
openrouter:ring-2.6-1t-20260508:free
Strictly a display change in chat-page externalModels useMemo — the
backend wire id stays `openrouter/free`, the runtime store still
caches the full `inclusionai/...:free` value, and the model-switch
clearing logic is unchanged.
* studio/providers: switch OpenRouter to remote listing with org allowlist + cap
Same shape as Hugging Face Inference. The curated list had only four
entries; remote listing fetches OpenRouter's full ~300-model
catalog via /v1/models and the new allowlist + limit scope it back
down to a usable picker.
- model_list_mode: remote (was curated)
- model_id_allowlist matches the prefixes:
openrouter | openai | anthropic | google | meta-llama | qwen
| mistralai | deepseek | moonshotai | inclusionai | zai-org
| z-ai
Anything outside drops out.
- model_id_limit: 20 — first 20 post-filter matches from the live
fetch; default_models stays seeded so the most useful canonical
ids are always visible regardless of API response order.
- default_models seed extended from 4 to 6 (openrouter/free,
openai/gpt-4o, anthropic/claude-sonnet-4-5, google/gemini-2.5-flash,
mistralai/mistral-large-2411, deepseek/deepseek-r1).
openrouter/free remains the first entry, so the dialog's
loadModels() union-merge (registryDefaults first, then remote,
deduped via Set) keeps it at the top of the picker.
* feat: external mistral thinking toggle
* studio/chat: fix TS2540 by replacing readonly ContentPart instead of mutating
The ContentPart type from @assistant-ui/react marks `text` as readonly,
so the coalesce-adjacent-same-type-part optimization in
parseAssistantContent failed the tsc build with:
parse-assistant-content.ts(15,10): error TS2540: Cannot assign to
'text' because it is a read-only property.
parse-assistant-content.ts(25,10): error TS2540: ...
This broke npm run build, the Studio installer's `building frontend...`
step, and every downstream CI job that runs against an installed
Studio (Mac/Windows/Linux variants of Studio API CI, GGUF CI, UI CI,
Tauri CI, Wheel CI).
Replace the last element with a fresh merged object instead of
mutating its `text` field. Same allocation profile as the previous
path (one object swap per merge), type-safe under the readonly
declaration. Behaviour unchanged.
* studio/backend: restore summary='auto' on OpenAI Responses reasoning body
A recent refactor dropped the `summary: 'auto'` field from the
reasoning config we send to /v1/responses. Without it OpenAI does
not emit reasoning summary events on most reasoning models, which
means our SSE handler has no <think>…</think> to wrap and the chat
reasoning panel stays blank for any gpt-5.x / o3 response.
The expected wire shape is:
body['reasoning'] = {'effort': '<level>', 'summary': 'auto'}
Two backend tests pin this:
- test_responses_reasoning_effort_included_when_requested (high)
- test_responses_reasoning_effort_xhigh_passthrough (xhigh)
Both were failing with AssertionError because the produced body
omitted `summary: auto`.
Restore the field. Skip it only for the explicit "off" case
(effort: 'none'), where summaries serve no purpose. The
enable_thinking=True fallback (no explicit effort) also pairs
medium effort with summary='auto' so that branch produces
reasoning text too.
* chat: external reasoning, OpenRouter curation, Think toggle fixes
* fix: opus and sonnet 4.6 xhigh --> max
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Roland Tannous