ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

studio: API external provider support for chat (OpenAI, Mistral, Gemini, Cohere, Anthropic, OpenRouter, DeepSeek, custom providers) (#4706)

Browse source 
* studio: add external provider support for chat inference

Adds the ability to connect to OpenAI, Mistral, Google, Cohere, Together,
Fireworks, and Perplexity from the Studio chat interface.

- Provider configs stored in SQLite (no API keys persisted)
- RSA-2048 key pair generated at startup for client-side key encryption
- httpx proxy client streams SSE responses in OpenAI-compatible format
- New /api/providers routes: registry, CRUD, test, models
- /v1/chat/completions routes to external provider when provider fields present
- Integration test suite covering CRUD, connection, model listing, and inference
- Frontend spec doc with full API contract

* remove frontend spec doc from branch

* fix auth fixture: handle forced password change on fresh install

* fix tests: default port 8000, allow 400 for no-model-loaded

* fix: update Cohere models to current (command-r retired Sept 2025)

* feat: add OpenRouter as 8th provider

* feat: add native Anthropic provider with Messages API translation

* fix: correct Anthropic base URL and drop top_p (conflicts with temperature)

* feat: add DeepSeek provider (deepseek-chat, deepseek-reasoner)

* feat: rename google -> gemini, refresh model list to 2.5 series

* feat: remove together, fireworks, perplexity providers

* feat: multimodal image support for external providers

- Add _build_external_messages() that preserves image_url parts for
  vision-capable providers instead of stripping them
- Update _proxy_to_external_provider() to use new helper
- Translate image_url content parts to Anthropic native image format
  in _stream_anthropic()
- Add TestVisionInference pytest class (1x1 PNG smoke test)

* test: use sloth photo URL for vision test, add Anthropic remote URL support

* fix: update Mistral model to mistral-small-2506

* update mistral default model to mistral-large-2512

* fix gemini vision test: download image as base64 data URI instead of remote URL

* add gemini-3-flash-preview as default gemini model

* fix gemini truncated reply (max_tokens 16->64) and suppress GeneratorExit on client disconnect

* increase vision test max_tokens to 215

* fix GeneratorExit: aclose stream generator before closing httpx client

* fix httpcore GeneratorExit: explicitly aclose aiter_lines before response closes

* fix duplicate [DONE] and suppress httpcore RuntimeError on Python 3.13 asyncgen cleanup

* fix: call response.aclose() before lines_gen.aclose() to prevent httpcore RuntimeError on Python 3.13

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Potential fix for code scanning alert no. 36: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* review: add comments for manual iteration rationale, mask password in test print, clarify Anthropic URL/models support

* perf: use shared module-level httpx client for connection pooling across requests

* studio: add API provider UI and integrate wiring (#4737)

* feat: expose external models in selector and chat settings

* feat(chat): wire external providers to backend + RSA key flow

- Fetch registry/configs; create/update/delete saved providers
- Encrypt API keys (Web Crypto RSA-OAEP) for test/models/chat
- External model selection + chat payload (provider_id/type, external_model, encrypted key, optional base URL)
- Local storage for keys + provider list; small UX/copy and guardrails

* add missing providers-api.ts file by Imagineer99

* fix: address PR review comments — system prompt visibility, retry loop, test logging

* feat(studio): encrypt external provider API keys at rest in localStorage

API keys for external providers (OpenAI, Mistral, etc.) were stored as
plaintext in localStorage, vulnerable to browser extensions and XSS.

Add password-derived AES-256-GCM encryption: on login the user's password
is used via PBKDF2 (100k iterations, SHA-256) to derive an in-memory
encryption key. API keys are encrypted before writing to localStorage and
decrypted on read. The derived key is never persisted — cleared on logout,
re-derived on next login.

Legacy plaintext keys are transparently migrated on first access. Password
changes re-encrypt all stored keys. No backend changes required — the
existing RSA-OAEP transit encryption is unaffected.

* fix: cast PBKDF2 salt to BufferSource for strict TypeScript lib types

* fix: persist session password in sessionStorage to survive page refreshes

* feat(studio): preserve image parts in external provider chat requests

toOpenAIMessage() now returns multimodal content arrays (OpenAI vision
format) when messages contain images, instead of always flattening to
plain text. This enables vision-capable external providers (OpenAI,
Gemini, Anthropic, etc.) to receive user images. The backend already
handles image_url content parts in _build_external_messages().

* studio: fix external models selectable in chat-only mode (#4779)

* fix: external models selectable in chat-only mode

* fix: model selector tabs default to active model kind

* Studio: API external provider registry + curated catalogs (HF/OpenRouter) and chat UX (#4787)

* fix: external models selectable in chat-only mode

* fix: model selector tabs default to active model kind

* feat(studio): expand provider registry, curated catalogs, and chat UX

- Add Hugging Face, Kimi, Qwen; remove Cohere; reorder registry
- model_list_mode curated for HF/OpenRouter; lightweight /models check
- API returns default models for curated providers; expose model_list_mode
- Frontend: provider logos in model picker, providerType on external models
- Chat providers dialog: curated vs remote flows, motion polish
- Thread: LayoutGroup + composer motion alignment with app easing

* fix(studio): disable Anthropic tool-calling flag and preselect curated defaults

* feat(studio): add external provider logos and ApiProviderLogo helper

* Studio: Polish API Providers dialog  (#4899)

* fix: lower verbage in API providers page

* fix: fix(studio): tune API Providers dialog width with rem-based responsive caps

* feat: add custom provider support (#4902)

* fix: replace crypto.subtle with node-forge for HTTP compatibility

crypto.subtle is only available in secure contexts (HTTPS/localhost),
which breaks provider API key encryption when Studio is accessed over
plain HTTP on remote GPU VMs. Switch to node-forge for RSA-OAEP and
AES-256-GCM operations — same algorithms, works on any origin.

* fix: store provider API keys as plaintext in localStorage

Drop AES-256-GCM at-rest encryption for provider API keys. The
session-password-derived encryption broke on auto-login via refresh
token (password never captured), causing keys to silently vanish.
API keys are still RSA-encrypted in transit via node-forge. At-rest
encryption in localStorage added no real security since the
decryption key also had to live client-side.

Removes crypto-storage.ts, session password plumbing, and
reEncryptAllKeys.

* fix: use max_completion_tokens for OpenAI provider

Newer OpenAI models (gpt-4o, gpt-5.x) reject the max_tokens param
and require max_completion_tokens instead. Other providers still use
max_tokens.

* fix: skip empty assistant messages in external provider requests

Some providers (Mistral) reject assistant messages with empty content.
Filter them out when building the message list for external providers.

* Update model-selector.tsx

* Update model-selector.tsx

* Update model-selector.tsx

* Update chat-adapter.ts

* Update chat-adapter.ts

* Update chat-page.tsx

* Update chat-settings-sheet.tsx

* Update chat-settings-sheet.tsx

* Update chat-settings-sheet.tsx

* Update chat-providers-dialog.tsx

* feat: polish providers settings form UI

* style: polish provider row icon sizing and alignment

* style: stabilize provider layout

* style: add provider API key visibility toggle

* fix: add provider render on empty list

* studio/frontend: sync package-lock.json with package.json

npm ci was failing because node-forge and @types/node-forge were
declared in package.json but missing from the lockfile. Ran
npm install to regenerate.

* studio/backend: fix backend CI failures for providers router

- test_desktop_auth: include providers_router in the routes stub so
  studio.backend.main imports cleanly under the monkeypatched module
- test_providers_api: skip the whole module when STUDIO_TEST_PASSWORD
  is unset (it is an integration test against a live Studio server,
  same shape as the already-ignored test_studio_api.py)

* studio/chat: drive ChatSettingsPanel from a per-provider capability map

Replace the binary isExternalModel toggle in the sampling section with a
provider-aware capability map. Each external provider type advertises
which of top_k / min_p / repetition_penalty / presence_penalty its
chat-completions API actually accepts, so the panel only renders the
knobs that map onto the active provider's request body.

Anthropic now exposes top_k; DeepSeek hides presence_penalty (deprecated
in their docs); OpenRouter and custom providers continue to show every
knob (OpenRouter drops unsupported server-side, custom assumes
OpenAI-compat or a permissive vLLM/Ollama backend). Local models are
unaffected — null capabilities means 'show everything'.

chat-adapter.ts now forwards top_k / presence_penalty to the external
proxy only when the active provider's capabilities permit it, so the
request body matches what the UI shows.

* studio/backend: forward top_k to Anthropic; filter OpenAI model list

Two paired changes so the frontend capability map has matching backend
behaviour:

1. ExternalProviderClient.stream_chat_completion now accepts top_k and
   forwards it to the Anthropic Messages body. OpenAI-compat providers
   (which all reject unknown sampling params) still receive only the
   fields they document. The proxy route in routes/inference.py passes
   payload.top_k through, so a UI request with top_k actually reaches
   Anthropic instead of being silently dropped at the boundary.

2. PROVIDER_REGISTRY['openai'] gains a model_id_allowlist regex that
   scopes the /models picker to current-gen ids (gpt-5.5 / gpt-5.4 /
   gpt-5.3 / gpt-4.5 / o3 families). The remote /v1/models listing
   otherwise returns dozens of historical snapshots, fine-tunes and
   non-chat models (embeddings, TTS, image, moderation) that we never
   want in the chat UI. default_models is refreshed to match.

* studio/chat: relax presence_penalty to optional on OpenAIChatCompletionsRequest

Followup to 1fbf445a — chat-adapter now omits presence_penalty for
providers that do not accept it (Anthropic / DeepSeek), but the
request type still required it as a non-optional number, breaking
tsc. The backend pydantic model already defaults presence_penalty
to 0, so making it optional client-side matches reality.

* studio/backend: route OpenAI traffic through /v1/responses

OpenAI's new flagship models (gpt-5.x) return 404 'This is not a chat
model' on /v1/chat/completions and are only reachable via /v1/responses.
Add a dedicated _stream_openai_responses path in ExternalProviderClient
that:

- Translates outbound messages into the Responses shape: system messages
  are folded into the top-level 'instructions' field, user/assistant
  messages become {role, content} items with input_text / input_image
  content parts (data URLs and https URLs both pass through).
- Drops presence_penalty / top_k / frequency_penalty, none of which the
  Responses contract accepts.
- Translates inbound SSE events back into OpenAI Chat Completions
  chunks so the frontend keeps a single SSE shape:
    response.output_text.delta  -> delta chunk with content
    response.completed          -> chunk with finish_reason='stop'
    response.incomplete         -> chunk with finish_reason='length'
    response.failed / error     -> propagated error SSE line
  Stream terminates with data: [DONE] (Responses emits this verbatim).

stream_chat_completion dispatches all provider_type='openai' calls to
this path; other OpenAI-compatible providers (mistral, gemini, etc.)
continue to use /v1/chat/completions.

Frontend provider-capabilities map updated to hide presence_penalty for
OpenAI in the chat settings panel, matching the new request contract.

Includes unit coverage in tests/test_openai_responses_translation.py
exercising the request body translation, image-part rewriting, and
SSE-to-chat-completions translation via httpx.MockTransport.

* studio/chat: clamp external max_tokens to 32k to stay within provider caps

The chat settings slider already capped maxTokens at 32768 for external
models, but a value persisted from a prior local-model session (where
the cap can be 128k+) was sent verbatim to the provider — Claude Opus
returns 'max_tokens: 131072 > 128000' on requests like that, and other
providers have stricter limits still.

Expose EXTERNAL_MAX_OUTPUT_TOKENS from provider-capabilities (32k) and
use it both for the slider max and as the clamp inside chat-adapter's
external-request body. 32k sits below the tightest declared output
limit across the providers we ship and well above what a typical chat
reply needs; the local-model path is unaffected.

* studio: drop temperature/top_p for OpenAI reasoning models

gpt-5.x / o3 / gpt-4.5 are reasoning-class models served via
/v1/responses, and reject temperature and top_p with
'Unsupported parameter' 400s. The OpenAI registry allowlist already
scopes the picker to those families, so neither knob ever applies on
this branch.

- external_provider._stream_openai_responses no longer puts
  temperature or top_p in the request body (kept on the method
  signature for API symmetry with the other stream methods).
- ProviderCapabilities gains temperature/topP flags; OpenAI sets both
  to false. ChatSettingsPanel hides the sliders for OpenAI so the user
  does not see inert controls.
- chat-adapter omits temperature/top_p from the external request body
  when the active provider does not advertise them.
- OpenAIChatCompletionsRequest type marks both as optional, matching
  the new chat-adapter shape.
- test_responses_request_body_uses_input_and_instructions: assertions
  flipped to confirm temperature / top_p are absent from the body.

* studio: stop forwarding top_k to Anthropic

Claude 4.x (Opus / Sonnet / Haiku 4.x) returns 400 'top_k is
deprecated for this model' on any request that includes top_k. It
was always optional on the older 3.x line, so dropping it
unconditionally for every Anthropic call is the simplest path —
no per-model gate to maintain.

- external_provider._stream_anthropic no longer adds top_k to the
  Messages body (kept on the method signature for API symmetry).
- provider-capabilities sets anthropic.topK = false so the chat
  settings panel hides the Top K slider for Anthropic providers
  and chat-adapter does not send top_k in the external request.

* studio: gate Anthropic top_k drop to Claude 4.7 only

Previous commit (b5aa6ffd) dropped top_k for every Anthropic call,
but only Claude 4.7 (Opus/Sonnet/Haiku) actually rejects it. 4.6, 4.5,
and the 3.x line still accept top_k and use it as documented.

Backend: _stream_anthropic matches the model id against
^claude-(opus|sonnet|haiku)-4-7(-|.|$) and only strips top_k when it
hits. Every other Claude generation continues to receive the value
from the chat settings panel.

Frontend: anthropic.topK is restored to true so the Top K slider is
visible again — the backend handles the per-model drop, and the
4.7 case is silent (request still succeeds without top_k).

* chore: hide dated openai models in provider select

* studio/providers: apply model_id_denylist when listing remote models

The OpenAI registry entry gained a model_id_denylist regex matching
dated snapshot ids (-YYYY-MM-DD) in 048d73bf, but the list-models
route was never consulting it, so the snapshots still showed up
alongside their canonical ids (gpt-5.5 and gpt-5.5-2026-04-23 both
listed). Apply the denylist with .search() right after the allowlist
filter so dated entries are dropped before the response is built.

* studio/chat: seed registry default_models for remote providers in picker

The Anthropic provider runs in remote model-list mode, so the picker
started with an empty availableModels until the user clicked
'Load Models'. If that /api/providers/models call fails (e.g. the
known transient decryption error during key rotation), the user sees
no models at all — claude-haiku-4-5 in particular was missing from
the dialog even though it is seeded in the registry.

Always pre-populate availableModels with the registry's default_models
when a provider type is selected (curated and remote alike), and have
loadModels() return the union of defaults + the live /models response
so registry-seeded ids are reachable regardless of what the provider's
endpoint returns or whether the call succeeds at all.

* studio/backend: diagnostic logging on provider key decryption

Decryption failures currently log just 'Failed to decrypt API key:
Decryption failed', which leaves no way to tell whether the cause is
a stale public key in the browser, a corrupted ciphertext, an
unexpected exception class, or a server-side keypair rotation. That's
the gap the next reproduction needs to close.

- key_exchange now publishes a short SHA256 fingerprint of the public
  key PEM. init_key_pair logs the fingerprint on generation and warns
  if it is ever called a second time (re-init silently invalidates
  every browser that cached the previous public key).
- decrypt_api_key wraps both the base64 decode and the RSA decrypt
  in dedicated try/excepts that log exception type, ciphertext byte
  length (RSA-2048 should be exactly 256), input string length, and
  the current public-key fingerprint.
- GET /api/providers/public-key returns the fingerprint alongside the
  PEM so the frontend can correlate a future encrypt-time fingerprint
  against the decrypt-time fingerprint and prove or rule out a
  keypair rotation as the cause.
- The /test and /models route-level decrypt warnings now include the
  exception class name (alongside the existing message).

* studio/providers: hide dated Anthropic snapshots from the model picker

Anthropic's /v1/models returns dated snapshot ids (e.g.
claude-3-5-sonnet-20241022, claude-3-5-haiku-20241022) alongside
the canonical names users actually want to pick. Same intent as
the OpenAI denylist added in 048d73bf, just a different date
format — Anthropic uses -YYYYMMDD (no dashes) while OpenAI uses
-YYYY-MM-DD.

- Add model_id_denylist = re.compile(r'-\d{8}$') to the anthropic
  registry entry. The /api/providers/models route already applies
  any denylist after fetching, so dated ids drop out automatically.
- Strip the dated 3.5 ids from default_models so the seeded picker
  no longer surfaces them; keep claude-opus-4-7 and the 4.5 family
  as the curated set.

Net effect: the picker shows opus-4-7 / opus-4-5 / sonnet-4-5 /
haiku-4-5 only, regardless of whether the remote /models call
succeeds or fails.

* fix: provider dialog and mistral short list

* style: fix provider dialog curated list styling

* fix: provider dialog curated model ids placeholder reference

* style: rename Providers to Cloud and tighten dialog header spacing

* UX: rename Providers to Cloud, remove header shortcut

* studio/chat: normalize structured delta.content from reasoning providers

Mistral's magistral (and similarly-shaped reasoning models) stream
chat-completion deltas where choices[0].delta.content is an array of
structured parts rather than a plain string, e.g.
  [{ type: 'text', text: '...' }, { type: 'thinking', thinking: '...' }]
The accumulator did 'cumulativeText += delta', which coerced each
part to '[object Object]' and produced output like
  '[object Object][object Object]...Hey there!'.

Add extractDeltaText() to normalize delta.content before append:
- string → returned as-is
- array of parts → text/output_text parts contribute their .text or
  .content; thinking/reasoning parts are re-wrapped inline as
  <think>...</think> so the downstream parseAssistantContent lifts
  them into a reasoning part the same way it does for providers that
  emit thinking inline. magistral keeps its thinking panel; no other
  provider's output shape changes.
- unknown shapes → dropped rather than stringified, so a stray field
  cannot pollute the rendered chat with '[object Object]'.

* Studio: restore Cloud icon shortcut in chat header

Brings back the header chip that opens Settings -> Cloud (external
providers) directly from the chat view. Same button as before the
bf24e604 removal: single-mode only, opens useSettingsDialogStore on
the 'connections' tab, tooltip 'API providers'.

* studio/chat: strip trailing template literal from external provider streams

Mistral's magistral occasionally appends a literal '${response}' token
after its actual answer — likely a training-format artifact, since it
keeps happening with an empty system prompt and only on that model.

Apply a tight strip in the chat-adapter SSE accumulator: when the
active provider is external, drop a trailing '${...}' template literal
(with optional whitespace) from cumulativeText after each chunk. The
regex anchors to end-of-string, so mid-stream fragments ('${re')
remain untouched and only collapse once the closing brace arrives.
Local-model output is unaffected.

* studio/providers: scope Kimi picker to kimi-k2.6 / kimi-k2.5

Mirror what the live Kimi docs surface as the current models
(https://platform.kimi.ai/docs/models). Everything else the
remote /v1/models call returns — moonshot-v1-* legacy ids and
dated k2 previews like kimi-k2-0711-preview — is filtered out.

- default_models: ['kimi-k2.6', 'kimi-k2.5'] (was four
  legacy moonshot-v1 ids plus the dated k2 preview)
- model_id_allowlist: ^kimi-k2\.[56]$ applied in the
  /api/providers/models route after the live fetch
- doc-link comments point at platform.kimi.ai overview /
  models / list-models for the next refresh

* studio: drop temperature/top_p for Kimi reasoning models

Kimi k2.5/k2.6 are reasoning-class. The API locks temperature and
top_p to fixed defaults and 400s on any other value with
'invalid temperature: only 1 is allowed for this model'.

The frontend capability map already gated these knobs out of the
external request body, but the OpenAI-compat path on the backend
unconditionally re-adds them from the pydantic ChatCompletionRequest
defaults (temperature=0.7 etc), so the gate was bypassed end-to-end.

Add a generic body_omit hook on the provider registry that
stream_chat_completion consults after building the body, and use it
to strip temperature/top_p for Kimi. Frontend provider-capabilities
flips kimi.temperature and kimi.topP to false so the sliders are
hidden in the chat settings panel as well.

* studio/providers: scope Gemini picker to current 3.x + *-latest aliases

Google's /v1beta/openai/models returns dozens of historical,
experimental, and non-chat ids that we never want in the chat UI.
Cap the picker to the current curated set:

- gemini-3.1-pro-preview
- gemini-3.1-flash-lite
- gemini-3-flash-preview
- gemini-pro-latest
- gemini-flash-latest
- gemini-flash-lite-latest

Default_models seeded with these, model_id_allowlist applied in
the /api/providers/models route to drop anything else the live
fetch returns.

* studio/providers: switch Hugging Face to remote model listing

Per the Inference Providers docs
(https://huggingface.co/docs/inference-providers/index),
GET https://router.huggingface.co/v1/models returns the full
chat-model catalog across all providers, including per-provider
metadata. The OpenAI-compatible endpoint we already use for
chat completions accepts the same Bearer token, so flipping
model_list_mode from 'curated' to 'remote' lets users discover
models via the existing list_models() path without any new
wiring.

- model_list_mode: 'remote' (was 'curated')
- default_models refreshed with current popular ids
  (gpt-oss-120b, DeepSeek-V3, Llama-3.3-70B, Qwen2.5-72B) so the
  picker still has a sensible seed if /v1/models fails
- notes updated to reference the docs page and clarify the
  endpoint is chat-only

* UX: chat cloud icon changed to model select signifier

* studio/providers: org allowlist + count cap for HF Inference picker

The HF /v1/models response is the full cross-provider catalog (hundreds
of ids — community fine-tunes, mirrors, fp8 variants, dated snapshots).
Scope the picker to the first-party org repos worth surfacing and cap
the post-filter list.

- model_id_allowlist matches the org prefixes openai/, deepseek-ai/,
  google/, meta-llama/, Qwen/, moonshotai/, mistralai/, zai-org/.
  Anything outside those orgs is dropped.
- model_id_limit (new registry field) caps the post-filter list. The
  list-models route now slices [:limit] after allowlist/denylist; set
  to 15 for HF Inference. Other providers leave it unset and behave
  exactly as before.
- default_models stays as the seed so the flagship ids users care
  about (gpt-oss-120b, DeepSeek-V3, Llama-3.3-70B, Qwen2.5-72B) are
  always reachable regardless of the API's response order.

Dedup is already handled in loadModels() via Set, so no additional
work needed there.

* style: adjust cloud icon right margin with rem spacing

* Studio: cloud openai reasoning level toggle (#5402)

* feat: cloud openai reasoning level toggle

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: honor enable_thinking=false

* fix: prevent local reasoning toggle regressions and align OpenAI effort levels

* fix: isolate external OpenAI reasoning toggle state

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>

* fix: clamp reasoning effort

* fix: align OpenAI reasoning effort

* fix: clear stale GGUF badge state

* ui: new badge on cloud setting

* fix: separate selected models from cached provider model list

* Studio: anthropic effort by model family (#5412)

* feat: external thinking control and Anthropic effort mapping

* fix: anthropic thinking constraints and 4.6 max effort mapping

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: harden Anthropic thinking params and effort mapping

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* studio/backend: drop top_p from Anthropic body when thinking is enabled

PR 5412 added body['top_p'] = max(0.95, min(top_p, 1.0)) inside the
thinking branch of _stream_anthropic, but Anthropic returns 400 on
extended/adaptive thinking when both temperature and top_p are set:

  invalid_request_error: temperature and top_p cannot both be
  specified for this model. Please use only one.

(Observed on Claude Opus 4.6.) The contract for thinking-enabled
requests is temperature=1 with neither top_p nor top_k allowed.

Replace the body['top_p'] = ... line with body.pop('top_p', None).
Defensive pop rather than a bare delete: the base body construction
above does not currently set top_p, but a future edit that adds it
would silently reintroduce the regression.

* studio/chat: force reasoningEnabled=true on local reasoning-effort models

Followup to PR 5402 / 5412. The model-status refresh path in
use-chat-model-runtime carried reasoningEnabled forward verbatim for
every reasoning-capable model. That left one observable edge case:

  1. user picks an external model that supports Off (gpt-5.x, Claude
     4.x), clicks Off — store sets reasoningEnabled=false
  2. user switches back to a local reasoning-effort model
     (gpt-oss / Harmony-style) which does NOT support Off
  3. composer's effectiveReasoningEnabled override paints the UI as
     'Think: <level>' (on)
  4. chat-adapter sees reasoningEnabled=false on the local branch
     and sends '{}', so the backend's _request_reasoning_kwargs
     returns None and the Harmony template falls back to its own
     default effort instead of the displayed level

Mirror the composer's override in the store on load: for local
reasoning-effort models (where supportsReasoningOff is false), force
reasoningEnabled=true so the store and the UI agree on every send.
Other reasoning styles still inherit prior state — only the
reasoning-effort family changes.

* studio/backend: align Anthropic thinking with the extended-thinking docs

Two compliance fixes against
https://platform.claude.com/docs/en/build-with-claude/extended-thinking

1. Adaptive-mode effort field shape
   The docs spell adaptive thinking as:
     {'thinking': {'type': 'adaptive'}, 'effort': {'type': '<level>'}}
   We had been sending the legacy 'output_config: {effort: <level>}'
   shape, which Anthropic appears to silently ignore — adaptive ran
   at the server default effort regardless of the user's selection.
   Rename to 'effort: {type: <level>}'.

2. thinking_delta event translation
   The Messages-API streams reasoning content as
   content_block_delta events with delta.type == 'thinking_delta',
   which our SSE loop was dropping entirely. On Claude 4.5/4.6 with
   display=summarized (the default), the user would see the answer
   text but never the reasoning panel. Wrap thinking_delta.thinking
   as inline <think>...</think> chunks (same pattern as the OpenAI
   Responses path) so the frontend's parseAssistantContent lifts it
   into the reasoning channel. The </think> closer fires on the
   first text_delta transition, on content_block_stop for the
   thinking block, on message_delta, and on message_stop —
   whichever arrives first — so no model path can leak an
   unclosed <think> into chat output.
   signature_delta events are left as no-ops; they carry
   verification metadata, not user-visible content.

Adds test_anthropic_thinking_translation.py with httpx.MockTransport
coverage of: effort shape on adaptive (Claude 4.6), budget_tokens
shape on manual (Claude 4.5), thinking_delta wrapping with signature
suppression, and thinking-only turns (display=omitted on Opus 4.7).

* studio/backend: revert Anthropic adaptive effort to output_config nesting

The previous commit (0a664df4) moved the adaptive-thinking effort
field to a top-level 'effort: {type: <level>}' based on a misread of
the docs page. The actual Messages API schema nests it under
output_config:

  thinking:       optional ThinkingConfigParam   ({type: 'adaptive'})
  output_config:  optional OutputConfig
    effort:       optional 'low' | 'medium' | 'high' | 'xhigh' | 'max'

Sending the top-level field produced:
  400 invalid_request_error: effort: Extra inputs are not permitted

Restore the body to:
  body['thinking'] = {'type': 'adaptive'}
  body['output_config'] = {'effort': effort}

This was the shape PR 5412 originally shipped (and the author
validated against live APIs). My 'compliance fix' was a regression.

The companion thinking_delta SSE translation added in 0a664df4 stays
— that part WAS missing from the previous shape and is unchanged
by this revert. Test pinning the body shape flipped to assert
output_config.effort, top-level effort is asserted absent.

* studio/backend: opt in to summarized thinking display on adaptive

Per the adaptive-thinking docs, the 'display' field on the thinking
config defaults to 'omitted' on Claude Opus 4.7 (and Mythos Preview).
With 'omitted' the API still emits a thinking content block, but its
'thinking' field is empty — only the signature_delta arrives.

Our SSE handler would then surface a stray '<think></think>' for the
empty block and the reasoning panel would stay blank for the entire
response. Set 'display': 'summarized' explicitly on the adaptive
thinking config so Opus 4.7 emits thinking_delta events the same way
Opus 4.6 / Sonnet 4.6 do (where 'summarized' is the default, making
the explicit setting a no-op there).

The manual-thinking branch (Claude 4.5) is unaffected — its default
is also 'summarized', and we have no reason to override it.

* studio/backend: log Anthropic SSE event counts for thinking diagnostics

Reports of 'no reasoning panel content on Anthropic' have two
distinct causes that produce the same symptom:

  1. Anthropic streamed thinking_delta events but our frontend
     dropped them somewhere on the rendering side.
  2. Anthropic did not emit thinking_delta at all (adaptive mode
     can skip thinking for simple prompts even with effort=high,
     and display=summarized only re-enables the *content* — it
     does not force thinking to happen).

Tally each event type for the duration of one stream and log the
counts in the finally branch, so the next 'no reasoning content'
report shows immediately whether thinking_delta was even on the
wire. Zero counts → upstream (model/effort/prompt choice).
Non-zero counts → triage moves to chat-adapter / parse-assistant
-content / the reasoning component.

* studio/backend: route external_provider logs through structlog

The studio backend wires structlog as the active logger (via
LogConfig.setup_logging at main.py:262), but external_provider.py
was using stdlib logging.getLogger(__name__) for every diagnostic.
The stdlib root logger defaults to WARNING with no handlers
attached, so plain logger.info('...') and logger.debug('...') from
this module were being silently dropped — including the
'Proxying chat completion to <url>' and the new
'Anthropic stream event counts' lines. Only WARNING/ERROR survived
(via the implicit fallthrough that the user actually observed
when an Anthropic call 400'd).

Switch the module-level logger to structlog.get_logger(__name__),
matching the routes/providers.py and routes/inference.py pattern.
All existing call sites use printf-style positional args, which
structlog accepts unchanged — no other edits needed.

* studio/backend: disable read timeout on SSE streams to external providers

Anthropic Opus 4.7 (adaptive thinking) and OpenAI gpt-5.x (/v1/responses)
can pause for tens of seconds between bytes while the model is
internally reasoning. httpx's read timeout is the *gap* between
successive reads, not a wall clock on the whole request — so the
shared 120s default was cutting streams mid-response:

  log: Anthropic stream event counts (... text_delta: 11)
       Read timeout from anthropic

(eleven text deltas in, no content_block_stop, no message_stop)

Add a separate _stream_timeout on ExternalProviderClient with
read = None (no gap timeout) and the same 10s / 120s connect/write/
pool bounds, then use it at the three SSE streaming call sites:
default OpenAI-compat chat completions, _stream_anthropic, and
_stream_openai_responses. Non-streaming call sites (chat_completion,
list_models, verify_models_endpoint_lightweight) keep self._timeout
because a stuck non-streaming response should still fail fast.

* studio/backend: log outbound Anthropic request shape for thinking debug

After bumping to Xhigh effort the user still saw zero thinking_delta
events and only one content_block_start, meaning Anthropic Opus 4.7
opened no thinking block at all. Per the effort docs that should be
impossible — Xhigh always thinks. Two open hypotheses:

  1. Our adaptive branch is not wiring output_config.effort onto the
     outbound body for this code path (regex miss, frontend never
     propagated reasoning_effort, etc).
  2. Anthropic is silently accepting output_config as an unknown
     field and falling back to high default effort regardless.

Add a single-line structlog INFO right before the stream POST that
echoes the keys actually present on the body (thinking, output_config,
temperature, presence of top_p / top_k, max_tokens). Messages are
deliberately excluded to keep PII out of the log. With this in place
the next 'no thinking on 4.7 at Xhigh' report shows immediately
whether we sent the effort knob — separating client bug from
provider behaviour.

* studio/chat: surface delta.reasoning_content from Kimi / DeepSeek thinking

Kimi (kimi-k2.6, kimi-k2-thinking) and DeepSeek's reasoner stream
their thinking content via a separate top-level field on the
chat-completion delta — choices[0].delta.reasoning_content — rather
than as a structured part inside delta.content. Per Kimi docs:

    In streaming output (stream=True), the reasoning_content field
    will always appear before the content field.

Our chat-adapter SSE loop only read delta.content (via
extractDeltaText), so the entire reasoning channel from these
providers was being silently dropped — kimi-k2.6 thinks by default
yet the chat UI showed no reasoning panel.

In the adapter:
- Read both delta.content and delta.reasoning_content per chunk
- When reasoning_content arrives, open a <think> block in
  cumulativeText (mirrors how the backend wraps Anthropic
  thinking_delta and OpenAI Responses reasoning summaries)
- When content arrives after reasoning, close </think> first
- On stream end, force-close any still-open <think> so
  parseAssistantContent can lift it into a reasoning part cleanly

Anthropic and OpenAI Responses paths are unaffected — they already
wrap as <think> on the backend and never set reasoning_content.

* studio: Kimi thinking toggle + 16k max_tokens floor

Two coordinated changes so Kimi's thinking is user-controllable and
the response budget meets the docs' floor.

Toggle (frontend + backend):
- getExternalReasoningCapabilities now handles provider=='kimi':
  kimi-k2.6 -> reasoning_style=enable_thinking, reasoningOff allowed
  kimi-k2-thinking -> always on (reasoningAlwaysOn=true, no off)
  kimi-k2.5 (and anything else) -> no reasoning controls
- chat-adapter already forwards enable_thinking on the
  enable_thinking-style branch, so the user toggle reaches the
  backend without additional wiring there.
- external_provider stream_chat_completion now translates the
  boolean into Kimi's wire shape on the default OAI-compat path:
    enable_thinking=True  -> body['thinking'] = {type: enabled, keep: all}
    enable_thinking=False -> body['thinking'] = {type: disabled}
  kimi-k2-thinking ignores the toggle so the API never gets a
  disabled value it would reject. Other providers on the same
  path are unaffected (gated on provider_type == 'kimi').

Max tokens floor:
- New EXTERNAL_MIN_OUTPUT_TOKENS_BY_PROVIDER table and
  getExternalMinOutputTokens helper. Kimi entry = 16000 per docs:
  'Set max_tokens >= 16,000 to ensure the full reasoning_content
  and final content can be returned without truncation.'
- chat-adapter clamps the outbound max_tokens to
  min(max(stored, providerMin), EXTERNAL_MAX_OUTPUT_TOKENS),
  so a stored value of 4096 still becomes 16000 when sending to
  Kimi (other providers unaffected, min stays effectively 64).
- chat-settings-sheet's Max Tokens slider min mirrors the same
  floor when an external Kimi model is selected, so the slider
  cannot show a value lower than what we'd actually send.
- chat-page threads activeExternalProviderType down to the panel.

* fix: stabilize external reasoning controls for Anthropic 4.6 and OpenAI o3

normalize Anthropic 4.6 reasoning effort handling by accepting max as an alias and mapping it to xhigh, while keeping Sonnet/Opus 4.6 in default model suggestions.
broaden reasoning effort typing across backend/frontend and migrate persisted max selections to xhigh for compatibility.
remove reasoning.summary=\"auto\" from OpenAI /v1/responses payloads to avoid o3 eligibility/gating errors.
tighten provider model filtering to hide retired gpt-5.3 IDs and add exact/prefix filtering support in provider routes.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: add openrouter/free + full reasoning passthrough on OpenRouter

Four-layer wire-up so the OpenRouter free-router model (which picks
a free model at random per request, filtered by needed capabilities)
shows up in the picker and its reasoning channel surfaces in the
chat UI.

Registry:
- providers.py: openrouter/free seeded at the top of openrouter
  default_models. Curated list, so picker shows it immediately.

Frontend capability map:
- provider-capabilities.ts: getExternalReasoningCapabilities now
  treats openrouter as enable_thinking style with off support. The
  Think dropdown appears for every OpenRouter model; the gateway
  silently no-ops the parameter for models that do not reason, so
  surfacing one toggle on every model is safe.

Backend reasoning passthrough:
- external_provider.py stream_chat_completion (default OAI-compat
  branch): for provider_type=='openrouter', translate the request:
    reasoning_effort in {low,medium,high} -> body['reasoning'] =
        {'effort': <level>}
    enable_thinking=True  -> body['reasoning'] = {'enabled': True}
    enable_thinking=False -> body['reasoning'] = {'enabled': False}
  Matches the documented shape at
  https://openrouter.ai/docs/guides/best-practices/reasoning-tokens
  with effort and max_tokens mutually exclusive.

Frontend SSE reader:
- chat-adapter.ts: OpenRouter streams reasoning as a third shape we
  did not handle yet: delta.reasoning_details is an array of parts
  like {type: 'reasoning.text', text: '...'}. Pull text from every
  part, merge with the existing delta.reasoning_content channel
  used by Kimi/DeepSeek, and feed the combined string through the
  same <think>...</think> wrap path so parseAssistantContent lifts
  it into the reasoning panel. Anthropic/OpenAI Responses paths
  already wrap on the backend, so they never set this field — no
  cross-provider interference.

* studio/backend: surface OpenRouter SSE errors and router-chosen model in logs

The frontend showed 'Provider returned error' for some openrouter/free
requests with nothing on the backend side to triage from — the
existing 4xx error log only fires when the upstream returns a non-200
status code, but OpenRouter (and most OAI-compat providers) return
200 OK and emit the actual failure as an SSE error event mid-stream,
which our default-path stream loop forwarded verbatim without
logging.

Best-effort diagnostics on the default OpenAI-compat stream path:
- Peek at every `data:` line in the inner forward loop, parse JSON
  best-effort (silently skip on failure so nothing is dropped).
- Count event types: delta / error / done.
- On any chunk containing an `error` field, emit a structlog WARNING
  with the provider type and the error payload — same trail the
  user would otherwise have to dig out of browser devtools.
- Latch the first non-empty `chunk.model` field. OpenRouter reports
  the router-picked underlying model there per request, so the
  finally-block summary log shows which free model handled the call.

In the finally block:

    'openrouter stream complete (model=openrouter/free,
     chosen=google/gemini-2.5-flash, events={delta: 47, done: 1})'

Zero overhead for non-error streams (a json.loads per chunk +
dict-key lookups). The structlog logger is already configured at
INFO; ERROR and WARNING surface in JSON logs without further setup.

Hoists `import json as _json` to module top so the default path can
reuse it; the existing in-function imports in _stream_anthropic and
_stream_openai_responses are now redundant but harmless.

* studio/chat: show router-picked model after 'openrouter/free:' in chip

When the user picks openrouter/free, the gateway routes each request
to a different underlying free model. Until now there was no way to
tell which one actually replied without reading the backend logs.

Surface the picked model in the active-model chip:

- chat-runtime-store gains lastOpenRouterChosenModel: string|null
  plus a setter. Reset on every model switch unless the user stays
  on openrouter/free.
- chat-adapter SSE loop latches chunk.model into the store on
  every chunk whose top-level model differs from
  openrouter/free, gated on the active checkpoint being
  openrouter/free under an OpenRouter provider.
- chat-page externalModels useMemo appends :<chosen> to the display
  name for the openrouter/free option when the store has a value,
  so ModelSelector renders e.g.
    'openrouter/free:google/gemini-2.5-flash'
  in the chip. Other models unaffected.
- Model-switch callback in chat-page clears the cached value when
  the user moves to any model other than openrouter/free, so the
  chip never shows a stale suffix from a previous session.

* studio/chat: shorten openrouter/free chip to openrouter:<short-chosen>

The full display name in use was:
  openrouter/free:inclusionai/ring-2.6-1t-20260508:free

The `:free` suffix on the underlying id already conveys 'free model',
which made the leading `/free` on the router id redundant, and the
`inclusionai/` org prefix was just noise crowding the chip.

Trim both. Now the chip renders as:
  openrouter:ring-2.6-1t-20260508:free

Strictly a display change in chat-page externalModels useMemo — the
backend wire id stays `openrouter/free`, the runtime store still
caches the full `inclusionai/...:free` value, and the model-switch
clearing logic is unchanged.

* studio/providers: switch OpenRouter to remote listing with org allowlist + cap

Same shape as Hugging Face Inference. The curated list had only four
entries; remote listing fetches OpenRouter's full ~300-model
catalog via /v1/models and the new allowlist + limit scope it back
down to a usable picker.

- model_list_mode: remote (was curated)
- model_id_allowlist matches the prefixes:
    openrouter | openai | anthropic | google | meta-llama | qwen
    | mistralai | deepseek | moonshotai | inclusionai | zai-org
    | z-ai
  Anything outside drops out.
- model_id_limit: 20 — first 20 post-filter matches from the live
  fetch; default_models stays seeded so the most useful canonical
  ids are always visible regardless of API response order.
- default_models seed extended from 4 to 6 (openrouter/free,
  openai/gpt-4o, anthropic/claude-sonnet-4-5, google/gemini-2.5-flash,
  mistralai/mistral-large-2411, deepseek/deepseek-r1).
  openrouter/free remains the first entry, so the dialog's
  loadModels() union-merge (registryDefaults first, then remote,
  deduped via Set) keeps it at the top of the picker.

* feat: external mistral thinking toggle

* studio/chat: fix TS2540 by replacing readonly ContentPart instead of mutating

The ContentPart type from @assistant-ui/react marks `text` as readonly,
so the coalesce-adjacent-same-type-part optimization in
parseAssistantContent failed the tsc build with:

  parse-assistant-content.ts(15,10): error TS2540: Cannot assign to
      'text' because it is a read-only property.
  parse-assistant-content.ts(25,10): error TS2540: ...

This broke npm run build, the Studio installer's `building frontend...`
step, and every downstream CI job that runs against an installed
Studio (Mac/Windows/Linux variants of Studio API CI, GGUF CI, UI CI,
Tauri CI, Wheel CI).

Replace the last element with a fresh merged object instead of
mutating its `text` field. Same allocation profile as the previous
path (one object swap per merge), type-safe under the readonly
declaration. Behaviour unchanged.

* studio/backend: restore summary='auto' on OpenAI Responses reasoning body

A recent refactor dropped the `summary: 'auto'` field from the
reasoning config we send to /v1/responses. Without it OpenAI does
not emit reasoning summary events on most reasoning models, which
means our SSE handler has no <think>…</think> to wrap and the chat
reasoning panel stays blank for any gpt-5.x / o3 response.

The expected wire shape is:
    body['reasoning'] = {'effort': '<level>', 'summary': 'auto'}

Two backend tests pin this:
- test_responses_reasoning_effort_included_when_requested (high)
- test_responses_reasoning_effort_xhigh_passthrough (xhigh)
Both were failing with AssertionError because the produced body
omitted `summary: auto`.

Restore the field. Skip it only for the explicit "off" case
(effort: 'none'), where summaries serve no purpose. The
enable_thinking=True fallback (no explicit effort) also pairs
medium effort with summary='auto' so that branch produces
reasoning text too.

* chat: external reasoning, OpenRouter curation, Think toggle fixes

* fix: opus and sonnet 4.6 xhigh --> max

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
This commit is contained in:
Roland Tannous 2026-05-14 16:13:59 +04:00 • committed by GitHub
commit 9a0d6f80cb
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
54 changed files with 8205 additions and 271 deletions
Download patch file Download diff file Expand all files Collapse all files

1238
studio/backend/core/inference/external_provider.py Normal file
View file

File diff suppressed because it is too large Load diff

127
studio/backend/core/inference/key_exchange.py Normal file
View file

@ -0,0 +1,127 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
RSA key pair for encrypting API keys in transit.
The frontend encrypts API keys with the server's public key before
including them in requests. The backend decrypts with its private key
before forwarding to external providers.
The key pair is generated at server startup and lives only in memory —
it is regenerated on each restart. The frontend fetches the public key
via GET /api/providers/public-key on load.
"""
import base64
import hashlib
import logging
from cryptography.hazmat.primitives.asymmetric import rsa, padding
from cryptography.hazmat.primitives import serialization, hashes
logger = logging.getLogger(__name__)
_private_key: rsa.RSAPrivateKey | None = None
_public_key_pem: str | None = None
_public_key_fingerprint: str | None = None
def _compute_fingerprint(pem: str) -> str:
"""SHA256 of the PEM bytes, truncated for log compactness."""
return hashlib.sha256(pem.encode("utf-8")).hexdigest()[:16]
def init_key_pair() -> None:
"""Generate an RSA-2048 key pair. Called once at server startup."""
global _private_key, _public_key_pem, _public_key_fingerprint
if _private_key is not None:
# Re-entry is suspicious — every fresh keypair invalidates all
# in-flight ciphertext encrypted against the previous public key.
# Log loudly so a regression that calls init twice is visible.
logger.warning(
"init_key_pair called again — replacing existing RSA keypair "
"(previous fingerprint=%s). Any frontend that cached the old "
"public key will start hitting decryption failures.",
_public_key_fingerprint,
)
_private_key = rsa.generate_private_key(
public_exponent = 65537,
key_size = 2048,
)
_public_key_pem = (
_private_key.public_key()
.public_bytes(
serialization.Encoding.PEM,
serialization.PublicFormat.SubjectPublicKeyInfo,
)
.decode("utf-8")
)
_public_key_fingerprint = _compute_fingerprint(_public_key_pem)
logger.info(
"RSA key pair generated for API key encryption (fingerprint=%s)",
_public_key_fingerprint,
)
def get_public_key_fingerprint() -> str | None:
"""Short SHA256 of the current public key PEM; None before init."""
return _public_key_fingerprint
def get_public_key_pem() -> str:
"""Return the PEM-encoded public key for the frontend."""
if _public_key_pem is None:
raise RuntimeError("Key pair not initialized. Call init_key_pair() first.")
return _public_key_pem
def decrypt_api_key(encrypted_b64: str) -> str:
"""
Decrypt an API key that was encrypted with the public key.
Args:
encrypted_b64: Base64-encoded RSA-OAEP ciphertext.
Returns:
The plaintext API key string.
"""
if _private_key is None:
raise RuntimeError("Key pair not initialized. Call init_key_pair() first.")
try:
ciphertext = base64.b64decode(encrypted_b64)
except Exception as exc:
logger.warning(
"decrypt_api_key: base64 decode failed (input_len=%d, fingerprint=%s): %s: %s",
len(encrypted_b64),
_public_key_fingerprint,
type(exc).__name__,
exc,
)
raise
try:
plaintext = _private_key.decrypt(
ciphertext,
padding.OAEP(
mgf = padding.MGF1(algorithm = hashes.SHA256()),
algorithm = hashes.SHA256(),
label = None,
),
)
except Exception as exc:
# Surface enough state to distinguish key mismatch (wrong public key
# used on encrypt) from a padding/algo mismatch or corrupted bytes.
# Expected ciphertext length for RSA-2048 is exactly 256 bytes.
logger.warning(
"decrypt_api_key: RSA decrypt failed (ciphertext_len=%d, expected=256, "
"fingerprint=%s, exc=%s): %s",
len(ciphertext),
_public_key_fingerprint,
type(exc).__name__,
exc,
)
raise
return plaintext.decode("utf-8")

287
studio/backend/core/inference/providers.py Normal file
View file

@ -0,0 +1,287 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Static registry of supported external LLM providers.
All providers expose OpenAI-compatible /v1/chat/completions endpoints
with Bearer token authentication and SSE streaming support.
"""
import re
from typing import Any
PROVIDER_REGISTRY: dict[str, dict[str, Any]] = {
"openai": {
"display_name": "OpenAI",
"base_url": "https://api.openai.com/v1",
"default_models": [
"gpt-5.5",
"gpt-5.4",
"gpt-5.4-mini",
"o3",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
# Keep the model picker scoped to the current generation. The remote
# /v1/models listing returns dozens of historical snapshots, fine-tunes
# and non-chat models (embeddings, TTS, image, moderation) that we
# never want to surface in the chat UI. Filtering here so backend
# is the single source of truth.
"model_id_allowlist": re.compile(r"^(gpt-5\.[345]|gpt-4\.5|o3)(?:[-.]|$)"),
# Hide dated snapshots and the retired plain gpt-5.3 id.
"model_id_denylist": re.compile(r"^(gpt-5\.3)$|-\d{4}-\d{2}-\d{2}$"),
},
"anthropic": {
"display_name": "Anthropic",
"base_url": "https://api.anthropic.com/v1",
"default_models": [
"claude-opus-4-7",
"claude-opus-4-6",
"claude-sonnet-4-6",
"claude-opus-4-5",
"claude-sonnet-4-5",
"claude-haiku-4-5",
],
# Anthropic /v1/models returns dated snapshot ids alongside the
# canonical names (e.g. claude-3-5-sonnet-20241022). Hide the
# YYYYMMDD-suffixed variants from the picker — same intent as the
# OpenAI denylist, just a different date format (no dashes between
# year/month/day).
"model_id_denylist": re.compile(r"-\d{8}$"),
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": False,
"auth_header": "x-api-key",
"auth_prefix": "",
"extra_headers": {
"anthropic-version": "2023-06-01",
},
"openai_compatible": False,
"notes": "Native Anthropic Messages API. Uses x-api-key header and /v1/messages endpoint with SSE translation.",
},
"gemini": {
"display_name": "Google Gemini",
"base_url": "https://generativelanguage.googleapis.com/v1beta/openai",
# Curated lineup — Google's /v1beta/openai/models returns dozens
# of historical / experimental / embedding ids. Cap to the current
# 3.x family plus the rolling `*-latest` aliases.
"default_models": [
"gemini-3.1-pro-preview",
"gemini-3.1-flash-lite",
"gemini-3-flash-preview",
"gemini-pro-latest",
"gemini-flash-latest",
"gemini-flash-lite-latest",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"notes": "OpenAI-compatible endpoint. API key from https://aistudio.google.com/apikey.",
"model_id_allowlist": re.compile(
r"^(gemini-3\.1-flash-lite|gemini-3-flash-preview|"
r"gemini-3\.1-pro-preview|gemini-pro-latest|"
r"gemini-flash-latest|gemini-flash-lite-latest)$"
),
},
"deepseek": {
"display_name": "DeepSeek",
"base_url": "https://api.deepseek.com/v1",
"default_models": [
"deepseek-chat",
"deepseek-reasoner",
],
"supports_streaming": True,
"supports_vision": False,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"notes": "OpenAI-compatible API. deepseek-chat = V3, deepseek-reasoner = R1 thinking mode.",
},
"mistral": {
"display_name": "Mistral AI",
"base_url": "https://api.mistral.ai/v1",
"default_models": [
"codestral-latest",
"devstral-latest",
"devstral-medium-latest",
"magistral-medium-latest",
"ministral-14b-latest",
"ministral-3b-latest",
"ministral-8b-latest",
"mistral-large-latest",
"mistral-medium-latest",
"mistral-small-latest",
"mistral-tiny-latest",
"mistral-vibe-cli-latest",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"model_id_allowlist": re.compile(
r"^(codestral-latest|devstral-latest|devstral-medium-latest|"
r"magistral-medium-latest|ministral-(?:14b|3b|8b)-latest|"
r"mistral-(?:large|medium|small|tiny)-latest|"
r"mistral-vibe-cli-latest)$"
),
},
"kimi": {
"display_name": "Kimi",
"base_url": "https://api.moonshot.ai/v1",
# Current Kimi model lineup per the official docs:
# https://platform.kimi.ai/docs/models
# Listing/overview endpoints used to enumerate them:
# https://platform.kimi.ai/docs/api/list-models
# https://platform.kimi.ai/docs/api/overview
# kimi-k2.6 and kimi-k2.5 are the two SoTA multimodal models we
# surface in the picker; everything else (moonshot-v1-*, dated
# k2 previews) is filtered out by model_id_allowlist below.
"default_models": [
"kimi-k2.6",
"kimi-k2.5",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"notes": "Moonshot API key. China: use base URL https://api.moonshot.cn/v1",
"model_id_allowlist": re.compile(r"^kimi-k2\.[56]$"),
# Both k2.6 and k2.5 are reasoning-class. The API rejects custom
# sampling: "invalid temperature: only 1 is allowed for this model"
# (and the same shape for top_p). Strip both fields from the
# outbound body so the server falls back to its required defaults.
"body_omit": ("temperature", "top_p"),
},
"qwen": {
"display_name": "Qwen",
"base_url": "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
"default_models": [
"qwen-plus",
"qwen-turbo",
"qwen-max",
"qwen2.5-72b-instruct",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"notes": "DashScope API key. China mainland: override base URL to https://dashscope.aliyuncs.com/compatible-mode/v1",
},
"huggingface": {
"display_name": "Hugging Face",
"base_url": "https://router.huggingface.co/v1",
# Seed the picker with a few popular ids so something is selectable
# before the live /v1/models call resolves. The remote listing is
# the source of truth — see model_list_mode below.
"default_models": [
"openai/gpt-oss-120b",
"deepseek-ai/DeepSeek-V3",
"meta-llama/Llama-3.3-70B-Instruct",
"Qwen/Qwen2.5-72B-Instruct",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"notes": (
"HF token from huggingface.co/settings/tokens. Uses the "
"OpenAI-compatible router at /v1/chat/completions; /v1/models "
"returns the cross-provider chat catalog. See "
"https://huggingface.co/docs/inference-providers/index."
),
# /v1/models works on the HF router and returns the full chat-model
# catalog (state.org/model[:policy] ids). Switch to remote so users
# see live availability — the picker has a search box, and
# loadModels() merges defaults so default_models entries remain
# visible if the remote call fails.
"model_list_mode": "remote",
# Scope the catalog to first-party org repos we trust as primary
# sources. The HF /v1/models response is otherwise hundreds of
# ids long (community fine-tunes, mirrors, fp8 variants, etc.).
"model_id_allowlist": re.compile(
r"^(openai|deepseek-ai|google|meta-llama|Qwen|moonshotai|"
r"mistralai|zai-org)/"
),
# Cap the post-filter list. /v1/models has no server-side limit
# or popularity sort, so this is just "first N matches" — pair it
# with the default_models seed so the most useful flagship ids
# are always among the top regardless of the API's order.
"model_id_limit": 15,
},
"openrouter": {
"display_name": "OpenRouter",
"base_url": "https://openrouter.ai/api/v1",
# Curated list for Studio's picker (explicitly locked, not live /models).
"default_models": [
"openrouter/free",
"openai/gpt-4o",
"anthropic/claude-sonnet-4-5",
"google/gemini-2.5-flash",
"mistralai/mistral-large-2411",
"deepseek/deepseek-r1",
"mistralai/mistral-small-3.1-24b-instruct",
"perceptron/perceptron-mk1",
"inclusionai/ring-2.6-1t:free",
"google/gemini-3.1-flash-lite",
"baidu/cobuddy:free",
"openai/gpt-chat-latest",
"x-ai/grok-4.3",
"ibm-granite/granite-4.1-8b",
"openrouter/owl-alpha",
"poolside/laguna-xs.2:free",
"~google/gemini-pro-latest",
"~moonshotai/kimi-latest",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"extra_headers": {
"HTTP-Referer": "https://unsloth.ai",
"X-Title": "Unsloth Studio",
},
"notes": "Unified gateway to 300+ models across all major providers. HTTP-Referer and X-Title headers sent for attribution.",
"model_list_mode": "curated",
},
}
def get_provider_info(provider_type: str) -> dict[str, Any] | None:
"""Return the registry entry for a provider type, or None if unknown."""
return PROVIDER_REGISTRY.get(provider_type)
def get_base_url(provider_type: str) -> str | None:
"""Return the default base URL for a provider type."""
info = PROVIDER_REGISTRY.get(provider_type)
return info["base_url"] if info else None
def list_available_providers() -> list[dict[str, Any]]:
"""Return all registered providers (for the /registry endpoint)."""
result = []
for provider_type, info in PROVIDER_REGISTRY.items():
result.append(
{
"provider_type": provider_type,
"display_name": info["display_name"],
"base_url": info["base_url"],
"default_models": info["default_models"],
"supports_streaming": info["supports_streaming"],
"supports_vision": info.get("supports_vision", False),
"supports_tool_calling": info.get("supports_tool_calling", False),
"model_list_mode": info.get("model_list_mode", "remote"),
}
)
return result

7
studio/backend/main.py
View file

@ -120,6 +120,7 @@ from routes import (
inference_router,
inference_studio_router,
models_router,
providers_router,
training_history_router,
training_router,
)
@ -222,6 +223,11 @@ async def lifespan(app: FastAPI):
threading.Thread(target = _precache, daemon = True).start()
# Initialize RSA key pair for API key encryption (external providers)
from core.inference.key_exchange import init_key_pair
init_key_pair()
if storage.ensure_default_admin():
bootstrap_pw = storage.get_bootstrap_password()
app.state.bootstrap_password = bootstrap_pw
@ -474,6 +480,7 @@ app.include_router(inference_studio_router, prefix = "/api/inference", tags = ["
# so external tools (Open WebUI, SillyTavern, etc.) can use the
# standard /v1/chat/completions path.
app.include_router(inference_router, prefix = "/v1", tags = ["openai-compat"])
app.include_router(providers_router, prefix = "/api/providers", tags = ["providers"])
app.include_router(datasets_router, prefix = "/api/datasets", tags = ["datasets"])
app.include_router(data_recipe_router, prefix = "/api/data-recipe", tags = ["data-recipe"])
app.include_router(export_router, prefix = "/api/export", tags = ["export"])

28
studio/backend/models/inference.py
View file

@ -531,9 +531,11 @@ class ChatCompletionRequest(BaseModel):
None,
description = "[x-unsloth] Enable/disable thinking/reasoning mode for supported models",
)
reasoning_effort: Optional[Literal["low", "medium", "high"]] = Field(
reasoning_effort: Optional[
Literal["none", "minimal", "low", "medium", "high", "max", "xhigh"]
] = Field(
None,
description = "[x-unsloth] Reasoning effort level ('low'|'medium'|'high') for Harmony-style reasoning models (e.g. gpt-oss). Overrides enable_thinking when the active model uses reasoning_effort style.",
description = "[x-unsloth] Reasoning effort level ('none'|'minimal'|'low'|'medium'|'high'|'max'|'xhigh'). OpenAI `/v1/responses` accepts model-dependent subsets; Anthropic adaptive thinking uses `max` as the top tier on Claude 4.6 Opus/Sonnet (inbound `xhigh` is mapped to `max`) and `xhigh` on Claude 4.7 Opus; local Harmony/gpt-oss templates support low|medium|high.",
)
preserve_thinking: Optional[bool] = Field(
None,
@ -570,6 +572,28 @@ class ChatCompletionRequest(BaseModel):
description = "[x-unsloth] Per-request cancellation token. Frontend sends a fresh UUID per run so /inference/cancel matches one specific generation.",
)
# ── External provider routing (x-unsloth extensions) ──────────
provider_id: Optional[str] = Field(
None,
description = "[x-unsloth] Saved provider config ID. If set with encrypted_api_key, routes to external LLM.",
)
provider_type: Optional[str] = Field(
None,
description = "[x-unsloth] Provider type (e.g. 'openai', 'mistral'). Used if provider_id is not set.",
)
external_model: Optional[str] = Field(
None,
description = "[x-unsloth] Model ID at the external provider.",
)
encrypted_api_key: Optional[str] = Field(
None,
description = "[x-unsloth] RSA-encrypted, base64-encoded API key for the external provider.",
)
provider_base_url: Optional[str] = Field(
None,
description = "[x-unsloth] Override base URL for the external provider.",
)
# ── Streaming response chunks ────────────────────────────────────

128
studio/backend/models/providers.py Normal file
View file

@ -0,0 +1,128 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Pydantic schemas for the external LLM providers API.
"""
from typing import Literal, Optional
from pydantic import BaseModel, Field
# ── Registry (static provider info) ───────────────────────────────
class ProviderRegistryEntry(BaseModel):
"""A supported provider type with its default configuration."""
provider_type: str = Field(
..., description = "Provider identifier (e.g. 'openai', 'mistral')"
)
display_name: str = Field(..., description = "Human-readable provider name")
base_url: str = Field(..., description = "Default API base URL")
default_models: list[str] = Field(
default_factory = list, description = "Well-known model IDs for this provider"
)
supports_streaming: bool = Field(
True, description = "Whether this provider supports SSE streaming"
)
supports_vision: bool = Field(
False, description = "Whether this provider supports vision/image input"
)
supports_tool_calling: bool = Field(
False, description = "Whether this provider supports tool/function calling"
)
model_list_mode: Literal["remote", "curated"] = Field(
"remote",
description = "remote = fetch /models; curated = huge catalogs — UI uses defaults + manual IDs only",
)
# ── Provider config CRUD ──────────────────────────────────────────
class ProviderCreate(BaseModel):
"""Request to create a saved provider configuration."""
provider_type: str = Field(..., description = "Provider type from the registry")
display_name: str = Field(
..., description = "User-chosen label (e.g. 'My OpenAI Key')"
)
base_url: Optional[str] = Field(
None,
description = "Custom base URL (overrides registry default). Omit to use the default.",
)
class ProviderUpdate(BaseModel):
"""Request to update a saved provider configuration."""
display_name: Optional[str] = Field(None, description = "New display name")
base_url: Optional[str] = Field(None, description = "New base URL")
is_enabled: Optional[bool] = Field(
None, description = "Enable or disable this provider"
)
class ProviderResponse(BaseModel):
"""A saved provider configuration (returned by list/get endpoints)."""
id: str = Field(..., description = "Unique provider config ID")
provider_type: str = Field(..., description = "Provider type (e.g. 'openai')")
display_name: str = Field(..., description = "User-chosen label")
base_url: str = Field(..., description = "API base URL")
is_enabled: bool = Field(True, description = "Whether this provider is enabled")
created_at: str = Field(..., description = "ISO 8601 creation timestamp")
updated_at: str = Field(..., description = "ISO 8601 last-update timestamp")
# ── Model listing ─────────────────────────────────────────────────
class ProviderModelInfo(BaseModel):
"""A model available from an external provider."""
id: str = Field(..., description = "Model ID as expected by the provider API")
display_name: str = Field("", description = "Human-readable model name")
context_length: Optional[int] = Field(
None, description = "Maximum context length in tokens"
)
owned_by: Optional[str] = Field(None, description = "Model owner/organization")
class ProviderModelsRequest(BaseModel):
"""Request to list models from an external provider."""
provider_type: str = Field(..., description = "Provider type from the registry")
encrypted_api_key: str = Field(
..., description = "RSA-encrypted, base64-encoded API key"
)
base_url: Optional[str] = Field(
None, description = "Custom base URL (overrides registry default)"
)
# ── Connection testing ────────────────────────────────────────────
class ProviderTestRequest(BaseModel):
"""Request to test connectivity to an external provider."""
provider_type: str = Field(..., description = "Provider type from the registry")
encrypted_api_key: str = Field(
..., description = "RSA-encrypted, base64-encoded API key"
)
base_url: Optional[str] = Field(
None, description = "Custom base URL (overrides registry default)"
)
class ProviderTestResult(BaseModel):
"""Result of a provider connectivity test."""
success: bool = Field(..., description = "Whether the test succeeded")
message: str = Field(..., description = "Human-readable result message")
models_count: Optional[int] = Field(
None, description = "Number of models found (if test succeeded)"
)

2
studio/backend/requirements/studio.txt
View file

@ -16,3 +16,5 @@ huggingface-hub==0.36.2
structlog>=24.1.0
diceware
ddgs
cryptography>=42.0.0
httpx>=0.27.0

2
studio/backend/routes/__init__.py
View file

@ -14,6 +14,7 @@ from routes.auth import router as auth_router
from routes.data_recipe import router as data_recipe_router
from routes.export import router as export_router
from routes.training_history import router as training_history_router
from routes.providers import router as providers_router
__all__ = [
"training_router",
@ -25,4 +26,5 @@ __all__ = [
"data_recipe_router",
"export_router",
"training_history_router",
"providers_router",
]

164
studio/backend/routes/inference.py
View file

@ -204,6 +204,11 @@ from core.inference.anthropic_compat import (
)
from auth.authentication import get_current_subject
from core.inference.key_exchange import decrypt_api_key
from core.inference.providers import get_provider_info, get_base_url
from core.inference.external_provider import ExternalProviderClient
from storage import providers_db
import io
import wave
import base64
@ -1464,6 +1469,161 @@ def _extract_content_parts(
return system_prompt, chat_messages, first_image_b64
# ── External provider proxy ──────────────────────────────────────
def _build_external_messages(
messages: list,
supports_vision: bool,
) -> list[dict]:
"""
Convert ChatMessage list to OpenAI-compatible dicts for external providers.
- Vision providers: preserve multimodal content arrays (image_url parts intact).
- Non-vision providers: flatten to text-only (images silently dropped).
"""
result = []
for msg in messages:
if isinstance(msg.content, str):
# Skip assistant messages with empty content (some providers reject them)
if msg.role == "assistant" and not msg.content.strip():
continue
result.append({"role": msg.role, "content": msg.content})
elif isinstance(msg.content, list):
if supports_vision:
parts = []
for part in msg.content:
if part.type == "text":
parts.append({"type": "text", "text": part.text})
elif part.type == "image_url":
parts.append(
{
"type": "image_url",
"image_url": {"url": part.image_url.url},
}
)
result.append({"role": msg.role, "content": parts})
else:
# Non-vision provider — strip images, keep text only
text = "\n".join(p.text for p in msg.content if p.type == "text")
result.append({"role": msg.role, "content": text})
return result
async def _proxy_to_external_provider(
payload: ChatCompletionRequest,
request: Request,
) -> StreamingResponse:
"""
Proxy a chat completion request to an external LLM provider.
Resolves provider config (from DB or registry), decrypts the API key,
and streams the response back in OpenAI SSE format.
"""
# Resolve provider type and base URL
provider_type = payload.provider_type
base_url = payload.provider_base_url
if payload.provider_id:
config = providers_db.get_provider(payload.provider_id)
if config is None:
raise HTTPException(
status_code = 404,
detail = f"Provider config not found: {payload.provider_id}",
)
if not config["is_enabled"]:
raise HTTPException(
status_code = 400,
detail = f"Provider '{config['display_name']}' is disabled.",
)
provider_type = provider_type or config["provider_type"]
base_url = base_url or config["base_url"]
if not provider_type:
raise HTTPException(
status_code = 400,
detail = "Either provider_id or provider_type is required for external provider routing.",
)
# Fall back to registry default base URL
if not base_url:
base_url = get_base_url(provider_type)
if not base_url:
raise HTTPException(
status_code = 400,
detail = f"Unknown provider type: {provider_type}",
)
# Decrypt the API key
try:
api_key = decrypt_api_key(payload.encrypted_api_key)
except Exception as exc:
logger.warning("external_provider.decrypt_failed", error = str(exc))
raise HTTPException(
status_code = 400,
detail = "Failed to decrypt API key. The server key may have changed — try refreshing the page.",
)
model = payload.external_model or payload.model
if model == "default":
raise HTTPException(
status_code = 400,
detail = "external_model is required when using an external provider.",
)
# Build messages preserving multimodal content for vision-capable providers
from core.inference.providers import get_provider_info as _get_provider_info
_pinfo = _get_provider_info(provider_type) or {}
_supports_vision = _pinfo.get("supports_vision", False)
chat_messages = _build_external_messages(payload.messages, _supports_vision)
client = ExternalProviderClient(
provider_type = provider_type,
base_url = base_url,
api_key = api_key,
)
async def _stream():
gen = client.stream_chat_completion(
messages = chat_messages,
model = model,
temperature = payload.temperature,
top_p = payload.top_p,
max_tokens = payload.max_tokens,
presence_penalty = payload.presence_penalty,
top_k = payload.top_k,
enable_thinking = payload.enable_thinking,
reasoning_effort = payload.reasoning_effort,
stream = payload.stream,
)
try:
sent_done = False
async for line in gen:
yield f"{line}\n\n"
if "[DONE]" in line:
sent_done = True
if not sent_done:
yield "data: [DONE]\n\n"
except Exception as exc:
logger.error("external_provider.stream_error", error = str(exc))
finally:
try:
await gen.aclose()
except RuntimeError:
pass # suppress httpcore asyncgen cleanup error (Python 3.13 + httpcore 1.0.x)
await client.close()
return StreamingResponse(
_stream(),
media_type = "text/event-stream",
headers = {
"Cache-Control": "no-cache",
"X-Accel-Buffering": "no",
},
)
@router.post("/chat/completions")
async def openai_chat_completions(
payload: ChatCompletionRequest,
@ -1483,6 +1643,10 @@ async def openai_chat_completions(
- GGUF models → llama-server via LlamaCppBackend
- Other models → Unsloth/transformers via InferenceBackend
"""
# ── External provider routing ────────────────────────────────
if payload.encrypted_api_key and (payload.provider_id or payload.provider_type):
return await _proxy_to_external_provider(payload, request)
llama_backend = get_llama_cpp_backend()
using_gguf = llama_backend.is_loaded

338
studio/backend/routes/providers.py Normal file
View file

@ -0,0 +1,338 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
API routes for external LLM provider management.
Provides endpoints for:
- Discovering available provider types (registry)
- CRUD for saved provider configurations (no API keys stored)
- Fetching the RSA public key for API key encryption
- Testing provider connectivity
- Listing models from a provider
"""
import uuid
import structlog
from fastapi import APIRouter, Depends, HTTPException
from auth.authentication import get_current_subject
from core.inference.key_exchange import (
decrypt_api_key,
get_public_key_fingerprint,
get_public_key_pem,
)
from core.inference.providers import (
get_base_url,
get_provider_info,
list_available_providers,
)
from core.inference.external_provider import ExternalProviderClient
from models.providers import (
ProviderCreate,
ProviderModelsRequest,
ProviderModelInfo,
ProviderResponse,
ProviderRegistryEntry,
ProviderTestRequest,
ProviderTestResult,
ProviderUpdate,
)
from storage import providers_db
logger = structlog.get_logger(__name__)
router = APIRouter()
# ── Public key for API key encryption ─────────────────────────────
@router.get("/public-key")
async def get_public_key(
current_subject: str = Depends(get_current_subject),
):
"""Return the RSA public key PEM for client-side API key encryption.
The ``fingerprint`` field is a short SHA256 of the PEM and is meant
purely for diagnostics — a mismatch between what the frontend
captured at encrypt time and what the server reports here is a
clear signal that the keypair rotated mid-flight (e.g. the server
re-ran ``init_key_pair`` for any reason).
"""
return {
"public_key": get_public_key_pem(),
"fingerprint": get_public_key_fingerprint(),
}
# ── Provider registry (static) ───────────────────────────────────
@router.get("/registry", response_model = list[ProviderRegistryEntry])
async def list_registry(
current_subject: str = Depends(get_current_subject),
):
"""List all supported provider types with their default configurations."""
return list_available_providers()
# ── Provider config CRUD ──────────────────────────────────────────
@router.get("/", response_model = list[ProviderResponse])
async def list_provider_configs(
current_subject: str = Depends(get_current_subject),
):
"""List all saved provider configurations."""
rows = providers_db.list_providers()
return [
ProviderResponse(
id = row["id"],
provider_type = row["provider_type"],
display_name = row["display_name"],
base_url = row["base_url"],
is_enabled = bool(row["is_enabled"]),
created_at = row["created_at"],
updated_at = row["updated_at"],
)
for row in rows
]
@router.post("/", response_model = ProviderResponse, status_code = 201)
async def create_provider_config(
payload: ProviderCreate,
current_subject: str = Depends(get_current_subject),
):
"""Create a new saved provider configuration (no API key stored)."""
info = get_provider_info(payload.provider_type)
if info is None:
raise HTTPException(
status_code = 400,
detail = f"Unknown provider type: {payload.provider_type}. "
f"Use GET /api/providers/registry to see available types.",
)
provider_id = uuid.uuid4().hex[:16]
base_url = payload.base_url or info["base_url"]
providers_db.create_provider(
id = provider_id,
provider_type = payload.provider_type,
display_name = payload.display_name,
base_url = base_url,
)
row = providers_db.get_provider(provider_id)
return ProviderResponse(
id = row["id"],
provider_type = row["provider_type"],
display_name = row["display_name"],
base_url = row["base_url"],
is_enabled = bool(row["is_enabled"]),
created_at = row["created_at"],
updated_at = row["updated_at"],
)
@router.put("/{provider_id}", response_model = ProviderResponse)
async def update_provider_config(
provider_id: str,
payload: ProviderUpdate,
current_subject: str = Depends(get_current_subject),
):
"""Update a saved provider configuration."""
existing = providers_db.get_provider(provider_id)
if not existing:
raise HTTPException(status_code = 404, detail = "Provider not found")
updated = providers_db.update_provider(
id = provider_id,
display_name = payload.display_name,
base_url = payload.base_url,
is_enabled = payload.is_enabled,
)
if not updated:
raise HTTPException(status_code = 400, detail = "No fields to update")
row = providers_db.get_provider(provider_id)
return ProviderResponse(
id = row["id"],
provider_type = row["provider_type"],
display_name = row["display_name"],
base_url = row["base_url"],
is_enabled = bool(row["is_enabled"]),
created_at = row["created_at"],
updated_at = row["updated_at"],
)
@router.delete("/{provider_id}", status_code = 204)
async def delete_provider_config(
provider_id: str,
current_subject: str = Depends(get_current_subject),
):
"""Delete a saved provider configuration."""
deleted = providers_db.delete_provider(provider_id)
if not deleted:
raise HTTPException(status_code = 404, detail = "Provider not found")
# ── Test connectivity ─────────────────────────────────────────────
@router.post("/test", response_model = ProviderTestResult)
async def test_provider(
payload: ProviderTestRequest,
current_subject: str = Depends(get_current_subject),
):
"""
Test connectivity to an external provider.
Makes a lightweight GET /models call to verify the API key works.
The encrypted_api_key is decrypted server-side and never stored.
"""
info = get_provider_info(payload.provider_type)
if info is None:
raise HTTPException(
status_code = 400,
detail = f"Unknown provider type: {payload.provider_type}",
)
try:
api_key = decrypt_api_key(payload.encrypted_api_key)
except Exception as exc:
logger.warning("Failed to decrypt API key (%s): %s", type(exc).__name__, exc)
raise HTTPException(
status_code = 400,
detail = "Failed to decrypt API key. The public key may have changed — try refreshing the page.",
)
base_url = payload.base_url or info["base_url"]
client = ExternalProviderClient(
provider_type = payload.provider_type,
base_url = base_url,
api_key = api_key,
timeout = 15.0,
)
try:
if info.get("model_list_mode") == "curated":
await client.verify_models_endpoint_lightweight()
return ProviderTestResult(
success = True,
message = (
"Connected successfully. Full model list is not fetched for this provider — "
"use suggestions and manual model IDs in the dialog."
),
models_count = None,
)
models = await client.list_models()
return ProviderTestResult(
success = True,
message = f"Connected successfully. Found {len(models)} model(s).",
models_count = len(models),
)
except Exception as exc:
logger.warning("Provider test failed for %s: %s", payload.provider_type, exc)
return ProviderTestResult(
success = False,
message = f"Connection failed: {exc}",
models_count = None,
)
finally:
await client.close()
# ── List models from provider ─────────────────────────────────────
@router.post("/models", response_model = list[ProviderModelInfo])
async def list_provider_models(
payload: ProviderModelsRequest,
current_subject: str = Depends(get_current_subject),
):
"""
List models available from an external provider.
The encrypted_api_key is decrypted server-side and never stored.
"""
info = get_provider_info(payload.provider_type)
if info is None:
raise HTTPException(
status_code = 400,
detail = f"Unknown provider type: {payload.provider_type}",
)
try:
api_key = decrypt_api_key(payload.encrypted_api_key)
except Exception as exc:
logger.warning("Failed to decrypt API key (%s): %s", type(exc).__name__, exc)
raise HTTPException(
status_code = 400,
detail = "Failed to decrypt API key. The public key may have changed — try refreshing the page.",
)
if info.get("model_list_mode") == "curated":
return [
ProviderModelInfo(
id = m,
display_name = m,
context_length = None,
owned_by = None,
)
for m in info.get("default_models", [])
]
base_url = payload.base_url or info["base_url"]
client = ExternalProviderClient(
provider_type = payload.provider_type,
base_url = base_url,
api_key = api_key,
timeout = 15.0,
)
try:
models = await client.list_models()
allow_prefixes = info.get("model_id_allow_prefixes")
if allow_prefixes is not None:
prefix_tuple = tuple(str(p) for p in allow_prefixes if str(p))
if prefix_tuple:
models = [m for m in models if m.get("id", "").startswith(prefix_tuple)]
allowlist = info.get("model_id_allowlist")
if allowlist is not None:
models = [m for m in models if allowlist.match(m.get("id", ""))]
deny_exact = info.get("model_id_deny_exact")
if deny_exact is not None:
deny_ids = {str(m) for m in deny_exact if str(m)}
if deny_ids:
models = [m for m in models if m.get("id", "") not in deny_ids]
denylist = info.get("model_id_denylist")
if denylist is not None:
models = [m for m in models if not denylist.search(m.get("id", ""))]
# Apply an optional cap after filtering so registry entries with a
# large remote catalog (e.g. HF Inference Providers) can stay
# picker-sized. No popularity sort happens server-side, so this is
# "first N matches" — pair with default_models for any must-have
# flagship ids.
limit = info.get("model_id_limit")
if isinstance(limit, int) and limit > 0:
models = models[:limit]
return [
ProviderModelInfo(
id = m.get("id", ""),
display_name = m.get("id", ""),
context_length = m.get("context_length") or m.get("context_window"),
owned_by = m.get("owned_by"),
)
for m in models
]
except Exception as exc:
logger.error("Failed to list models from %s: %s", payload.provider_type, exc)
raise HTTPException(
status_code = 502,
detail = f"Failed to list models from {payload.provider_type}: {exc}",
)
finally:
await client.close()

153
studio/backend/storage/providers_db.py Normal file
View file

@ -0,0 +1,153 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
SQLite storage for external LLM provider configurations.
Follows the same pattern as studio_db.py — module-level functions,
raw sqlite3, WAL mode, per-function connections.
NOTE: API keys are NOT stored here. They live only in the browser
(localStorage) and are sent encrypted per-request.
"""
import logging
import sqlite3
import threading
from datetime import datetime, timezone
from typing import Optional
logger = logging.getLogger(__name__)
from utils.paths import studio_db_path, ensure_dir
_schema_lock = threading.Lock()
_schema_ready = False
def _ensure_schema(conn: sqlite3.Connection) -> None:
"""Create the llm_providers table if it doesn't exist. Called once per process."""
conn.execute("PRAGMA journal_mode=WAL")
conn.execute(
"""
CREATE TABLE IF NOT EXISTS llm_providers (
id TEXT NOT NULL PRIMARY KEY,
provider_type TEXT NOT NULL,
display_name TEXT NOT NULL,
base_url TEXT NOT NULL,
is_enabled INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
)
"""
)
def get_connection() -> sqlite3.Connection:
"""Open studio.db with WAL mode, create table once per process."""
global _schema_ready
db_path = studio_db_path()
ensure_dir(db_path.parent)
conn = sqlite3.connect(str(db_path))
conn.row_factory = sqlite3.Row
if not _schema_ready:
with _schema_lock:
if not _schema_ready:
try:
_ensure_schema(conn)
_schema_ready = True
except Exception:
conn.close()
raise
return conn
def create_provider(
id: str,
provider_type: str,
display_name: str,
base_url: str,
) -> None:
"""Insert a new provider configuration."""
now = datetime.now(timezone.utc).isoformat()
conn = get_connection()
try:
conn.execute(
"""
INSERT INTO llm_providers (id, provider_type, display_name, base_url, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?)
""",
(id, provider_type, display_name, base_url, now, now),
)
conn.commit()
finally:
conn.close()
def update_provider(
id: str,
display_name: Optional[str] = None,
base_url: Optional[str] = None,
is_enabled: Optional[bool] = None,
) -> bool:
"""Update fields on an existing provider. Returns True if a row was updated."""
updates = []
params = []
if display_name is not None:
updates.append("display_name = ?")
params.append(display_name)
if base_url is not None:
updates.append("base_url = ?")
params.append(base_url)
if is_enabled is not None:
updates.append("is_enabled = ?")
params.append(1 if is_enabled else 0)
if not updates:
return False
updates.append("updated_at = ?")
params.append(datetime.now(timezone.utc).isoformat())
params.append(id)
conn = get_connection()
try:
cursor = conn.execute(
f"UPDATE llm_providers SET {', '.join(updates)} WHERE id = ?",
params,
)
conn.commit()
return cursor.rowcount > 0
finally:
conn.close()
def delete_provider(id: str) -> bool:
"""Delete a provider by ID. Returns True if a row was deleted."""
conn = get_connection()
try:
cursor = conn.execute("DELETE FROM llm_providers WHERE id = ?", (id,))
conn.commit()
return cursor.rowcount > 0
finally:
conn.close()
def get_provider(id: str) -> Optional[dict]:
"""Fetch a single provider by ID."""
conn = get_connection()
try:
row = conn.execute("SELECT * FROM llm_providers WHERE id = ?", (id,)).fetchone()
return dict(row) if row else None
finally:
conn.close()
def list_providers() -> list[dict]:
"""List all provider configurations, ordered by creation time."""
conn = get_connection()
try:
rows = conn.execute(
"SELECT * FROM llm_providers ORDER BY created_at"
).fetchall()
return [dict(row) for row in rows]
finally:
conn.close()

404
studio/backend/tests/test_anthropic_thinking_translation.py Normal file
View file

@ -0,0 +1,404 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Unit tests for the Anthropic extended-thinking translation in
external_provider.
Covers:
- Adaptive-mode request body nests effort under
``output_config: {effort: "<level>"}`` per the Messages API
reference (a top-level ``effort`` field 400s with
"effort: Extra inputs are not permitted").
- Streaming SSE: ``content_block_delta`` with
``delta.type == "thinking_delta"`` is translated into inline
``<think>...</think>`` chat-completion chunks so the frontend's
reasoning-panel pipeline lifts it correctly.
- The ``<think>`` tag closes when the first ``text_delta`` arrives,
on ``content_block_stop``, on ``message_delta``, or on
``message_stop``.
- Thinking is paired with ``temperature=1`` and no ``top_p`` /
``top_k`` on the wire (Anthropic extended-thinking contract).
"""
import asyncio
import json
import httpx
from core.inference import external_provider as ep_mod
from core.inference.external_provider import ExternalProviderClient
def _drive(coro):
return asyncio.new_event_loop().run_until_complete(coro)
async def _collect(agen):
out = []
async for line in agen:
out.append(line)
return out
def _mock_http_client(monkeypatch, handler):
transport = httpx.MockTransport(handler)
monkeypatch.setattr(ep_mod, "_http_client", httpx.AsyncClient(transport = transport))
def _make_client() -> ExternalProviderClient:
return ExternalProviderClient(
provider_type = "anthropic",
base_url = "https://api.anthropic.com/v1",
api_key = "sk-ant-test",
)
def _anthropic_sse(events: list[dict]) -> bytes:
"""Serialize a list of Messages-API event dicts as an SSE byte stream."""
chunks: list[str] = []
for event in events:
chunks.append(f"event: {event['type']}")
chunks.append(f"data: {json.dumps(event)}")
chunks.append("")
return ("\n".join(chunks) + "\n").encode("utf-8")
def _payloads_from_lines(lines: list[str]) -> list:
out = []
for line in lines:
if not line.startswith("data:"):
continue
raw = line[len("data:") :].strip()
if not raw:
continue
if raw == "[DONE]":
out.append("[DONE]")
else:
out.append(json.loads(raw))
return out
def test_adaptive_thinking_body_uses_output_config_effort_shape(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-6",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = None,
reasoning_effort = "medium",
):
pass
await client.close()
_drive(run())
body = captured["body"]
# display=summarized is set explicitly so Opus 4.7 (which defaults to
# "omitted") still emits thinking_delta events for the reasoning panel.
assert body["thinking"] == {"type": "adaptive", "display": "summarized"}
# Documented shape: effort is nested under output_config.
# A top-level `effort` field produces a 400:
# "effort: Extra inputs are not permitted".
assert body["output_config"] == {"effort": "medium"}
assert "effort" not in body
# Extended-thinking contract: temperature=1, no top_p / top_k.
assert body["temperature"] == 1
assert "top_p" not in body
assert "top_k" not in body
def test_adaptive_thinking_maps_xhigh_to_max_on_claude_4_6(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-sonnet-4-6",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = None,
reasoning_effort = "xhigh",
):
pass
await client.close()
_drive(run())
assert captured["body"]["output_config"] == {"effort": "max"}
def test_adaptive_thinking_keeps_max_on_claude_4_6(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-6",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = None,
reasoning_effort = "max",
):
pass
await client.close()
_drive(run())
assert captured["body"]["output_config"] == {"effort": "max"}
def test_adaptive_thinking_keeps_xhigh_on_claude_4_7(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-7",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = None,
reasoning_effort = "xhigh",
):
pass
await client.close()
_drive(run())
body = captured["body"]
assert body["output_config"] == {"effort": "xhigh"}
assert "effort" not in body
def test_manual_thinking_body_uses_budget_tokens_on_4_5(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 1024,
top_k = None,
enable_thinking = None,
reasoning_effort = "high",
):
pass
await client.close()
_drive(run())
body = captured["body"]
assert body["thinking"] == {"type": "enabled", "budget_tokens": 4096}
# max_tokens must be strictly greater than budget_tokens; we shipped 1024
# and budget is 4096, so the wrapper should bump max_tokens.
assert body["max_tokens"] > body["thinking"]["budget_tokens"]
# Manual-thinking path does not use output_config / effort — those are
# the adaptive-mode controls (Claude 4.6 / 4.7).
assert "effort" not in body
assert "output_config" not in body
def test_thinking_delta_wrapped_in_think_tags(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
events = [
{
"type": "content_block_start",
"index": 0,
"content_block": {"type": "thinking", "thinking": "", "signature": ""},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {"type": "thinking_delta", "thinking": "First "},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {"type": "thinking_delta", "thinking": "I plan."},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {"type": "signature_delta", "signature": "abc123"},
},
{"type": "content_block_stop", "index": 0},
{
"type": "content_block_start",
"index": 1,
"content_block": {"type": "text", "text": ""},
},
{
"type": "content_block_delta",
"index": 1,
"delta": {"type": "text_delta", "text": "Answer."},
},
{"type": "content_block_stop", "index": 1},
{"type": "message_delta", "delta": {"stop_reason": "end_turn"}},
{"type": "message_stop"},
]
return httpx.Response(
200,
content = _anthropic_sse(events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
lines = await _collect(
client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-6",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = True,
reasoning_effort = None,
)
)
await client.close()
return lines
lines = _drive(run())
payloads = _payloads_from_lines(lines)
combined = "".join(
p["choices"][0]["delta"].get("content", "")
for p in payloads
if isinstance(p, dict) and p["choices"][0]["delta"]
)
# Reasoning text should be wrapped in <think>...</think>, followed by the
# answer text, and the stream should terminate with [DONE].
assert "<think>First I plan.</think>" in combined
assert combined.endswith("Answer.")
# signature_delta is intentionally dropped — no leaked signature text.
assert "abc123" not in combined
assert "[DONE]" in payloads
def test_thinking_only_turn_closes_tag_without_text_delta(monkeypatch):
"""display=omitted on Claude 4.7 emits a signature_delta and no text.
The <think> open is still triggered by the (synthetic) thinking_delta;
we want content_block_stop to close it cleanly so the tag never leaks
into the next chunk."""
def handler(request: httpx.Request) -> httpx.Response:
events = [
{
"type": "content_block_start",
"index": 0,
"content_block": {"type": "thinking", "thinking": "", "signature": ""},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {"type": "thinking_delta", "thinking": "internal"},
},
{"type": "content_block_stop", "index": 0},
{"type": "message_delta", "delta": {"stop_reason": "end_turn"}},
{"type": "message_stop"},
]
return httpx.Response(
200,
content = _anthropic_sse(events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
lines = await _collect(
client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-7",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = True,
reasoning_effort = None,
)
)
await client.close()
return lines
payloads = _payloads_from_lines(_drive(run()))
combined = "".join(
p["choices"][0]["delta"].get("content", "")
for p in payloads
if isinstance(p, dict) and p["choices"][0]["delta"]
)
assert combined == "<think>internal</think>"

1
studio/backend/tests/test_desktop_auth.py
View file

@ -437,6 +437,7 @@ def test_health_response_reports_desktop_capability_fields(monkeypatch):
inference_router = APIRouter(),
inference_studio_router = APIRouter(),
models_router = APIRouter(),
providers_router = APIRouter(),
training_history_router = APIRouter(),
training_router = APIRouter(),
)

432
studio/backend/tests/test_openai_responses_translation.py Normal file
View file

@ -0,0 +1,432 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Unit tests for the OpenAI `/v1/responses` translation in external_provider.
Covers:
- Request body shape: system messages collapse into `instructions`, user/
assistant messages go into `input`, sampling knobs Responses does not
support (presence_penalty, top_k) are not forwarded.
- SSE translation: `response.output_text.delta` events become OpenAI Chat
Completions chunks, `response.completed` emits a `finish_reason: stop`
chunk, the stream terminates with `data: [DONE]`.
- Image parts in user content are rewritten from Chat Completions
`{type: image_url, image_url: {url}}` into Responses
`{type: input_image, image_url: <url>}`.
"""
import asyncio
import json
import httpx
from core.inference import external_provider as ep_mod
from core.inference.external_provider import ExternalProviderClient
def _drive(coro):
return asyncio.new_event_loop().run_until_complete(coro)
async def _collect(agen):
out = []
async for line in agen:
out.append(line)
return out
def _mock_http_client(monkeypatch, handler):
transport = httpx.MockTransport(handler)
monkeypatch.setattr(ep_mod, "_http_client", httpx.AsyncClient(transport = transport))
def _make_client() -> ExternalProviderClient:
return ExternalProviderClient(
provider_type = "openai",
base_url = "https://api.openai.com/v1",
api_key = "sk-test",
)
def _responses_sse(events: list[dict]) -> bytes:
"""Serialize a list of Responses-API event dicts as an SSE byte stream."""
chunks: list[str] = []
for event in events:
chunks.append(f"event: {event['type']}")
chunks.append(f"data: {json.dumps(event)}")
chunks.append("")
chunks.append("data: [DONE]")
chunks.append("")
return ("\n".join(chunks) + "\n").encode("utf-8")
def test_responses_request_body_uses_input_and_instructions(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["url"] = str(request.url)
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [
{"role": "system", "content": "You are concise."},
{"role": "user", "content": "Hi"},
],
model = "gpt-5.5",
temperature = 0.5,
top_p = 0.9,
max_tokens = 512,
enable_thinking = None,
reasoning_effort = None,
):
pass
await client.close()
_drive(run())
assert captured["url"] == "https://api.openai.com/v1/responses"
body = captured["body"]
assert body["model"] == "gpt-5.5"
assert body["instructions"] == "You are concise."
assert body["input"] == [{"role": "user", "content": "Hi"}]
assert body["max_output_tokens"] == 512
assert body["stream"] is True
# Responses API on reasoning-class models (gpt-5.x / o3 / gpt-4.5 — the
# only OpenAI ids the registry allowlist exposes) rejects these as
# `Unsupported parameter`. Make sure we never silently forward them.
assert "temperature" not in body
assert "top_p" not in body
assert "presence_penalty" not in body
assert "frequency_penalty" not in body
assert "top_k" not in body
assert "messages" not in body
def test_responses_translates_image_parts(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "What is this?"},
{
"type": "image_url",
"image_url": {"url": "data:image/png;base64,AAA"},
},
],
}
],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = None,
):
pass
await client.close()
_drive(run())
parts = captured["body"]["input"][0]["content"]
assert parts[0] == {"type": "input_text", "text": "What is this?"}
assert parts[1] == {
"type": "input_image",
"image_url": "data:image/png;base64,AAA",
}
# No max_output_tokens key when caller passes max_tokens=None.
assert "max_output_tokens" not in captured["body"]
def test_responses_sse_translates_to_chat_completions_chunks(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
events = [
{"type": "response.created"},
{"type": "response.output_text.delta", "delta": "Hello"},
{"type": "response.output_text.delta", "delta": ", world"},
{"type": "response.completed", "response": {}},
]
return httpx.Response(
200,
content = _responses_sse(events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
lines = await _collect(
client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = None,
)
)
await client.close()
return lines
lines = _drive(run())
# Drop empty / non-data lines for assertion clarity.
data_lines = [line for line in lines if line.startswith("data:")]
payloads = []
for line in data_lines:
raw = line[len("data:") :].strip()
if raw == "[DONE]":
payloads.append("[DONE]")
else:
payloads.append(json.loads(raw))
# Two text deltas, one terminal chunk, then [DONE].
assert payloads[0]["choices"][0]["delta"]["content"] == "Hello"
assert payloads[0]["choices"][0]["finish_reason"] is None
assert payloads[1]["choices"][0]["delta"]["content"] == ", world"
assert payloads[2]["choices"][0]["delta"] == {}
assert payloads[2]["choices"][0]["finish_reason"] == "stop"
assert payloads[-1] == "[DONE]"
def test_responses_response_incomplete_maps_to_length_finish_reason(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
events = [
{"type": "response.output_text.delta", "delta": "partial"},
{"type": "response.incomplete", "response": {}},
]
return httpx.Response(
200,
content = _responses_sse(events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
lines = await _collect(
client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4,
enable_thinking = None,
reasoning_effort = None,
)
)
await client.close()
return lines
lines = _drive(run())
finish_reasons = [
json.loads(line[len("data:") :].strip())["choices"][0]["finish_reason"]
for line in lines
if line.startswith("data:")
and line[len("data:") :].strip() not in ("", "[DONE]")
]
assert "length" in finish_reasons
def test_responses_reasoning_effort_included_when_requested(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = "high",
):
pass
await client.close()
_drive(run())
assert captured["body"]["reasoning"] == {"effort": "high", "summary": "auto"}
def test_responses_reasoning_effort_none_omits_summary(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = "none",
):
pass
await client.close()
_drive(run())
assert captured["body"]["reasoning"] == {"effort": "none"}
def test_responses_reasoning_effort_xhigh_passthrough(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = "xhigh",
):
pass
await client.close()
_drive(run())
assert captured["body"]["reasoning"] == {"effort": "xhigh", "summary": "auto"}
def test_responses_enable_thinking_false_maps_to_reasoning_none(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = False,
reasoning_effort = None,
):
pass
await client.close()
_drive(run())
assert captured["body"]["reasoning"] == {"effort": "none"}
def test_responses_reasoning_summary_wrapped_in_think_tags(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
events = [
{
"type": "response.output_item.done",
"item": {
"type": "reasoning",
"summary": [{"type": "summary_text", "text": "plan"}],
},
},
{"type": "response.output_text.delta", "delta": "answer"},
{"type": "response.completed", "response": {}},
]
return httpx.Response(
200,
content = _responses_sse(events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
lines = await _collect(
client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = None,
)
)
await client.close()
return lines
lines = _drive(run())
data_lines = [
line[len("data:") :].strip()
for line in lines
if line.startswith("data:")
and line[len("data:") :].strip() not in ("", "[DONE]")
]
payloads = [json.loads(raw) for raw in data_lines]
combined = "".join(
payload["choices"][0]["delta"].get("content", "")
for payload in payloads
if payload["choices"][0]["delta"]
)
assert "<think>plan</think>answer" in combined

609
studio/backend/tests/test_providers_api.py Normal file
View file

@ -0,0 +1,609 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Integration tests for the external providers API.
Requires a running Unsloth Studio server. Configure via environment variables:
export STUDIO_TEST_URL="http://localhost:8888" # default
export STUDIO_TEST_USER="unsloth" # default
export STUDIO_TEST_PASSWORD="..." # required — see .bootstrap_password
# Provider API keys — any left unset will have their tests automatically skipped
export OPENAI_API_KEY="sk-..."
export MISTRAL_API_KEY="..."
export GOOGLE_API_KEY="..."
export TOGETHER_API_KEY="..."
export FIREWORKS_API_KEY="..."
export PERPLEXITY_API_KEY="..."
Run:
cd studio/backend
pytest tests/test_providers_api.py -v -s
"""
import base64
import json
import os
import pytest
import requests
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import padding
# ── Configuration ─────────────────────────────────────────────────
BASE_URL = os.getenv("STUDIO_TEST_URL", "http://localhost:8000")
USERNAME = os.getenv("STUDIO_TEST_USER", "unsloth")
PASSWORD = os.getenv("STUDIO_TEST_PASSWORD", "")
# These tests require a live Studio server reachable at BASE_URL with a known
# bootstrap password. Skip the whole module when that environment is missing
# (e.g. on CI runners) so pytest discovery does not error out.
pytestmark = pytest.mark.skipif(
not PASSWORD,
reason = "Integration test requires a running Studio server; set STUDIO_TEST_PASSWORD to enable.",
)
# Map provider_type → (env var name, model to use for inference test)
_PROVIDER_CONFIGS: dict[str, tuple[str, str]] = {
"openai": ("OPENAI_API_KEY", "gpt-4o-mini"),
"mistral": ("MISTRAL_API_KEY", "mistral-small-2506"),
"gemini": ("GEMINI_API_KEY", "gemini-3-flash-preview"),
"openrouter": ("OPENROUTER_API_KEY", "openai/gpt-4o-mini"),
"anthropic": ("ANTHROPIC_API_KEY", "claude-haiku-4-5"),
"deepseek": ("DEEPSEEK_API_KEY", "deepseek-chat"),
"huggingface": ("HUGGINGFACE_API_KEY", "meta-llama/Llama-3.3-70B-Instruct"),
"kimi": ("MOONSHOT_API_KEY", "moonshot-v1-8k"),
"qwen": ("DASHSCOPE_API_KEY", "qwen-turbo"),
}
PROVIDER_KEYS: dict[str, str] = {
ptype: os.getenv(env_var, "") for ptype, (env_var, _) in _PROVIDER_CONFIGS.items()
}
EXPECTED_PROVIDER_TYPES = set(_PROVIDER_CONFIGS.keys())
# ── Helpers ────────────────────────────────────────────────────────
def _url(path: str) -> str:
return f"{BASE_URL}/{path.lstrip('/')}"
def _parse_sse_stream(response: requests.Response) -> tuple[str, bool]:
"""
Read a streaming SSE response and return (assembled_text, saw_done).
Each chunk is a JSON object with choices[0].delta.content.
The stream ends with `data: [DONE]`.
"""
reply_parts: list[str] = []
saw_done = False
for raw_line in response.iter_lines():
if isinstance(raw_line, bytes):
raw_line = raw_line.decode("utf-8")
if not raw_line.startswith("data:"):
continue
data = raw_line[len("data:") :].strip()
if data == "[DONE]":
saw_done = True
break
try:
chunk = json.loads(data)
# Handle both error payloads and normal chunks
if "error" in chunk:
raise RuntimeError(f"Provider error in stream: {chunk['error']}")
delta = chunk.get("choices", [{}])[0].get("delta", {})
content = delta.get("content") or ""
if content:
reply_parts.append(content)
except (json.JSONDecodeError, IndexError, KeyError):
pass # skip malformed lines
return "".join(reply_parts), saw_done
# ── Session-scoped fixtures ────────────────────────────────────────
@pytest.fixture(scope = "session")
def auth_headers() -> dict[str, str]:
"""
Log in once per session and return auth headers.
On a fresh Studio install the bootstrap password triggers a forced password
change (must_change_password=True). Any subsequent API call using that token
returns 403 "Password change required". This fixture detects that state,
automatically completes the change-password flow, and re-logs in so all other
tests get a fully usable token.
The new password used during auto-change is:
STUDIO_TEST_NEW_PASSWORD (env var, optional)
or PASSWORD + "-test" (derived default)
On the second run, set STUDIO_TEST_PASSWORD to the new password.
"""
assert PASSWORD, (
"STUDIO_TEST_PASSWORD is not set.\n"
"Run: export STUDIO_TEST_PASSWORD=$(cat studio/backend/.bootstrap_password)"
)
resp = requests.post(
_url("/api/auth/login"),
json = {"username": USERNAME, "password": PASSWORD},
timeout = 10,
)
assert resp.status_code == 200, f"Login failed ({resp.status_code}): {resp.text}"
body = resp.json()
token = body["access_token"]
assert token, "access_token is empty"
if body.get("must_change_password"):
# Bootstrap token is restricted — only /api/auth/change-password works with it.
# Auto-complete the forced change so the rest of the tests get a full token.
new_password = os.getenv("STUDIO_TEST_NEW_PASSWORD") or f"{PASSWORD}-test"
change_resp = requests.post(
_url("/api/auth/change-password"),
headers = {"Authorization": f"Bearer {token}"},
json = {"current_password": PASSWORD, "new_password": new_password},
timeout = 10,
)
assert (
change_resp.status_code == 200
), f"Auto password-change failed ({change_resp.status_code}): {change_resp.text}"
token = change_resp.json()["access_token"]
return {"Authorization": f"Bearer {token}"}
@pytest.fixture(scope = "session")
def public_key_pem(auth_headers: dict[str, str]) -> str:
"""Fetch RSA public key PEM once per session."""
resp = requests.get(
_url("/api/providers/public-key"),
headers = auth_headers,
timeout = 10,
)
assert resp.status_code == 200, f"Public key fetch failed: {resp.text}"
pem = resp.json().get("public_key", "")
assert pem.startswith("-----BEGIN PUBLIC KEY-----"), "Not a valid PEM public key"
return pem
@pytest.fixture(scope = "session")
def vision_image_data_url() -> str:
"""
Download the sloth image once per session and return it as a base64 data URI.
Using a data URI instead of a remote URL ensures every provider receives
the image inline — Gemini's OpenAI-compatible layer does not fetch external
HTTP URLs, so raw image_url links silently produce empty replies for Gemini.
"""
resp = requests.get(_VISION_IMAGE_URL, timeout = 30)
resp.raise_for_status()
content_type = resp.headers.get("Content-Type", "image/jpeg").split(";")[0].strip()
b64 = base64.b64encode(resp.content).decode("utf-8")
return f"data:{content_type};base64,{b64}"
@pytest.fixture(scope = "session")
def encrypt_key(public_key_pem: str):
"""
Return a callable encrypt_key(plaintext: str) -> str (base64 RSA-OAEP ciphertext).
Uses the backend's RSA public key — mirrors what the frontend does.
"""
# Decode PEM → load RSA public key
pem_bytes = public_key_pem.encode("utf-8")
rsa_pub = serialization.load_pem_public_key(pem_bytes)
def _encrypt(plaintext: str) -> str:
ciphertext = rsa_pub.encrypt(
plaintext.encode("utf-8"),
padding.OAEP(
mgf = padding.MGF1(algorithm = hashes.SHA256()),
algorithm = hashes.SHA256(),
label = None,
),
)
return base64.b64encode(ciphertext).decode("utf-8")
return _encrypt
# ── TestAuth ────────────────────────────────────────────────────────
class TestAuth:
def test_login_returns_token(self):
"""POST /api/auth/login returns a non-empty access_token."""
assert PASSWORD, "STUDIO_TEST_PASSWORD not set"
resp = requests.post(
_url("/api/auth/login"),
json = {"username": USERNAME, "password": PASSWORD},
timeout = 10,
)
assert (
resp.status_code == 200
), f"Login failed ({resp.status_code}): {resp.text}"
body = resp.json()
assert body.get("access_token"), "access_token is missing or empty"
assert body.get("token_type") == "bearer"
# ── TestPublicKey ────────────────────────────────────────────────────
class TestPublicKey:
def test_public_key_is_valid_pem(
self, auth_headers: dict[str, str], public_key_pem: str
):
"""GET /api/providers/public-key returns an importable RSA PEM key."""
pem_bytes = public_key_pem.encode("utf-8")
key = serialization.load_pem_public_key(pem_bytes)
key_size = key.key_size # type: ignore[attr-defined]
assert key_size >= 2048, f"Key size too small: {key_size}"
print(f"\n RSA-{key_size} public key OK")
# ── TestRegistry ────────────────────────────────────────────────────
class TestRegistry:
def test_registry_returns_all_providers(self, auth_headers: dict[str, str]):
"""GET /api/providers/registry returns all supported providers."""
resp = requests.get(
_url("/api/providers/registry"),
headers = auth_headers,
timeout = 10,
)
assert resp.status_code == 200, f"Registry failed: {resp.text}"
providers = resp.json()
assert (
len(providers) == 9
), f"Expected 9 providers, got {len(providers)}: {providers}"
print(f"\n {'Provider':<12} {'Base URL'}")
print(f" {'-'*12} {'-'*45}")
for p in providers:
print(f" {p['provider_type']:<12} {p['base_url']}")
def test_registry_has_expected_types(self, auth_headers: dict[str, str]):
"""All expected provider_type values are present in the registry."""
resp = requests.get(
_url("/api/providers/registry"),
headers = auth_headers,
timeout = 10,
)
assert resp.status_code == 200
returned_types = {p["provider_type"] for p in resp.json()}
missing = EXPECTED_PROVIDER_TYPES - returned_types
assert not missing, f"Missing provider types: {missing}"
def test_registry_entries_have_required_fields(self, auth_headers: dict[str, str]):
"""Each registry entry has provider_type, display_name, base_url, default_models."""
resp = requests.get(
_url("/api/providers/registry"), headers = auth_headers, timeout = 10
)
assert resp.status_code == 200
for entry in resp.json():
for field in (
"provider_type",
"display_name",
"base_url",
"default_models",
"model_list_mode",
):
assert field in entry, f"Missing field '{field}' in entry: {entry}"
assert entry["model_list_mode"] in ("remote", "curated")
assert isinstance(entry["default_models"], list)
assert len(entry["default_models"]) > 0
# ── TestProviderCRUD ────────────────────────────────────────────────
class TestProviderCRUD:
"""
These tests run sequentially within the class and share state via class variables.
They create, read, update, and delete a single test provider config.
"""
_created_id: str = ""
def test_create_provider(self, auth_headers: dict[str, str]):
"""POST /api/providers/ creates a provider config and returns 201."""
resp = requests.post(
_url("/api/providers/"),
headers = auth_headers,
json = {"provider_type": "openai", "display_name": "Test OpenAI (pytest)"},
timeout = 10,
)
assert (
resp.status_code == 201
), f"Create failed ({resp.status_code}): {resp.text}"
body = resp.json()
assert body.get("id"), "No id in response"
assert body["provider_type"] == "openai"
assert body["display_name"] == "Test OpenAI (pytest)"
assert body["is_enabled"] is True
TestProviderCRUD._created_id = body["id"]
print(f"\n created id={body['id']}")
def test_list_includes_created(self, auth_headers: dict[str, str]):
"""GET /api/providers/ includes the newly created config."""
assert (
TestProviderCRUD._created_id
), "No created_id (run test_create_provider first)"
resp = requests.get(_url("/api/providers/"), headers = auth_headers, timeout = 10)
assert resp.status_code == 200
ids = [p["id"] for p in resp.json()]
assert (
TestProviderCRUD._created_id in ids
), f"Created id {TestProviderCRUD._created_id!r} not found in list: {ids}"
print(f"\n found id={TestProviderCRUD._created_id} in list of {len(ids)}")
def test_update_display_name(self, auth_headers: dict[str, str]):
"""PUT /api/providers/{id} updates the display_name."""
assert TestProviderCRUD._created_id, "No created_id"
new_name = "Test OpenAI (pytest updated)"
resp = requests.put(
_url(f"/api/providers/{TestProviderCRUD._created_id}"),
headers = auth_headers,
json = {"display_name": new_name},
timeout = 10,
)
assert (
resp.status_code == 200
), f"Update failed ({resp.status_code}): {resp.text}"
assert resp.json()["display_name"] == new_name
print(f"\n updated display_name to '{new_name}'")
def test_delete_provider(self, auth_headers: dict[str, str]):
"""DELETE /api/providers/{id} removes the config (204) and it's gone from list."""
assert TestProviderCRUD._created_id, "No created_id"
resp = requests.delete(
_url(f"/api/providers/{TestProviderCRUD._created_id}"),
headers = auth_headers,
timeout = 10,
)
assert (
resp.status_code == 204
), f"Delete failed ({resp.status_code}): {resp.text}"
# Confirm gone from list
list_resp = requests.get(
_url("/api/providers/"), headers = auth_headers, timeout = 10
)
ids = [p["id"] for p in list_resp.json()]
assert TestProviderCRUD._created_id not in ids, "Deleted provider still in list"
print(f"\n deleted id={TestProviderCRUD._created_id} confirmed gone")
# ── TestProviderInference ────────────────────────────────────────────
# Build parametrize list: (provider_type, model, api_key) for configured providers only
_INFERENCE_PARAMS = [
pytest.param(
ptype,
model,
PROVIDER_KEYS.get(ptype, ""),
id = ptype,
marks = pytest.mark.skipif(
not PROVIDER_KEYS.get(ptype, ""),
reason = f"no {env_var} set",
),
)
for ptype, (env_var, model) in _PROVIDER_CONFIGS.items()
]
class TestProviderInference:
"""
Live inference tests — one parametrized set per provider.
Each test is automatically skipped when the provider's API key env var is not set.
"""
@pytest.mark.parametrize("provider_type,model,api_key", _INFERENCE_PARAMS)
def test_connection(
self,
auth_headers: dict[str, str],
encrypt_key,
provider_type: str,
model: str,
api_key: str,
):
"""POST /api/providers/test → success: true."""
encrypted = encrypt_key(api_key)
resp = requests.post(
_url("/api/providers/test"),
headers = auth_headers,
json = {"provider_type": provider_type, "encrypted_api_key": encrypted},
timeout = 30,
)
assert (
resp.status_code == 200
), f"Request failed ({resp.status_code}): {resp.text}"
body = resp.json()
assert (
body["success"] is True
), f"Connection test failed for {provider_type}: {body.get('message')}"
print(f"\n [{provider_type}] connection OK — {body['message']}")
@pytest.mark.parametrize("provider_type,model,api_key", _INFERENCE_PARAMS)
def test_list_models(
self,
auth_headers: dict[str, str],
encrypt_key,
provider_type: str,
model: str,
api_key: str,
):
"""POST /api/providers/models → non-empty list, print first 3."""
encrypted = encrypt_key(api_key)
resp = requests.post(
_url("/api/providers/models"),
headers = auth_headers,
json = {"provider_type": provider_type, "encrypted_api_key": encrypted},
timeout = 30,
)
assert (
resp.status_code == 200
), f"Request failed ({resp.status_code}): {resp.text}"
models = resp.json()
assert isinstance(models, list), f"Expected list, got {type(models)}"
assert len(models) > 0, f"No models returned for {provider_type}"
preview = [m["id"] for m in models[:3]]
print(f"\n [{provider_type}] {len(models)} models — first 3: {preview}")
@pytest.mark.parametrize("provider_type,model,api_key", _INFERENCE_PARAMS)
def test_chat_inference(
self,
auth_headers: dict[str, str],
encrypt_key,
provider_type: str,
model: str,
api_key: str,
):
"""POST /v1/chat/completions with provider fields → streamed reply."""
encrypted = encrypt_key(api_key)
payload = {
"messages": [{"role": "user", "content": "Say hello in one sentence."}],
"stream": True,
"temperature": 0.7,
"max_tokens": 64,
"provider_type": provider_type,
"external_model": model,
"encrypted_api_key": encrypted,
}
with requests.post(
_url("/v1/chat/completions"),
headers = {**auth_headers, "Content-Type": "application/json"},
json = payload,
stream = True,
timeout = 60,
) as resp:
assert (
resp.status_code == 200
), f"Chat completions failed ({resp.status_code}): {resp.text[:500]}"
reply, saw_done = _parse_sse_stream(resp)
assert reply.strip(), f"Empty reply from {provider_type}/{model}"
assert saw_done, f"Stream did not end with [DONE] for {provider_type}/{model}"
print(f'\n [{provider_type}/{model}] reply: "{reply.strip()}"')
# ── TestVisionInference ─────────────────────────────────────────────
# Sloth photo — used to test vision routing across providers
_VISION_IMAGE_URL = (
"https://www.travelexcellence.com/images/where-to-see-sloths-in-costa-rica.jpg"
)
_VISION_PARAMS = [
pytest.param(
ptype,
model,
PROVIDER_KEYS.get(ptype, ""),
id = ptype,
marks = pytest.mark.skipif(
not PROVIDER_KEYS.get(ptype, ""),
reason = f"no key for {ptype}",
),
)
for ptype, (_, model) in _PROVIDER_CONFIGS.items()
if ptype in {"openai", "mistral", "gemini", "anthropic", "openrouter"}
]
class TestVisionInference:
"""
Send a 1×1 white PNG alongside a text question to each vision-capable provider.
Verifies that image content parts survive the proxy and the provider replies.
"""
@pytest.mark.parametrize("provider_type,model,api_key", _VISION_PARAMS)
def test_vision_chat_inference(
self,
auth_headers: dict[str, str],
encrypt_key,
vision_image_data_url: str,
provider_type: str,
model: str,
api_key: str,
):
"""Image URL + text message → non-empty streamed reply."""
encrypted = encrypt_key(api_key)
payload = {
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Which animal is in this image? Reply in one word.",
},
{
"type": "image_url",
"image_url": {"url": vision_image_data_url},
},
],
}
],
"stream": True,
"max_tokens": 215,
"provider_type": provider_type,
"external_model": model,
"encrypted_api_key": encrypted,
}
with requests.post(
_url("/v1/chat/completions"),
headers = {**auth_headers, "Content-Type": "application/json"},
json = payload,
stream = True,
timeout = 60,
) as resp:
assert (
resp.status_code == 200
), f"Vision request failed ({resp.status_code}): {resp.text[:300]}"
reply, saw_done = _parse_sse_stream(resp)
assert reply.strip(), f"Empty reply from {provider_type}/{model}"
assert saw_done, f"Stream did not end with [DONE] for {provider_type}/{model}"
print(f"\n [{provider_type}/{model}] vision reply: {reply.strip()!r}")
# ── TestLocalInferenceUnaffected ────────────────────────────────────
class TestLocalInferenceUnaffected:
def test_chat_without_provider(self, auth_headers: dict[str, str]):
"""
POST /v1/chat/completions without provider fields must not return 422 or 500.
200 = a local model is loaded and responded.
503 = no model loaded (expected in test environment — that's fine).
Any other 4xx/5xx (except 503) = regression in request handling.
"""
resp = requests.post(
_url("/v1/chat/completions"),
headers = {**auth_headers, "Content-Type": "application/json"},
json = {
"messages": [{"role": "user", "content": "Hello"}],
"stream": False,
},
timeout = 15,
)
allowed = {200, 400, 503}
assert resp.status_code in allowed, (
f"Unexpected status {resp.status_code} for local inference path: {resp.text[:300]}\n"
f"This likely means the provider fields broke the base request schema."
)
status_label = (
"local model responded"
if resp.status_code == 200
else "no model loaded (expected)"
)
print(f"\n status={resp.status_code} ({status_label}) — local path unaffected")

21
studio/frontend/package-lock.json generated
View file

@ -58,6 +58,7 @@
"motion": "^12.34.0",
"next": "^16.1.6",
"next-themes": "^0.4.6",
"node-forge": "^1.4.0",
"radix-ui": "^1.4.3",
"react": "^19.2.4",
"react-day-picker": "^9.13.2",
@ -80,6 +81,7 @@
"@eslint/js": "^9.39.1",
"@types/js-yaml": "^4.0.9",
"@types/node": "^25.5.2",
"@types/node-forge": "^1.3.14",
"@types/react": "^19.2.5",
"@types/react-dom": "^19.2.3",
"@vitejs/plugin-react": "^6.0.1",
@ -7377,6 +7379,16 @@
"undici-types": "~7.19.0"
}
},
"node_modules/@types/node-forge": {
"version": "1.3.14",
"resolved": "https://registry.npmjs.org/@types/node-forge/-/node-forge-1.3.14.tgz",
"integrity": "sha512-mhVF2BnD4BO+jtOp7z1CdzaK4mbuK0LLQYAvdOLqHTavxFNq4zA1EmYkpnFjP8HOUzedfQkRnp0E2ulSAYSzAw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/react": {
"version": "19.2.14",
"resolved": "https://registry.npmjs.org/@types/react/-/react-19.2.14.tgz",
@ -13285,6 +13297,15 @@
"url": "https://opencollective.com/node-fetch"
}
},
"node_modules/node-forge": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/node-forge/-/node-forge-1.4.0.tgz",
"integrity": "sha512-LarFH0+6VfriEhqMMcLX2F7SwSXeWwnEAJEsYm5QKWchiVYVvJyV9v7UDvUv+w5HO23ZpQTXDv/GxdDdMyOuoQ==",
"license": "(BSD-3-Clause OR GPL-2.0)",
"engines": {
"node": ">= 6.13.0"
}
},
"node_modules/node-releases": {
"version": "2.0.38",
"resolved": "https://registry.npmjs.org/node-releases/-/node-releases-2.0.38.tgz",

2
studio/frontend/package.json
View file

@ -66,6 +66,7 @@
"motion": "^12.34.0",
"next": "^16.1.6",
"next-themes": "^0.4.6",
"node-forge": "^1.4.0",
"radix-ui": "^1.4.3",
"react": "^19.2.4",
"react-day-picker": "^9.13.2",
@ -92,6 +93,7 @@
"@biomejs/biome": "^1.9.4",
"@eslint/js": "^9.39.1",
"@types/js-yaml": "^4.0.9",
"@types/node-forge": "^1.3.14",
"@types/node": "^25.5.2",
"@types/react": "^19.2.5",
"@types/react-dom": "^19.2.3",

6
studio/frontend/public/provider-logos/anthropic.svg Normal file
View file

@ -0,0 +1,6 @@
<svg xmlns="http://www.w3.org/2000/svg" width="900" height="900" viewBox="0 0 900 900">
<g>
<path d="M 222.16 664.50 L 212.35 691.50 L 206.42 691.53 C163.39,691.75 102.00,690.80 102.00,689.92 C102.00,689.34 103.32,685.63 104.94,681.68 C106.56,677.73 124.74,632.20 145.34,580.50 C165.94,528.80 205.07,430.70 232.29,362.50 C259.51,294.30 284.65,231.20 288.14,222.28 L 294.50 206.05 L 404.68 206.00 L 405.94 208.75 C407.23,211.56 413.90,227.98 437.50,286.50 C444.82,304.65 453.36,325.80 456.49,333.50 C459.61,341.20 475.70,381.02 492.24,422.00 C508.78,462.98 528.13,510.90 535.25,528.50 C542.36,546.10 553.72,574.22 560.48,591.00 C567.25,607.78 575.81,628.92 579.50,638.00 C585.05,651.65 598.92,686.11 600.73,690.76 C601.12,691.76 590.44,691.97 547.95,691.76 L 494.69 691.50 L 489.31 678.00 C486.35,670.58 481.94,659.33 479.52,653.00 C477.10,646.67 471.91,633.17 468.00,623.00 C464.08,612.83 459.67,601.24 458.19,597.25 L 455.51 590.00 L 249.31 590.00 L 245.56 600.25 C237.09,623.44 231.46,638.87 222.16,664.50 ZM 798.00 691.05 C798.00,691.64 777.31,692.00 743.50,692.00 C703.78,692.00 689.00,691.69 689.00,690.87 C689.00,690.26 685.87,681.82 682.04,672.12 C678.21,662.43 670.76,643.47 665.49,630.00 C660.21,616.53 652.36,596.50 648.03,585.50 C638.84,562.16 624.19,524.76 620.02,514.00 C615.63,502.69 600.65,464.57 593.49,446.50 C590.00,437.70 582.57,418.80 576.98,404.50 C562.77,368.16 549.76,334.97 543.27,318.50 C540.24,310.80 536.29,300.67 534.50,296.00 C532.71,291.33 526.45,275.35 520.59,260.50 C514.73,245.65 507.70,227.83 504.97,220.91 C502.23,213.98 500.00,207.85 500.00,207.29 C500.00,204.88 522.83,204.39 576.12,205.66 L 603.75 206.32 L 624.52 257.91 C635.94,286.28 646.96,313.77 649.01,319.00 C651.05,324.23 663.76,355.95 677.26,389.50 C699.90,445.80 737.77,540.07 780.62,646.80 C790.18,670.61 798.00,690.53 798.00,691.05 ZM 285.33 500.42 C285.62,501.18 305.07,501.42 351.82,501.24 L 417.90 500.97 L 415.37 494.24 C413.98,490.53 410.81,482.33 408.32,476.00 C405.83,469.67 403.42,463.38 402.98,462.00 C402.53,460.62 398.95,451.17 395.03,441.00 C391.11,430.83 384.57,413.73 380.50,403.00 C376.42,392.27 369.89,375.17 365.97,365.00 C362.05,354.83 357.46,342.77 355.77,338.20 C354.08,333.64 352.38,330.26 352.00,330.70 C351.61,331.14 347.21,341.85 342.21,354.50 C333.07,377.64 317.55,416.89 296.36,470.42 C290.06,486.32 285.10,499.82 285.33,500.42 Z" fill="rgb(37,37,36)"/>
<path d="M 0.00 450.00 L 0.00 0.00 L 450.00 0.00 L 900.00 0.00 L 900.00 450.00 L 900.00 900.00 L 450.00 900.00 L 0.00 900.00 L 0.00 450.00 ZM 222.16 664.50 C231.46,638.87 237.09,623.44 245.56,600.25 L 249.31 590.00 L 352.41 590.00 L 455.51 590.00 L 458.19 597.25 C459.67,601.24 464.08,612.83 468.00,623.00 C471.91,633.17 477.10,646.67 479.52,653.00 C481.94,659.33 486.35,670.58 489.31,678.00 L 494.69 691.50 L 547.95 691.76 C590.44,691.97 601.12,691.76 600.73,690.76 C598.92,686.11 585.05,651.65 579.50,638.00 C575.81,628.92 567.25,607.78 560.48,591.00 C553.72,574.22 542.36,546.10 535.25,528.50 C528.13,510.90 508.78,462.98 492.24,422.00 C475.70,381.02 459.61,341.20 456.49,333.50 C453.36,325.80 444.82,304.65 437.50,286.50 C413.90,227.98 407.23,211.56 405.94,208.75 L 404.68 206.00 L 349.59 206.03 L 294.50 206.05 L 288.14 222.28 C284.65,231.20 259.51,294.30 232.29,362.50 C205.07,430.70 165.94,528.80 145.34,580.50 C124.74,632.20 106.56,677.73 104.94,681.68 C103.32,685.63 102.00,689.34 102.00,689.92 C102.00,690.80 163.39,691.75 206.42,691.53 L 212.35 691.50 L 222.16 664.50 ZM 798.00 691.05 C798.00,690.53 790.18,670.61 780.62,646.80 C737.77,540.07 699.90,445.80 677.26,389.50 C663.76,355.95 651.05,324.23 649.01,319.00 C646.96,313.77 635.94,286.28 624.52,257.91 L 603.75 206.32 L 576.12 205.66 C522.83,204.39 500.00,204.88 500.00,207.29 C500.00,207.85 502.23,213.98 504.97,220.91 C507.70,227.83 514.73,245.65 520.59,260.50 C526.45,275.35 532.71,291.33 534.50,296.00 C536.29,300.67 540.24,310.80 543.27,318.50 C549.76,334.97 562.77,368.16 576.98,404.50 C582.57,418.80 590.00,437.70 593.49,446.50 C600.65,464.57 615.63,502.69 620.02,514.00 C624.19,524.76 638.84,562.16 648.03,585.50 C652.36,596.50 660.21,616.53 665.49,630.00 C670.76,643.47 678.21,662.43 682.04,672.12 C685.87,681.82 689.00,690.26 689.00,690.87 C689.00,691.69 703.78,692.00 743.50,692.00 C777.31,692.00 798.00,691.64 798.00,691.05 ZM 285.33 500.42 C285.10,499.82 290.06,486.32 296.36,470.42 C317.55,416.89 333.07,377.64 342.21,354.50 C347.21,341.85 351.61,331.14 352.00,330.70 C352.38,330.26 354.08,333.64 355.77,338.20 C357.46,342.77 362.05,354.83 365.97,365.00 C369.89,375.17 376.42,392.27 380.50,403.00 C384.57,413.73 391.11,430.83 395.03,441.00 C398.95,451.17 402.53,460.62 402.98,462.00 C403.42,463.38 405.83,469.67 408.32,476.00 C410.81,482.33 413.98,490.53 415.37,494.24 L 417.90 500.97 L 351.82 501.24 C305.07,501.42 285.62,501.18 285.33,500.42 Z" fill="rgb(209,155,118)"/>
</g>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 4.7 KiB

14
studio/frontend/public/provider-logos/deepseek.svg Normal file
View file

@ -0,0 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg id="Layer_2" data-name="Layer 2" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 377.1 277.86">
<defs>
<style>
.cls-1 {
fill: #4d6bfe;
stroke-width: 0px;
}
</style>
</defs>
<g id="Layer_1-2" data-name="Layer 1">
<path id="path" class="cls-1" d="M373.15,23.32c-4-1.95-5.72,1.77-8.06,3.66-.79.62-1.47,1.43-2.14,2.14-5.85,6.26-12.67,10.36-21.57,9.86-13.04-.71-24.16,3.38-33.99,13.37-2.09-12.31-9.04-19.66-19.6-24.38-5.54-2.45-11.13-4.9-14.99-10.23-2.71-3.78-3.44-8-4.81-12.16-.85-2.51-1.72-5.09-4.6-5.52-3.13-.5-4.36,2.14-5.58,4.34-4.93,8.99-6.82,18.92-6.65,28.97.43,22.58,9.97,40.56,28.89,53.37,2.16,1.46,2.71,2.95,2.03,5.09-1.29,4.4-2.82,8.68-4.19,13.09-.85,2.82-2.14,3.44-5.15,2.2-10.39-4.34-19.37-10.76-27.29-18.55-13.46-13.02-25.63-27.41-40.81-38.67-3.57-2.64-7.12-5.09-10.81-7.41-15.49-15.07,2.03-27.45,6.08-28.9,4.25-1.52,1.47-6.79-12.23-6.73-13.69.06-26.24,4.65-42.21,10.76-2.34.93-4.79,1.61-7.32,2.14-14.5-2.73-29.55-3.35-45.29-1.58-29.62,3.32-53.28,17.34-70.68,41.28C1.29,88.2-3.63,120.88,2.39,155c6.33,35.91,24.64,65.68,52.8,88.94,29.18,24.1,62.8,35.91,101.15,33.65,23.29-1.33,49.23-4.46,78.48-29.24,7.38,3.66,15.12,5.12,27.97,6.23,9.89.93,19.41-.5,26.79-2.02,11.55-2.45,10.75-13.15,6.58-15.13-33.87-15.78-26.44-9.36-33.2-14.54,17.21-20.41,43.15-41.59,53.3-110.19.79-5.46.11-8.87,0-13.3-.06-2.67.54-3.72,3.61-4.03,8.48-.96,16.72-3.29,24.28-7.47,21.94-12,30.78-31.69,32.87-55.33.31-3.6-.06-7.35-3.86-9.24ZM181.96,235.97c-32.83-25.83-48.74-34.33-55.31-33.96-6.14.34-5.04,7.38-3.69,11.97,1.41,4.53,3.26,7.66,5.85,11.63,1.78,2.64,3.01,6.57-1.78,9.49-10.57,6.58-28.95-2.2-29.82-2.64-21.38-12.59-39.26-29.24-51.87-52.01-12.16-21.92-19.23-45.43-20.39-70.52-.31-6.08,1.47-8.22,7.49-9.3,7.92-1.46,16.11-1.77,24.03-.62,33.49,4.9,62.01,19.91,85.9,43.63,13.65,13.55,23.97,29.71,34.61,45.49,11.3,16.78,23.48,32.75,38.97,45.84,5.46,4.59,9.83,8.09,14,10.67-12.59,1.4-33.62,1.71-47.99-9.68ZM197.69,134.65c0-2.7,2.15-4.84,4.87-4.84.6,0,1.16.12,1.66.31.67.25,1.29.62,1.77,1.18.87.84,1.36,2.08,1.36,3.35,0,2.7-2.15,4.84-4.85,4.84s-4.81-2.14-4.81-4.84ZM246.55,159.77c-3.13,1.27-6.26,2.39-9.27,2.51-4.67.22-9.77-1.68-12.55-4-4.3-3.6-7.36-5.61-8.67-11.94-.54-2.7-.23-6.85.25-9.24,1.12-5.15-.12-8.44-3.74-11.44-2.96-2.45-6.7-3.1-10.82-3.1-1.54,0-2.95-.68-4-1.24-1.72-.87-3.13-3.01-1.78-5.64.43-.84,2.53-2.92,3.02-3.29,5.58-3.19,12.03-2.14,18,.25,5.54,2.26,9.71,6.42,15.72,12.28,6.16,7.1,7.26,9.09,10.76,14.39,2.76,4.19,5.29,8.47,7.01,13.37,1.04,3.04-.31,5.55-3.94,7.1Z"/>
</g>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 2.5 KiB

72
studio/frontend/public/provider-logos/gemini.svg Normal file
View file

File diff suppressed because one or more lines are too long
Side by side

After

Width:  |  Height:  |  Size: 3 MiB

8
studio/frontend/public/provider-logos/huggingface.svg Normal file
View file

File diff suppressed because one or more lines are too long
Side by side

After

Width:  |  Height:  |  Size: 34 KiB

BIN
studio/frontend/public/provider-logos/kimi.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 15 KiB

19
studio/frontend/public/provider-logos/misc/meta.svg Normal file
View file

@ -0,0 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cc="http://creativecommons.org/ns#" width="287.56" height="191">
<desc>Logo of Meta Platforms -- Graphic created by Detmar Owen</desc>
<defs>
<linearGradient id="Grad_Logo1" x1="61" y1="117" x2="259" y2="127" gradientUnits="userSpaceOnUse">
<stop style="stop-color:#0064e1" offset="0"/>
<stop style="stop-color:#0064e1" offset="0.4"/>
<stop style="stop-color:#0073ee" offset="0.83"/>
<stop style="stop-color:#0082fb" offset="1"/>
</linearGradient>
<linearGradient id="Grad_Logo2" x1="45" y1="139" x2="45" y2="66" gradientUnits="userSpaceOnUse">
<stop style="stop-color:#0082fb" offset="0"/>
<stop style="stop-color:#0064e0" offset="1"/>
</linearGradient>
</defs>
<path id="Logo0" style="fill:#0081fb" d="m31.06,125.96c0,10.98 2.41,19.41 5.56,24.51 4.13,6.68 10.29,9.51 16.57,9.51 8.1,0 15.51-2.01 29.79-21.76 11.44-15.83 24.92-38.05 33.99-51.98l15.36-23.6c10.67-16.39 23.02-34.61 37.18-46.96 11.56-10.08 24.03-15.68 36.58-15.68 21.07,0 41.14,12.21 56.5,35.11 16.81,25.08 24.97,56.67 24.97,89.27 0,19.38-3.82,33.62-10.32,44.87-6.28,10.88-18.52,21.75-39.11,21.75l0-31.02c17.63,0 22.03-16.2 22.03-34.74 0-26.42-6.16-55.74-19.73-76.69-9.63-14.86-22.11-23.94-35.84-23.94-14.85,0-26.8,11.2-40.23,31.17-7.14,10.61-14.47,23.54-22.7,38.13l-9.06,16.05c-18.2,32.27-22.81,39.62-31.91,51.75-15.95,21.24-29.57,29.29-47.5,29.29-21.27,0-34.72-9.21-43.05-23.09-6.8-11.31-10.14-26.15-10.14-43.06z"/>
<path id="Logo1" style="fill:url(#Grad_Logo1)" d="m24.49,37.3c14.24-21.95 34.79-37.3 58.36-37.3 13.65,0 27.22,4.04 41.39,15.61 15.5,12.65 32.02,33.48 52.63,67.81l7.39,12.32c17.84,29.72 27.99,45.01 33.93,52.22 7.64,9.26 12.99,12.02 19.94,12.02 17.63,0 22.03-16.2 22.03-34.74l27.4-.86c0,19.38-3.82,33.62-10.32,44.87-6.28,10.88-18.52,21.75-39.11,21.75-12.8,0-24.14-2.78-36.68-14.61-9.64-9.08-20.91-25.21-29.58-39.71l-25.79-43.08c-12.94-21.62-24.81-37.74-31.68-45.04-7.39-7.85-16.89-17.33-32.05-17.33-12.27,0-22.69,8.61-31.41,21.78z"/>
<path id="Logo2" style="fill:url(#Grad_Logo2)" d="m82.35,31.23c-12.27,0-22.69,8.61-31.41,21.78-12.33,18.61-19.88,46.33-19.88,72.95 0,10.98 2.41,19.41 5.56,24.51l-26.48,17.44c-6.8-11.31-10.14-26.15-10.14-43.06 0-30.75 8.44-62.8 24.49-87.55 14.24-21.95 34.79-37.3 58.36-37.3z"/>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 2.3 KiB

1
studio/frontend/public/provider-logos/misc/microsoft.svg Normal file
View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 23 23"><path fill="#f3f3f3" d="M0 0h23v23H0z"/><path fill="#f35325" d="M1 1h10v10H1z"/><path fill="#81bc06" d="M12 1h10v10H12z"/><path fill="#05a6f0" d="M1 12h10v10H1z"/><path fill="#ffba08" d="M12 12h10v10H12z"/></svg>
Side by side

After

Width:  |  Height:  |  Size: 272 B

BIN
studio/frontend/public/provider-logos/misc/minimax.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 8.5 KiB

1
studio/frontend/public/provider-logos/misc/nvidia.svg Normal file
View file

@ -0,0 +1 @@
<svg viewBox="0 0 271.7 179.7" xmlns="http://www.w3.org/2000/svg" width="2500" height="1653"><path d="M101.3 53.6V37.4c1.6-.1 3.2-.2 4.8-.2 44.4-1.4 73.5 38.2 73.5 38.2S148.2 119 114.5 119c-4.5 0-8.9-.7-13.1-2.1V67.7c17.3 2.1 20.8 9.7 31.1 27l23.1-19.4s-16.9-22.1-45.3-22.1c-3-.1-6 .1-9 .4m0-53.6v24.2l4.8-.3c61.7-2.1 102 50.6 102 50.6s-46.2 56.2-94.3 56.2c-4.2 0-8.3-.4-12.4-1.1v15c3.4.4 6.9.7 10.3.7 44.8 0 77.2-22.9 108.6-49.9 5.2 4.2 26.5 14.3 30.9 18.7-29.8 25-99.3 45.1-138.7 45.1-3.8 0-7.4-.2-11-.6v21.1h170.2V0H101.3zm0 116.9v12.8c-41.4-7.4-52.9-50.5-52.9-50.5s19.9-22 52.9-25.6v14h-.1c-17.3-2.1-30.9 14.1-30.9 14.1s7.7 27.3 31 35.2M27.8 77.4s24.5-36.2 73.6-40V24.2C47 28.6 0 74.6 0 74.6s26.6 77 101.3 84v-14c-54.8-6.8-73.5-67.2-73.5-67.2z" fill="#76b900"/></svg>
Side by side

After

Width:  |  Height:  |  Size: 771 B

BIN
studio/frontend/public/provider-logos/misc/perplexity.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 6.5 KiB

1
studio/frontend/public/provider-logos/misc/xai.svg Normal file
View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 466.04 516.93"><polygon points="0.12 182.71 234.14 516.92 338.15 516.92 104.13 182.71 0.12 182.71"/><polygon points="0 516.92 104.08 516.92 156.08 442.67 104.04 368.34 0 516.92"/><polygon points="466.04 0 361.96 0 182.1 256.86 234.15 331.18 466.04 0"/><polygon points="380.78 516.92 466.04 516.92 466.04 37.16 380.78 158.92 380.78 516.92"/></svg>
Side by side

After

Width:  |  Height:  |  Size: 399 B

215
studio/frontend/public/provider-logos/misc/z-ai.svg Normal file
View file

@ -0,0 +1,215 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" x="0px" y="0px" viewBox="0.0 0.0 30.0 30.0" style="enable-background:new 0 0 30 30;" xml:space="preserve" width="316.22776601683796" height="316.22776601683796">
<style type="text/css">
.st0{opacity:0.3;fill:#E2E4E7;}
.st1{opacity:0.8;fill:#E2E4E7;stroke:#FFFFFF;stroke-width:5;stroke-miterlimit:10;}
.st2{fill:url(#SVGID_1_);}
.st3{fill:none;stroke:#E0E4E9;stroke-width:0.25;stroke-miterlimit:10;}
.st4{fill:none;}
.st5{fill:#9DA1A5;}
.st6{fill-rule:evenodd;clip-rule:evenodd;fill:none;}
.st7{fill-rule:evenodd;clip-rule:evenodd;fill:#DFE2E7;}
.st8{fill-rule:evenodd;clip-rule:evenodd;fill:#CDD4DA;}
.st9{fill-rule:evenodd;clip-rule:evenodd;fill:#B3BCC7;}
.st10{fill-rule:evenodd;clip-rule:evenodd;fill:#9DAAB7;}
.st11{fill-rule:evenodd;clip-rule:evenodd;fill:#8698A8;}
.st12{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_2_);}
.st13{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_3_);}
.st14{fill:#1F63EC;}
.st15{fill:#2D2D2D;}
.st16{fill:none;stroke:#E0E4E9;stroke-width:0.5;stroke-miterlimit:10;}
.st17{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_4_);}
.st18{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_5_);}
.st19{fill:none;stroke:#677380;stroke-width:0.5;stroke-miterlimit:10;}
.st20{fill:none;stroke:url(#SVGID_6_);stroke-width:2;stroke-miterlimit:10;}
.st21{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_7_);}
.st22{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_8_);}
.st23{fill:#FFFFFF;}
.st24{fill-rule:evenodd;clip-rule:evenodd;fill:#2D2D2D;}
.st25{clip-path:url(#SVGID_10_);}
.st26{clip-path:url(#SVGID_12_);}
.st27{fill:url(#SVGID_13_);}
.st28{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_14_);}
.st29{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_15_);}
.st30{clip-path:url(#SVGID_17_);}
.st31{clip-path:url(#SVGID_19_);}
.st32{fill:url(#SVGID_20_);}
.st33{fill:none;stroke:url(#SVGID_21_);stroke-width:2;stroke-miterlimit:10;}
.st34{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_22_);}
.st35{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_23_);}
.st36{clip-path:url(#SVGID_25_);}
.st37{clip-path:url(#SVGID_27_);}
.st38{fill:url(#SVGID_28_);}
.st39{clip-path:url(#SVGID_30_);}
.st40{clip-path:url(#SVGID_32_);}
.st41{fill:url(#SVGID_33_);}
.st42{fill-rule:evenodd;clip-rule:evenodd;fill:#126EF6;}
.st43{fill-rule:evenodd;clip-rule:evenodd;fill:#FFFFFF;}
.st44{clip-path:url(#SVGID_35_);}
.st45{clip-path:url(#SVGID_37_);}
.st46{fill:url(#SVGID_38_);}
.st47{fill-rule:evenodd;clip-rule:evenodd;fill:#9DA1A5;}
.st48{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_39_);}
.st49{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_40_);}
.st50{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_41_);}
.st51{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_42_);}
.st52{fill:none;stroke:url(#SVGID_43_);stroke-width:2;stroke-miterlimit:10;}
.st53{fill-rule:evenodd;clip-rule:evenodd;fill:none;stroke:#E0E4E9;stroke-width:0.5;stroke-miterlimit:10;}
.st54{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_44_);}
.st55{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_45_);}
.st56{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_46_);}
.st57{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_47_);}
.st58{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_48_);}
.st59{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_49_);}
.st60{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_50_);}
.st61{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_51_);}
.st62{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_52_);}
.st63{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_53_);}
.st64{clip-path:url(#SVGID_55_);}
.st65{clip-path:url(#SVGID_57_);}
.st66{fill:url(#SVGID_58_);}
.st67{clip-path:url(#SVGID_60_);}
.st68{clip-path:url(#SVGID_62_);}
.st69{fill:url(#SVGID_63_);}
.st70{fill:none;stroke:url(#SVGID_64_);stroke-width:2;stroke-miterlimit:10;}
.st71{clip-path:url(#SVGID_66_);}
.st72{clip-path:url(#SVGID_68_);}
.st73{fill:url(#SVGID_69_);}
.st74{clip-path:url(#SVGID_71_);}
.st75{clip-path:url(#SVGID_73_);}
.st76{fill:url(#SVGID_74_);}
.st77{clip-path:url(#SVGID_76_);}
.st78{clip-path:url(#SVGID_78_);}
.st79{fill:url(#SVGID_79_);}
.st80{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_80_);}
.st81{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_81_);}
.st82{clip-path:url(#SVGID_83_);}
.st83{clip-path:url(#SVGID_85_);}
.st84{fill:url(#SVGID_86_);}
.st85{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_87_);}
.st86{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_88_);}
.st87{clip-path:url(#SVGID_90_);}
.st88{clip-path:url(#SVGID_92_);}
.st89{fill:url(#SVGID_93_);}
.st90{fill:none;stroke:url(#SVGID_94_);stroke-width:2;stroke-miterlimit:10;}
.st91{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_95_);}
.st92{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_96_);}
.st93{clip-path:url(#SVGID_98_);}
.st94{clip-path:url(#SVGID_100_);}
.st95{fill:url(#SVGID_101_);}
.st96{clip-path:url(#SVGID_103_);}
.st97{clip-path:url(#SVGID_105_);}
.st98{fill:url(#SVGID_106_);}
.st99{clip-path:url(#SVGID_108_);}
.st100{clip-path:url(#SVGID_110_);}
.st101{fill:url(#SVGID_111_);}
.st102{fill:#FFFFFF;stroke:#B3BCC7;stroke-width:0.275;stroke-miterlimit:10;}
.st103{clip-path:url(#SVGID_113_);}
.st104{fill:#FDD138;}
.st105{fill:#FCA62F;}
.st106{fill:#FB7927;}
.st107{fill:#F44B22;}
.st108{fill:#D81915;}
.st109{fill:#2D2D2D;stroke:#FFFFFF;stroke-width:0.3354;stroke-miterlimit:10;}
.st110{fill:none;stroke:#65727F;stroke-width:2;stroke-miterlimit:10;}
.st111{fill:none;stroke:#65727F;stroke-width:0.75;stroke-miterlimit:10;}
.st112{fill:url(#SVGID_114_);}
.st113{fill:#D06C50;}
.st114{fill:#2D2D2D;stroke:#B3BCC7;stroke-width:0.275;stroke-miterlimit:10;}
.st115{opacity:0.2;}
.st116{fill:none;stroke:#677380;stroke-width:0.3564;stroke-miterlimit:10;}
.st117{fill:none;stroke:#677380;stroke-width:0.3564;stroke-miterlimit:10;stroke-dasharray:1.0212,1.0212;}
.st118{fill:none;stroke:#677380;stroke-width:0.3564;stroke-miterlimit:10;stroke-dasharray:1.0205,1.0205;}
.st119{opacity:0.2;fill:none;}
.st120{fill:none;stroke:#677380;stroke-width:0.3689;stroke-miterlimit:10;}
.st121{fill:none;stroke:#677380;stroke-width:0.3689;stroke-miterlimit:10;stroke-dasharray:1.0509,1.0509;}
.st122{opacity:0.3;fill:#1F63EC;}
.st123{fill:#2D2D2D;stroke:#FFFFFF;stroke-width:0.3162;stroke-miterlimit:10;}
.st124{fill:#FFFFFF;stroke:#B3BCC7;stroke-width:0.3162;stroke-miterlimit:10;}
.st125{clip-path:url(#SVGID_118_);}
.st126{fill:url(#SVGID_119_);}
.st127{fill:none;stroke:#DFE2E7;stroke-width:0.75;stroke-miterlimit:10;}
.st128{fill:#9DA1A5;stroke:#FFFFFF;stroke-miterlimit:10;}
.st129{fill:url(#SVGID_120_);}
.st130{fill:none;stroke:#677380;stroke-width:0.75;stroke-miterlimit:10;}
.st131{opacity:0.4;}
.st132{clip-path:url(#SVGID_122_);}
.st133{clip-path:url(#SVGID_124_);}
.st134{fill:url(#SVGID_125_);}
.st135{fill:none;stroke:#8392A3;stroke-width:0.35;stroke-miterlimit:10;}
.st136{fill:none;stroke:#8392A3;stroke-width:0.35;stroke-miterlimit:10;stroke-dasharray:0.9951,0.9951;}
.st137{fill:none;stroke:#8392A3;stroke-width:0.35;stroke-miterlimit:10;stroke-dasharray:1.004,1.004;}
.st138{fill:none;stroke:url(#SVGID_126_);stroke-width:1.5;stroke-miterlimit:10;}
.st139{fill:url(#SVGID_127_);}
.st140{fill:none;stroke:#DDE0E4;stroke-width:0.35;stroke-miterlimit:10;}
.st141{fill:#2D2D2D;stroke:#A9B3BE;stroke-width:0.275;stroke-miterlimit:10;}
.st142{fill-rule:evenodd;clip-rule:evenodd;fill:#126EF4;}
.st143{fill:#FFFFFF;stroke:#B1BAC4;stroke-width:0.275;stroke-miterlimit:10;}
.st144{fill:#CE6C50;}
.st145{fill:#5B5B5B;}
.st146{fill:#8392A3;}
.st147{fill:none;stroke:url(#SVGID_128_);stroke-width:1.5;stroke-miterlimit:10;}
.st148{fill:url(#SVGID_129_);}
.st149{fill:none;stroke:#B5BDC4;stroke-width:0.7;stroke-miterlimit:10;}
.st150{opacity:0.6;fill:none;stroke:#78838E;stroke-width:0.35;stroke-miterlimit:10;}
.st151{opacity:0.2;fill:none;stroke:#8392A3;stroke-width:0.35;stroke-miterlimit:10;stroke-dasharray:1,1;}
.st152{fill:none;stroke:#DDE0E4;stroke-width:0.75;stroke-miterlimit:10;}
.st153{fill:none;stroke:#8392A3;stroke-width:0.5;stroke-miterlimit:10;}
.st154{opacity:0.2;fill:none;stroke:#677380;stroke-width:0.3564;stroke-miterlimit:10;stroke-dasharray:1.0182,1.0182;}
.st155{fill:none;stroke:#DDE0E4;stroke-width:0.765;stroke-miterlimit:10;}
.st156{fill:url(#SVGID_130_);}
.st157{fill:url(#SVGID_131_);}
.st158{fill:#B1BAC4;}
.st159{fill:#CBD1D8;}
.st160{fill:#0B1B2B;}
.st161{fill:#91D119;}
.st162{opacity:0.7;}
.st163{fill:#FFFFFF;stroke:#000000;stroke-width:0.4418;stroke-miterlimit:10;}
.st164{fill:none;stroke:#939CAA;stroke-width:0.2209;stroke-miterlimit:10;}
.st165{fill:none;stroke:#FFFFFF;stroke-width:3.0924;stroke-miterlimit:10;}
.st166{fill:url(#SVGID_132_);}
.st167{fill:none;stroke:url(#SVGID_133_);stroke-width:1.714;stroke-miterlimit:10;}
.st168{fill:url(#SVGID_134_);}
.st169{fill:url(#SVGID_135_);}
.st170{fill:url(#SVGID_136_);}
.st171{fill:url(#SVGID_137_);}
.st172{fill:url(#SVGID_138_);}
.st173{fill:url(#SVGID_139_);}
.st174{fill:url(#SVGID_140_);}
.st175{fill:url(#SVGID_141_);}
.st176{fill:url(#SVGID_142_);}
.st177{fill:url(#SVGID_143_);}
.st178{fill:url(#SVGID_144_);}
.st179{fill:none;stroke:#1F63EC;stroke-width:4;stroke-miterlimit:10;}
.st180{fill:none;stroke:#0B1B2B;stroke-width:4;stroke-miterlimit:10;}
.st181{fill:none;stroke:#677380;stroke-width:0.3989;stroke-miterlimit:10;}
.st182{fill:none;stroke:#677380;stroke-width:0.3989;stroke-miterlimit:10;stroke-dasharray:1.14,1.14;}
.st183{fill:#257AF1;}
.st184{opacity:0.3;fill:#FFFFFF;}
.st185{fill:none;stroke:#98A5B2;stroke-width:4;stroke-miterlimit:10;}
.st186{fill:none;stroke:#65727F;stroke-width:0.3989;stroke-miterlimit:10;}
.st187{fill:none;stroke:#65727F;stroke-width:0.3989;stroke-miterlimit:10;stroke-dasharray:1.14,1.14;}
.st188{fill:none;stroke:#DDDFE4;stroke-width:0.75;stroke-miterlimit:10;}
.st189{fill:#9A9EA2;}
.st190{fill-rule:evenodd;clip-rule:evenodd;fill:#3267AC;}
.st191{fill:#FFFFFF;stroke:#AFB8C3;stroke-width:0.275;stroke-miterlimit:10;}
.st192{fill:#C5694E;}
.st193{fill:#8192A2;}
.st194{fill:#2D2D2D;stroke:#FFFFFF;stroke-width:0.6317;stroke-miterlimit:10;}
</style>
<g id="图层_2">
</g>
<g id="图层_1">
<path class="st194" d="M24.51,28.51H5.49c-2.21,0-4-1.79-4-4V5.49c0-2.21,1.79-4,4-4h19.03c2.21,0,4,1.79,4,4v19.03 C28.51,26.72,26.72,28.51,24.51,28.51z"/>
<g>
<g>
<g>
<g>
<path class="st23" d="M15.47,7.1l-1.3,1.85c-0.2,0.29-0.54,0.47-0.9,0.47h-7.1V7.09C6.16,7.1,15.47,7.1,15.47,7.1z"/>
<polygon class="st23" points="24.3,7.1 13.14,22.91 5.7,22.91 16.86,7.1 "/>
<path class="st23" d="M14.53,22.91l1.31-1.86c0.2-0.29,0.54-0.47,0.9-0.47h7.09v2.33H14.53z"/>
</g>
</g>
</g>
</g>
</g>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 11 KiB

19
studio/frontend/public/provider-logos/mistral.svg Normal file
View file

@ -0,0 +1,19 @@
<svg width="191" height="135" viewBox="0 0 191 135" fill="none" xmlns="http://www.w3.org/2000/svg">
<g clip-path="url(#clip0_134_208)">
<path d="M54.3221 0H27.1531V27.0892H54.3221V0Z" fill="#FFD800"/>
<path d="M162.984 0H135.815V27.0892H162.984V0Z" fill="#FFD800"/>
<path d="M81.4823 27.0913H27.1531V54.1805H81.4823V27.0913Z" fill="#FFAF00"/>
<path d="M162.99 27.0913H108.661V54.1805H162.99V27.0913Z" fill="#FFAF00"/>
<path d="M162.972 54.168H27.1531V81.2572H162.972V54.168Z" fill="#FF8205"/>
<path d="M54.3221 81.2593H27.1531V108.349H54.3221V81.2593Z" fill="#FA500F"/>
<path d="M108.661 81.2593H81.4917V108.349H108.661V81.2593Z" fill="#FA500F"/>
<path d="M162.984 81.2593H135.815V108.349H162.984V81.2593Z" fill="#FA500F"/>
<path d="M81.4879 108.339H-0.00146484V135.429H81.4879V108.339Z" fill="#E10500"/>
<path d="M190.159 108.339H108.661V135.429H190.159V108.339Z" fill="#E10500"/>
</g>
<defs>
<clipPath id="clip0_134_208">
<rect width="190.141" height="135" fill="white"/>
</clipPath>
</defs>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 1,001 B

5
studio/frontend/public/provider-logos/openai.svg Normal file
View file

@ -0,0 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg id="Layer_1" xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 158.7128 157.296">
<!-- Generator: Adobe Illustrator 29.2.1, SVG Export Plug-In . SVG Version: 2.1.0 Build 116) -->
<path d="M60.8734,57.2556v-14.9432c0-1.2586.4722-2.2029,1.5728-2.8314l30.0443-17.3023c4.0899-2.3593,8.9662-3.4599,13.9988-3.4599,18.8759,0,30.8307,14.6289,30.8307,30.2006,0,1.1007,0,2.3593-.158,3.6178l-31.1446-18.2467c-1.8872-1.1006-3.7754-1.1006-5.6629,0l-39.4812,22.9651ZM131.0276,115.4561v-35.7074c0-2.2028-.9446-3.7756-2.8318-4.8763l-39.481-22.9651,12.8982-7.3934c1.1007-.6285,2.0453-.6285,3.1458,0l30.0441,17.3024c8.6523,5.0341,14.4708,15.7296,14.4708,26.1107,0,11.9539-7.0769,22.965-18.2461,27.527v.0021ZM51.593,83.9964l-12.8982-7.5497c-1.1007-.6285-1.5728-1.5728-1.5728-2.8314v-34.6048c0-16.8303,12.8982-29.5722,30.3585-29.5722,6.607,0,12.7403,2.2029,17.9324,6.1349l-30.987,17.9324c-1.8871,1.1007-2.8314,2.6735-2.8314,4.8764v45.6159l-.0014-.0015ZM79.3562,100.0403l-18.4829-10.3811v-22.0209l18.4829-10.3811,18.4812,10.3811v22.0209l-18.4812,10.3811ZM91.2319,147.8591c-6.607,0-12.7403-2.2031-17.9324-6.1344l30.9866-17.9333c1.8872-1.1005,2.8318-2.6728,2.8318-4.8759v-45.616l13.0564,7.5498c1.1005.6285,1.5723,1.5728,1.5723,2.8314v34.6051c0,16.8297-13.0564,29.5723-30.5147,29.5723v.001ZM53.9522,112.7822l-30.0443-17.3024c-8.652-5.0343-14.471-15.7296-14.471-26.1107,0-12.1119,7.2356-22.9652,18.403-27.5272v35.8634c0,2.2028.9443,3.7756,2.8314,4.8763l39.3248,22.8068-12.8982,7.3938c-1.1007.6287-2.045.6287-3.1456,0ZM52.2229,138.5791c-17.7745,0-30.8306-13.3713-30.8306-29.8871,0-1.2585.1578-2.5169.3143-3.7754l30.987,17.9323c1.8871,1.1005,3.7757,1.1005,5.6628,0l39.4811-22.807v14.9435c0,1.2585-.4721,2.2021-1.5728,2.8308l-30.0443,17.3025c-4.0898,2.359-8.9662,3.4605-13.9989,3.4605h.0014ZM91.2319,157.296c19.0327,0,34.9188-13.5272,38.5383-31.4594,17.6164-4.562,28.9425-21.0779,28.9425-37.908,0-11.0112-4.719-21.7066-13.2133-29.4143.7867-3.3035,1.2595-6.607,1.2595-9.909,0-22.4929-18.2471-39.3247-39.3251-39.3247-4.2461,0-8.3363.6285-12.4262,2.045-7.0792-6.9213-16.8318-11.3254-27.5271-11.3254-19.0331,0-34.9191,13.5268-38.5384,31.4591C11.3255,36.0212,0,52.5373,0,69.3675c0,11.0112,4.7184,21.7065,13.2125,29.4142-.7865,3.3035-1.2586,6.6067-1.2586,9.9092,0,22.4923,18.2466,39.3241,39.3248,39.3241,4.2462,0,8.3362-.6277,12.426-2.0441,7.0776,6.921,16.8302,11.3251,27.5271,11.3251Z"/>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 2.4 KiB

1
studio/frontend/public/provider-logos/openrouter.svg Normal file
View file

@ -0,0 +1 @@
<svg fill="none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><g clip-path="url(#prefix__clip0_8_13)"><path fill-rule="evenodd" clip-rule="evenodd" d="M358.485 41.75l154.027 87.573v1.856l-155.605 86.634.362-45.162-17.514-.64c-22.592-.598-34.368.042-48.384 2.346-22.699 3.734-43.478 12.31-67.136 28.843l-46.208 32.107c-6.059 4.16-10.56 7.168-14.507 9.706l-10.987 6.87-8.469 4.992 8.213 4.906 11.307 7.211c10.155 6.699 24.96 16.981 57.621 39.808 23.68 16.533 44.438 25.109 67.136 28.843l6.4.96c14.806 1.941 29.334 2.005 60.267.704l.469-46.059 154.027 87.573v1.856l-155.605 86.656.298-39.722-13.546.469c-29.568.896-45.59.043-66.944-3.456-36.139-5.973-69.547-19.755-104.128-43.925l-46.038-32a467.072 467.072 0 00-16.106-10.624l-9.963-5.974c-5.38-3.1-10.785-6.157-16.213-9.173C62.037 314.24 12.01 301.141 0 301.141v-90.197l2.987.085c12.032-.149 62.08-13.269 81.258-23.978l21.675-12.374 9.344-5.845c9.131-5.973 22.869-15.488 57.301-39.531 34.582-24.17 67.968-37.973 104.128-43.925 24.576-4.053 42.112-4.544 81.366-2.944l.426-40.683z" fill="#000"/></g><defs><clipPath id="prefix__clip0_8_13"><path fill="#fff" d="M0 0h512v512H0z"/></clipPath></defs></svg>
Side by side

After

Width:  |  Height:  |  Size: 1.1 KiB

BIN
studio/frontend/public/provider-logos/qwen.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 114 KiB

246
studio/frontend/src/components/assistant-ui/model-selector.tsx
View file

@ -13,21 +13,71 @@ import { usePlatformStore } from "@/config/env";
import { cn } from "@/lib/utils";
import {
ArrowDown01Icon,
CloudIcon,
FolderSearchIcon,
Logout01Icon,
Search01Icon,
} from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
import { useMemo, useState } from "react";
import type {
DeletedModelRef,
ExternalModelOption,
LoraModelOption,
ModelOption,
ModelSelectorChangeMeta,
} from "./model-selector/types";
import { HubModelPicker, LoraModelPicker } from "./model-selector/pickers";
import { Input } from "../ui/input";
const PROVIDER_LOGO_EXT: Record<string, "svg" | "png" | "jpg"> = {
openai: "svg",
mistral: "svg",
gemini: "svg",
anthropic: "svg",
deepseek: "svg",
huggingface: "svg",
kimi: "jpg",
qwen: "png",
openrouter: "svg",
};
function providerLogoSrc(providerType: string | undefined): string | undefined {
if (!providerType) return undefined;
const ext = PROVIDER_LOGO_EXT[providerType];
if (!ext) return undefined;
return `${import.meta.env.BASE_URL}provider-logos/${providerType}.${ext}`;
}
function ExternalProviderLogo({
providerType,
className,
title,
}: {
providerType: string | undefined;
className?: string;
title?: string;
}) {
const src = providerLogoSrc(providerType);
if (!src) return null;
return (
<img
src={src}
alt=""
title={title}
aria-hidden={true}
className={cn(
"shrink-0 object-contain",
providerType === "openai" && "dark:invert",
className,
)}
/>
);
}
export type {
DeletedModelRef,
ExternalModelOption,
LoraModelOption,
ModelOption,
ModelSelectorChangeMeta,
@ -36,6 +86,7 @@ export type {
interface ModelSelectorProps {
models: ModelOption[];
loraModels?: LoraModelOption[];
externalModels?: ExternalModelOption[];
value?: string;
defaultValue?: string;
activeGgufVariant?: string | null;
@ -53,11 +104,13 @@ interface ModelSelectorProps {
onOpenChange?: (open: boolean) => void;
triggerDataTour?: string;
contentDataTour?: string;
showCloudIndicator?: boolean;
}
function ModelSelectorTrigger({
currentModel,
isLoaded,
showCloudIndicator = false,
variant = "outline",
size = "default",
className,
@ -65,6 +118,7 @@ function ModelSelectorTrigger({
}: {
currentModel?: ModelOption;
isLoaded: boolean;
showCloudIndicator?: boolean;
variant?: "outline" | "ghost" | "muted";
size?: "sm" | "default" | "lg";
className?: string;
@ -90,12 +144,27 @@ function ModelSelectorTrigger({
{isLoaded && (
<span className="size-2 shrink-0 rounded-full bg-emerald-500" />
)}
<span className="flex min-w-0 flex-1 items-baseline gap-2">
<span className="min-w-0 flex-1 truncate font-heading text-[16px] font-medium leading-tight text-black dark:text-white">
{currentModel?.icon ? (
<span className="flex shrink-0 items-center">{currentModel.icon}</span>
) : null}
<span className="flex min-w-0 flex-1 items-baseline">
<span className="min-w-0 flex flex-1 items-baseline truncate font-heading text-[16px] font-medium leading-tight text-black dark:text-white">
{currentModel?.name ?? "Select model"}
{showCloudIndicator ? (
<HugeiconsIcon
icon={CloudIcon}
strokeWidth={1.75}
className="relative top-[0.15625rem] ml-1.5 mr-[0.36rem] size-3.5 shrink-0 text-muted-foreground"
/>
) : null}
</span>
{currentModel?.description && (
<span className="shrink-0 text-xs leading-none text-muted-foreground">
<span
className={cn(
"shrink-0 text-xs leading-none text-muted-foreground",
showCloudIndicator ? "" : "ml-2",
)}
>
{currentModel.description}
</span>
)}
@ -115,6 +184,7 @@ function ModelSelectorTrigger({
function ModelSelectorContent({
models,
loraModels,
externalModels,
value,
onSelect,
onEject,
@ -127,6 +197,7 @@ function ModelSelectorContent({
}: {
models: ModelOption[];
loraModels: LoraModelOption[];
externalModels: ExternalModelOption[];
value?: string;
onSelect: (id: string, meta: ModelSelectorChangeMeta) => void;
onEject?: () => void;
@ -139,6 +210,20 @@ function ModelSelectorContent({
}) {
const hasSelection = Boolean(value);
const chatOnly = usePlatformStore((s) => s.isChatOnly());
const hasExternal = externalModels.length > 0;
const chatOnlyTabsDefault = useMemo(
() => (value && externalModels.some((model) => model.id === value) ? "external" : "hub"),
[externalModels, value],
);
const studioTabsDefault = useMemo((): "hub" | "lora" | "external" => {
if (value && externalModels.some((model) => model.id === value)) {
return "external";
}
if (value && loraModels.some((model) => model.id === value)) {
return "lora";
}
return "hub";
}, [externalModels, loraModels, value]);
return (
<PopoverContent
@ -150,12 +235,32 @@ function ModelSelectorContent({
)}
>
{chatOnly ? (
<HubModelPicker models={models} value={value} onSelect={onSelect} onFoldersChange={onFoldersChange} />
hasExternal ? (
<Tabs defaultValue={chatOnlyTabsDefault} className="w-full">
<TabsList className="mb-2 w-full">
<TabsTrigger value="hub">Hub models</TabsTrigger>
<TabsTrigger value="external">External</TabsTrigger>
</TabsList>
<TabsContent value="hub" className="m-0">
<HubModelPicker models={models} value={value} onSelect={onSelect} onFoldersChange={onFoldersChange} />
</TabsContent>
<TabsContent value="external" className="m-0">
<ExternalModelPicker
externalModels={externalModels}
value={value}
onSelect={onSelect}
/>
</TabsContent>
</Tabs>
) : (
<HubModelPicker models={models} value={value} onSelect={onSelect} onFoldersChange={onFoldersChange} />
)
) : (
<Tabs defaultValue="hub" className="w-full">
<Tabs defaultValue={studioTabsDefault} className="w-full">
<TabsList className="mb-2 w-full">
<TabsTrigger value="hub">Hub models</TabsTrigger>
<TabsTrigger value="lora">Fine-tuned</TabsTrigger>
{hasExternal ? <TabsTrigger value="external">External</TabsTrigger> : null}
</TabsList>
<TabsContent value="hub" className="m-0">
@ -171,6 +276,16 @@ function ModelSelectorContent({
deleteDisabled={deleteDisabled}
/>
</TabsContent>
{hasExternal ? (
<TabsContent value="external" className="m-0">
<ExternalModelPicker
externalModels={externalModels}
value={value}
onSelect={onSelect}
/>
</TabsContent>
) : null}
</Tabs>
)}
@ -207,6 +322,7 @@ function ModelSelectorContent({
export function ModelSelector({
models,
loraModels = [],
externalModels = [],
value,
defaultValue,
activeGgufVariant,
@ -224,6 +340,7 @@ export function ModelSelector({
onOpenChange,
triggerDataTour,
contentDataTour,
showCloudIndicator = false,
}: ModelSelectorProps) {
const [uncontrolledOpen, setUncontrolledOpen] = useState(false);
const open = controlledOpen ?? uncontrolledOpen;
@ -266,8 +383,21 @@ export function ModelSelector({
description: tag,
});
}
for (const externalModel of externalModels) {
all.set(externalModel.id, {
...externalModel,
description: externalModel.providerName,
icon: (
<ExternalProviderLogo
providerType={externalModel.providerType}
className="size-4"
title={externalModel.providerName}
/>
),
});
}
return all;
}, [loraModels, models]);
}, [externalModels, loraModels, models]);
const currentModel = useMemo(() => {
if (!selected) return undefined;
@ -303,6 +433,7 @@ export function ModelSelector({
<ModelSelectorTrigger
currentModel={currentModel}
isLoaded={isLoaded}
showCloudIndicator={showCloudIndicator}
variant={variant}
size={size}
className={className}
@ -311,6 +442,7 @@ export function ModelSelector({
<ModelSelectorContent
models={models}
loraModels={loraModels}
externalModels={externalModels}
value={selected}
onSelect={handleSelect}
onEject={onEject ? handleEject : undefined}
@ -327,3 +459,105 @@ export function ModelSelector({
ModelSelector.Trigger = ModelSelectorTrigger;
ModelSelector.Content = ModelSelectorContent;
function normalizeForSearch(value: string): string {
return value.toLowerCase().replace(/[\s_.-]/g, "");
}
function ExternalModelPicker({
externalModels,
value,
onSelect,
}: {
externalModels: ExternalModelOption[];
value?: string;
onSelect: (id: string, meta: ModelSelectorChangeMeta) => void;
}) {
const [query, setQuery] = useState("");
const grouped = useMemo(() => {
const needle = normalizeForSearch(query.trim());
const byProvider = new Map<
string,
{ providerName: string; models: ExternalModelOption[] }
>();
for (const model of externalModels) {
const searchText = normalizeForSearch(
`${model.name} ${model.providerName} ${model.id}`,
);
if (needle && !searchText.includes(needle)) continue;
const prev = byProvider.get(model.providerId);
if (prev) {
prev.models.push(model);
} else {
byProvider.set(model.providerId, {
providerName: model.providerName,
models: [model],
});
}
}
return [...byProvider.entries()]
.map(([providerId, group]) => ({
providerId,
providerName: group.providerName,
models: group.models.sort((a, b) => a.name.localeCompare(b.name)),
}))
.sort((a, b) => a.providerName.localeCompare(b.providerName));
}, [externalModels, query]);
return (
<div className="space-y-2">
<div className="relative">
<HugeiconsIcon
icon={Search01Icon}
className="pointer-events-none absolute left-2.5 top-2.5 size-4 text-muted-foreground"
/>
<Input
value={query}
onChange={(event) => setQuery(event.target.value)}
placeholder="Search external models"
className="h-9 pl-8"
/>
</div>
<div className="max-h-64 overflow-y-auto">
<div className="space-y-2 p-1">
{grouped.length === 0 ? (
<div className="px-2.5 py-2 text-xs text-muted-foreground">
No external models configured.
</div>
) : (
grouped.map((group) => (
<div key={group.providerId}>
<div className="flex items-center gap-2 px-2.5 py-1.5 text-[10px] font-semibold uppercase tracking-wider text-muted-foreground">
<ExternalProviderLogo
providerType={group.models[0]?.providerType}
className="size-3.5"
title={group.providerName}
/>
<span className="min-w-0 truncate">{group.providerName}</span>
</div>
{group.models.map((model) => (
<button
key={model.id}
type="button"
onClick={() =>
onSelect(model.id, {
source: "external",
isLora: false,
})
}
className={cn(
"flex w-full items-center rounded-md px-2.5 py-1.5 text-left text-sm transition-colors hover:bg-accent",
value === model.id && "bg-accent/60",
)}
>
<span className="min-w-0 truncate">{model.name}</span>
</button>
))}
</div>
))
)}
</div>
</div>
</div>
);
}

9
studio/frontend/src/components/assistant-ui/model-selector/types.ts
View file

@ -18,8 +18,15 @@ export interface LoraModelOption extends ModelOption {
exportType?: "lora" | "merged" | "gguf";
}
export interface ExternalModelOption extends ModelOption {
providerId: string;
providerName: string;
/** Registry key (e.g. openai, gemini) for provider branding. */
providerType: string;
}
export interface ModelSelectorChangeMeta {
source: "hub" | "lora" | "exported" | "local";
source: "hub" | "lora" | "exported" | "local" | "external";
isLora: boolean;
ggufVariant?: string;
isDownloaded?: boolean;

125
studio/frontend/src/components/assistant-ui/thread.tsx
View file

@ -31,6 +31,9 @@ import {
DropdownMenuTrigger,
} from "@/components/ui/dropdown-menu";
import { sentAudioNames } from "@/features/chat/api/chat-adapter";
import { parseExternalModelId } from "@/features/chat/external-providers";
import { getExternalReasoningCapabilities } from "@/features/chat/provider-capabilities";
import { useExternalProvidersStore } from "@/features/chat/stores/external-providers-store";
import { useChatRuntimeStore } from "@/features/chat/stores/chat-runtime-store";
import { applyQwenThinkingParams } from "@/features/chat/utils/qwen-params";
import { isTauri } from "@/lib/api-base";
@ -474,15 +477,69 @@ const ReasoningToggle: FC = () => {
const modelLoaded = useChatRuntimeStore(
(s) => !!s.params.checkpoint && !s.modelLoading,
);
const checkpoint = useChatRuntimeStore((s) => s.params.checkpoint);
const supportsReasoning = useChatRuntimeStore((s) => s.supportsReasoning);
const reasoningAlwaysOn = useChatRuntimeStore((s) => s.reasoningAlwaysOn);
const reasoningEnabled = useChatRuntimeStore((s) => s.reasoningEnabled);
const setReasoningEnabled = useChatRuntimeStore((s) => s.setReasoningEnabled);
const reasoningStyle = useChatRuntimeStore((s) => s.reasoningStyle);
const reasoningEffort = useChatRuntimeStore((s) => s.reasoningEffort);
const supportsReasoningOff = useChatRuntimeStore((s) => s.supportsReasoningOff);
const reasoningEffortLevels = useChatRuntimeStore((s) => s.reasoningEffortLevels);
const setReasoningEffort = useChatRuntimeStore((s) => s.setReasoningEffort);
const disabled = !(modelLoaded && supportsReasoning);
const lastOpenRouterChosenModel = useChatRuntimeStore(
(s) => s.lastOpenRouterChosenModel,
);
const externalProviders = useExternalProvidersStore((s) => s.providers);
const externalSelection = parseExternalModelId(checkpoint);
const selectedExternalProvider =
externalSelection != null
? externalProviders.find((p) => p.id === externalSelection.providerId)
: undefined;
const effectiveExternalModelId =
selectedExternalProvider?.providerType === "openrouter" &&
externalSelection?.modelId === "openrouter/free" &&
lastOpenRouterChosenModel
? lastOpenRouterChosenModel
: externalSelection?.modelId;
const externalReasoningCaps =
externalSelection != null
? getExternalReasoningCapabilities(
selectedExternalProvider?.providerType,
effectiveExternalModelId,
)
: null;
const effectiveReasoningStyle =
externalReasoningCaps?.reasoningStyle ?? reasoningStyle;
const effectiveReasoningAlwaysOn =
externalReasoningCaps?.reasoningAlwaysOn ?? reasoningAlwaysOn;
const effectiveSupportsReasoningOff =
externalReasoningCaps?.supportsReasoningOff ?? supportsReasoningOff;
const effectiveReasoningEffortLevels =
externalReasoningCaps?.reasoningEffortLevels ?? reasoningEffortLevels;
const effectiveSupportsReasoning =
externalReasoningCaps?.supportsReasoning ?? supportsReasoning;
const reasoningLockedOn =
effectiveSupportsReasoning &&
(effectiveReasoningAlwaysOn || !effectiveSupportsReasoningOff);
const effectiveReasoningEnabled = reasoningLockedOn ? true : reasoningEnabled;
const effectiveReasoningVisualEnabled =
effectiveReasoningEnabled && reasoningEffort !== "none";
const disabled = !(modelLoaded && effectiveSupportsReasoning);
const formatEffortLabel = (level: typeof reasoningEffort): string => {
if (level !== "xhigh") return level.charAt(0).toUpperCase() + level.slice(1);
const normalized = externalSelection?.modelId?.trim().toLowerCase() ?? "";
if (
normalized.startsWith("claude-opus-4-6") ||
normalized.startsWith("claude-sonnet-4-6")
) {
return "Max";
}
return "Extra High";
};
const effortLabel = formatEffortLabel(reasoningEffort);
if (reasoningStyle === "reasoning_effort") {
if (effectiveReasoningStyle === "reasoning_effort") {
return (
<DropdownMenu>
<DropdownMenuTrigger asChild={true}>
@ -493,26 +550,47 @@ const ReasoningToggle: FC = () => {
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
disabled
? "cursor-not-allowed opacity-40"
: "bg-primary/10 text-primary hover:bg-primary/20",
: effectiveReasoningVisualEnabled
? "bg-primary/10 text-primary hover:bg-primary/20"
: "text-muted-foreground hover:bg-muted-foreground/15",
)}
aria-label={`Reasoning effort: ${reasoningEffort}`}
>
<LightbulbIcon className="size-3.5" />
{effectiveReasoningVisualEnabled ? (
<LightbulbIcon className="size-3.5" />
) : (
<LightbulbOffIcon className="size-3.5" />
)}
<span>
Think:{" "}
{reasoningEffort.charAt(0).toUpperCase() +
reasoningEffort.slice(1)}
Think: {effectiveReasoningVisualEnabled ? effortLabel : "None"}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
{(["low", "medium", "high"] as const).map((level) => (
{effectiveSupportsReasoningOff && (
<DropdownMenuItem
onSelect={() => {
setReasoningEnabled(false);
applyQwenThinkingParams(false);
}}
>
None
{!effectiveReasoningVisualEnabled ? " \u2713" : ""}
</DropdownMenuItem>
)}
{effectiveReasoningEffortLevels
.filter((level) => level !== "none")
.map((level) => (
<DropdownMenuItem
key={level}
onSelect={() => setReasoningEffort(level)}
onSelect={() => {
setReasoningEffort(level);
setReasoningEnabled(true);
applyQwenThinkingParams(true);
}}
>
{level.charAt(0).toUpperCase() + level.slice(1)}
{reasoningEffort === level ? " \u2713" : ""}
{formatEffortLabel(level)}
{effectiveReasoningVisualEnabled && reasoningEffort === level ? " \u2713" : ""}
</DropdownMenuItem>
))}
</DropdownMenuContent>
@ -523,17 +601,34 @@ const ReasoningToggle: FC = () => {
return (
<button
type="button"
disabled={disabled}
disabled={disabled || reasoningLockedOn}
aria-disabled={disabled || reasoningLockedOn}
title={
reasoningLockedOn
? "This model requires reasoning to stay on."
: undefined
}
onClick={() => {
if (reasoningLockedOn) return;
const next = !reasoningEnabled;
setReasoningEnabled(next);
applyQwenThinkingParams(next);
}}
className="composer-pill-btn"
data-active={reasoningEnabled && !disabled ? "true" : "false"}
aria-label={reasoningEnabled ? "Disable thinking" : "Enable thinking"}
data-active={
reasoningLockedOn || (effectiveReasoningEnabled && !disabled)
? "true"
: "false"
}
aria-label={
reasoningLockedOn
? "Thinking is required for this model"
: effectiveReasoningEnabled
? "Disable thinking"
: "Enable thinking"
}
>
{reasoningEnabled && !disabled ? (
{reasoningLockedOn || (effectiveReasoningEnabled && !disabled) ? (
<LightbulbIcon className="size-3.5" />
) : (
<LightbulbOffIcon className="size-3.5" />

68
studio/frontend/src/features/chat/api-provider-logo.tsx Normal file
View file

@ -0,0 +1,68 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { cn } from "@/lib/utils";
import { DashboardSquare01Icon } from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
/**
* Registry logos live at `public/provider-logos/{provider_type}.{ext}` where `provider_type`
* matches `PROVIDER_REGISTRY` keys exactly (lowercase). Extension varies by asset (svg preferred).
*/
const PROVIDER_LOGO_EXT: Record<string, "svg" | "png" | "jpg"> = {
openai: "svg",
mistral: "svg",
gemini: "svg",
anthropic: "svg",
deepseek: "svg",
huggingface: "svg",
kimi: "jpg",
qwen: "png",
openrouter: "svg",
};
export function apiProviderLogoSrc(
providerType: string | undefined | null,
): string | undefined {
if (!providerType) return undefined;
const ext = PROVIDER_LOGO_EXT[providerType];
if (!ext) return undefined;
return `${import.meta.env.BASE_URL}provider-logos/${providerType}.${ext}`;
}
interface ApiProviderLogoProps {
providerType: string | undefined | null;
className?: string;
title?: string;
}
/**
* Renders the logo for a registry provider type when `provider_type.{ext}` exists under
* `public/provider-logos/`.
* OpenAI's asset is black-on-transparent; it is inverted in dark mode for contrast.
*/
export function ApiProviderLogo({ providerType, className, title }: ApiProviderLogoProps) {
if (providerType === "custom") {
return (
<span title={title} aria-hidden className="inline-flex shrink-0">
<HugeiconsIcon icon={DashboardSquare01Icon} className={cn("shrink-0", className)} />
</span>
);
}
const src = apiProviderLogoSrc(providerType);
if (!src) return null;
return (
<img
src={src}
alt=""
title={title}
aria-hidden
className={cn(
"shrink-0 object-contain",
providerType === "openai" && "dark:invert",
className,
)}
/>
);
}

604
studio/frontend/src/features/chat/api/chat-adapter.ts
View file

@ -15,7 +15,27 @@ import {
streamChatCompletions,
validateModel,
} from "./chat-api";
import {
encryptProviderApiKey,
isProviderKeyRotationError,
} from "./providers-api";
import { db } from "../db";
import type {
OpenAIChatCompletionsRequest,
OpenAIMessageContent,
} from "../types/api";
import {
getExternalProviderApiKey,
loadExternalProviders,
parseExternalModelId,
} from "../external-providers";
import {
EXTERNAL_MAX_OUTPUT_TOKENS,
clampReasoningEffortToLevels,
getExternalMinOutputTokens,
getExternalReasoningCapabilities,
getProviderCapabilities,
} from "../provider-capabilities";
import { useChatRuntimeStore } from "../stores/chat-runtime-store";
import { isMultimodalResponse } from "../types/api";
import type { ChatModelSummary } from "../types/runtime";
@ -118,6 +138,70 @@ function estimateTokenCount(text: string): number | undefined {
return Math.max(1, Math.round(trimmed.length / 4));
}
/**
* Normalize a streamed `delta.content` to a plain text string.
*
* OpenAI Chat Completions originally typed `delta.content` as a string, but
* a number of providers now emit it as an array of structured content parts.
* Concatenating that with `cumulativeText += delta` would stringify each
* part as `[object Object]` — this function is the guard against that.
*
* Handled part shapes:
* { type: "text" | "output_text", text | content: "..." } → text body
* { type: "thinking" | "reasoning", thinking | text: "..." } → wrapped as
* inline `<think>...</think>` so the downstream parser
* (`parseAssistantContent`) lifts it into a reasoning part the same way
* it does for providers that emit thinking inline. Without this wrap,
* Mistral magistral and similar reasoning-part providers would lose
* their thinking panel.
*
* Unknown part types are skipped — better to drop a stray field than to
* stringify an object and pollute the rendered chat with `[object Object]`.
*/
function extractDeltaText(delta: unknown): string {
const extractReasoningText = (payload: unknown): string => {
if (typeof payload === "string") return payload;
if (Array.isArray(payload)) {
return payload.map((item) => extractReasoningText(item)).join("");
}
if (!payload || typeof payload !== "object") return "";
const obj = payload as Record<string, unknown>;
for (const key of ["thinking", "text", "content", "reasoning", "summary"]) {
if (key in obj) {
const text = extractReasoningText(obj[key]);
if (text) return text;
}
}
return "";
};
if (typeof delta === "string") return delta;
if (!Array.isArray(delta)) return "";
let out = "";
for (const part of delta) {
if (typeof part === "string") {
out += part;
continue;
}
if (!part || typeof part !== "object") continue;
const obj = part as {
type?: string;
text?: string;
content?: string;
thinking?: string;
};
if (obj.type === "text" || obj.type === "output_text") {
if (typeof obj.text === "string") out += obj.text;
else if (typeof obj.content === "string") out += obj.content;
} else if (obj.type === "thinking" || obj.type === "reasoning") {
const thinking = extractReasoningText(obj);
if (thinking) out += `<think>${thinking}</think>`;
}
}
return out;
}
function buildTiming(
streamStartTime: number,
totalChunks: number,
@ -162,9 +246,51 @@ function collectTextParts(message: RunMessage): string[] {
return textParts;
}
function collectImageParts(
message: RunMessage,
): Array<{ type: "image_url"; image_url: { url: string } }> {
const parts: Array<{ type: "image_url"; image_url: { url: string } }> = [];
for (const part of message.content ?? []) {
if (part.type === "image" && "image" in part) {
const src = (part as { image: string }).image;
if (src) {
parts.push({
type: "image_url",
image_url: {
url: src.startsWith("data:") ? src : `data:image/png;base64,${src}`,
},
});
}
}
}
if ("attachments" in message && (message.attachments?.length ?? 0) > 0) {
for (const attachment of message.attachments ?? []) {
for (const part of attachment.content ?? []) {
if (part.type === "image" && "image" in part) {
const src = (part as { image: string }).image;
if (src) {
parts.push({
type: "image_url",
image_url: {
url: src.startsWith("data:")
? src
: `data:image/png;base64,${src}`,
},
});
}
}
}
}
}
return parts;
}
function toOpenAIMessage(message: RunMessage): {
role: "system" | "user" | "assistant";
content: string;
content: OpenAIMessageContent;
} | null {
if (
message.role !== "system" &&
@ -174,17 +300,25 @@ function toOpenAIMessage(message: RunMessage): {
return null;
}
let content = collectTextParts(message).join("\n");
let textContent = collectTextParts(message).join("\n");
// Strip inline audio base64 from prior assistant messages to avoid
// inflating token counts (e.g. audio-player responses with embedded WAV).
if (message.role === "assistant") {
content = content.replace(
textContent = textContent.replace(
/data:audio\/[a-z0-9.+-]+;base64,[A-Za-z0-9+/=]+/g,
"[audio]",
);
}
return { role: message.role, content };
const imageParts = collectImageParts(message);
if (imageParts.length > 0) {
return {
role: message.role,
content: [{ type: "text", text: textContent }, ...imageParts],
};
}
return { role: message.role, content: textContent };
}
function extractImageBase64(input: string): string | undefined {
@ -594,6 +728,29 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
toolsEnabled,
codeToolsEnabled,
} = runtime;
const externalSelection = parseExternalModelId(params.checkpoint);
const isExternalRequest = externalSelection !== null;
const externalProvider = isExternalRequest
? loadExternalProviders().find(
(provider) => provider.id === externalSelection.providerId,
)
: null;
const externalApiKey = externalProvider
? getExternalProviderApiKey(externalProvider.id).trim()
: "";
if (isExternalRequest && !externalProvider) {
toast.error("External provider not found.", {
description: "Open API Providers and re-add this provider.",
});
throw new Error("External provider not found.");
}
if (isExternalRequest && !externalApiKey) {
toast.error("Missing API key for selected external provider.", {
description: "Open API Providers and set the API key again.",
});
throw new Error("Missing external provider API key.");
}
const outboundMessages = messages
.map(toOpenAIMessage)
@ -711,6 +868,17 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
let cumulativeText = "";
let reasoningStartAt: number | null = null;
let reasoningDuration = 0;
// Tracks whether we are currently inside a `<think>` block opened by
// a `delta.reasoning_content` chunk. Kimi (kimi-k2.6, kimi-k2-thinking)
// and DeepSeek's reasoner stream their thinking as a separate
// `reasoning_content` field on the chat-completion delta — not as
// `content`, not as a structured part. We wrap those chunks with
// inline `<think>...</think>` so the existing parseAssistantContent
// lifts them into the reasoning panel the same way it does for
// local Harmony models. State has to live outside the SSE loop
// because the close tag fires when the next chunk carries content
// (or when the stream ends).
let reasoningContentOpen = false;
// Tool call content parts — accumulated and yielded cumulatively.
// result is set directly on the tool-call part when tool_end arrives.
const toolCallParts: ToolCallMessagePart[] = [];
@ -760,8 +928,106 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
supportsPreserveThinking,
preserveThinking,
} = runtime;
const stream = streamChatCompletions(
{
const externalBackendProviderType =
externalProvider?.providerType === "custom"
? "openai"
: externalProvider?.providerType;
const externalCapabilities = getProviderCapabilities(
externalProvider?.providerType,
);
const externalReasoningCaps: ReturnType<
typeof getExternalReasoningCapabilities
> =
externalSelection && externalProvider
? getExternalReasoningCapabilities(
externalProvider.providerType,
externalSelection.modelId,
)
: {
supportsReasoning,
reasoningStyle,
reasoningAlwaysOn: false,
supportsReasoningOff: false,
reasoningEffortLevels: ["low", "medium", "high"] as const,
};
type RequestReasoningEffort = Extract<
NonNullable<OpenAIChatCompletionsRequest["reasoning_effort"]>,
"none" | "minimal" | "low" | "medium" | "high" | "max" | "xhigh"
>;
const fallbackExternalEffort =
(externalReasoningCaps.reasoningEffortLevels[0] ??
"low") as RequestReasoningEffort;
const selectedExternalEffort: RequestReasoningEffort =
clampReasoningEffortToLevels(
reasoningEffort,
externalReasoningCaps.reasoningEffortLevels,
) as RequestReasoningEffort;
const localReasoningEffort =
reasoningEffort === "low" || reasoningEffort === "medium" || reasoningEffort === "high"
? reasoningEffort
: "low";
const externalReasoningEnabled =
!externalReasoningCaps.supportsReasoningOff ? true : reasoningEnabled;
const buildRequestPayload = async (
forceRefreshPublicKey = false,
): Promise<OpenAIChatCompletionsRequest> => {
if (externalSelection && externalProvider) {
return {
model: externalSelection.modelId,
messages: outboundMessages,
stream: true,
// Reasoning-class models (OpenAI gpt-5.x / o3) reject temperature
// and top_p; only forward when the active provider supports them.
...(externalCapabilities?.temperature !== false
? { temperature: params.temperature }
: {}),
...(externalCapabilities?.topP !== false
? { top_p: params.topP }
: {}),
// Clamp to the cross-provider output cap so a maxTokens value
// carried over from a local-model session does not blow past
// provider limits (e.g. Claude Opus 400s on >128k). Also
// floor to the provider's documented minimum — Kimi's
// thinking models need >=16k or the response truncates
// before the answer fits alongside reasoning_content.
max_tokens: Math.min(
Math.max(
params.maxTokens,
getExternalMinOutputTokens(externalProvider?.providerType),
),
EXTERNAL_MAX_OUTPUT_TOKENS,
),
// Only forward sampling knobs the provider actually accepts; the
// backend's external-provider proxy is param-permissive and would
// surface a 400 from providers that reject unknown fields (e.g.
// OpenAI rejects top_k, Anthropic/DeepSeek reject presence_penalty).
...(externalCapabilities?.topK ? { top_k: params.topK } : {}),
...(externalCapabilities?.presencePenalty
? { presence_penalty: params.presencePenalty }
: {}),
provider_id: externalProvider.id,
provider_type: externalBackendProviderType,
external_model: externalSelection.modelId,
encrypted_api_key: await encryptProviderApiKey(
externalApiKey,
forceRefreshPublicKey,
),
provider_base_url: externalProvider.baseUrl || null,
...(externalReasoningCaps.supportsReasoning
? externalReasoningCaps.reasoningStyle === "reasoning_effort"
? externalReasoningEnabled
? { reasoning_effort: selectedExternalEffort }
: externalReasoningCaps.supportsReasoningOff
? { reasoning_effort: "none" }
: {
reasoning_effort: fallbackExternalEffort,
}
: { enable_thinking: reasoningEnabled }
: {}),
};
}
return {
model: params.checkpoint,
messages: outboundMessages,
stream: true,
@ -779,7 +1045,9 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
...(useAdapter === undefined ? {} : { use_adapter: useAdapter }),
...(supportsReasoning
? reasoningStyle === "reasoning_effort"
? { reasoning_effort: reasoningEffort }
? reasoningEnabled
? { reasoning_effort: localReasoningEffort }
: {}
: { enable_thinking: reasoningEnabled }
: {}),
...(supportsPreserveThinking ? { preserve_thinking: preserveThinking } : {}),
@ -798,116 +1066,234 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
})(),
}
: {}),
},
abortSignal,
);
};
};
for await (const chunk of stream) {
// Handle tool status events
const toolStatusText = (chunk as unknown as { _toolStatus?: string })._toolStatus;
if (toolStatusText !== undefined) {
runtime.setToolStatus(toolStatusText || null);
continue;
}
let retriedWithRefreshedKey = false;
while (true) {
try {
const stream = streamChatCompletions(
await buildRequestPayload(retriedWithRefreshedKey),
abortSignal,
);
// Emit tool-call content parts for assistant-ui.
// On tool_start: add a new tool-call part (renders in "running" state).
// On tool_end: set result on the existing part (transitions to "complete").
const toolEvent = (chunk as unknown as { _toolEvent?: Record<string, unknown> })._toolEvent;
if (toolEvent !== undefined) {
if (toolEvent.type === "tool_start") {
const id = (toolEvent.tool_call_id as string) || `${toolEvent.tool_name}_${Date.now()}`;
const toolArgs = (toolEvent.arguments ?? {}) as ToolCallMessagePart["args"];
toolCallParts.push({
type: "tool-call" as const,
toolCallId: id,
toolName: toolEvent.tool_name as string,
argsText: JSON.stringify(toolArgs),
args: toolArgs,
});
} else if (toolEvent.type === "tool_end") {
const id = (toolEvent.tool_call_id as string) ||
toolCallParts[toolCallParts.length - 1]?.toolCallId || "";
const idx = toolCallParts.findIndex((p) => p.toolCallId === id);
if (idx !== -1) {
const rawResult = (toolEvent.result as string) ?? "";
const imgMarker = "\n__IMAGES__:";
const imgIdx = rawResult.lastIndexOf(imgMarker);
let parsedResult: string | { text: string; images: string[]; sessionId: string };
if (imgIdx !== -1) {
const text = rawResult.slice(0, imgIdx);
// Fall back to "_default" to match the backend sandbox directory
// used when no session_id is provided (see tools.py _get_workdir).
const sessionId = resolvedThreadId || "_default";
try {
const images = JSON.parse(rawResult.slice(imgIdx + imgMarker.length)) as string[];
parsedResult = { text, images, sessionId };
} catch {
parsedResult = rawResult;
for await (const chunk of stream) {
// Handle tool status events
const toolStatusText = (chunk as unknown as { _toolStatus?: string })._toolStatus;
if (toolStatusText !== undefined) {
runtime.setToolStatus(toolStatusText || null);
continue;
}
// Emit tool-call content parts for assistant-ui.
// On tool_start: add a new tool-call part (renders in "running" state).
// On tool_end: set result on the existing part (transitions to "complete").
const toolEvent = (chunk as unknown as { _toolEvent?: Record<string, unknown> })._toolEvent;
if (toolEvent !== undefined) {
if (toolEvent.type === "tool_start") {
const id = (toolEvent.tool_call_id as string) || `${toolEvent.tool_name}_${Date.now()}`;
const toolArgs = (toolEvent.arguments ?? {}) as ToolCallMessagePart["args"];
toolCallParts.push({
type: "tool-call" as const,
toolCallId: id,
toolName: toolEvent.tool_name as string,
argsText: JSON.stringify(toolArgs),
args: toolArgs,
});
} else if (toolEvent.type === "tool_end") {
const id = (toolEvent.tool_call_id as string) ||
toolCallParts[toolCallParts.length - 1]?.toolCallId || "";
const idx = toolCallParts.findIndex((p) => p.toolCallId === id);
if (idx !== -1) {
const rawResult = (toolEvent.result as string) ?? "";
const imgMarker = "\n__IMAGES__:";
const imgIdx = rawResult.lastIndexOf(imgMarker);
let parsedResult: string | { text: string; images: string[]; sessionId: string };
if (imgIdx !== -1) {
const text = rawResult.slice(0, imgIdx);
// Fall back to "_default" to match the backend sandbox directory
// used when no session_id is provided (see tools.py _get_workdir).
const sessionId = resolvedThreadId || "_default";
try {
const images = JSON.parse(rawResult.slice(imgIdx + imgMarker.length)) as string[];
parsedResult = { text, images, sessionId };
} catch {
parsedResult = rawResult;
}
} else {
parsedResult = rawResult;
}
toolCallParts[idx] = { ...toolCallParts[idx], result: parsedResult };
}
} else {
parsedResult = rawResult;
}
toolCallParts[idx] = { ...toolCallParts[idx], result: parsedResult };
// Yield cumulative state so tool UI updates (tools first, text after)
const textParts = parseAssistantContent(cumulativeText);
yield {
content: [...toolCallParts, ...textParts],
metadata: {
timing: buildTiming(streamStartTime, totalChunks, firstTokenTime),
custom: { reasoningDuration },
},
};
continue;
}
// OpenAI-standard usage chunk: choices=[], usage populated
if (chunk.choices?.length === 0 && chunk.usage) {
serverMetadata = {
usage: chunk.usage,
timings: (chunk as Record<string, unknown>).timings as ServerTimings | undefined,
};
continue;
}
totalChunks += 1;
// OpenRouter's free router (openrouter/free) picks a different
// underlying free model per request and reports it in every
// chunk's top-level `model` field. Latch the first non-empty
// value that differs from the requested checkpoint so the
// header chip can render "openrouter/free:<chosen>".
if (
isExternalRequest &&
externalProvider?.providerType === "openrouter" &&
externalSelection?.modelId === "openrouter/free"
) {
const chunkModel = (chunk as { model?: unknown }).model;
if (
typeof chunkModel === "string" &&
chunkModel.length > 0 &&
chunkModel !== externalSelection.modelId
) {
const storeState = useChatRuntimeStore.getState();
if (storeState.lastOpenRouterChosenModel !== chunkModel) {
storeState.setLastOpenRouterChosenModel(chunkModel);
}
}
}
const rawDelta = chunk.choices?.[0]?.delta?.content;
// Providers like Mistral's magistral return delta.content as an
// array of structured parts; normalize to text (with thinking
// parts re-wrapped as inline <think> tags) so the rest of the
// accumulator stays string-based.
const delta = extractDeltaText(rawDelta);
// Kimi (kimi-k2.6, kimi-k2-thinking) and DeepSeek reasoner
// stream thinking via `delta.reasoning_content` as a plain
// string field — separate from `delta.content` which carries
// the answer. Wrap reasoning chunks inline as <think>...
// </think> so parseAssistantContent treats them like any
// other reasoning. The close tag fires when the next chunk
// brings content, or when the stream ends.
const rawReasoning = (
chunk.choices?.[0]?.delta as
| { reasoning_content?: unknown }
| undefined
)?.reasoning_content;
// OpenRouter uses a third reasoning shape: a structured
// `delta.reasoning_details` array of parts (each carrying
// `text`). The router emits this regardless of which
// underlying provider it picked, so we extract here and
// merge into the same <think>...</think> wrap path used
// for Kimi / DeepSeek reasoning_content. See
// https://openrouter.ai/docs/guides/best-practices/reasoning-tokens
const rawReasoningDetails = (
chunk.choices?.[0]?.delta as
| { reasoning_details?: unknown }
| undefined
)?.reasoning_details;
const reasoningFromDetails = Array.isArray(rawReasoningDetails)
? rawReasoningDetails
.map((part) => {
if (!part || typeof part !== "object") return "";
const text = (part as { text?: unknown }).text;
return typeof text === "string" ? text : "";
})
.join("")
: "";
const reasoning =
(typeof rawReasoning === "string" ? rawReasoning : "") +
reasoningFromDetails;
if (!delta && !reasoning) {
continue;
}
if (waitingFirstChunk) {
waitingFirstChunk = false;
firstTokenTime = Date.now() - streamStartTime;
settleFirstTokenOk();
runtime.setGeneratingStatus(null);
}
if (reasoning) {
if (!reasoningContentOpen) {
cumulativeText += `<think>${reasoning}`;
reasoningContentOpen = true;
} else {
cumulativeText += reasoning;
}
}
if (delta) {
if (reasoningContentOpen) {
cumulativeText += "</think>";
reasoningContentOpen = false;
}
cumulativeText += delta;
}
// Mistral's magistral occasionally emits a trailing
// template-literal artifact (e.g. "${response}") at the end of
// an otherwise complete answer. It is never part of a real
// reply, so strip a trailing `${...}` token from external
// provider streams. The regex anchors to end-of-string and is
// idempotent — fragments mid-stream (e.g. "${re") leave the
// string untouched and only collapse once the closing brace
// arrives. Local-model output is left alone.
if (isExternalRequest) {
cumulativeText = cumulativeText.replace(
/\s*\$\{[^}]*\}\s*$/,
"",
);
}
const parts = parseAssistantContent(cumulativeText);
if (parts.some((part) => part.type === "reasoning") && !reasoningStartAt) {
reasoningStartAt = Date.now();
}
if (hasClosedThinkTag(cumulativeText) && reasoningStartAt && !reasoningDuration) {
reasoningDuration = Math.round((Date.now() - reasoningStartAt) / 1000);
}
if (parts.length > 0 || toolCallParts.length > 0) {
yield {
content: [...toolCallParts, ...parts],
metadata: {
timing: buildTiming(
streamStartTime,
totalChunks,
firstTokenTime,
),
custom: { reasoningDuration },
},
};
}
}
// Yield cumulative state so tool UI updates (tools first, text after)
const textParts = parseAssistantContent(cumulativeText);
yield {
content: [...toolCallParts, ...textParts],
metadata: {
timing: buildTiming(streamStartTime, totalChunks, firstTokenTime),
custom: { reasoningDuration },
},
};
continue;
}
// OpenAI-standard usage chunk: choices=[], usage populated
if (chunk.choices?.length === 0 && chunk.usage) {
serverMetadata = {
usage: chunk.usage,
timings: (chunk as Record<string, unknown>).timings as ServerTimings | undefined,
};
continue;
}
totalChunks += 1;
const delta = chunk.choices?.[0]?.delta?.content;
if (!delta) {
continue;
}
if (waitingFirstChunk) {
waitingFirstChunk = false;
firstTokenTime = Date.now() - streamStartTime;
settleFirstTokenOk();
runtime.setGeneratingStatus(null);
}
cumulativeText += delta;
const parts = parseAssistantContent(cumulativeText);
if (parts.some((part) => part.type === "reasoning") && !reasoningStartAt) {
reasoningStartAt = Date.now();
}
if (hasClosedThinkTag(cumulativeText) && reasoningStartAt && !reasoningDuration) {
reasoningDuration = Math.round((Date.now() - reasoningStartAt) / 1000);
}
if (parts.length > 0 || toolCallParts.length > 0) {
yield {
content: [...toolCallParts, ...parts],
metadata: {
timing: buildTiming(
streamStartTime,
totalChunks,
firstTokenTime,
),
custom: { reasoningDuration },
},
};
break;
} catch (streamError) {
if (
isExternalRequest &&
!retriedWithRefreshedKey &&
isProviderKeyRotationError(streamError)
) {
retriedWithRefreshedKey = true;
continue;
}
throw streamError;
}
}
// If the stream ended while we were still inside a
// delta.reasoning_content block (Kimi / DeepSeek path), close
// the open <think> tag so the reasoning panel parses cleanly.
if (reasoningContentOpen) {
cumulativeText += "</think>";
reasoningContentOpen = false;
}
settleFirstTokenOk();
// Extract source parts from completed web_search tool calls

230
studio/frontend/src/features/chat/api/providers-api.ts Normal file
View file

@ -0,0 +1,230 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import forge from "node-forge";
import { authFetch } from "@/features/auth";
export interface ProviderRegistryEntry {
provider_type: string;
display_name: string;
base_url: string;
default_models: string[];
supports_streaming: boolean;
supports_vision: boolean;
supports_tool_calling: boolean;
/** remote = fetch /models; curated = huge catalogs — UI uses defaults + manual IDs only */
model_list_mode?: "remote" | "curated";
}
export interface ProviderConfig {
id: string;
provider_type: string;
display_name: string;
base_url: string;
is_enabled: boolean;
created_at: string;
updated_at: string;
}
export interface ProviderModelInfo {
id: string;
display_name: string;
context_length?: number | null;
owned_by?: string | null;
}
export interface ProviderTestResult {
success: boolean;
message: string;
models_count?: number | null;
}
function parseErrorText(status: number, body: unknown): string {
if (
body &&
typeof body === "object" &&
"detail" in body &&
typeof body.detail === "string"
) {
return body.detail;
}
if (
body &&
typeof body === "object" &&
"message" in body &&
typeof body.message === "string"
) {
return body.message;
}
return `Request failed (${status})`;
}
async function parseJsonOrThrow<T>(response: Response): Promise<T> {
const body = await response.json().catch(() => null);
if (!response.ok) {
throw new Error(parseErrorText(response.status, body));
}
return body as T;
}
export function isProviderKeyRotationError(error: unknown): boolean {
if (!(error instanceof Error)) return false;
const normalized = error.message.toLowerCase();
return (
normalized.includes("public key may have changed") ||
normalized.includes("server key may have changed")
);
}
let cachedPublicKeyPem: string | null = null;
let cachedForgeKey: forge.pki.rsa.PublicKey | null = null;
export function clearProviderPublicKeyCache(): void {
cachedPublicKeyPem = null;
cachedForgeKey = null;
}
async function importProviderPublicKey(
forceRefresh = false,
): Promise<forge.pki.rsa.PublicKey> {
if (!forceRefresh && cachedForgeKey) {
return cachedForgeKey;
}
const response = await authFetch("/api/providers/public-key");
const body = await parseJsonOrThrow<{ public_key: string }>(response);
const publicKeyPem = body.public_key?.trim();
if (!publicKeyPem) {
throw new Error("Provider public key is missing.");
}
if (!forceRefresh && cachedPublicKeyPem === publicKeyPem && cachedForgeKey) {
return cachedForgeKey;
}
const forgeKey = forge.pki.publicKeyFromPem(publicKeyPem);
cachedPublicKeyPem = publicKeyPem;
cachedForgeKey = forgeKey;
return forgeKey;
}
export async function encryptProviderApiKey(
plaintextApiKey: string,
forceRefresh = false,
): Promise<string> {
const key = await importProviderPublicKey(forceRefresh);
const encrypted = key.encrypt(plaintextApiKey, "RSA-OAEP", {
md: forge.md.sha256.create(),
mgf1: { md: forge.md.sha256.create() },
});
return forge.util.encode64(encrypted);
}
export async function listProviderRegistry(): Promise<ProviderRegistryEntry[]> {
const response = await authFetch("/api/providers/registry");
return parseJsonOrThrow<ProviderRegistryEntry[]>(response);
}
export async function listProviderConfigs(): Promise<ProviderConfig[]> {
const response = await authFetch("/api/providers/");
return parseJsonOrThrow<ProviderConfig[]>(response);
}
export async function createProviderConfig(payload: {
providerType: string;
displayName: string;
baseUrl?: string | null;
}): Promise<ProviderConfig> {
const response = await authFetch("/api/providers/", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
provider_type: payload.providerType,
display_name: payload.displayName,
base_url: payload.baseUrl ?? null,
}),
});
return parseJsonOrThrow<ProviderConfig>(response);
}
export async function deleteProviderConfig(providerId: string): Promise<void> {
const response = await authFetch(`/api/providers/${providerId}`, {
method: "DELETE",
});
if (!response.ok) {
const body = await response.json().catch(() => null);
throw new Error(parseErrorText(response.status, body));
}
}
export async function updateProviderConfig(
providerId: string,
payload: {
displayName?: string;
baseUrl?: string | null;
isEnabled?: boolean;
},
): Promise<ProviderConfig> {
const response = await authFetch(`/api/providers/${providerId}`, {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
...(payload.displayName === undefined ? {} : { display_name: payload.displayName }),
...(payload.baseUrl === undefined ? {} : { base_url: payload.baseUrl }),
...(payload.isEnabled === undefined ? {} : { is_enabled: payload.isEnabled }),
}),
});
return parseJsonOrThrow<ProviderConfig>(response);
}
async function withApiKeyEncryptionRetry<T>(
plaintextApiKey: string,
call: (encryptedApiKey: string) => Promise<T>,
): Promise<T> {
try {
const encrypted = await encryptProviderApiKey(plaintextApiKey, false);
return await call(encrypted);
} catch (error) {
if (!isProviderKeyRotationError(error)) {
throw error;
}
clearProviderPublicKeyCache();
const encrypted = await encryptProviderApiKey(plaintextApiKey, true);
return await call(encrypted);
}
}
export async function testProviderConnection(payload: {
providerType: string;
apiKey: string;
baseUrl?: string | null;
}): Promise<ProviderTestResult> {
return withApiKeyEncryptionRetry(payload.apiKey, async (encryptedApiKey) => {
const response = await authFetch("/api/providers/test", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
provider_type: payload.providerType,
encrypted_api_key: encryptedApiKey,
base_url: payload.baseUrl ?? null,
}),
});
return parseJsonOrThrow<ProviderTestResult>(response);
});
}
export async function listProviderModels(payload: {
providerType: string;
apiKey: string;
baseUrl?: string | null;
}): Promise<ProviderModelInfo[]> {
return withApiKeyEncryptionRetry(payload.apiKey, async (encryptedApiKey) => {
const response = await authFetch("/api/providers/models", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
provider_type: payload.providerType,
encrypted_api_key: encryptedApiKey,
base_url: payload.baseUrl ?? null,
}),
});
return parseJsonOrThrow<ProviderModelInfo[]>(response);
});
}

240
studio/frontend/src/features/chat/chat-page.tsx
View file

@ -3,6 +3,7 @@
import {
type DeletedModelRef,
type ExternalModelOption,
type LoraModelOption,
type ModelOption,
ModelSelector,
@ -40,6 +41,16 @@ import { ChatSettingsPanel } from "./chat-settings-sheet";
import { ContextUsageBar } from "./components/context-usage-bar";
import { ModelLoadInlineStatus } from "./components/model-load-status";
import { db } from "./db";
import {
buildExternalModelId,
isExternalModelId,
parseExternalModelId,
} from "./external-providers";
import {
clampReasoningEffortToLevels,
getExternalReasoningCapabilities,
getProviderCapabilities,
} from "./provider-capabilities";
import { useChatModelRuntime } from "./hooks/use-chat-model-runtime";
import {
clearTrainingCompareHandoff,
@ -54,6 +65,7 @@ import {
SharedComposer,
} from "./shared-composer";
import { useChatRuntimeStore } from "./stores/chat-runtime-store";
import { useExternalProvidersStore } from "./stores/external-providers-store";
import { buildChatTourSteps } from "./tour";
import type { ChatView, MessageRecord } from "./types";
@ -536,6 +548,7 @@ export function ChatPage(): ReactElement {
const settingsOpen = useChatRuntimeStore((s) => s.settingsPanelOpen);
const setSettingsOpen = useChatRuntimeStore((s) => s.setSettingsPanelOpen);
const externalProviders = useExternalProvidersStore((s) => s.providers);
useEffect(() => {
const threadId = search.thread;
@ -596,7 +609,9 @@ export function ChatPage(): ReactElement {
loadProgress,
loadToastDismissed,
} = useChatModelRuntime();
const pendingNativeModelIntent = useNativeIntentStore((state) => state.pendingModelIntent);
const pendingNativeModelIntent = useNativeIntentStore(
(state) => state.pendingModelIntent,
);
const nativePathLeasesSupported = useNativePathLeasesSupported();
const refreshRef = useRef(refresh);
const selectModelRef = useRef(selectModel);
@ -605,9 +620,85 @@ export function ChatPage(): ReactElement {
refreshRef.current = refresh;
selectModelRef.current = selectModel;
}, [refresh, selectModel]);
const isExternalModel = useMemo(
() => isExternalModelId(inferenceParams.checkpoint),
[inferenceParams.checkpoint],
);
const reasoningEnabled = useChatRuntimeStore((s) => s.reasoningEnabled);
const reasoningStyle = useChatRuntimeStore((s) => s.reasoningStyle);
const reasoningEffort = useChatRuntimeStore((s) => s.reasoningEffort);
const supportsReasoningOff = useChatRuntimeStore((s) => s.supportsReasoningOff);
const activeExternalProviderType = useMemo(() => {
const selection = parseExternalModelId(inferenceParams.checkpoint);
if (!selection) return null;
const provider = externalProviders.find(
(p) => p.id === selection.providerId,
);
return provider?.providerType ?? null;
}, [externalProviders, inferenceParams.checkpoint]);
const activeProviderCapabilities = useMemo(() => {
const selection = parseExternalModelId(inferenceParams.checkpoint);
if (!selection) return null;
const provider = externalProviders.find(
(p) => p.id === selection.providerId,
);
const baseCapabilities = getProviderCapabilities(provider?.providerType);
if (!baseCapabilities) return baseCapabilities;
const anthropicThinkingEnabled =
provider?.providerType === "anthropic" &&
reasoningStyle === "reasoning_effort" &&
(supportsReasoningOff ? reasoningEnabled : true) &&
reasoningEffort !== "none";
if (!anthropicThinkingEnabled) return baseCapabilities;
return {
...baseCapabilities,
temperature: false,
topK: false,
};
}, [
externalProviders,
inferenceParams.checkpoint,
reasoningEnabled,
reasoningStyle,
reasoningEffort,
supportsReasoningOff,
]);
useEffect(() => {
const selection = parseExternalModelId(inferenceParams.checkpoint);
if (!selection) return;
const provider = externalProviders.find((p) => p.id === selection.providerId);
const reasoningCaps = getExternalReasoningCapabilities(
provider?.providerType,
selection.modelId,
);
const state = useChatRuntimeStore.getState();
const preferredEffort = state.reasoningEffort;
const effortLevels = reasoningCaps.reasoningEffortLevels;
const clampedEffort = clampReasoningEffortToLevels(
preferredEffort,
effortLevels,
);
const nextReasoningEffort = reasoningCaps.supportsReasoning
? clampedEffort
: state.reasoningEffort;
useChatRuntimeStore.setState({
supportsReasoning: reasoningCaps.supportsReasoning,
reasoningAlwaysOn: reasoningCaps.reasoningAlwaysOn,
reasoningStyle: reasoningCaps.reasoningStyle,
supportsReasoningOff: reasoningCaps.supportsReasoningOff,
reasoningEffortLevels: effortLevels,
reasoningEffort: nextReasoningEffort,
reasoningEnabled: reasoningCaps.supportsReasoning
? reasoningCaps.supportsReasoningOff
? state.reasoningEnabled
: true
: state.reasoningEnabled,
supportsPreserveThinking: false,
});
}, [externalProviders, inferenceParams.checkpoint]);
const canCompare = useMemo(() => {
return Boolean(inferenceParams.checkpoint);
}, [inferenceParams.checkpoint]);
return Boolean(inferenceParams.checkpoint) && !isExternalModel;
}, [inferenceParams.checkpoint, isExternalModel]);
// Derive view from URL search params
const view = useMemo<ChatView>(() => {
@ -632,7 +723,8 @@ export function ChatPage(): ReactElement {
const hasActiveModel = Boolean(inferenceParams.checkpoint);
const loadNativeModelIntent = useCallback(
async (intent: NativeIntent, loadingDescription: string) => {
const label = intent.path.displayLabel || intent.displayLabel || "Local GGUF model";
const label =
intent.path.displayLabel || intent.displayLabel || "Local GGUF model";
await selectModel({
id: label,
nativePathToken: intent.path.token,
@ -687,6 +779,7 @@ export function ChatPage(): ReactElement {
(
value: string,
meta?: {
source?: string;
isLora: boolean;
ggufVariant?: string;
isDownloaded?: boolean;
@ -702,6 +795,58 @@ export function ChatPage(): ReactElement {
(meta?.ggufVariant ?? null) === (currentVariant ?? null))
)
return;
if (meta?.source === "external" || isExternalModelId(value)) {
const selectedExternal = parseExternalModelId(value);
const selectedProvider = selectedExternal
? externalProviders.find((p) => p.id === selectedExternal.providerId)
: null;
const reasoningCaps = getExternalReasoningCapabilities(
selectedProvider?.providerType,
selectedExternal?.modelId,
);
const preferredEffort = store.reasoningEffort;
const effortLevels = reasoningCaps.reasoningEffortLevels;
const clampedEffort = clampReasoningEffortToLevels(
preferredEffort,
effortLevels,
);
const nextReasoningEffort = reasoningCaps.supportsReasoning
? clampedEffort
: store.reasoningEffort;
// Clear any cached router-picked openrouter/free model unless the
// user is staying on openrouter/free — otherwise the chip would
// keep showing a stale ":<chosen>" suffix from a previous model.
const stillOnOpenRouterFree =
selectedProvider?.providerType === "openrouter" &&
selectedExternal?.modelId === "openrouter/free";
setInferenceParams({
...store.params,
checkpoint: value,
});
useChatRuntimeStore.setState({
activeGgufVariant: null,
ggufContextLength: null,
ggufMaxContextLength: null,
ggufNativeContextLength: null,
activeNativePathToken: null,
supportsReasoning: reasoningCaps.supportsReasoning,
reasoningAlwaysOn: reasoningCaps.reasoningAlwaysOn,
reasoningStyle: reasoningCaps.reasoningStyle,
supportsReasoningOff: reasoningCaps.supportsReasoningOff,
reasoningEffortLevels: effortLevels,
reasoningEffort: nextReasoningEffort,
reasoningEnabled: reasoningCaps.supportsReasoning
? reasoningCaps.supportsReasoningOff
? store.reasoningEnabled
: true
: store.reasoningEnabled,
supportsPreserveThinking: false,
...(stillOnOpenRouterFree ? {} : { lastOpenRouterChosenModel: null }),
});
return;
}
// Local model picked → drop any cached openrouter/free chosen model.
useChatRuntimeStore.setState({ lastOpenRouterChosenModel: null });
void (async () => {
let showImageCompatibilityWarning = false;
if (view.mode === "single" && activeThreadId) {
@ -738,7 +883,14 @@ export function ChatPage(): ReactElement {
});
})();
},
[activeThreadId, modelsFromStore, selectModel, view],
[
activeThreadId,
externalProviders,
modelsFromStore,
selectModel,
setInferenceParams,
view,
],
);
const handleEject = useCallback(() => {
void ejectModel();
@ -813,6 +965,47 @@ export function ChatPage(): ReactElement {
})),
[modelsFromStore],
);
const lastOpenRouterChosenModel = useChatRuntimeStore(
(s) => s.lastOpenRouterChosenModel,
);
const externalModels = useMemo<ExternalModelOption[]>(
() =>
externalProviders.flatMap((provider) =>
provider.models.map((model) => {
// For OpenRouter's free router we know which underlying free
// model the gateway actually picked once a stream completes
// (chat-adapter latches `chunk.model` into the runtime store).
// Render the chip as `openrouter:<short-chosen>` — drop the
// redundant `/free` from the router id and the org prefix
// from the chosen id (e.g.
// openrouter/free + inclusionai/ring-2.6-1t-20260508:free
// -> openrouter:ring-2.6-1t-20260508:free
// ). The `:free` suffix on the chosen id already conveys
// 'free model', so the leading `/free` is noise.
let displayName = model;
if (
provider.providerType === "openrouter" &&
model === "openrouter/free" &&
lastOpenRouterChosenModel
) {
const lastSlash = lastOpenRouterChosenModel.lastIndexOf("/");
const shortChosen =
lastSlash >= 0
? lastOpenRouterChosenModel.slice(lastSlash + 1)
: lastOpenRouterChosenModel;
displayName = `openrouter:${shortChosen}`;
}
return {
id: buildExternalModelId(provider.id, model),
name: displayName,
providerId: provider.id,
providerName: provider.name,
providerType: provider.providerType,
};
}),
),
[externalProviders, lastOpenRouterChosenModel],
);
const [localModels, setLocalModels] = useState<LoraModelOption[]>([]);
@ -847,20 +1040,24 @@ export function ChatPage(): ReactElement {
.catch(() => {});
}, [navigate]);
const refreshModelLists = useCallback((deletedModel?: DeletedModelRef) => {
const { checkpoint } = useChatRuntimeStore.getState().params;
const activeGgufVariant = useChatRuntimeStore.getState().activeGgufVariant;
if (
modelMatchesDeleted(
{ id: checkpoint, ggufVariant: activeGgufVariant },
deletedModel,
)
) {
useChatRuntimeStore.getState().clearCheckpoint();
}
void refresh();
refreshLocalModels();
}, [refresh, refreshLocalModels]);
const refreshModelLists = useCallback(
(deletedModel?: DeletedModelRef) => {
const { checkpoint } = useChatRuntimeStore.getState().params;
const activeGgufVariant =
useChatRuntimeStore.getState().activeGgufVariant;
if (
modelMatchesDeleted(
{ id: checkpoint, ggufVariant: activeGgufVariant },
deletedModel,
)
) {
useChatRuntimeStore.getState().clearCheckpoint();
}
void refresh();
refreshLocalModels();
},
[refresh, refreshLocalModels],
);
const loraModels = useMemo<LoraModelOption[]>(() => {
const fromLoras = lorasFromStore.map((lora) => ({
@ -1001,6 +1198,7 @@ export function ChatPage(): ReactElement {
<ModelSelector
models={models}
loraModels={loraModels}
externalModels={externalModels}
value={inferenceParams.checkpoint}
activeGgufVariant={activeGgufVariant}
onValueChange={handleCheckpointChange}
@ -1014,6 +1212,7 @@ export function ChatPage(): ReactElement {
onOpenChange={handleModelSelectorOpenChange}
triggerDataTour="chat-model-selector"
contentDataTour="chat-model-selector-popover"
showCloudIndicator={isExternalModel}
className="max-w-[62vw] !pr-3 sm:max-w-none !h-[34px]"
/>
)}
@ -1120,6 +1319,9 @@ export function ChatPage(): ReactElement {
onOpenChange={setSettingsOpen}
params={inferenceParams}
onParamsChange={setInferenceParams}
isExternalModel={isExternalModel}
providerCapabilities={activeProviderCapabilities}
externalProviderType={activeExternalProviderType}
onReloadModel={() => {
const state = useChatRuntimeStore.getState();
if (state.params.checkpoint) {

1361
studio/frontend/src/features/chat/chat-providers-dialog.tsx Normal file
View file

File diff suppressed because it is too large Load diff

199
studio/frontend/src/features/chat/chat-settings-sheet.tsx
View file

@ -80,6 +80,11 @@ import {
toPresetParams,
type Preset,
} from "./presets/preset-policy";
import {
EXTERNAL_MAX_OUTPUT_TOKENS,
getExternalMinOutputTokens,
type ProviderCapabilities,
} from "./provider-capabilities";
import type { InferenceParams } from "./types/runtime";
export { defaultInferenceParams, type Preset } from "./presets/preset-policy";
@ -505,6 +510,19 @@ interface ChatSettingsPanelProps {
onOpenChange?: (open: boolean) => void;
params: InferenceParams;
onParamsChange: (params: InferenceParams) => void;
isExternalModel?: boolean;
/**
* Sampling-param capability set for the active external provider, or `null`
* for local models (in which case every knob is rendered). Drives the
* per-param visibility in the sampling section.
*/
providerCapabilities?: ProviderCapabilities | null;
/**
* Backend provider type for the active external model (e.g. "kimi",
* "anthropic", "openai"), or `null` for local models. Drives the
* per-provider Max Tokens floor in the slider.
*/
externalProviderType?: string | null;
onReloadModel?: () => void;
}
@ -513,11 +531,28 @@ export function ChatSettingsPanel({
onOpenChange,
params,
onParamsChange,
isExternalModel = false,
providerCapabilities = null,
externalProviderType = null,
onReloadModel,
}: ChatSettingsPanelProps) {
// For non-external (local) models we show every knob — providerCapabilities
// is only consulted when `isExternalModel` is true. An external model with an
// unknown provider falls back to the OpenAI-compat shape via
// getProviderCapabilities, so these flags never undercount support.
const showTemperature =
!isExternalModel || Boolean(providerCapabilities?.temperature);
const showTopP = !isExternalModel || Boolean(providerCapabilities?.topP);
const showTopK = !isExternalModel || Boolean(providerCapabilities?.topK);
const showMinP = !isExternalModel || Boolean(providerCapabilities?.minP);
const showRepetitionPenalty =
!isExternalModel || Boolean(providerCapabilities?.repetitionPenalty);
const showPresencePenalty =
!isExternalModel || Boolean(providerCapabilities?.presencePenalty);
const isMobile = useIsMobile();
const isGguf = useChatRuntimeStore((s) => s.activeGgufVariant) != null;
const hasModelContent = isGguf || Boolean(params.checkpoint);
const hasModelContent =
!isExternalModel && (isGguf || Boolean(params.checkpoint));
const speculativeType = useChatRuntimeStore((s) => s.speculativeType);
const setSpeculativeType = useChatRuntimeStore((s) => s.setSpeculativeType);
const loadedSpeculativeType = useChatRuntimeStore(
@ -1131,65 +1166,79 @@ export function ChatSettingsPanel({
<CollapsibleSection label="Sampling" defaultOpen={true}>
<div className="flex flex-col gap-5 pt-1">
<ParamSlider
label="Temperature"
value={params.temperature}
min={0}
max={2}
step={0.01}
onChange={set("temperature")}
info="Controls randomness. Lower values make output focused and deterministic; higher values increase variety and creativity."
/>
<ParamSlider
label="Top P"
value={params.topP}
min={0}
max={1}
step={0.05}
onChange={set("topP")}
displayValue={params.topP === 1 ? "Off" : undefined}
info="Nucleus sampling. Restricts choices to the smallest set of tokens whose cumulative probability reaches this threshold. 1.0 = off."
/>
<ParamSlider
label="Top K"
value={params.topK}
min={0}
max={100}
step={1}
onChange={set("topK")}
displayValue={params.topK === 0 ? "Off" : undefined}
info="Limits sampling to the K most likely tokens at each step. 0 = off."
/>
<ParamSlider
label="Min P"
value={params.minP}
min={0}
max={1}
step={0.01}
onChange={set("minP")}
info="Drops tokens whose probability is below this fraction of the top token's probability. Filters unlikely candidates."
/>
<ParamSlider
label="Repetition Penalty"
value={params.repetitionPenalty}
min={1}
max={2}
step={0.05}
onChange={set("repetitionPenalty")}
displayValue={params.repetitionPenalty === 1 ? "Off" : undefined}
info="Down-weights tokens that have already appeared, reducing repetition. 1.0 = off; higher values penalize more strongly."
/>
<ParamSlider
label="Presence Penalty"
value={params.presencePenalty}
min={0}
max={2}
step={0.1}
onChange={set("presencePenalty")}
displayValue={params.presencePenalty === 0 ? "Off" : undefined}
info="Penalizes any token that has already appeared at least once, encouraging the model to introduce new topics. 0 = off."
/>
{!isGguf && (
{showTemperature ? (
<ParamSlider
label="Temperature"
value={params.temperature}
min={0}
max={2}
step={0.01}
onChange={set("temperature")}
info="Controls randomness. Lower values make output focused and deterministic; higher values increase variety and creativity."
/>
) : null}
{showTopP ? (
<ParamSlider
label="Top P"
value={params.topP}
min={0}
max={1}
step={0.05}
onChange={set("topP")}
displayValue={params.topP === 1 ? "Off" : undefined}
info="Nucleus sampling. Restricts choices to the smallest set of tokens whose cumulative probability reaches this threshold. 1.0 = off."
/>
) : null}
{showTopK ? (
<ParamSlider
label="Top K"
value={params.topK}
min={0}
max={100}
step={1}
onChange={set("topK")}
displayValue={params.topK === 0 ? "Off" : undefined}
info="Limits sampling to the K most likely tokens at each step. 0 = off."
/>
) : null}
{showMinP ? (
<ParamSlider
label="Min P"
value={params.minP}
min={0}
max={1}
step={0.01}
onChange={set("minP")}
info="Drops tokens whose probability is below this fraction of the top token's probability. Filters unlikely candidates."
/>
) : null}
{showRepetitionPenalty ? (
<ParamSlider
label="Repetition Penalty"
value={params.repetitionPenalty}
min={1}
max={2}
step={0.05}
onChange={set("repetitionPenalty")}
displayValue={
params.repetitionPenalty === 1 ? "Off" : undefined
}
info="Down-weights tokens that have already appeared, reducing repetition. 1.0 = off; higher values penalize more strongly."
/>
) : null}
{showPresencePenalty ? (
<ParamSlider
label="Presence Penalty"
value={params.presencePenalty}
min={0}
max={2}
step={0.1}
onChange={set("presencePenalty")}
displayValue={params.presencePenalty === 0 ? "Off" : undefined}
info="Penalizes any token that has already appeared at least once, encouraging the model to introduce new topics. 0 = off."
/>
) : null}
{!isExternalModel && !isGguf && (
<ParamSlider
label="Max Seq Length"
value={params.maxSeqLength}
@ -1203,8 +1252,18 @@ export function ChatSettingsPanel({
<ParamSlider
label="Max Tokens"
value={params.maxTokens}
min={64}
max={isGguf && ggufContextLength ? ggufContextLength : 32768}
min={
isExternalModel
? getExternalMinOutputTokens(externalProviderType)
: 64
}
max={
isExternalModel
? EXTERNAL_MAX_OUTPUT_TOKENS
: isGguf && ggufContextLength
? ggufContextLength
: 32768
}
step={64}
onChange={set("maxTokens")}
displayValue={
@ -1219,13 +1278,15 @@ export function ChatSettingsPanel({
</div>
</CollapsibleSection>
<CollapsibleSection label="Tools">
<div className="flex flex-col gap-5 pt-1">
<AutoHealToolCallsToggle />
<MaxToolCallsSlider />
<ToolCallTimeoutSlider />
</div>
</CollapsibleSection>
{!isExternalModel ? (
<CollapsibleSection label="Tools">
<div className="flex flex-col gap-5 pt-1">
<AutoHealToolCallsToggle />
<MaxToolCallsSlider />
<ToolCallTimeoutSlider />
</div>
</CollapsibleSection>
) : null}
</div>
</div>
<Dialog

230
studio/frontend/src/features/chat/external-providers.ts Normal file
View file

@ -0,0 +1,230 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
export interface ExternalProviderConfig {
id: string;
/** Backend provider type (e.g. openai, mistral, gemini). */
providerType: string;
/** Display name in UI. */
name: string;
/** Provider base URL (default from registry or backend-saved override). */
baseUrl: string;
/** Model ids user enabled from `/api/providers/models`. */
models: string[];
/** Cached available model ids from the provider's /models response. */
availableModels?: string[];
createdAt: number;
updatedAt: number;
}
const EXTERNAL_PROVIDERS_KEY = "unsloth_chat_external_providers";
const EXTERNAL_PROVIDER_KEYS_KEY = "unsloth_chat_external_provider_keys";
const EXTERNAL_MODEL_PREFIX = "external::";
function canUseStorage(): boolean {
return typeof window !== "undefined";
}
export function isExternalModelId(
value: string | null | undefined,
): value is string {
return typeof value === "string" && value.startsWith(EXTERNAL_MODEL_PREFIX);
}
export function buildExternalModelId(providerId: string, modelId: string): string {
return `${EXTERNAL_MODEL_PREFIX}${providerId}::${encodeURIComponent(modelId)}`;
}
export function parseExternalModelId(
value: string | null | undefined,
): { providerId: string; modelId: string } | null {
if (!isExternalModelId(value)) return null;
const payload = value.slice(EXTERNAL_MODEL_PREFIX.length);
const separator = payload.indexOf("::");
if (separator < 0) return null;
const providerId = payload.slice(0, separator);
const encodedModelId = payload.slice(separator + 2);
if (!providerId || !encodedModelId) return null;
try {
return { providerId, modelId: decodeURIComponent(encodedModelId) };
} catch {
return null;
}
}
function isExternalProviderConfig(value: unknown): value is ExternalProviderConfig {
if (!value || typeof value !== "object") return false;
const maybe = value as Partial<ExternalProviderConfig>;
return (
typeof maybe.id === "string" &&
typeof maybe.providerType === "string" &&
typeof maybe.name === "string" &&
typeof maybe.baseUrl === "string" &&
Array.isArray(maybe.models)
);
}
function mapLegacyPresetToProviderType(presetId: string): string {
if (presetId === "google") return "gemini";
return presetId;
}
function normalizeProvider(raw: ExternalProviderConfig): ExternalProviderConfig {
return {
...raw,
providerType: raw.providerType.trim(),
name: raw.name.trim(),
baseUrl: raw.baseUrl.trim(),
models: raw.models
.map((model) => model.trim())
.filter((model) => model.length > 0),
availableModels: (raw.availableModels ?? [])
.map((model) => model.trim())
.filter((model) => model.length > 0),
};
}
function isCompleteProvider(provider: ExternalProviderConfig): boolean {
if (!provider.id || !provider.name || !provider.providerType) return false;
return true;
}
type LegacyProviderConfig = {
id?: unknown;
presetId?: unknown;
name?: unknown;
baseUrl?: unknown;
models?: unknown;
createdAt?: unknown;
updatedAt?: unknown;
};
function fromUnknownProvider(value: unknown): ExternalProviderConfig | null {
if (!value || typeof value !== "object") return null;
if (isExternalProviderConfig(value)) {
return value;
}
const legacy = value as LegacyProviderConfig;
const id = typeof legacy.id === "string" ? legacy.id : "";
const presetId = typeof legacy.presetId === "string" ? legacy.presetId : "";
if (!id || !presetId || presetId === "custom") return null;
const providerType = mapLegacyPresetToProviderType(presetId);
if (!providerType) return null;
return {
id,
providerType,
name: typeof legacy.name === "string" ? legacy.name : providerType,
baseUrl: typeof legacy.baseUrl === "string" ? legacy.baseUrl : "",
models: Array.isArray(legacy.models)
? legacy.models.filter((item): item is string => typeof item === "string")
: [],
createdAt: typeof legacy.createdAt === "number" ? legacy.createdAt : Date.now(),
updatedAt: typeof legacy.updatedAt === "number" ? legacy.updatedAt : Date.now(),
};
}
export function loadExternalProviders(): ExternalProviderConfig[] {
if (!canUseStorage()) return [];
try {
const raw = localStorage.getItem(EXTERNAL_PROVIDERS_KEY);
if (!raw) return [];
const parsed = JSON.parse(raw) as unknown;
if (!Array.isArray(parsed)) return [];
return parsed
.map(fromUnknownProvider)
.filter((provider): provider is ExternalProviderConfig => provider !== null)
.map(normalizeProvider)
.filter(isCompleteProvider);
} catch {
return [];
}
}
/**
* Load the raw (encrypted or legacy plaintext) key map from localStorage.
* Values are opaque strings — either AES-GCM ciphertext or legacy plaintext.
*/
function loadRawKeyMap(): Record<string, string> {
if (!canUseStorage()) return {};
try {
const raw = localStorage.getItem(EXTERNAL_PROVIDER_KEYS_KEY);
if (!raw) return {};
const parsed = JSON.parse(raw) as unknown;
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) return {};
const out: Record<string, string> = {};
for (const [providerId, value] of Object.entries(parsed)) {
if (typeof providerId === "string" && typeof value === "string") {
out[providerId] = value;
}
}
return out;
} catch {
return {};
}
}
function saveRawKeyMap(map: Record<string, string>): void {
if (!canUseStorage()) return;
try {
localStorage.setItem(EXTERNAL_PROVIDER_KEYS_KEY, JSON.stringify(map));
} catch {
// ignore
}
}
export function saveExternalProviders(
providers: ExternalProviderConfig[],
): void {
if (!canUseStorage()) return;
try {
localStorage.setItem(EXTERNAL_PROVIDERS_KEY, JSON.stringify(providers));
// Prune keys for removed providers — works on raw ciphertext, no decryption needed
const allowedIds = new Set(providers.map((provider) => provider.id));
const keys = loadRawKeyMap();
const pruned: Record<string, string> = {};
for (const [providerId, value] of Object.entries(keys)) {
if (allowedIds.has(providerId)) {
pruned[providerId] = value;
}
}
saveRawKeyMap(pruned);
} catch {
// ignore
}
}
/**
* Retrieve a provider API key from localStorage.
* Returns "" if no key is stored.
*/
export function getExternalProviderApiKey(
providerId: string,
): string {
const keys = loadRawKeyMap();
return keys[providerId] ?? "";
}
/**
* Store a provider API key in localStorage.
*/
export function setExternalProviderApiKey(
providerId: string,
apiKey: string,
): void {
if (!canUseStorage()) return;
const keys = loadRawKeyMap();
keys[providerId] = apiKey;
saveRawKeyMap(keys);
}
export function removeExternalProviderApiKey(providerId: string): void {
if (!canUseStorage()) return;
try {
const keys = loadRawKeyMap();
delete keys[providerId];
saveRawKeyMap(keys);
} catch {
// ignore
}
}

61
studio/frontend/src/features/chat/hooks/use-chat-model-runtime.ts
View file

@ -23,7 +23,10 @@ import {
validateModel,
} from "../api/chat-api";
import { formatEta, formatRate } from "../utils/format-transfer";
import { useChatRuntimeStore } from "../stores/chat-runtime-store";
import {
type ReasoningEffort,
useChatRuntimeStore,
} from "../stores/chat-runtime-store";
import {
mergeBackendRecommendedInference,
resolveLoadMaxSeqLength,
@ -31,6 +34,7 @@ import {
import {
isMultimodalResponse,
} from "../types/api";
import { isExternalModelId } from "../external-providers";
import type {
ChatLoraSummary,
ChatModelSummary,
@ -143,6 +147,15 @@ function normalizeSpeculativeType(v: string | null | undefined): string | null {
return "default";
}
type LocalReasoningEffort = Extract<ReasoningEffort, "low" | "medium" | "high">;
function clampLocalReasoningEffort(value: ReasoningEffort): LocalReasoningEffort {
if (value === "low" || value === "medium" || value === "high") {
return value;
}
return "low";
}
export function useChatModelRuntime() {
const params = useChatRuntimeStore((state) => state.params);
const models = useChatRuntimeStore((state) => state.models);
@ -221,7 +234,9 @@ export function useChatModelRuntime() {
setModels(listRes.models.map(toChatModelSummary));
setLoras(lorasRes.loras.map(toLoraSummary));
if (statusRes.active_model) {
const selectedCheckpoint = useChatRuntimeStore.getState().params.checkpoint;
const isExternalSelectionActive = isExternalModelId(selectedCheckpoint);
if (statusRes.active_model && !isExternalSelectionActive) {
setCheckpoint(statusRes.active_model, statusRes.gguf_variant);
// Apply inference defaults on reconnect (page refresh with model already loaded)
@ -241,6 +256,10 @@ export function useChatModelRuntime() {
const supportsReasoning = statusRes.supports_reasoning ?? false;
const reasoningAlwaysOn = statusRes.reasoning_always_on ?? false;
const reasoningStyle = statusRes.reasoning_style ?? "enable_thinking";
const reasoningEffortLevels =
reasoningStyle === "reasoning_effort"
? (["low", "medium", "high"] as const)
: (["low", "medium", "high"] as const);
const supportsPreserveThinking = statusRes.supports_preserve_thinking ?? false;
const supportsTools = statusRes.supports_tools ?? false;
const currentGgufContextLength = statusRes.is_gguf
@ -262,6 +281,9 @@ export function useChatModelRuntime() {
// Otherwise we'd clobber the values the load path just applied and
// the UI would appear to revert the user's changes.
const prevState = useChatRuntimeStore.getState();
const clampedReasoningEffort = clampLocalReasoningEffort(
prevState.reasoningEffort,
);
const nextDefaultChatTemplate =
statusRes.chat_template === undefined
? prevState.defaultChatTemplate
@ -270,12 +292,25 @@ export function useChatModelRuntime() {
supportsReasoning,
reasoningAlwaysOn,
reasoningStyle,
supportsReasoningOff: reasoningStyle !== "reasoning_effort",
reasoningEffortLevels,
reasoningEffort: clampedReasoningEffort,
supportsPreserveThinking,
supportsTools,
// Reset per-turn reasoning flag so models that do not support
// reasoning do not inherit a stale off state from a prior model.
// Reset per-turn reasoning flag so:
// 1. models that do not support reasoning do not inherit a stale
// off state from a prior model, and
// 2. local reasoning-effort models (where the composer hides
// the Off option via supportsReasoningOff=false) cannot end
// up with reasoningEnabled=false carried over from an
// external model where Off was selected — the composer would
// keep showing "Think: <level>" via effectiveReasoningEnabled,
// but the chat-adapter would omit the kwarg and the Harmony
// template would fall back to its own default effort.
reasoningEnabled: supportsReasoning
? useChatRuntimeStore.getState().reasoningEnabled
? reasoningStyle === "reasoning_effort"
? true
: useChatRuntimeStore.getState().reasoningEnabled
: true,
ggufContextLength: currentGgufContextLength,
ggufMaxContextLength,
@ -313,7 +348,7 @@ export function useChatModelRuntime() {
}
useChatRuntimeStore.getState().setReasoningEnabled(reasoningDefault);
}
} else {
} else if (!statusRes.active_model && !isExternalSelectionActive) {
useChatRuntimeStore.setState({
modelRequiresTrustRemoteCode: false,
loadedIsMultimodal: false,
@ -569,6 +604,15 @@ export function useChatModelRuntime() {
// context state and display the backend-reported effective context.
const keepCustomCtx = null;
const reasoningAlwaysOn = loadResponse.reasoning_always_on ?? false;
const reasoningStyle = loadResponse.reasoning_style ?? "enable_thinking";
const reasoningEffortLevels =
reasoningStyle === "reasoning_effort"
? (["low", "medium", "high"] as const)
: (["low", "medium", "high"] as const);
const existingReasoningEffort = useChatRuntimeStore.getState().reasoningEffort;
const clampedReasoningEffort = clampLocalReasoningEffort(
existingReasoningEffort,
);
const ggufMaxContextLength = reportedMaxCtx;
useChatRuntimeStore.setState({
ggufContextLength: nativeCtx,
@ -579,7 +623,10 @@ export function useChatModelRuntime() {
supportsReasoning: loadResponse.supports_reasoning ?? false,
reasoningAlwaysOn,
reasoningEnabled: reasoningAlwaysOn ? true : reasoningDefault,
reasoningStyle: loadResponse.reasoning_style ?? "enable_thinking",
reasoningStyle,
supportsReasoningOff: reasoningStyle !== "reasoning_effort",
reasoningEffortLevels,
reasoningEffort: clampedReasoningEffort,
supportsPreserveThinking: loadResponse.supports_preserve_thinking ?? false,
supportsTools: loadResponse.supports_tools ?? false,
toolsEnabled: loadResponse.supports_tools ?? false,

448
studio/frontend/src/features/chat/provider-capabilities.ts Normal file
View file

@ -0,0 +1,448 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
/**
* Per-provider sampling parameter capability matrix.
*
* Values are derived from each provider's published chat-completion docs as of
* 2026-05. They describe which of our UI knobs map cleanly onto the provider's
* request body; the panel hides params a provider does not accept so users
* cannot dial a value that gets silently dropped or rejected.
*
* "Local" models (anything that is not an external provider) are represented by
* a null capability — every knob renders for them.
*/
export interface ProviderCapabilities {
/**
* Temperature sampling. Reasoning-class models (OpenAI's gpt-5.x / o3 via
* /v1/responses) reject this with `Unsupported parameter`.
*/
temperature: boolean;
/** Nucleus (top_p) sampling. Same restriction as `temperature` on OpenAI. */
topP: boolean;
/** top-k token sampling (only Anthropic on the providers we ship). */
topK: boolean;
/** min-p token cutoff (no SaaS provider currently exposes this). */
minP: boolean;
/** Repetition penalty (no SaaS provider currently exposes this). */
repetitionPenalty: boolean;
/** OpenAI-style presence penalty. */
presencePenalty: boolean;
}
export type ExternalReasoningCapabilities = {
supportsReasoning: boolean;
reasoningStyle: "enable_thinking" | "reasoning_effort";
reasoningAlwaysOn: boolean;
supportsReasoningOff: boolean;
reasoningEffortLevels: readonly (
| "none"
| "minimal"
| "low"
| "medium"
| "high"
| "max"
| "xhigh"
)[];
};
/**
* Prefer a stored reasoning effort level that exists in ``effortLevels``,
* mapping legacy "xhigh" to "max" when the model only exposes the latter
* (Claude 4.6 adaptive thinking).
*/
export function clampReasoningEffortToLevels(
preferred: ExternalReasoningCapabilities["reasoningEffortLevels"][number],
effortLevels: ExternalReasoningCapabilities["reasoningEffortLevels"],
): ExternalReasoningCapabilities["reasoningEffortLevels"][number] {
let candidate = preferred;
if (
candidate === "xhigh" &&
!effortLevels.includes("xhigh") &&
effortLevels.includes("max")
) {
candidate = "max";
}
if (effortLevels.includes(candidate)) {
return candidate;
}
return effortLevels[0] ?? "low";
}
/**
* Output-token cap for any external provider request. Picked to stay below the
* tightest declared limit across the providers we ship (Anthropic Claude Opus
* tops out at 128k, GPT-5.x ~128k, Gemini 2.5 ~65k, DeepSeek 8k) while staying
* well above what a typical chat reply needs. The local-model path is not
* subject to this — local backends honour whatever the loaded context allows.
*
* If a user's stored maxTokens (e.g. carried over from a prior local-model
* session with a 128k+ context) exceeds this, chat-adapter clamps the
* outbound request so the provider does not 400 on it.
*/
export const EXTERNAL_MAX_OUTPUT_TOKENS = 32768;
/**
* Per-provider minimum on the outbound max_tokens. Kimi's docs require
* `max_tokens >= 16000` whenever a thinking model is in use so the
* reasoning_content and final answer both fit in the budget — anything
* lower truncates the response mid-stream. Other providers don't have a
* documented floor, so they fall through to the generic min of 64 in
* the slider.
*
* The chat-adapter resolves the effective floor on send and bumps the
* outbound max_tokens up to this value if the user's stored maxTokens
* sits below it. The settings panel reflects the same floor as the
* slider min so the displayed value never drifts from what's sent.
*/
const EXTERNAL_MIN_OUTPUT_TOKENS_BY_PROVIDER: Record<string, number> = {
kimi: 16000,
};
export function getExternalMinOutputTokens(
providerType: string | null | undefined,
): number {
if (!providerType) return 64;
return EXTERNAL_MIN_OUTPUT_TOKENS_BY_PROVIDER[providerType] ?? 64;
}
const OPENAI_COMPAT_BASE: ProviderCapabilities = {
temperature: true,
topP: true,
topK: false,
minP: false,
repetitionPenalty: false,
presencePenalty: true,
};
const ALL_SUPPORTED: ProviderCapabilities = {
temperature: true,
topP: true,
topK: true,
minP: true,
repetitionPenalty: true,
presencePenalty: true,
};
const PROVIDER_CAPABILITIES: Record<string, ProviderCapabilities> = {
// OpenAI's flagship models (gpt-5.x / o3 / gpt-4.5) are reasoning-class
// models served via /v1/responses, which rejects temperature, top_p, and
// presence/frequency penalty. See backend
// external_provider._stream_openai_responses for the proxy.
openai: {
temperature: false,
topP: false,
topK: false,
minP: false,
repetitionPenalty: false,
presencePenalty: false,
},
// Anthropic's Messages API accepts top_k on 3.x and 4.5/4.6, but Claude
// 4.7 (Opus/Sonnet/Haiku) deprecated it and returns 400 if it is set.
// We surface top_k in the panel for all Anthropic providers and let the
// backend strip it per-model — see _stream_anthropic in
// studio/backend/core/inference/external_provider.py.
// Presence/frequency penalty is not part of the Messages API on any
// Claude generation.
anthropic: {
temperature: true,
topP: true,
topK: true,
minP: false,
repetitionPenalty: false,
presencePenalty: false,
},
mistral: OPENAI_COMPAT_BASE,
gemini: OPENAI_COMPAT_BASE,
// Kimi k2.5/k2.6 are reasoning-class — the API locks temperature and
// top_p to fixed defaults and 400s on any other value:
// "invalid temperature: only 1 is allowed for this model".
// Hide both sliders so the user is not offered knobs the model
// silently overrides. Backend additionally strips these fields via
// PROVIDER_REGISTRY['kimi']['body_omit'].
kimi: {
temperature: false,
topP: false,
topK: false,
minP: false,
repetitionPenalty: false,
presencePenalty: true,
},
// DeepSeek deprecated presence/frequency penalty in their current docs.
deepseek: {
temperature: true,
topP: true,
topK: false,
minP: false,
repetitionPenalty: false,
presencePenalty: false,
},
qwen: OPENAI_COMPAT_BASE,
huggingface: OPENAI_COMPAT_BASE,
// OpenRouter silently drops params the target model does not support, so we
// surface every knob and let the gateway handle the per-model fan-out.
openrouter: ALL_SUPPORTED,
// Custom providers are assumed OpenAI-compatible by the backend; users who
// point at vLLM/Ollama backends often want top_k / min_p / repetition,
// so be permissive.
custom: ALL_SUPPORTED,
};
const DEFAULT_EXTERNAL_CAPABILITIES = OPENAI_COMPAT_BASE;
/**
* Resolve the capability set for an external provider. Returns `null` for
* a local model (i.e. when `providerType` is null/undefined), which callers
* should treat as "every knob applies".
*/
export function getProviderCapabilities(
providerType: string | null | undefined,
): ProviderCapabilities | null {
if (!providerType) return null;
return PROVIDER_CAPABILITIES[providerType] ?? DEFAULT_EXTERNAL_CAPABILITIES;
}
const DEFAULT_EFFORT_LEVELS = ["low", "medium", "high"] as const;
const OPENROUTER_MANDATORY_REASONING_MODELS = new Set([
"google/gemini-pro-latest",
"baidu/cobuddy:free",
"inclusionai/ring-2.6-1t:free",
"deepseek/deepseek-r1",
]);
function isOpenRouterMandatoryReasoningModel(modelId: string): boolean {
const normalized = modelId.trim().toLowerCase();
const canonical = normalized.startsWith("~") ? normalized.slice(1) : normalized;
return OPENROUTER_MANDATORY_REASONING_MODELS.has(canonical);
}
type ReasoningCaps = {
supportsReasoning: boolean;
supportsReasoningOff: boolean;
reasoningEffortLevels: ExternalReasoningCapabilities["reasoningEffortLevels"];
};
const DEFAULT_EXTERNAL_REASONING_CAPABILITIES: ExternalReasoningCapabilities = {
supportsReasoning: false,
reasoningStyle: "enable_thinking",
reasoningAlwaysOn: false,
supportsReasoningOff: false,
reasoningEffortLevels: DEFAULT_EFFORT_LEVELS,
};
const NO_REASONING_CAPS: ReasoningCaps = {
supportsReasoning: false,
supportsReasoningOff: false,
reasoningEffortLevels: DEFAULT_EFFORT_LEVELS,
};
const ANTHROPIC_REASONING_MODELS = [
{
prefixes: ["claude-opus-4-7"],
levels: ["none", "low", "medium", "high", "xhigh"],
},
{
prefixes: ["claude-opus-4-6", "claude-sonnet-4-6"],
levels: ["none", "low", "medium", "high", "max"],
},
{
prefixes: ["claude-opus-4-5", "claude-sonnet-4-5", "claude-haiku-4-5"],
// Backend maps semantic levels to manual budget_tokens.
levels: ["none", "low", "medium", "high"],
},
] as const;
function matchesModelPrefix(
modelId: string,
prefixes: readonly string[],
): boolean {
return prefixes.some((prefix) => modelId.startsWith(prefix));
}
function resolveAnthropicReasoningEffortCapabilities(modelId: string): ReasoningCaps {
const normalized = modelId.trim().toLowerCase();
const matched = ANTHROPIC_REASONING_MODELS.find((entry) =>
matchesModelPrefix(normalized, entry.prefixes),
);
if (matched) {
return {
supportsReasoning: true,
supportsReasoningOff: true,
reasoningEffortLevels: matched.levels,
};
}
return NO_REASONING_CAPS;
}
const OPENAI_REASONING_MODELS = [
{
prefixes: ["gpt-5.5-pro", "gpt-5.4-pro"],
supportsOff: false,
levels: ["medium", "high", "xhigh"],
},
{
prefixes: ["gpt-5.5", "gpt-5.4"],
supportsOff: true,
levels: ["none", "low", "medium", "high", "xhigh"],
},
{
prefixes: ["gpt-5.3-chat-latest"],
supportsOff: false,
levels: ["medium"],
},
{
prefixes: ["gpt-5.3-codex"],
supportsOff: true,
levels: ["none", "low", "medium", "high", "xhigh"],
},
{
prefixes: ["gpt-5", "gpt-5.1", "gpt-5.2"],
supportsOff: false,
levels: ["minimal", "low", "medium", "high"],
},
{
prefixes: ["o3"],
supportsOff: false,
levels: DEFAULT_EFFORT_LEVELS,
},
] as const;
function resolveOpenAIReasoningEffortCapabilities(modelId: string): ReasoningCaps {
const normalized = modelId.trim().toLowerCase();
const matched = OPENAI_REASONING_MODELS.find((entry) =>
matchesModelPrefix(normalized, entry.prefixes),
);
if (matched) {
return {
supportsReasoning: true,
supportsReasoningOff: matched.supportsOff,
reasoningEffortLevels: matched.levels,
};
}
return NO_REASONING_CAPS;
}
function withEnableThinkingStyle(
overrides?: Partial<ExternalReasoningCapabilities>,
): ExternalReasoningCapabilities {
return {
...DEFAULT_EXTERNAL_REASONING_CAPABILITIES,
...overrides,
reasoningStyle: "enable_thinking",
};
}
function withReasoningEffortStyle(caps: ReasoningCaps): ExternalReasoningCapabilities {
return {
...DEFAULT_EXTERNAL_REASONING_CAPABILITIES,
supportsReasoning: true,
reasoningStyle: "reasoning_effort",
supportsReasoningOff: caps.supportsReasoningOff,
reasoningEffortLevels: caps.reasoningEffortLevels,
};
}
function resolveKimiReasoningCapabilities(modelId: string): ExternalReasoningCapabilities {
// Kimi exposes a boolean thinking toggle rather than an effort scale.
// - kimi-k2.6: thinking enabled by default, toggleable
// via extra_body: {thinking: {type: enabled|disabled}}
// - kimi-k2-thinking: thinking always on, no off switch
// - kimi-k2.5 (and anything else): no thinking
if (modelId === "kimi-k2-thinking") {
return withEnableThinkingStyle({
supportsReasoning: true,
reasoningAlwaysOn: true,
});
}
if (modelId === "kimi-k2.6") {
return withEnableThinkingStyle({
supportsReasoning: true,
supportsReasoningOff: true,
});
}
return withEnableThinkingStyle();
}
function resolveMistralReasoningCapabilities(modelId: string): ExternalReasoningCapabilities {
if (modelId === "magistral-medium-latest") {
return withReasoningEffortStyle({
supportsReasoning: true,
supportsReasoningOff: false,
// Native reasoning model: present baseline as Medium in the UI.
reasoningEffortLevels: ["medium", "high"] as const,
});
}
if (modelId === "mistral-small-latest" || modelId === "mistral-vibe-cli-latest") {
return withReasoningEffortStyle({
supportsReasoning: true,
supportsReasoningOff: true,
reasoningEffortLevels: ["none", "high"] as const,
});
}
return withEnableThinkingStyle();
}
/**
* resolve external-model thinking capabilities.
* provider-specific matching lives in the OpenAI/Anthropic resolvers.
* other providers default to no reasoning controls.
*/
export function getExternalReasoningCapabilities(
providerType: string | null | undefined,
modelId: string | null | undefined,
): ExternalReasoningCapabilities {
const normalizedModel = modelId?.trim().toLowerCase() ?? "";
const normalizedProvider = providerType?.trim().toLowerCase() ?? "";
if (!normalizedModel) {
return withEnableThinkingStyle();
}
// Some OpenRouter-routed ids are mandatory-reasoning and must stay on even
// if they arrive through aliased/custom provider routes.
if (isOpenRouterMandatoryReasoningModel(normalizedModel)) {
return withEnableThinkingStyle({
supportsReasoning: true,
reasoningAlwaysOn: true,
supportsReasoningOff: false,
});
}
// OpenRouter ids are namespaced (e.g. "openai/gpt-5.5").
const modelForMatching =
normalizedProvider === "openrouter" && normalizedModel.includes("/")
? normalizedModel.split("/").at(-1) ?? normalizedModel
: normalizedModel;
const isOpenAIProvider = normalizedProvider === "openai";
const isAnthropicProvider = normalizedProvider === "anthropic";
const isKimiProvider = normalizedProvider === "kimi";
const isMistralProvider = normalizedProvider === "mistral";
const isOpenRouterProvider = normalizedProvider === "openrouter";
if (isOpenRouterProvider) {
// OpenRouter's unified `reasoning` parameter is accepted on every
// chat-completion request; the gateway silently no-ops for models
// that don't reason. Mandatory-reasoning ids are handled by the
// early guard above; everything else exposes a toggleable control.
return {
supportsReasoning: true,
reasoningStyle: "enable_thinking",
reasoningAlwaysOn: false,
supportsReasoningOff: true,
reasoningEffortLevels: DEFAULT_EFFORT_LEVELS,
};
}
if (isKimiProvider) return resolveKimiReasoningCapabilities(modelForMatching);
if (isMistralProvider) return resolveMistralReasoningCapabilities(modelForMatching);
if (!isOpenAIProvider && !isAnthropicProvider) {
return withEnableThinkingStyle();
}
const providerCaps = isOpenAIProvider
? resolveOpenAIReasoningEffortCapabilities(modelForMatching)
: resolveAnthropicReasoningEffortCapabilities(modelForMatching);
if (providerCaps.supportsReasoning) {
return withReasoningEffortStyle(providerCaps);
}
return withEnableThinkingStyle();
}

173
studio/frontend/src/features/chat/shared-composer.tsx
View file

@ -18,7 +18,13 @@ import { useAui } from "@assistant-ui/react";
import { ArrowUpIcon, GlobeIcon, HeadphonesIcon, LightbulbIcon, LightbulbOffIcon, MicIcon, PlusIcon, SquareIcon, XIcon } from "lucide-react";
import { toast } from "sonner";
import { loadModel, validateModel } from "./api/chat-api";
import { useChatRuntimeStore } from "./stores/chat-runtime-store";
import { parseExternalModelId } from "./external-providers";
import { useExternalProvidersStore } from "./stores/external-providers-store";
import {
type ReasoningEffort,
useChatRuntimeStore,
} from "./stores/chat-runtime-store";
import { getExternalReasoningCapabilities } from "./provider-capabilities";
import {
type CompositionEvent,
type KeyboardEvent,
@ -66,6 +72,33 @@ function fileToBase64DataURL(file: File): Promise<string> {
});
}
function formatReasoningEffortLabel(level: ReasoningEffort, modelId?: string): string {
if (level === "max") return "Max";
if (level === "xhigh") {
const normalized = modelId?.trim().toLowerCase() ?? "";
if (
normalized.startsWith("claude-opus-4-6") ||
normalized.startsWith("claude-sonnet-4-6")
) {
return "Max";
}
return "Extra High";
}
return level.charAt(0).toUpperCase() + level.slice(1);
}
function formatReasoningDisabledLabel(
supportsReasoningOff: boolean,
isExternalOpenAIReasoning: boolean,
modelId?: string,
): string {
const normalized = modelId?.trim().toLowerCase() ?? "";
// Magistral keeps the "none" wire value, but UX should present this floor
// as "Medium" rather than a disabled state label.
if (normalized.includes("magistral-medium-latest")) return "Medium";
return supportsReasoningOff && isExternalOpenAIReasoning ? "None" : "Off";
}
function useDictation(
setText: (value: string | ((prev: string) => string)) => void,
) {
@ -253,6 +286,8 @@ export function SharedComposer({
const checkpoint = s.params.checkpoint;
return s.models.find((m) => m.id === checkpoint);
});
const checkpoint = useChatRuntimeStore((s) => s.params.checkpoint);
const externalProviders = useExternalProvidersStore((s) => s.providers);
const modelLoaded = useChatRuntimeStore(
(s) => !!s.params.checkpoint && !s.modelLoading,
);
@ -262,6 +297,8 @@ export function SharedComposer({
const setReasoningEnabled = useChatRuntimeStore((s) => s.setReasoningEnabled);
const reasoningStyle = useChatRuntimeStore((s) => s.reasoningStyle);
const reasoningEffort = useChatRuntimeStore((s) => s.reasoningEffort);
const supportsReasoningOff = useChatRuntimeStore((s) => s.supportsReasoningOff);
const reasoningEffortLevels = useChatRuntimeStore((s) => s.reasoningEffortLevels);
const setReasoningEffort = useChatRuntimeStore((s) => s.setReasoningEffort);
const supportsPreserveThinking = useChatRuntimeStore((s) => s.supportsPreserveThinking);
const preserveThinking = useChatRuntimeStore((s) => s.preserveThinking);
@ -271,7 +308,49 @@ export function SharedComposer({
const setToolsEnabled = useChatRuntimeStore((s) => s.setToolsEnabled);
const codeToolsEnabled = useChatRuntimeStore((s) => s.codeToolsEnabled);
const setCodeToolsEnabled = useChatRuntimeStore((s) => s.setCodeToolsEnabled);
const reasoningDisabled = !modelLoaded || !supportsReasoning;
const lastOpenRouterChosenModel = useChatRuntimeStore(
(s) => s.lastOpenRouterChosenModel,
);
const externalSelection = parseExternalModelId(checkpoint);
const selectedExternalProvider =
externalSelection != null
? externalProviders.find((p) => p.id === externalSelection.providerId)
: undefined;
const effectiveExternalModelId =
selectedExternalProvider?.providerType === "openrouter" &&
externalSelection?.modelId === "openrouter/free" &&
lastOpenRouterChosenModel
? lastOpenRouterChosenModel
: externalSelection?.modelId;
const externalReasoningCaps =
externalSelection != null
? getExternalReasoningCapabilities(
selectedExternalProvider?.providerType,
effectiveExternalModelId,
)
: null;
const isExternalOpenAIReasoning =
externalReasoningCaps?.supportsReasoning === true &&
externalReasoningCaps.reasoningStyle === "reasoning_effort";
const effectiveReasoningStyle =
externalReasoningCaps?.reasoningStyle ?? reasoningStyle;
const effectiveReasoningAlwaysOn =
externalReasoningCaps?.reasoningAlwaysOn ?? reasoningAlwaysOn;
const effectiveSupportsReasoningOff =
externalReasoningCaps?.supportsReasoningOff ?? supportsReasoningOff;
const effectiveReasoningEffortLevels =
externalReasoningCaps?.reasoningEffortLevels ?? reasoningEffortLevels;
const effectiveSupportsReasoning =
externalReasoningCaps?.supportsReasoning ?? supportsReasoning;
const reasoningLockedOn =
effectiveSupportsReasoning &&
(effectiveReasoningAlwaysOn || !effectiveSupportsReasoningOff);
const effectiveReasoningEnabled = reasoningLockedOn ? true : reasoningEnabled;
const effectiveReasoningVisualEnabled =
effectiveReasoningEnabled && reasoningEffort !== "none";
const reasoningDisabled = !modelLoaded || !effectiveSupportsReasoning;
const showReasoningControl =
effectiveSupportsReasoning || effectiveReasoningAlwaysOn;
const toolsDisabled = !modelLoaded || !supportsTools;
const setPendingAudioStore = useChatRuntimeStore((s) => s.setPendingAudio);
const clearPendingAudioStore = useChatRuntimeStore((s) => s.clearPendingAudio);
@ -625,7 +704,8 @@ export function SharedComposer({
</TooltipIconButton>
</>
)}
{reasoningStyle === "reasoning_effort" ? (
{showReasoningControl ? (
effectiveReasoningStyle === "reasoning_effort" ? (
<DropdownMenu>
<DropdownMenuTrigger asChild={true}>
<button
@ -635,26 +715,61 @@ export function SharedComposer({
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
reasoningDisabled
? "cursor-not-allowed opacity-40"
: "bg-primary/10 text-primary hover:bg-primary/20",
: effectiveReasoningVisualEnabled
? "bg-primary/10 text-primary hover:bg-primary/20"
: "text-muted-foreground hover:bg-muted-foreground/15",
)}
aria-label={`Reasoning effort: ${reasoningEffort}`}
>
<LightbulbIcon className="size-3.5" />
{effectiveReasoningVisualEnabled ? (
<LightbulbIcon className="size-3.5" />
) : (
<LightbulbOffIcon className="size-3.5" />
)}
<span>
Think:{" "}
{reasoningEffort.charAt(0).toUpperCase() +
reasoningEffort.slice(1)}
{effectiveReasoningVisualEnabled
? formatReasoningEffortLabel(
reasoningEffort,
externalSelection?.modelId,
)
: formatReasoningDisabledLabel(
effectiveSupportsReasoningOff,
isExternalOpenAIReasoning,
checkpoint,
)}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
{(["low", "medium", "high"] as const).map((level) => (
{effectiveSupportsReasoningOff && (
<DropdownMenuItem
onSelect={() => {
setReasoningEnabled(false);
applyQwenThinkingParams(false);
}}
>
{formatReasoningDisabledLabel(
effectiveSupportsReasoningOff,
isExternalOpenAIReasoning,
checkpoint,
)}
{!effectiveReasoningVisualEnabled ? " \u2713" : ""}
</DropdownMenuItem>
)}
{effectiveReasoningEffortLevels
.filter((level) => level !== "none")
.map((level) => (
<DropdownMenuItem
key={level}
onSelect={() => setReasoningEffort(level)}
onSelect={() => {
setReasoningEffort(level);
setReasoningEnabled(true);
applyQwenThinkingParams(true);
}}
>
{level.charAt(0).toUpperCase() + level.slice(1)}
{reasoningEffort === level ? " \u2713" : ""}
{formatReasoningEffortLabel(level, externalSelection?.modelId)}
{effectiveReasoningVisualEnabled && reasoningEffort === level ? " \u2713" : ""}
</DropdownMenuItem>
))}
</DropdownMenuContent>
@ -662,31 +777,47 @@ export function SharedComposer({
) : (
<button
type="button"
disabled={reasoningDisabled}
disabled={reasoningDisabled || reasoningLockedOn}
aria-disabled={reasoningDisabled || reasoningLockedOn}
title={
reasoningLockedOn
? "This model requires reasoning to stay on."
: undefined
}
onClick={() => {
if (reasoningAlwaysOn) return;
if (reasoningLockedOn) return;
const next = !reasoningEnabled;
setReasoningEnabled(next);
applyQwenThinkingParams(next);
}}
className={cn(
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
reasoningDisabled
? "cursor-not-allowed opacity-40"
: (reasoningEnabled || reasoningAlwaysOn)
? "bg-primary/10 text-primary hover:bg-primary/20"
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
reasoningLockedOn
? "cursor-not-allowed bg-primary/10 text-primary"
: reasoningDisabled
? "cursor-not-allowed opacity-40"
: effectiveReasoningEnabled
? "bg-primary/10 text-primary hover:bg-primary/20"
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
)}
aria-label={reasoningEnabled ? "Disable thinking" : "Enable thinking"}
aria-label={
reasoningLockedOn
? "Thinking is required for this model"
: effectiveReasoningEnabled
? "Disable thinking"
: "Enable thinking"
}
>
{(reasoningEnabled || reasoningAlwaysOn) && !reasoningDisabled ? (
{reasoningLockedOn ||
(effectiveReasoningEnabled && !reasoningDisabled) ? (
<LightbulbIcon className="size-3.5" />
) : (
<LightbulbOffIcon className="size-3.5" />
)}
<span>Think</span>
</button>
)}
)
) : null}
{supportsPreserveThinking && (
<button
type="button"

40
studio/frontend/src/features/chat/stores/chat-runtime-store.ts
View file

@ -26,13 +26,30 @@ const REASONING_EFFORT_KEY = "unsloth_reasoning_effort";
const PRESERVE_THINKING_KEY = "unsloth_preserve_thinking";
export type ReasoningStyle = "enable_thinking" | "reasoning_effort";
export type ReasoningEffort = "low" | "medium" | "high";
export type ReasoningEffort =
| "none"
| "minimal"
| "low"
| "medium"
| "high"
| "max"
| "xhigh";
function loadReasoningEffort(fallback: ReasoningEffort): ReasoningEffort {
if (!canUseStorage()) return fallback;
try {
const raw = localStorage.getItem(REASONING_EFFORT_KEY);
if (raw === "low" || raw === "medium" || raw === "high") return raw;
if (
raw === "none" ||
raw === "minimal" ||
raw === "low" ||
raw === "medium" ||
raw === "high" ||
raw === "max" ||
raw === "xhigh"
) {
return raw;
}
return fallback;
} catch {
return fallback;
@ -196,8 +213,19 @@ type ChatRuntimeStore = {
supportsReasoning: boolean;
reasoningAlwaysOn: boolean;
reasoningEnabled: boolean;
/**
* The model id the OpenRouter router actually picked for the most recent
* stream when the active checkpoint is the openrouter/free meta-model.
* Updated each time a chunk arrives carrying a non-empty `model` field
* that differs from the requested id. Cleared when a non-OpenRouter
* model is selected. Used purely for UI display — appended after
* `openrouter/free:` in the active model chip.
*/
lastOpenRouterChosenModel: string | null;
reasoningStyle: ReasoningStyle;
reasoningEffort: ReasoningEffort;
supportsReasoningOff: boolean;
reasoningEffortLevels: readonly ReasoningEffort[];
supportsPreserveThinking: boolean;
preserveThinking: boolean;
supportsTools: boolean;
@ -246,6 +274,7 @@ type ChatRuntimeStore = {
setSettingsPanelOpen: (open: boolean) => void;
clearCheckpoint: () => void;
setReasoningEnabled: (enabled: boolean) => void;
setLastOpenRouterChosenModel: (chosen: string | null) => void;
setReasoningStyle: (style: ReasoningStyle) => void;
setReasoningEffort: (effort: ReasoningEffort) => void;
setPreserveThinking: (value: boolean) => void;
@ -285,6 +314,9 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set) => ({
reasoningEnabled: true,
reasoningStyle: "enable_thinking",
reasoningEffort: loadReasoningEffort("medium"),
supportsReasoningOff: false,
reasoningEffortLevels: ["low", "medium", "high"],
lastOpenRouterChosenModel: null,
supportsPreserveThinking: false,
preserveThinking: loadBool(PRESERVE_THINKING_KEY, false),
supportsTools: false,
@ -394,6 +426,8 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set) => ({
reasoningAlwaysOn: false,
reasoningEnabled: true,
reasoningStyle: "enable_thinking",
supportsReasoningOff: false,
reasoningEffortLevels: ["low", "medium", "high"],
supportsPreserveThinking: false,
supportsTools: false,
toolsEnabled: false,
@ -410,6 +444,8 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set) => ({
loadedChatTemplateOverride: null,
})),
setReasoningEnabled: (reasoningEnabled) => set({ reasoningEnabled }),
setLastOpenRouterChosenModel: (lastOpenRouterChosenModel) =>
set({ lastOpenRouterChosenModel }),
setReasoningStyle: (reasoningStyle) => set({ reasoningStyle }),
setReasoningEffort: (reasoningEffort) =>
set(() => {

24
studio/frontend/src/features/chat/stores/external-providers-store.ts Normal file
View file

@ -0,0 +1,24 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { create } from "zustand";
import {
loadExternalProviders,
saveExternalProviders,
type ExternalProviderConfig,
} from "../external-providers";
interface ExternalProvidersState {
providers: ExternalProviderConfig[];
setProviders: (providers: ExternalProviderConfig[]) => void;
}
export const useExternalProvidersStore = create<ExternalProvidersState>(
(set) => ({
providers: loadExternalProviders(),
setProviders: (providers) => {
set({ providers });
saveExternalProviders(providers);
},
}),
);

37
studio/frontend/src/features/chat/types/api.ts
View file

@ -174,27 +174,43 @@ export interface AudioGenerationResponse {
}>;
}
export type OpenAIMessageContent =
| string
| Array<
| { type: "text"; text: string }
| { type: "image_url"; image_url: { url: string } }
>;
export interface OpenAIChatMessage {
role: "system" | "user" | "assistant";
content: string;
content: OpenAIMessageContent;
}
export interface OpenAIChatCompletionsRequest {
model: string;
messages: OpenAIChatMessage[];
stream: boolean;
temperature: number;
top_p: number;
/** Reasoning-class OpenAI models reject these — caller may omit. */
temperature?: number;
top_p?: number;
max_tokens: number;
top_k: number;
min_p: number;
repetition_penalty: number;
presence_penalty: number;
top_k?: number;
min_p?: number;
repetition_penalty?: number;
presence_penalty?: number;
image_base64?: string;
audio_base64?: string;
use_adapter?: boolean | string | null;
enable_thinking?: boolean | null;
reasoning_effort?: "low" | "medium" | "high" | null;
reasoning_effort?:
| "none"
| "minimal"
| "low"
| "medium"
| "high"
| "max"
| "xhigh"
| null;
preserve_thinking?: boolean | null;
enable_tools?: boolean | null;
enabled_tools?: string[];
@ -203,6 +219,11 @@ export interface OpenAIChatCompletionsRequest {
tool_call_timeout?: number;
session_id?: string;
cancel_id?: string;
provider_id?: string;
provider_type?: string;
external_model?: string;
encrypted_api_key?: string;
provider_base_url?: string | null;
}
export interface OpenAIChatDelta {

22
studio/frontend/src/features/chat/utils/parse-assistant-content.ts
View file

@ -8,16 +8,30 @@ type ContentPart = NonNullable<ChatModelRunResult["content"]>[number];
const THINK_OPEN_TAG = "<think>";
const THINK_CLOSE_TAG = "</think>";
// ContentPart from @assistant-ui/react has readonly fields, so we cannot
// do `last.text += text` to coalesce adjacent same-type parts — tsc fails
// with TS2540 "Cannot assign to 'text' because it is a read-only property".
// Instead, replace the last element with a fresh merged object: same
// allocation cost as the mutation path but type-safe.
function appendTextPart(parts: ContentPart[], text: string): void {
if (text) {
parts.push({ type: "text", text });
if (!text) return;
const last = parts.at(-1);
if (last?.type === "text") {
parts[parts.length - 1] = { type: "text", text: last.text + text };
return;
}
parts.push({ type: "text", text });
}
function appendReasoningPart(parts: ContentPart[], text: string): void {
if (text) {
parts.push({ type: "reasoning", text });
if (!text) return;
const last = parts.at(-1);
if (last?.type === "reasoning") {
parts[parts.length - 1] = { type: "reasoning", text: last.text + text };
return;
}
parts.push({ type: "reasoning", text });
}
export function parseAssistantContent(

26
studio/frontend/src/features/settings/settings-dialog.tsx
View file

@ -10,6 +10,7 @@ import {
import { cn } from "@/lib/utils";
import {
Cancel01Icon,
CloudIcon,
Globe02Icon,
HelpCircleIcon,
Message01Icon,
@ -20,11 +21,15 @@ import {
import { HugeiconsIcon } from "@hugeicons/react";
import { motion, useReducedMotion } from "motion/react";
import { useEffect, useRef } from "react";
import { useSettingsDialogStore, type SettingsTab } from "./stores/settings-dialog-store";
import {
useSettingsDialogStore,
type SettingsTab,
} from "./stores/settings-dialog-store";
import { AboutTab } from "./tabs/about-tab";
import { ApiKeysTab } from "./tabs/api-keys-tab";
import { AppearanceTab } from "./tabs/appearance-tab";
import { ChatTab } from "./tabs/chat-tab";
import { ConnectionsTab } from "./tabs/connections-tab";
import { GeneralTab } from "./tabs/general-tab";
import { ProfileTab } from "./tabs/profile-tab";
@ -40,6 +45,7 @@ const TABS: TabDef[] = [
{ id: "profile", label: "Profile", icon: UserIcon },
{ id: "appearance", label: "Appearance", icon: PaintBrush02Icon },
{ id: "chat", label: "Chat", icon: Message01Icon },
{ id: "connections", label: "Cloud", icon: CloudIcon, badge: "New" },
{ id: "api-keys", label: "API", icon: Globe02Icon, badge: "New" },
{ id: "about", label: "Help", icon: HelpCircleIcon },
];
@ -54,6 +60,8 @@ function renderTab(tab: SettingsTab) {
return <AppearanceTab />;
case "chat":
return <ChatTab />;
case "connections":
return <ConnectionsTab />;
case "api-keys":
return <ApiKeysTab />;
case "about":
@ -72,6 +80,7 @@ export function SettingsDialog() {
profile: null,
appearance: null,
chat: null,
connections: null,
"api-keys": null,
about: null,
});
@ -100,9 +109,9 @@ export function SettingsDialog() {
<DialogDescription className="sr-only">
Manage your Unsloth Studio preferences.
</DialogDescription>
<div className="flex h-full min-h-0">
<aside className="font-heading flex w-[200px] shrink-0 flex-col border-r border-border bg-muted/20 p-2">
<nav className="flex flex-col gap-0.5">
<div className="flex h-full min-h-0 max-sm:flex-col">
<aside className="font-heading flex w-[200px] shrink-0 flex-col border-r border-border bg-muted/20 p-2 max-sm:w-full max-sm:border-r-0 max-sm:border-b">
<nav className="flex flex-col gap-0.5 max-sm:flex-row max-sm:overflow-x-auto">
{TABS.map((tab) => {
const active = activeTab === tab.id;
return (
@ -115,6 +124,7 @@ export function SettingsDialog() {
onClick={() => setActiveTab(tab.id)}
className={cn(
"relative flex h-[32px] items-center gap-2.5 rounded-[8px] px-2.5 text-[14.5px] leading-[19px] tracking-nav font-medium transition-colors",
"max-sm:shrink-0",
"focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-1 focus-visible:ring-offset-background",
active
? "text-black dark:text-white"
@ -142,7 +152,9 @@ export function SettingsDialog() {
strokeWidth={1.75}
className="relative z-10 size-icon"
/>
<span className="relative z-10 min-w-0 truncate">{tab.label}</span>
<span className="relative z-10 min-w-0 truncate">
{tab.label}
</span>
{tab.badge ? (
<span className="relative z-10 ml-auto rounded-[6px] border border-emerald-500/25 bg-emerald-500/10 px-1.5 py-0.5 text-[10px] leading-none font-semibold text-emerald-700 dark:text-emerald-300">
{tab.badge}
@ -154,7 +166,7 @@ export function SettingsDialog() {
</nav>
</aside>
<main className="relative flex min-w-0 flex-1 flex-col">
<main className="relative flex min-h-0 min-w-0 flex-1 flex-col">
<button
type="button"
onClick={closeDialog}
@ -163,7 +175,7 @@ export function SettingsDialog() {
>
<HugeiconsIcon icon={Cancel01Icon} className="size-4" />
</button>
<div className="flex min-h-0 min-w-0 flex-1 flex-col overflow-y-auto p-6">
<div className="flex min-h-0 min-w-0 flex-1 flex-col overflow-y-auto p-6 [scrollbar-gutter:stable]">
{renderTab(activeTab)}
</div>
</main>

15
studio/frontend/src/features/settings/stores/settings-dialog-store.ts
View file

@ -8,6 +8,7 @@ export type SettingsTab =
| "profile"
| "appearance"
| "chat"
| "connections"
| "api-keys"
| "about";
@ -29,8 +30,18 @@ function loadInitialTab(): SettingsTab {
} catch {
return "general";
}
const valid: SettingsTab[] = ["general", "profile", "appearance", "chat", "api-keys", "about"];
return valid.includes(stored as SettingsTab) ? (stored as SettingsTab) : "general";
const valid: SettingsTab[] = [
"general",
"profile",
"appearance",
"chat",
"connections",
"api-keys",
"about",
];
return valid.includes(stored as SettingsTab)
? (stored as SettingsTab)
: "general";
}
export const useSettingsDialogStore = create<SettingsDialogState>((set) => ({

17
studio/frontend/src/features/settings/tabs/connections-tab.tsx Normal file
View file

@ -0,0 +1,17 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { ChatProvidersSettings } from "@/features/chat/chat-providers-dialog";
import { useExternalProvidersStore } from "@/features/chat/stores/external-providers-store";
export function ConnectionsTab() {
const providers = useExternalProvidersStore((s) => s.providers);
const setProviders = useExternalProvidersStore((s) => s.setProviders);
return (
<ChatProvidersSettings
providers={providers}
onProvidersChange={setProviders}
/>
);
}

4
tests/test_import_fixes_drift.py
View file

@ -315,9 +315,7 @@ def test_triton_compiled_kernel_has_num_ctas_and_cluster_dims():
code = getattr(init, "__code__", None)
freevars = set(getattr(code, "co_freevars", ()) or ())
co_names = set(getattr(code, "co_names", ()) or ())
if "_orig_init" in freevars or {"num_ctas", "cluster_dims"}.issubset(
co_names
):
if "_orig_init" in freevars or {"num_ctas", "cluster_dims"}.issubset(co_names):
return
pytest.fail(