From 9a0d6f80cb1bef2796ad6820479c7ccb990f9394 Mon Sep 17 00:00:00 2001
From: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Date: Thu, 14 May 2026 16:13:59 +0400
Subject: [PATCH] studio: API external provider support for chat (OpenAI,
Mistral, Gemini, Cohere, Anthropic, OpenRouter, DeepSeek, custom providers)
(#4706)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
* studio: add external provider support for chat inference
Adds the ability to connect to OpenAI, Mistral, Google, Cohere, Together,
Fireworks, and Perplexity from the Studio chat interface.
- Provider configs stored in SQLite (no API keys persisted)
- RSA-2048 key pair generated at startup for client-side key encryption
- httpx proxy client streams SSE responses in OpenAI-compatible format
- New /api/providers routes: registry, CRUD, test, models
- /v1/chat/completions routes to external provider when provider fields present
- Integration test suite covering CRUD, connection, model listing, and inference
- Frontend spec doc with full API contract
* remove frontend spec doc from branch
* fix auth fixture: handle forced password change on fresh install
* fix tests: default port 8000, allow 400 for no-model-loaded
* fix: update Cohere models to current (command-r retired Sept 2025)
* feat: add OpenRouter as 8th provider
* feat: add native Anthropic provider with Messages API translation
* fix: correct Anthropic base URL and drop top_p (conflicts with temperature)
* feat: add DeepSeek provider (deepseek-chat, deepseek-reasoner)
* feat: rename google -> gemini, refresh model list to 2.5 series
* feat: remove together, fireworks, perplexity providers
* feat: multimodal image support for external providers
- Add _build_external_messages() that preserves image_url parts for
vision-capable providers instead of stripping them
- Update _proxy_to_external_provider() to use new helper
- Translate image_url content parts to Anthropic native image format
in _stream_anthropic()
- Add TestVisionInference pytest class (1x1 PNG smoke test)
* test: use sloth photo URL for vision test, add Anthropic remote URL support
* fix: update Mistral model to mistral-small-2506
* update mistral default model to mistral-large-2512
* fix gemini vision test: download image as base64 data URI instead of remote URL
* add gemini-3-flash-preview as default gemini model
* fix gemini truncated reply (max_tokens 16->64) and suppress GeneratorExit on client disconnect
* increase vision test max_tokens to 215
* fix GeneratorExit: aclose stream generator before closing httpx client
* fix httpcore GeneratorExit: explicitly aclose aiter_lines before response closes
* fix duplicate [DONE] and suppress httpcore RuntimeError on Python 3.13 asyncgen cleanup
* fix: call response.aclose() before lines_gen.aclose() to prevent httpcore RuntimeError on Python 3.13
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Potential fix for code scanning alert no. 36: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* review: add comments for manual iteration rationale, mask password in test print, clarify Anthropic URL/models support
* perf: use shared module-level httpx client for connection pooling across requests
* studio: add API provider UI and integrate wiring (#4737)
* feat: expose external models in selector and chat settings
* feat(chat): wire external providers to backend + RSA key flow
- Fetch registry/configs; create/update/delete saved providers
- Encrypt API keys (Web Crypto RSA-OAEP) for test/models/chat
- External model selection + chat payload (provider_id/type, external_model, encrypted key, optional base URL)
- Local storage for keys + provider list; small UX/copy and guardrails
* add missing providers-api.ts file by Imagineer99
* fix: address PR review comments — system prompt visibility, retry loop, test logging
* feat(studio): encrypt external provider API keys at rest in localStorage
API keys for external providers (OpenAI, Mistral, etc.) were stored as
plaintext in localStorage, vulnerable to browser extensions and XSS.
Add password-derived AES-256-GCM encryption: on login the user's password
is used via PBKDF2 (100k iterations, SHA-256) to derive an in-memory
encryption key. API keys are encrypted before writing to localStorage and
decrypted on read. The derived key is never persisted — cleared on logout,
re-derived on next login.
Legacy plaintext keys are transparently migrated on first access. Password
changes re-encrypt all stored keys. No backend changes required — the
existing RSA-OAEP transit encryption is unaffected.
* fix: cast PBKDF2 salt to BufferSource for strict TypeScript lib types
* fix: persist session password in sessionStorage to survive page refreshes
* feat(studio): preserve image parts in external provider chat requests
toOpenAIMessage() now returns multimodal content arrays (OpenAI vision
format) when messages contain images, instead of always flattening to
plain text. This enables vision-capable external providers (OpenAI,
Gemini, Anthropic, etc.) to receive user images. The backend already
handles image_url content parts in _build_external_messages().
* studio: fix external models selectable in chat-only mode (#4779)
* fix: external models selectable in chat-only mode
* fix: model selector tabs default to active model kind
* Studio: API external provider registry + curated catalogs (HF/OpenRouter) and chat UX (#4787)
* fix: external models selectable in chat-only mode
* fix: model selector tabs default to active model kind
* feat(studio): expand provider registry, curated catalogs, and chat UX
- Add Hugging Face, Kimi, Qwen; remove Cohere; reorder registry
- model_list_mode curated for HF/OpenRouter; lightweight /models check
- API returns default models for curated providers; expose model_list_mode
- Frontend: provider logos in model picker, providerType on external models
- Chat providers dialog: curated vs remote flows, motion polish
- Thread: LayoutGroup + composer motion alignment with app easing
* fix(studio): disable Anthropic tool-calling flag and preselect curated defaults
* feat(studio): add external provider logos and ApiProviderLogo helper
* Studio: Polish API Providers dialog (#4899)
* fix: lower verbage in API providers page
* fix: fix(studio): tune API Providers dialog width with rem-based responsive caps
* feat: add custom provider support (#4902)
* fix: replace crypto.subtle with node-forge for HTTP compatibility
crypto.subtle is only available in secure contexts (HTTPS/localhost),
which breaks provider API key encryption when Studio is accessed over
plain HTTP on remote GPU VMs. Switch to node-forge for RSA-OAEP and
AES-256-GCM operations — same algorithms, works on any origin.
* fix: store provider API keys as plaintext in localStorage
Drop AES-256-GCM at-rest encryption for provider API keys. The
session-password-derived encryption broke on auto-login via refresh
token (password never captured), causing keys to silently vanish.
API keys are still RSA-encrypted in transit via node-forge. At-rest
encryption in localStorage added no real security since the
decryption key also had to live client-side.
Removes crypto-storage.ts, session password plumbing, and
reEncryptAllKeys.
* fix: use max_completion_tokens for OpenAI provider
Newer OpenAI models (gpt-4o, gpt-5.x) reject the max_tokens param
and require max_completion_tokens instead. Other providers still use
max_tokens.
* fix: skip empty assistant messages in external provider requests
Some providers (Mistral) reject assistant messages with empty content.
Filter them out when building the message list for external providers.
* Update model-selector.tsx
* Update model-selector.tsx
* Update model-selector.tsx
* Update chat-adapter.ts
* Update chat-adapter.ts
* Update chat-page.tsx
* Update chat-settings-sheet.tsx
* Update chat-settings-sheet.tsx
* Update chat-settings-sheet.tsx
* Update chat-providers-dialog.tsx
* feat: polish providers settings form UI
* style: polish provider row icon sizing and alignment
* style: stabilize provider layout
* style: add provider API key visibility toggle
* fix: add provider render on empty list
* studio/frontend: sync package-lock.json with package.json
npm ci was failing because node-forge and @types/node-forge were
declared in package.json but missing from the lockfile. Ran
npm install to regenerate.
* studio/backend: fix backend CI failures for providers router
- test_desktop_auth: include providers_router in the routes stub so
studio.backend.main imports cleanly under the monkeypatched module
- test_providers_api: skip the whole module when STUDIO_TEST_PASSWORD
is unset (it is an integration test against a live Studio server,
same shape as the already-ignored test_studio_api.py)
* studio/chat: drive ChatSettingsPanel from a per-provider capability map
Replace the binary isExternalModel toggle in the sampling section with a
provider-aware capability map. Each external provider type advertises
which of top_k / min_p / repetition_penalty / presence_penalty its
chat-completions API actually accepts, so the panel only renders the
knobs that map onto the active provider's request body.
Anthropic now exposes top_k; DeepSeek hides presence_penalty (deprecated
in their docs); OpenRouter and custom providers continue to show every
knob (OpenRouter drops unsupported server-side, custom assumes
OpenAI-compat or a permissive vLLM/Ollama backend). Local models are
unaffected — null capabilities means 'show everything'.
chat-adapter.ts now forwards top_k / presence_penalty to the external
proxy only when the active provider's capabilities permit it, so the
request body matches what the UI shows.
* studio/backend: forward top_k to Anthropic; filter OpenAI model list
Two paired changes so the frontend capability map has matching backend
behaviour:
1. ExternalProviderClient.stream_chat_completion now accepts top_k and
forwards it to the Anthropic Messages body. OpenAI-compat providers
(which all reject unknown sampling params) still receive only the
fields they document. The proxy route in routes/inference.py passes
payload.top_k through, so a UI request with top_k actually reaches
Anthropic instead of being silently dropped at the boundary.
2. PROVIDER_REGISTRY['openai'] gains a model_id_allowlist regex that
scopes the /models picker to current-gen ids (gpt-5.5 / gpt-5.4 /
gpt-5.3 / gpt-4.5 / o3 families). The remote /v1/models listing
otherwise returns dozens of historical snapshots, fine-tunes and
non-chat models (embeddings, TTS, image, moderation) that we never
want in the chat UI. default_models is refreshed to match.
* studio/chat: relax presence_penalty to optional on OpenAIChatCompletionsRequest
Followup to 1fbf445a — chat-adapter now omits presence_penalty for
providers that do not accept it (Anthropic / DeepSeek), but the
request type still required it as a non-optional number, breaking
tsc. The backend pydantic model already defaults presence_penalty
to 0, so making it optional client-side matches reality.
* studio/backend: route OpenAI traffic through /v1/responses
OpenAI's new flagship models (gpt-5.x) return 404 'This is not a chat
model' on /v1/chat/completions and are only reachable via /v1/responses.
Add a dedicated _stream_openai_responses path in ExternalProviderClient
that:
- Translates outbound messages into the Responses shape: system messages
are folded into the top-level 'instructions' field, user/assistant
messages become {role, content} items with input_text / input_image
content parts (data URLs and https URLs both pass through).
- Drops presence_penalty / top_k / frequency_penalty, none of which the
Responses contract accepts.
- Translates inbound SSE events back into OpenAI Chat Completions
chunks so the frontend keeps a single SSE shape:
response.output_text.delta -> delta chunk with content
response.completed -> chunk with finish_reason='stop'
response.incomplete -> chunk with finish_reason='length'
response.failed / error -> propagated error SSE line
Stream terminates with data: [DONE] (Responses emits this verbatim).
stream_chat_completion dispatches all provider_type='openai' calls to
this path; other OpenAI-compatible providers (mistral, gemini, etc.)
continue to use /v1/chat/completions.
Frontend provider-capabilities map updated to hide presence_penalty for
OpenAI in the chat settings panel, matching the new request contract.
Includes unit coverage in tests/test_openai_responses_translation.py
exercising the request body translation, image-part rewriting, and
SSE-to-chat-completions translation via httpx.MockTransport.
* studio/chat: clamp external max_tokens to 32k to stay within provider caps
The chat settings slider already capped maxTokens at 32768 for external
models, but a value persisted from a prior local-model session (where
the cap can be 128k+) was sent verbatim to the provider — Claude Opus
returns 'max_tokens: 131072 > 128000' on requests like that, and other
providers have stricter limits still.
Expose EXTERNAL_MAX_OUTPUT_TOKENS from provider-capabilities (32k) and
use it both for the slider max and as the clamp inside chat-adapter's
external-request body. 32k sits below the tightest declared output
limit across the providers we ship and well above what a typical chat
reply needs; the local-model path is unaffected.
* studio: drop temperature/top_p for OpenAI reasoning models
gpt-5.x / o3 / gpt-4.5 are reasoning-class models served via
/v1/responses, and reject temperature and top_p with
'Unsupported parameter' 400s. The OpenAI registry allowlist already
scopes the picker to those families, so neither knob ever applies on
this branch.
- external_provider._stream_openai_responses no longer puts
temperature or top_p in the request body (kept on the method
signature for API symmetry with the other stream methods).
- ProviderCapabilities gains temperature/topP flags; OpenAI sets both
to false. ChatSettingsPanel hides the sliders for OpenAI so the user
does not see inert controls.
- chat-adapter omits temperature/top_p from the external request body
when the active provider does not advertise them.
- OpenAIChatCompletionsRequest type marks both as optional, matching
the new chat-adapter shape.
- test_responses_request_body_uses_input_and_instructions: assertions
flipped to confirm temperature / top_p are absent from the body.
* studio: stop forwarding top_k to Anthropic
Claude 4.x (Opus / Sonnet / Haiku 4.x) returns 400 'top_k is
deprecated for this model' on any request that includes top_k. It
was always optional on the older 3.x line, so dropping it
unconditionally for every Anthropic call is the simplest path —
no per-model gate to maintain.
- external_provider._stream_anthropic no longer adds top_k to the
Messages body (kept on the method signature for API symmetry).
- provider-capabilities sets anthropic.topK = false so the chat
settings panel hides the Top K slider for Anthropic providers
and chat-adapter does not send top_k in the external request.
* studio: gate Anthropic top_k drop to Claude 4.7 only
Previous commit (b5aa6ffd) dropped top_k for every Anthropic call,
but only Claude 4.7 (Opus/Sonnet/Haiku) actually rejects it. 4.6, 4.5,
and the 3.x line still accept top_k and use it as documented.
Backend: _stream_anthropic matches the model id against
^claude-(opus|sonnet|haiku)-4-7(-|.|$) and only strips top_k when it
hits. Every other Claude generation continues to receive the value
from the chat settings panel.
Frontend: anthropic.topK is restored to true so the Top K slider is
visible again — the backend handles the per-model drop, and the
4.7 case is silent (request still succeeds without top_k).
* chore: hide dated openai models in provider select
* studio/providers: apply model_id_denylist when listing remote models
The OpenAI registry entry gained a model_id_denylist regex matching
dated snapshot ids (-YYYY-MM-DD) in 048d73bf, but the list-models
route was never consulting it, so the snapshots still showed up
alongside their canonical ids (gpt-5.5 and gpt-5.5-2026-04-23 both
listed). Apply the denylist with .search() right after the allowlist
filter so dated entries are dropped before the response is built.
* studio/chat: seed registry default_models for remote providers in picker
The Anthropic provider runs in remote model-list mode, so the picker
started with an empty availableModels until the user clicked
'Load Models'. If that /api/providers/models call fails (e.g. the
known transient decryption error during key rotation), the user sees
no models at all — claude-haiku-4-5 in particular was missing from
the dialog even though it is seeded in the registry.
Always pre-populate availableModels with the registry's default_models
when a provider type is selected (curated and remote alike), and have
loadModels() return the union of defaults + the live /models response
so registry-seeded ids are reachable regardless of what the provider's
endpoint returns or whether the call succeeds at all.
* studio/backend: diagnostic logging on provider key decryption
Decryption failures currently log just 'Failed to decrypt API key:
Decryption failed', which leaves no way to tell whether the cause is
a stale public key in the browser, a corrupted ciphertext, an
unexpected exception class, or a server-side keypair rotation. That's
the gap the next reproduction needs to close.
- key_exchange now publishes a short SHA256 fingerprint of the public
key PEM. init_key_pair logs the fingerprint on generation and warns
if it is ever called a second time (re-init silently invalidates
every browser that cached the previous public key).
- decrypt_api_key wraps both the base64 decode and the RSA decrypt
in dedicated try/excepts that log exception type, ciphertext byte
length (RSA-2048 should be exactly 256), input string length, and
the current public-key fingerprint.
- GET /api/providers/public-key returns the fingerprint alongside the
PEM so the frontend can correlate a future encrypt-time fingerprint
against the decrypt-time fingerprint and prove or rule out a
keypair rotation as the cause.
- The /test and /models route-level decrypt warnings now include the
exception class name (alongside the existing message).
* studio/providers: hide dated Anthropic snapshots from the model picker
Anthropic's /v1/models returns dated snapshot ids (e.g.
claude-3-5-sonnet-20241022, claude-3-5-haiku-20241022) alongside
the canonical names users actually want to pick. Same intent as
the OpenAI denylist added in 048d73bf, just a different date
format — Anthropic uses -YYYYMMDD (no dashes) while OpenAI uses
-YYYY-MM-DD.
- Add model_id_denylist = re.compile(r'-\d{8}$') to the anthropic
registry entry. The /api/providers/models route already applies
any denylist after fetching, so dated ids drop out automatically.
- Strip the dated 3.5 ids from default_models so the seeded picker
no longer surfaces them; keep claude-opus-4-7 and the 4.5 family
as the curated set.
Net effect: the picker shows opus-4-7 / opus-4-5 / sonnet-4-5 /
haiku-4-5 only, regardless of whether the remote /models call
succeeds or fails.
* fix: provider dialog and mistral short list
* style: fix provider dialog curated list styling
* fix: provider dialog curated model ids placeholder reference
* style: rename Providers to Cloud and tighten dialog header spacing
* UX: rename Providers to Cloud, remove header shortcut
* studio/chat: normalize structured delta.content from reasoning providers
Mistral's magistral (and similarly-shaped reasoning models) stream
chat-completion deltas where choices[0].delta.content is an array of
structured parts rather than a plain string, e.g.
[{ type: 'text', text: '...' }, { type: 'thinking', thinking: '...' }]
The accumulator did 'cumulativeText += delta', which coerced each
part to '[object Object]' and produced output like
'[object Object][object Object]...Hey there!'.
Add extractDeltaText() to normalize delta.content before append:
- string → returned as-is
- array of parts → text/output_text parts contribute their .text or
.content; thinking/reasoning parts are re-wrapped inline as
... so the downstream parseAssistantContent lifts
them into a reasoning part the same way it does for providers that
emit thinking inline. magistral keeps its thinking panel; no other
provider's output shape changes.
- unknown shapes → dropped rather than stringified, so a stray field
cannot pollute the rendered chat with '[object Object]'.
* Studio: restore Cloud icon shortcut in chat header
Brings back the header chip that opens Settings -> Cloud (external
providers) directly from the chat view. Same button as before the
bf24e604 removal: single-mode only, opens useSettingsDialogStore on
the 'connections' tab, tooltip 'API providers'.
* studio/chat: strip trailing template literal from external provider streams
Mistral's magistral occasionally appends a literal '${response}' token
after its actual answer — likely a training-format artifact, since it
keeps happening with an empty system prompt and only on that model.
Apply a tight strip in the chat-adapter SSE accumulator: when the
active provider is external, drop a trailing '${...}' template literal
(with optional whitespace) from cumulativeText after each chunk. The
regex anchors to end-of-string, so mid-stream fragments ('${re')
remain untouched and only collapse once the closing brace arrives.
Local-model output is unaffected.
* studio/providers: scope Kimi picker to kimi-k2.6 / kimi-k2.5
Mirror what the live Kimi docs surface as the current models
(https://platform.kimi.ai/docs/models). Everything else the
remote /v1/models call returns — moonshot-v1-* legacy ids and
dated k2 previews like kimi-k2-0711-preview — is filtered out.
- default_models: ['kimi-k2.6', 'kimi-k2.5'] (was four
legacy moonshot-v1 ids plus the dated k2 preview)
- model_id_allowlist: ^kimi-k2\.[56]$ applied in the
/api/providers/models route after the live fetch
- doc-link comments point at platform.kimi.ai overview /
models / list-models for the next refresh
* studio: drop temperature/top_p for Kimi reasoning models
Kimi k2.5/k2.6 are reasoning-class. The API locks temperature and
top_p to fixed defaults and 400s on any other value with
'invalid temperature: only 1 is allowed for this model'.
The frontend capability map already gated these knobs out of the
external request body, but the OpenAI-compat path on the backend
unconditionally re-adds them from the pydantic ChatCompletionRequest
defaults (temperature=0.7 etc), so the gate was bypassed end-to-end.
Add a generic body_omit hook on the provider registry that
stream_chat_completion consults after building the body, and use it
to strip temperature/top_p for Kimi. Frontend provider-capabilities
flips kimi.temperature and kimi.topP to false so the sliders are
hidden in the chat settings panel as well.
* studio/providers: scope Gemini picker to current 3.x + *-latest aliases
Google's /v1beta/openai/models returns dozens of historical,
experimental, and non-chat ids that we never want in the chat UI.
Cap the picker to the current curated set:
- gemini-3.1-pro-preview
- gemini-3.1-flash-lite
- gemini-3-flash-preview
- gemini-pro-latest
- gemini-flash-latest
- gemini-flash-lite-latest
Default_models seeded with these, model_id_allowlist applied in
the /api/providers/models route to drop anything else the live
fetch returns.
* studio/providers: switch Hugging Face to remote model listing
Per the Inference Providers docs
(https://huggingface.co/docs/inference-providers/index),
GET https://router.huggingface.co/v1/models returns the full
chat-model catalog across all providers, including per-provider
metadata. The OpenAI-compatible endpoint we already use for
chat completions accepts the same Bearer token, so flipping
model_list_mode from 'curated' to 'remote' lets users discover
models via the existing list_models() path without any new
wiring.
- model_list_mode: 'remote' (was 'curated')
- default_models refreshed with current popular ids
(gpt-oss-120b, DeepSeek-V3, Llama-3.3-70B, Qwen2.5-72B) so the
picker still has a sensible seed if /v1/models fails
- notes updated to reference the docs page and clarify the
endpoint is chat-only
* UX: chat cloud icon changed to model select signifier
* studio/providers: org allowlist + count cap for HF Inference picker
The HF /v1/models response is the full cross-provider catalog (hundreds
of ids — community fine-tunes, mirrors, fp8 variants, dated snapshots).
Scope the picker to the first-party org repos worth surfacing and cap
the post-filter list.
- model_id_allowlist matches the org prefixes openai/, deepseek-ai/,
google/, meta-llama/, Qwen/, moonshotai/, mistralai/, zai-org/.
Anything outside those orgs is dropped.
- model_id_limit (new registry field) caps the post-filter list. The
list-models route now slices [:limit] after allowlist/denylist; set
to 15 for HF Inference. Other providers leave it unset and behave
exactly as before.
- default_models stays as the seed so the flagship ids users care
about (gpt-oss-120b, DeepSeek-V3, Llama-3.3-70B, Qwen2.5-72B) are
always reachable regardless of the API's response order.
Dedup is already handled in loadModels() via Set, so no additional
work needed there.
* style: adjust cloud icon right margin with rem spacing
* Studio: cloud openai reasoning level toggle (#5402)
* feat: cloud openai reasoning level toggle
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: honor enable_thinking=false
* fix: prevent local reasoning toggle regressions and align OpenAI effort levels
* fix: isolate external OpenAI reasoning toggle state
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* fix: clamp reasoning effort
* fix: align OpenAI reasoning effort
* fix: clear stale GGUF badge state
* ui: new badge on cloud setting
* fix: separate selected models from cached provider model list
* Studio: anthropic effort by model family (#5412)
* feat: external thinking control and Anthropic effort mapping
* fix: anthropic thinking constraints and 4.6 max effort mapping
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: harden Anthropic thinking params and effort mapping
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio/backend: drop top_p from Anthropic body when thinking is enabled
PR 5412 added body['top_p'] = max(0.95, min(top_p, 1.0)) inside the
thinking branch of _stream_anthropic, but Anthropic returns 400 on
extended/adaptive thinking when both temperature and top_p are set:
invalid_request_error: temperature and top_p cannot both be
specified for this model. Please use only one.
(Observed on Claude Opus 4.6.) The contract for thinking-enabled
requests is temperature=1 with neither top_p nor top_k allowed.
Replace the body['top_p'] = ... line with body.pop('top_p', None).
Defensive pop rather than a bare delete: the base body construction
above does not currently set top_p, but a future edit that adds it
would silently reintroduce the regression.
* studio/chat: force reasoningEnabled=true on local reasoning-effort models
Followup to PR 5402 / 5412. The model-status refresh path in
use-chat-model-runtime carried reasoningEnabled forward verbatim for
every reasoning-capable model. That left one observable edge case:
1. user picks an external model that supports Off (gpt-5.x, Claude
4.x), clicks Off — store sets reasoningEnabled=false
2. user switches back to a local reasoning-effort model
(gpt-oss / Harmony-style) which does NOT support Off
3. composer's effectiveReasoningEnabled override paints the UI as
'Think: ' (on)
4. chat-adapter sees reasoningEnabled=false on the local branch
and sends '{}', so the backend's _request_reasoning_kwargs
returns None and the Harmony template falls back to its own
default effort instead of the displayed level
Mirror the composer's override in the store on load: for local
reasoning-effort models (where supportsReasoningOff is false), force
reasoningEnabled=true so the store and the UI agree on every send.
Other reasoning styles still inherit prior state — only the
reasoning-effort family changes.
* studio/backend: align Anthropic thinking with the extended-thinking docs
Two compliance fixes against
https://platform.claude.com/docs/en/build-with-claude/extended-thinking
1. Adaptive-mode effort field shape
The docs spell adaptive thinking as:
{'thinking': {'type': 'adaptive'}, 'effort': {'type': ''}}
We had been sending the legacy 'output_config: {effort: }'
shape, which Anthropic appears to silently ignore — adaptive ran
at the server default effort regardless of the user's selection.
Rename to 'effort: {type: }'.
2. thinking_delta event translation
The Messages-API streams reasoning content as
content_block_delta events with delta.type == 'thinking_delta',
which our SSE loop was dropping entirely. On Claude 4.5/4.6 with
display=summarized (the default), the user would see the answer
text but never the reasoning panel. Wrap thinking_delta.thinking
as inline ... chunks (same pattern as the OpenAI
Responses path) so the frontend's parseAssistantContent lifts it
into the reasoning channel. The closer fires on the
first text_delta transition, on content_block_stop for the
thinking block, on message_delta, and on message_stop —
whichever arrives first — so no model path can leak an
unclosed into chat output.
signature_delta events are left as no-ops; they carry
verification metadata, not user-visible content.
Adds test_anthropic_thinking_translation.py with httpx.MockTransport
coverage of: effort shape on adaptive (Claude 4.6), budget_tokens
shape on manual (Claude 4.5), thinking_delta wrapping with signature
suppression, and thinking-only turns (display=omitted on Opus 4.7).
* studio/backend: revert Anthropic adaptive effort to output_config nesting
The previous commit (0a664df4) moved the adaptive-thinking effort
field to a top-level 'effort: {type: }' based on a misread of
the docs page. The actual Messages API schema nests it under
output_config:
thinking: optional ThinkingConfigParam ({type: 'adaptive'})
output_config: optional OutputConfig
effort: optional 'low' | 'medium' | 'high' | 'xhigh' | 'max'
Sending the top-level field produced:
400 invalid_request_error: effort: Extra inputs are not permitted
Restore the body to:
body['thinking'] = {'type': 'adaptive'}
body['output_config'] = {'effort': effort}
This was the shape PR 5412 originally shipped (and the author
validated against live APIs). My 'compliance fix' was a regression.
The companion thinking_delta SSE translation added in 0a664df4 stays
— that part WAS missing from the previous shape and is unchanged
by this revert. Test pinning the body shape flipped to assert
output_config.effort, top-level effort is asserted absent.
* studio/backend: opt in to summarized thinking display on adaptive
Per the adaptive-thinking docs, the 'display' field on the thinking
config defaults to 'omitted' on Claude Opus 4.7 (and Mythos Preview).
With 'omitted' the API still emits a thinking content block, but its
'thinking' field is empty — only the signature_delta arrives.
Our SSE handler would then surface a stray '' for the
empty block and the reasoning panel would stay blank for the entire
response. Set 'display': 'summarized' explicitly on the adaptive
thinking config so Opus 4.7 emits thinking_delta events the same way
Opus 4.6 / Sonnet 4.6 do (where 'summarized' is the default, making
the explicit setting a no-op there).
The manual-thinking branch (Claude 4.5) is unaffected — its default
is also 'summarized', and we have no reason to override it.
* studio/backend: log Anthropic SSE event counts for thinking diagnostics
Reports of 'no reasoning panel content on Anthropic' have two
distinct causes that produce the same symptom:
1. Anthropic streamed thinking_delta events but our frontend
dropped them somewhere on the rendering side.
2. Anthropic did not emit thinking_delta at all (adaptive mode
can skip thinking for simple prompts even with effort=high,
and display=summarized only re-enables the *content* — it
does not force thinking to happen).
Tally each event type for the duration of one stream and log the
counts in the finally branch, so the next 'no reasoning content'
report shows immediately whether thinking_delta was even on the
wire. Zero counts → upstream (model/effort/prompt choice).
Non-zero counts → triage moves to chat-adapter / parse-assistant
-content / the reasoning component.
* studio/backend: route external_provider logs through structlog
The studio backend wires structlog as the active logger (via
LogConfig.setup_logging at main.py:262), but external_provider.py
was using stdlib logging.getLogger(__name__) for every diagnostic.
The stdlib root logger defaults to WARNING with no handlers
attached, so plain logger.info('...') and logger.debug('...') from
this module were being silently dropped — including the
'Proxying chat completion to ' and the new
'Anthropic stream event counts' lines. Only WARNING/ERROR survived
(via the implicit fallthrough that the user actually observed
when an Anthropic call 400'd).
Switch the module-level logger to structlog.get_logger(__name__),
matching the routes/providers.py and routes/inference.py pattern.
All existing call sites use printf-style positional args, which
structlog accepts unchanged — no other edits needed.
* studio/backend: disable read timeout on SSE streams to external providers
Anthropic Opus 4.7 (adaptive thinking) and OpenAI gpt-5.x (/v1/responses)
can pause for tens of seconds between bytes while the model is
internally reasoning. httpx's read timeout is the *gap* between
successive reads, not a wall clock on the whole request — so the
shared 120s default was cutting streams mid-response:
log: Anthropic stream event counts (... text_delta: 11)
Read timeout from anthropic
(eleven text deltas in, no content_block_stop, no message_stop)
Add a separate _stream_timeout on ExternalProviderClient with
read = None (no gap timeout) and the same 10s / 120s connect/write/
pool bounds, then use it at the three SSE streaming call sites:
default OpenAI-compat chat completions, _stream_anthropic, and
_stream_openai_responses. Non-streaming call sites (chat_completion,
list_models, verify_models_endpoint_lightweight) keep self._timeout
because a stuck non-streaming response should still fail fast.
* studio/backend: log outbound Anthropic request shape for thinking debug
After bumping to Xhigh effort the user still saw zero thinking_delta
events and only one content_block_start, meaning Anthropic Opus 4.7
opened no thinking block at all. Per the effort docs that should be
impossible — Xhigh always thinks. Two open hypotheses:
1. Our adaptive branch is not wiring output_config.effort onto the
outbound body for this code path (regex miss, frontend never
propagated reasoning_effort, etc).
2. Anthropic is silently accepting output_config as an unknown
field and falling back to high default effort regardless.
Add a single-line structlog INFO right before the stream POST that
echoes the keys actually present on the body (thinking, output_config,
temperature, presence of top_p / top_k, max_tokens). Messages are
deliberately excluded to keep PII out of the log. With this in place
the next 'no thinking on 4.7 at Xhigh' report shows immediately
whether we sent the effort knob — separating client bug from
provider behaviour.
* studio/chat: surface delta.reasoning_content from Kimi / DeepSeek thinking
Kimi (kimi-k2.6, kimi-k2-thinking) and DeepSeek's reasoner stream
their thinking content via a separate top-level field on the
chat-completion delta — choices[0].delta.reasoning_content — rather
than as a structured part inside delta.content. Per Kimi docs:
In streaming output (stream=True), the reasoning_content field
will always appear before the content field.
Our chat-adapter SSE loop only read delta.content (via
extractDeltaText), so the entire reasoning channel from these
providers was being silently dropped — kimi-k2.6 thinks by default
yet the chat UI showed no reasoning panel.
In the adapter:
- Read both delta.content and delta.reasoning_content per chunk
- When reasoning_content arrives, open a block in
cumulativeText (mirrors how the backend wraps Anthropic
thinking_delta and OpenAI Responses reasoning summaries)
- When content arrives after reasoning, close first
- On stream end, force-close any still-open so
parseAssistantContent can lift it into a reasoning part cleanly
Anthropic and OpenAI Responses paths are unaffected — they already
wrap as on the backend and never set reasoning_content.
* studio: Kimi thinking toggle + 16k max_tokens floor
Two coordinated changes so Kimi's thinking is user-controllable and
the response budget meets the docs' floor.
Toggle (frontend + backend):
- getExternalReasoningCapabilities now handles provider=='kimi':
kimi-k2.6 -> reasoning_style=enable_thinking, reasoningOff allowed
kimi-k2-thinking -> always on (reasoningAlwaysOn=true, no off)
kimi-k2.5 (and anything else) -> no reasoning controls
- chat-adapter already forwards enable_thinking on the
enable_thinking-style branch, so the user toggle reaches the
backend without additional wiring there.
- external_provider stream_chat_completion now translates the
boolean into Kimi's wire shape on the default OAI-compat path:
enable_thinking=True -> body['thinking'] = {type: enabled, keep: all}
enable_thinking=False -> body['thinking'] = {type: disabled}
kimi-k2-thinking ignores the toggle so the API never gets a
disabled value it would reject. Other providers on the same
path are unaffected (gated on provider_type == 'kimi').
Max tokens floor:
- New EXTERNAL_MIN_OUTPUT_TOKENS_BY_PROVIDER table and
getExternalMinOutputTokens helper. Kimi entry = 16000 per docs:
'Set max_tokens >= 16,000 to ensure the full reasoning_content
and final content can be returned without truncation.'
- chat-adapter clamps the outbound max_tokens to
min(max(stored, providerMin), EXTERNAL_MAX_OUTPUT_TOKENS),
so a stored value of 4096 still becomes 16000 when sending to
Kimi (other providers unaffected, min stays effectively 64).
- chat-settings-sheet's Max Tokens slider min mirrors the same
floor when an external Kimi model is selected, so the slider
cannot show a value lower than what we'd actually send.
- chat-page threads activeExternalProviderType down to the panel.
* fix: stabilize external reasoning controls for Anthropic 4.6 and OpenAI o3
normalize Anthropic 4.6 reasoning effort handling by accepting max as an alias and mapping it to xhigh, while keeping Sonnet/Opus 4.6 in default model suggestions.
broaden reasoning effort typing across backend/frontend and migrate persisted max selections to xhigh for compatibility.
remove reasoning.summary=\"auto\" from OpenAI /v1/responses payloads to avoid o3 eligibility/gating errors.
tighten provider model filtering to hide retired gpt-5.3 IDs and add exact/prefix filtering support in provider routes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: add openrouter/free + full reasoning passthrough on OpenRouter
Four-layer wire-up so the OpenRouter free-router model (which picks
a free model at random per request, filtered by needed capabilities)
shows up in the picker and its reasoning channel surfaces in the
chat UI.
Registry:
- providers.py: openrouter/free seeded at the top of openrouter
default_models. Curated list, so picker shows it immediately.
Frontend capability map:
- provider-capabilities.ts: getExternalReasoningCapabilities now
treats openrouter as enable_thinking style with off support. The
Think dropdown appears for every OpenRouter model; the gateway
silently no-ops the parameter for models that do not reason, so
surfacing one toggle on every model is safe.
Backend reasoning passthrough:
- external_provider.py stream_chat_completion (default OAI-compat
branch): for provider_type=='openrouter', translate the request:
reasoning_effort in {low,medium,high} -> body['reasoning'] =
{'effort': }
enable_thinking=True -> body['reasoning'] = {'enabled': True}
enable_thinking=False -> body['reasoning'] = {'enabled': False}
Matches the documented shape at
https://openrouter.ai/docs/guides/best-practices/reasoning-tokens
with effort and max_tokens mutually exclusive.
Frontend SSE reader:
- chat-adapter.ts: OpenRouter streams reasoning as a third shape we
did not handle yet: delta.reasoning_details is an array of parts
like {type: 'reasoning.text', text: '...'}. Pull text from every
part, merge with the existing delta.reasoning_content channel
used by Kimi/DeepSeek, and feed the combined string through the
same ... wrap path so parseAssistantContent lifts
it into the reasoning panel. Anthropic/OpenAI Responses paths
already wrap on the backend, so they never set this field — no
cross-provider interference.
* studio/backend: surface OpenRouter SSE errors and router-chosen model in logs
The frontend showed 'Provider returned error' for some openrouter/free
requests with nothing on the backend side to triage from — the
existing 4xx error log only fires when the upstream returns a non-200
status code, but OpenRouter (and most OAI-compat providers) return
200 OK and emit the actual failure as an SSE error event mid-stream,
which our default-path stream loop forwarded verbatim without
logging.
Best-effort diagnostics on the default OpenAI-compat stream path:
- Peek at every `data:` line in the inner forward loop, parse JSON
best-effort (silently skip on failure so nothing is dropped).
- Count event types: delta / error / done.
- On any chunk containing an `error` field, emit a structlog WARNING
with the provider type and the error payload — same trail the
user would otherwise have to dig out of browser devtools.
- Latch the first non-empty `chunk.model` field. OpenRouter reports
the router-picked underlying model there per request, so the
finally-block summary log shows which free model handled the call.
In the finally block:
'openrouter stream complete (model=openrouter/free,
chosen=google/gemini-2.5-flash, events={delta: 47, done: 1})'
Zero overhead for non-error streams (a json.loads per chunk +
dict-key lookups). The structlog logger is already configured at
INFO; ERROR and WARNING surface in JSON logs without further setup.
Hoists `import json as _json` to module top so the default path can
reuse it; the existing in-function imports in _stream_anthropic and
_stream_openai_responses are now redundant but harmless.
* studio/chat: show router-picked model after 'openrouter/free:' in chip
When the user picks openrouter/free, the gateway routes each request
to a different underlying free model. Until now there was no way to
tell which one actually replied without reading the backend logs.
Surface the picked model in the active-model chip:
- chat-runtime-store gains lastOpenRouterChosenModel: string|null
plus a setter. Reset on every model switch unless the user stays
on openrouter/free.
- chat-adapter SSE loop latches chunk.model into the store on
every chunk whose top-level model differs from
openrouter/free, gated on the active checkpoint being
openrouter/free under an OpenRouter provider.
- chat-page externalModels useMemo appends : to the display
name for the openrouter/free option when the store has a value,
so ModelSelector renders e.g.
'openrouter/free:google/gemini-2.5-flash'
in the chip. Other models unaffected.
- Model-switch callback in chat-page clears the cached value when
the user moves to any model other than openrouter/free, so the
chip never shows a stale suffix from a previous session.
* studio/chat: shorten openrouter/free chip to openrouter:
The full display name in use was:
openrouter/free:inclusionai/ring-2.6-1t-20260508:free
The `:free` suffix on the underlying id already conveys 'free model',
which made the leading `/free` on the router id redundant, and the
`inclusionai/` org prefix was just noise crowding the chip.
Trim both. Now the chip renders as:
openrouter:ring-2.6-1t-20260508:free
Strictly a display change in chat-page externalModels useMemo — the
backend wire id stays `openrouter/free`, the runtime store still
caches the full `inclusionai/...:free` value, and the model-switch
clearing logic is unchanged.
* studio/providers: switch OpenRouter to remote listing with org allowlist + cap
Same shape as Hugging Face Inference. The curated list had only four
entries; remote listing fetches OpenRouter's full ~300-model
catalog via /v1/models and the new allowlist + limit scope it back
down to a usable picker.
- model_list_mode: remote (was curated)
- model_id_allowlist matches the prefixes:
openrouter | openai | anthropic | google | meta-llama | qwen
| mistralai | deepseek | moonshotai | inclusionai | zai-org
| z-ai
Anything outside drops out.
- model_id_limit: 20 — first 20 post-filter matches from the live
fetch; default_models stays seeded so the most useful canonical
ids are always visible regardless of API response order.
- default_models seed extended from 4 to 6 (openrouter/free,
openai/gpt-4o, anthropic/claude-sonnet-4-5, google/gemini-2.5-flash,
mistralai/mistral-large-2411, deepseek/deepseek-r1).
openrouter/free remains the first entry, so the dialog's
loadModels() union-merge (registryDefaults first, then remote,
deduped via Set) keeps it at the top of the picker.
* feat: external mistral thinking toggle
* studio/chat: fix TS2540 by replacing readonly ContentPart instead of mutating
The ContentPart type from @assistant-ui/react marks `text` as readonly,
so the coalesce-adjacent-same-type-part optimization in
parseAssistantContent failed the tsc build with:
parse-assistant-content.ts(15,10): error TS2540: Cannot assign to
'text' because it is a read-only property.
parse-assistant-content.ts(25,10): error TS2540: ...
This broke npm run build, the Studio installer's `building frontend...`
step, and every downstream CI job that runs against an installed
Studio (Mac/Windows/Linux variants of Studio API CI, GGUF CI, UI CI,
Tauri CI, Wheel CI).
Replace the last element with a fresh merged object instead of
mutating its `text` field. Same allocation profile as the previous
path (one object swap per merge), type-safe under the readonly
declaration. Behaviour unchanged.
* studio/backend: restore summary='auto' on OpenAI Responses reasoning body
A recent refactor dropped the `summary: 'auto'` field from the
reasoning config we send to /v1/responses. Without it OpenAI does
not emit reasoning summary events on most reasoning models, which
means our SSE handler has no … to wrap and the chat
reasoning panel stays blank for any gpt-5.x / o3 response.
The expected wire shape is:
body['reasoning'] = {'effort': '', 'summary': 'auto'}
Two backend tests pin this:
- test_responses_reasoning_effort_included_when_requested (high)
- test_responses_reasoning_effort_xhigh_passthrough (xhigh)
Both were failing with AssertionError because the produced body
omitted `summary: auto`.
Restore the field. Skip it only for the explicit "off" case
(effort: 'none'), where summaries serve no purpose. The
enable_thinking=True fallback (no explicit effort) also pairs
medium effort with summary='auto' so that branch produces
reasoning text too.
* chat: external reasoning, OpenRouter curation, Think toggle fixes
* fix: opus and sonnet 4.6 xhigh --> max
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99
Co-authored-by: Daniel Han
---
.../core/inference/external_provider.py | 1238 +++++++++++++++
studio/backend/core/inference/key_exchange.py | 127 ++
studio/backend/core/inference/providers.py | 287 ++++
studio/backend/main.py | 7 +
studio/backend/models/inference.py | 28 +-
studio/backend/models/providers.py | 128 ++
studio/backend/requirements/studio.txt | 2 +
studio/backend/routes/__init__.py | 2 +
studio/backend/routes/inference.py | 164 ++
studio/backend/routes/providers.py | 338 ++++
studio/backend/storage/providers_db.py | 153 ++
.../test_anthropic_thinking_translation.py | 404 +++++
studio/backend/tests/test_desktop_auth.py | 1 +
.../test_openai_responses_translation.py | 432 ++++++
studio/backend/tests/test_providers_api.py | 609 ++++++++
studio/frontend/package-lock.json | 21 +
studio/frontend/package.json | 2 +
.../public/provider-logos/anthropic.svg | 6 +
.../public/provider-logos/deepseek.svg | 14 +
.../frontend/public/provider-logos/gemini.svg | 72 +
.../public/provider-logos/huggingface.svg | 8 +
.../frontend/public/provider-logos/kimi.jpg | Bin 0 -> 15554 bytes
.../public/provider-logos/misc/meta.svg | 19 +
.../public/provider-logos/misc/microsoft.svg | 1 +
.../public/provider-logos/misc/minimax.png | Bin 0 -> 8665 bytes
.../public/provider-logos/misc/nvidia.svg | 1 +
.../public/provider-logos/misc/perplexity.png | Bin 0 -> 6632 bytes
.../public/provider-logos/misc/xai.svg | 1 +
.../public/provider-logos/misc/z-ai.svg | 215 +++
.../public/provider-logos/mistral.svg | 19 +
.../frontend/public/provider-logos/openai.svg | 5 +
.../public/provider-logos/openrouter.svg | 1 +
.../frontend/public/provider-logos/qwen.png | Bin 0 -> 117172 bytes
.../assistant-ui/model-selector.tsx | 246 ++-
.../assistant-ui/model-selector/types.ts | 9 +-
.../src/components/assistant-ui/thread.tsx | 125 +-
.../src/features/chat/api-provider-logo.tsx | 68 +
.../src/features/chat/api/chat-adapter.ts | 604 ++++++--
.../src/features/chat/api/providers-api.ts | 230 +++
.../frontend/src/features/chat/chat-page.tsx | 240 ++-
.../features/chat/chat-providers-dialog.tsx | 1361 +++++++++++++++++
.../src/features/chat/chat-settings-sheet.tsx | 199 ++-
.../src/features/chat/external-providers.ts | 230 +++
.../chat/hooks/use-chat-model-runtime.ts | 61 +-
.../features/chat/provider-capabilities.ts | 448 ++++++
.../src/features/chat/shared-composer.tsx | 173 ++-
.../chat/stores/chat-runtime-store.ts | 40 +-
.../chat/stores/external-providers-store.ts | 24 +
.../frontend/src/features/chat/types/api.ts | 37 +-
.../chat/utils/parse-assistant-content.ts | 22 +-
.../src/features/settings/settings-dialog.tsx | 26 +-
.../settings/stores/settings-dialog-store.ts | 15 +-
.../settings/tabs/connections-tab.tsx | 17 +
tests/test_import_fixes_drift.py | 4 +-
54 files changed, 8209 insertions(+), 275 deletions(-)
create mode 100644 studio/backend/core/inference/external_provider.py
create mode 100644 studio/backend/core/inference/key_exchange.py
create mode 100644 studio/backend/core/inference/providers.py
create mode 100644 studio/backend/models/providers.py
create mode 100644 studio/backend/routes/providers.py
create mode 100644 studio/backend/storage/providers_db.py
create mode 100644 studio/backend/tests/test_anthropic_thinking_translation.py
create mode 100644 studio/backend/tests/test_openai_responses_translation.py
create mode 100644 studio/backend/tests/test_providers_api.py
create mode 100644 studio/frontend/public/provider-logos/anthropic.svg
create mode 100644 studio/frontend/public/provider-logos/deepseek.svg
create mode 100644 studio/frontend/public/provider-logos/gemini.svg
create mode 100644 studio/frontend/public/provider-logos/huggingface.svg
create mode 100644 studio/frontend/public/provider-logos/kimi.jpg
create mode 100644 studio/frontend/public/provider-logos/misc/meta.svg
create mode 100644 studio/frontend/public/provider-logos/misc/microsoft.svg
create mode 100644 studio/frontend/public/provider-logos/misc/minimax.png
create mode 100644 studio/frontend/public/provider-logos/misc/nvidia.svg
create mode 100644 studio/frontend/public/provider-logos/misc/perplexity.png
create mode 100644 studio/frontend/public/provider-logos/misc/xai.svg
create mode 100644 studio/frontend/public/provider-logos/misc/z-ai.svg
create mode 100644 studio/frontend/public/provider-logos/mistral.svg
create mode 100644 studio/frontend/public/provider-logos/openai.svg
create mode 100644 studio/frontend/public/provider-logos/openrouter.svg
create mode 100644 studio/frontend/public/provider-logos/qwen.png
create mode 100644 studio/frontend/src/features/chat/api-provider-logo.tsx
create mode 100644 studio/frontend/src/features/chat/api/providers-api.ts
create mode 100644 studio/frontend/src/features/chat/chat-providers-dialog.tsx
create mode 100644 studio/frontend/src/features/chat/external-providers.ts
create mode 100644 studio/frontend/src/features/chat/provider-capabilities.ts
create mode 100644 studio/frontend/src/features/chat/stores/external-providers-store.ts
create mode 100644 studio/frontend/src/features/settings/tabs/connections-tab.tsx
diff --git a/studio/backend/core/inference/external_provider.py b/studio/backend/core/inference/external_provider.py
new file mode 100644
index 0000000000..f5b67eef70
--- /dev/null
+++ b/studio/backend/core/inference/external_provider.py
@@ -0,0 +1,1238 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""
+Async HTTP client for proxying chat completions to external LLM providers.
+
+Most registry providers expose OpenAI-compatible /v1/chat/completions endpoints;
+Anthropic uses native Messages API with translation in this client.
+"""
+
+import json as _json
+import re
+from typing import Any, AsyncGenerator, Literal, NamedTuple, Optional
+
+import httpx
+import structlog
+
+# Use structlog so INFO-level diagnostics actually surface in the
+# studio backend's JSON log stream. The stdlib root logger defaults to
+# WARNING and is not configured with handlers, so plain
+# `logging.getLogger(__name__).info(...)` was being silently dropped —
+# only WARNING/ERROR made it through (because they bypassed the root
+# level threshold via uvicorn's stderr capture). All existing call
+# sites use printf-style positional args, which structlog accepts.
+logger = structlog.get_logger(__name__)
+
+# Claude 4.7 (Opus/Sonnet/Haiku) deprecated top_k and returns 400
+# "top_k is deprecated for this model" when it is set. 3.x and 4.5/4.6
+# still accept it. Match the 4-7 line specifically so we keep the knob
+# live on every other Claude generation.
+_ANTHROPIC_TOP_K_DEPRECATED = re.compile(r"^claude-(?:opus|sonnet|haiku)-4-7(?:[-.]|$)")
+
+
+class _AnthropicThinkingSpec(NamedTuple):
+ prefixes: tuple[str, ...]
+ kind: Literal["adaptive", "manual"]
+ efforts: tuple[str, ...]
+
+
+_ANTHROPIC_THINKING_SPECS = (
+ _AnthropicThinkingSpec(
+ prefixes = ("claude-opus-4-7",),
+ kind = "adaptive",
+ efforts = ("none", "low", "medium", "high", "xhigh"),
+ ),
+ _AnthropicThinkingSpec(
+ prefixes = ("claude-opus-4-6", "claude-sonnet-4-6"),
+ kind = "adaptive",
+ efforts = ("none", "low", "medium", "high", "xhigh", "max"),
+ ),
+ _AnthropicThinkingSpec(
+ prefixes = ("claude-opus-4-5", "claude-sonnet-4-5", "claude-haiku-4-5"),
+ kind = "manual",
+ efforts = ("none", "low", "medium", "high"),
+ ),
+)
+
+
+def _anthropic_thinking_spec(model: str) -> Optional[_AnthropicThinkingSpec]:
+ for spec in _ANTHROPIC_THINKING_SPECS:
+ if model.startswith(spec.prefixes):
+ return spec
+ return None
+
+
+class _MistralThinkingSpec(NamedTuple):
+ models: tuple[str, ...]
+ style: Literal["prompt_mode", "reasoning_effort", "disabled"]
+ efforts: tuple[str, ...] = ()
+
+
+_MISTRAL_THINKING_SPECS = (
+ _MistralThinkingSpec(
+ models = ("magistral-medium-latest",),
+ style = "prompt_mode",
+ ),
+ _MistralThinkingSpec(
+ models = ("mistral-small-latest", "mistral-vibe-cli-latest"),
+ style = "reasoning_effort",
+ efforts = ("none", "high"),
+ ),
+)
+
+_OPENROUTER_MANDATORY_REASONING_MODELS = frozenset(
+ {
+ "~google/gemini-pro-latest",
+ "baidu/cobuddy:free",
+ "inclusionai/ring-2.6-1t:free",
+ "deepseek/deepseek-r1",
+ }
+)
+
+
+def _mistral_thinking_spec(model: str) -> _MistralThinkingSpec:
+ for spec in _MISTRAL_THINKING_SPECS:
+ if model in spec.models:
+ return spec
+ return _MistralThinkingSpec(models = (), style = "disabled")
+
+
+def _apply_mistral_reasoning_controls(
+ body: dict[str, Any],
+ model: str,
+ enable_thinking: Optional[bool],
+ reasoning_effort: Optional[str],
+) -> None:
+ """
+ Translate generic reasoning controls into Mistral's model-specific shape.
+
+ Current contract:
+ - magistral-medium-latest: baseline (no extra field) or
+ `prompt_mode="reasoning"` for the explicit reasoning mode.
+ - mistral-small-latest / mistral-vibe-cli-latest:
+ `reasoning_effort` in {"none", "high"}.
+ - all other tested Mistral models: no reasoning/thinking params.
+ """
+ model_for_matching = model.rsplit("/", 1)[-1].strip().lower()
+ spec = _mistral_thinking_spec(model_for_matching)
+ body.pop("prompt_mode", None)
+ body.pop("reasoning_effort", None)
+
+ if spec.style == "prompt_mode":
+ # Magistral baseline is already reasoning-capable. The explicit
+ # prompt_mode path is only used for the "high" UI selection.
+ if enable_thinking is True or reasoning_effort == "high":
+ body["prompt_mode"] = "reasoning"
+ return
+
+ if spec.style == "reasoning_effort":
+ if reasoning_effort in spec.efforts:
+ body["reasoning_effort"] = reasoning_effort
+ elif enable_thinking is False:
+ body["reasoning_effort"] = "none"
+ elif enable_thinking is True:
+ body["reasoning_effort"] = "high"
+
+
+# Shared client reused across all requests for HTTP connection pooling.
+# Auth headers and timeouts are passed per-request, so a single client
+# handles every provider without storing credentials.
+_http_client = httpx.AsyncClient()
+
+
+class ExternalProviderClient:
+ """Async proxy for OpenAI-compatible external LLM APIs."""
+
+ def __init__(
+ self,
+ provider_type: str,
+ base_url: str,
+ api_key: str,
+ timeout: float = 120.0,
+ ):
+ self.provider_type = provider_type
+ self.base_url = base_url.rstrip("/")
+ self.api_key = api_key
+ self._timeout = httpx.Timeout(timeout, connect = 10.0)
+ # Separate timeout for SSE streams: reasoning-heavy providers
+ # (Anthropic Opus 4.7 with adaptive thinking, OpenAI gpt-5.x via
+ # /v1/responses) can pause for tens of seconds between bytes
+ # while the model is internally thinking. httpx's read timeout is
+ # the *gap* between successive reads, not a wall clock — so
+ # disabling it lets long thinks complete without cutting the
+ # stream prematurely. connect/write/pool keep the 10s / 120s
+ # bounds so genuine network failures still surface.
+ self._stream_timeout = httpx.Timeout(timeout, connect = 10.0, read = None)
+
+ def _auth_headers(self) -> dict[str, str]:
+ """Build authentication headers using the provider's registry config."""
+ from core.inference.providers import get_provider_info
+
+ provider_info = get_provider_info(self.provider_type) or {}
+ auth_header = provider_info.get("auth_header", "Authorization")
+ auth_prefix = provider_info.get("auth_prefix", "Bearer ")
+
+ headers = {
+ "Content-Type": "application/json",
+ auth_header: f"{auth_prefix}{self.api_key}",
+ }
+ # Merge any provider-specific extra headers (e.g. anthropic-version, OpenRouter attribution)
+ headers.update(provider_info.get("extra_headers", {}))
+ return headers
+
+ def _is_openai_compatible(self) -> bool:
+ """Return False for providers that need request/response translation (e.g. Anthropic)."""
+ from core.inference.providers import get_provider_info
+
+ info = get_provider_info(self.provider_type) or {}
+ return info.get("openai_compatible", True)
+
+ async def stream_chat_completion(
+ self,
+ messages: list[dict[str, Any]],
+ model: str,
+ temperature: float = 0.7,
+ top_p: float = 0.95,
+ max_tokens: Optional[int] = None,
+ presence_penalty: float = 0.0,
+ top_k: Optional[int] = None,
+ enable_thinking: Optional[bool] = None,
+ reasoning_effort: Optional[str] = None,
+ stream: bool = True,
+ ) -> AsyncGenerator[str, None]:
+ """
+ Yield OpenAI-format SSE lines from the external provider.
+
+ For OpenAI-compatible providers, lines are forwarded verbatim.
+ For Anthropic, the native Messages API SSE is translated to OpenAI format.
+
+ ``top_k`` and ``presence_penalty`` are forwarded only when the caller
+ supplies a value the provider accepts — the frontend's
+ provider-capability map already filters these per provider, so we
+ treat them as opt-in here.
+ """
+ if not self._is_openai_compatible():
+ async for line in self._stream_anthropic(
+ messages,
+ model,
+ temperature,
+ top_p,
+ max_tokens,
+ top_k,
+ enable_thinking,
+ reasoning_effort,
+ ):
+ yield line
+ return
+
+ # OpenAI moved their flagship models (gpt-5.x) off /v1/chat/completions
+ # — those endpoints return 404 with "This is not a chat model" for the
+ # new families. Route all OpenAI traffic through /v1/responses instead;
+ # we translate the Responses SSE back into Chat Completions chunks so
+ # the frontend stays endpoint-agnostic.
+ if self.provider_type == "openai":
+ async for line in self._stream_openai_responses(
+ messages,
+ model,
+ temperature,
+ top_p,
+ max_tokens,
+ enable_thinking,
+ reasoning_effort,
+ ):
+ yield line
+ return
+
+ body: dict[str, Any] = {
+ "model": model,
+ "messages": messages,
+ "stream": stream,
+ "temperature": temperature,
+ "top_p": top_p,
+ "presence_penalty": presence_penalty,
+ }
+ if max_tokens is not None:
+ # OpenAI newer models (gpt-4o, gpt-5.x) reject max_tokens
+ if self.provider_type == "openai":
+ body["max_completion_tokens"] = max_tokens
+ else:
+ body["max_tokens"] = max_tokens
+
+ # Strip body fields a provider's registry entry declares unusable —
+ # reasoning-class models that lock these to fixed defaults (e.g.
+ # Kimi k2.5/k2.6 only accept temperature=1, top_p=1) 400 otherwise.
+ # The frontend capability map already hides the matching sliders;
+ # this is the matching guard for the pydantic default that the
+ # route layer would otherwise still fill in.
+ from core.inference.providers import get_provider_info
+
+ provider_info = get_provider_info(self.provider_type) or {}
+ for field in provider_info.get("body_omit", ()):
+ body.pop(field, None)
+
+ # Kimi (kimi-k2.6, kimi-k2-thinking) accepts a boolean thinking toggle
+ # via a top-level `thinking` field (the docs show it nested under
+ # extra_body, but that is an OpenAI Python SDK convention; on the
+ # wire it merges into the request body).
+ # - kimi-k2.6 defaults to thinking enabled; clients can pass
+ # {"type": "disabled"} to suppress it.
+ # - kimi-k2-thinking is always on; we never send disabled there.
+ # `keep: all` retains every thinking chunk through the stream, which
+ # is what we need so our frontend can wrap reasoning_content into
+ # the chat reasoning panel.
+ if self.provider_type == "kimi" and enable_thinking is not None:
+ if model == "kimi-k2-thinking":
+ # Always on; ignore client toggle to avoid an API-level reject.
+ pass
+ elif enable_thinking:
+ body["thinking"] = {"type": "enabled", "keep": "all"}
+ else:
+ body["thinking"] = {"type": "disabled"}
+ elif self.provider_type == "mistral":
+ _apply_mistral_reasoning_controls(
+ body, model, enable_thinking, reasoning_effort
+ )
+
+ # OpenRouter exposes a unified `reasoning` parameter on every
+ # chat-completion request — the gateway routes it to whichever
+ # underlying model actually supports reasoning, and silently
+ # no-ops for ones that don't. Documented at
+ # https://openrouter.ai/docs/guides/best-practices/reasoning-tokens
+ # Shape: `reasoning: {enabled?: bool, effort?: low|medium|high,
+ # max_tokens?: N, exclude?: bool}` with effort and max_tokens
+ # mutually exclusive. We forward either an effort level (when
+ # the user picked one) or a bare {enabled: true}. A small set of
+ # known routes rejects explicit disable with 400 ("Reasoning is
+ # mandatory for this endpoint ..."), so only those omit "off".
+ if self.provider_type == "openrouter":
+ normalized_or_model = model.strip().lower()
+ if reasoning_effort in ("low", "medium", "high"):
+ body["reasoning"] = {"effort": reasoning_effort}
+ elif enable_thinking is True:
+ body["reasoning"] = {"enabled": True}
+ elif enable_thinking is False:
+ if normalized_or_model in _OPENROUTER_MANDATORY_REASONING_MODELS:
+ body.pop("reasoning", None)
+ else:
+ body["reasoning"] = {"enabled": False}
+
+ url = f"{self.base_url}/chat/completions"
+ logger.info(
+ "Proxying chat completion to %s (provider=%s, model=%s)",
+ url,
+ self.provider_type,
+ model,
+ )
+
+ try:
+ async with _http_client.stream(
+ "POST",
+ url,
+ json = body,
+ headers = self._auth_headers(),
+ timeout = self._stream_timeout,
+ ) as response:
+ if response.status_code != 200:
+ error_body = await response.aread()
+ error_text = error_body.decode("utf-8", errors = "replace")
+ logger.error(
+ "External provider returned %d: %s",
+ response.status_code,
+ error_text[:500],
+ )
+ yield _error_sse_line(
+ response.status_code, error_text, self.provider_type
+ )
+ return
+
+ # NOTE: manual __anext__ loop instead of `async for` is intentional.
+ # On Python 3.13 + httpcore 1.0.x, `async for` auto-calls aclose() on
+ # early exit (break/return/GeneratorExit) BEFORE our finally block runs.
+ # That propagates GeneratorExit into PoolByteStream.__aiter__() while it
+ # calls `await self.aclose()` inside `with AsyncShieldCancellation()`,
+ # triggering "RuntimeError: async generator ignored GeneratorExit".
+ # Fix: call response.aclose() FIRST (sets PoolByteStream._closed=True),
+ # then lines_gen.aclose() is a no-op and GeneratorExit re-raises cleanly.
+ lines_gen = response.aiter_lines().__aiter__()
+ # Best-effort diagnostics for the default OAI-compat path. Without
+ # this, OpenRouter mid-stream errors (200 OK + error event in the
+ # SSE body) and OpenRouter-router model selection were invisible
+ # in the backend logs — the user only saw "Provider returned
+ # error" in the UI with no trail on the server side.
+ event_counts: dict[str, int] = {}
+ chosen_model: Optional[str] = None
+ try:
+ while True:
+ try:
+ line = await lines_gen.__anext__()
+ except StopAsyncIteration:
+ break
+ if not line.strip():
+ continue
+ if line.startswith("data:"):
+ data_str = line[len("data:") :].strip()
+ if data_str == "[DONE]":
+ event_counts["done"] = event_counts.get("done", 0) + 1
+ elif data_str:
+ try:
+ parsed = _json.loads(data_str)
+ except Exception:
+ parsed = None
+ if isinstance(parsed, dict):
+ # Mid-stream provider error event. OpenRouter
+ # in particular returns 200 then surfaces the
+ # actual failure as an SSE error event.
+ if "error" in parsed:
+ event_counts["error"] = (
+ event_counts.get("error", 0) + 1
+ )
+ logger.warning(
+ "%s SSE error event: %s",
+ self.provider_type,
+ parsed.get("error"),
+ )
+ else:
+ event_counts["delta"] = (
+ event_counts.get("delta", 0) + 1
+ )
+ # OpenRouter (and most OAI-compat providers)
+ # report the underlying model that handled
+ # the request in every chunk's `model` field.
+ # Latch the first non-empty value so the
+ # router-picked model surfaces in logs and
+ # is available to the proxy caller.
+ if chosen_model is None and isinstance(
+ parsed.get("model"), str
+ ):
+ chosen_model = parsed["model"]
+ yield line
+ except GeneratorExit:
+ await response.aclose() # set PoolByteStream._closed=True FIRST
+ await lines_gen.aclose() # now safe — aclose() is a no-op
+ raise
+ finally:
+ logger.info(
+ "%s stream complete (model=%s, chosen=%s, events=%s)",
+ self.provider_type,
+ model,
+ chosen_model,
+ event_counts,
+ )
+ await response.aclose()
+ await lines_gen.aclose()
+
+ except httpx.ConnectError as exc:
+ logger.error("Connection error to %s: %s", self.provider_type, exc)
+ yield _error_sse_line(
+ 502,
+ f"Failed to connect to {self.provider_type}: {exc}",
+ self.provider_type,
+ )
+ except httpx.ReadTimeout as exc:
+ logger.error("Read timeout from %s: %s", self.provider_type, exc)
+ yield _error_sse_line(
+ 504,
+ f"Timeout waiting for {self.provider_type} response",
+ self.provider_type,
+ )
+ except httpx.HTTPError as exc:
+ logger.error("HTTP error from %s: %s", self.provider_type, exc)
+ yield _error_sse_line(
+ 502,
+ f"Error communicating with {self.provider_type}: {exc}",
+ self.provider_type,
+ )
+
+ async def _stream_anthropic(
+ self,
+ messages: list[dict[str, Any]],
+ model: str,
+ temperature: float,
+ top_p: float,
+ max_tokens: Optional[int],
+ top_k: Optional[int] = None,
+ enable_thinking: Optional[bool] = None,
+ reasoning_effort: Optional[str] = None,
+ ) -> AsyncGenerator[str, None]:
+ """
+ Call the Anthropic Messages API and translate its SSE to OpenAI format.
+
+ Anthropic SSE event types:
+ content_block_delta → OpenAI chunk with delta.content
+ message_delta → OpenAI chunk with finish_reason
+ message_stop → data: [DONE]
+ (all others skipped)
+ """
+ import json as _json
+
+ # Extract system prompt and translate image_url parts to Anthropic format
+ system: Optional[str] = None
+ filtered: list[dict[str, Any]] = []
+ for msg in messages:
+ if msg.get("role") == "system":
+ content = msg.get("content", "")
+ system = (
+ content
+ if isinstance(content, str)
+ else "\n".join(
+ p["text"] for p in content if p.get("type") == "text"
+ )
+ )
+ continue
+
+ content = msg.get("content")
+ if isinstance(content, list):
+ # Translate OpenAI image_url parts → Anthropic native image format
+ anthropic_parts: list[dict[str, Any]] = []
+ for part in content:
+ if part.get("type") == "text":
+ anthropic_parts.append({"type": "text", "text": part["text"]})
+ elif part.get("type") == "image_url":
+ url = part.get("image_url", {}).get("url", "")
+ if url.startswith("data:"):
+ # data:image/png;base64, → split header and data
+ header, _, b64data = url.partition(",")
+ media_type = (
+ header.split(";")[0].replace("data:", "")
+ or "image/jpeg"
+ )
+ anthropic_parts.append(
+ {
+ "type": "image",
+ "source": {
+ "type": "base64",
+ "media_type": media_type,
+ "data": b64data,
+ },
+ }
+ )
+ else:
+ # Remote URL — Anthropic supports url source type natively.
+ # See: https://docs.anthropic.com/en/docs/build-with-claude/vision#url-based-images
+ anthropic_parts.append(
+ {
+ "type": "image",
+ "source": {
+ "type": "url",
+ "url": url,
+ },
+ }
+ )
+ filtered.append({"role": msg["role"], "content": anthropic_parts})
+ else:
+ filtered.append(msg)
+
+ body: dict[str, Any] = {
+ "model": model,
+ "messages": filtered,
+ "max_tokens": max_tokens or 1024, # required by Anthropic
+ "temperature": temperature,
+ "stream": True,
+ }
+ # top_k is deprecated on Claude 4.7 (Opus/Sonnet/Haiku) — the API
+ # returns 400 "top_k is deprecated for this model" when it is set.
+ # 3.x and 4.5/4.6 still accept it, so gate strictly on the 4.7 ids.
+ if (
+ top_k is not None
+ and top_k > 0
+ and not _ANTHROPIC_TOP_K_DEPRECATED.match(model)
+ ):
+ body["top_k"] = top_k
+ if system:
+ body["system"] = system
+ thinking_spec = _anthropic_thinking_spec(model)
+ allowed_efforts = (
+ thinking_spec.efforts
+ if thinking_spec
+ else ("none", "low", "medium", "high")
+ )
+ effort = reasoning_effort if reasoning_effort in allowed_efforts else None
+ # Claude 4.6 Opus/Sonnet accept top-tier adaptive effort as "max" only;
+ # "xhigh" is rejected (supported on Claude 4.7). Map our shared "xhigh"
+ # semantic to "max" for 4.6 outbound requests while still accepting
+ # both in ``allowed_efforts`` for persisted / cross-provider UI state.
+ if effort == "xhigh" and model.startswith(
+ ("claude-opus-4-6", "claude-sonnet-4-6")
+ ):
+ effort = "max"
+ if effort is None:
+ if enable_thinking is False:
+ effort = "none"
+ elif enable_thinking is True:
+ effort = "medium"
+ # Normalize one semantic Thinking control into Anthropic's two model-era
+ # APIs: adaptive effort on Claude 4.6/4.7, manual budget_tokens on 4.5.
+ if effort and effort != "none":
+ # Anthropic rejects top_k whenever thinking is enabled.
+ body.pop("top_k", None)
+ # Anthropic requires temperature=1 whenever thinking is enabled,
+ # AND forbids top_p in the same request: setting both produces
+ # "temperature and top_p cannot both be specified for this
+ # model. Please use only one."
+ # The base body never sets top_p, but pop defensively in case
+ # an upstream edit ever adds it before this branch runs.
+ body["temperature"] = 1
+ body.pop("top_p", None)
+ if thinking_spec and thinking_spec.kind == "adaptive":
+ # `display` defaults to "omitted" on Claude Opus 4.7 (per the
+ # adaptive-thinking docs) — without an explicit opt-in the
+ # API emits an empty thinking block plus a signature_delta,
+ # so our SSE handler would surface a stray
+ # and the reasoning panel would stay blank. Force
+ # "summarized" so 4.7 streams thinking_delta events like
+ # 4.6 does. On 4.6 / Sonnet 4.6 this is the default, so
+ # setting it explicitly is harmless.
+ body["thinking"] = {"type": "adaptive", "display": "summarized"}
+ # Per the Messages API reference, the effort knob for
+ # adaptive thinking lives under `output_config.effort` —
+ # NOT as a top-level field. Sending `effort: ...` directly
+ # produces a 400 "effort: Extra inputs are not permitted".
+ # Allowed values: low | medium | high | xhigh | max. See:
+ # https://platform.claude.com/docs/en/api/messages
+ body["output_config"] = {"effort": effort}
+ elif thinking_spec and thinking_spec.kind == "manual":
+ budget_tokens = {"low": 1024, "medium": 2048, "high": 4096}[effort]
+ body["thinking"] = {
+ "type": "enabled",
+ "budget_tokens": budget_tokens,
+ }
+ # Anthropic requires max_tokens to be strictly greater than
+ # thinking.budget_tokens on the manual-thinking path.
+ if body.get("max_tokens", 0) <= budget_tokens:
+ body["max_tokens"] = budget_tokens + 1024
+
+ url = f"{self.base_url}/messages"
+ completion_id = f"chatcmpl-anthropic-{model.replace('/', '-')}"
+
+ # Log the outgoing config keys (not the messages themselves) so we
+ # can prove which thinking/effort fields actually reached the wire.
+ # If Anthropic skips reasoning despite a configured effort, this
+ # tells us whether we sent the field or dropped it on the floor.
+ logger.info(
+ "Anthropic request shape (model=%s, has_thinking=%s, thinking=%s, "
+ "output_config=%s, temperature=%s, has_top_p=%s, has_top_k=%s, "
+ "max_tokens=%s)",
+ model,
+ "thinking" in body,
+ body.get("thinking"),
+ body.get("output_config"),
+ body.get("temperature"),
+ "top_p" in body,
+ "top_k" in body,
+ body.get("max_tokens"),
+ )
+
+ _finish_reason_map = {
+ "end_turn": "stop",
+ "max_tokens": "length",
+ "stop_sequence": "stop",
+ }
+
+ logger.info("Proxying Anthropic Messages API to %s (model=%s)", url, model)
+
+ try:
+ async with _http_client.stream(
+ "POST",
+ url,
+ json = body,
+ headers = self._auth_headers(),
+ timeout = self._stream_timeout,
+ ) as response:
+ if response.status_code != 200:
+ error_body = await response.aread()
+ error_text = error_body.decode("utf-8", errors = "replace")
+ logger.error(
+ "Anthropic returned %d: %s",
+ response.status_code,
+ error_text[:500],
+ )
+ yield _error_sse_line(
+ response.status_code, error_text, self.provider_type
+ )
+ return
+
+ # NOTE: same manual __anext__ loop as stream_chat_completion — see comment there.
+ lines_gen = response.aiter_lines().__aiter__()
+ thinking_open = False
+ # Diagnostic counters for the next time the user reports
+ # "no thinking content" — distinguishes "Anthropic never sent
+ # thinking_delta" from "frontend didn't render the chunks".
+ event_counts: dict[str, int] = {}
+
+ def _content_chunk(text: str) -> str:
+ chunk = {
+ "id": completion_id,
+ "object": "chat.completion.chunk",
+ "choices": [
+ {
+ "index": 0,
+ "delta": {"content": text},
+ "finish_reason": None,
+ }
+ ],
+ }
+ return f"data: {_json.dumps(chunk)}"
+
+ try:
+ while True:
+ try:
+ line = await lines_gen.__anext__()
+ except StopAsyncIteration:
+ break
+ if not line or line.startswith("event:"):
+ continue
+ if not line.startswith("data:"):
+ continue
+
+ data_str = line[len("data:") :].strip()
+ if not data_str:
+ continue
+
+ try:
+ event = _json.loads(data_str)
+ except _json.JSONDecodeError:
+ continue
+
+ event_type = event.get("type")
+ if event_type == "content_block_delta":
+ delta_kind = (event.get("delta") or {}).get("type")
+ key = f"{event_type}:{delta_kind}"
+ else:
+ key = event_type or ""
+ event_counts[key] = event_counts.get(key, 0) + 1
+
+ if event_type == "content_block_delta":
+ delta = event.get("delta", {})
+ delta_type = delta.get("type")
+ if delta_type == "thinking_delta":
+ # Anthropic streams extended-thinking content as
+ # thinking_delta events on a separate content
+ # block. Wrap as inline ... so
+ # the frontend's parseAssistantContent lifts it
+ # into the reasoning panel — same pattern as
+ # the OpenAI Responses path.
+ thinking_text = delta.get("thinking", "")
+ if thinking_text:
+ if not thinking_open:
+ thinking_text = f"{thinking_text}"
+ thinking_open = True
+ yield _content_chunk(thinking_text)
+ elif delta_type == "text_delta":
+ # First text after a thinking block closes the
+ # tag we opened above. Anthropic emits
+ # a content_block_stop between blocks, but
+ # closing on the text_delta transition is more
+ # forgiving if events arrive out of order.
+ if thinking_open:
+ yield _content_chunk("")
+ thinking_open = False
+ text = delta.get("text", "")
+ if text:
+ yield _content_chunk(text)
+ # signature_delta and any other delta types are
+ # intentionally skipped — they carry trust /
+ # verification metadata, not user-visible content.
+
+ elif event_type == "content_block_stop":
+ # Close the tag when the thinking block
+ # ends, in case no text_delta follows (e.g.
+ # display=omitted on Claude 4.7, or thinking-only
+ # turns).
+ if thinking_open:
+ yield _content_chunk("")
+ thinking_open = False
+
+ elif event_type == "message_delta":
+ stop_reason = event.get("delta", {}).get("stop_reason")
+ if stop_reason:
+ if thinking_open:
+ yield _content_chunk("")
+ thinking_open = False
+ chunk = {
+ "id": completion_id,
+ "object": "chat.completion.chunk",
+ "choices": [
+ {
+ "index": 0,
+ "delta": {},
+ "finish_reason": _finish_reason_map.get(
+ stop_reason, "stop"
+ ),
+ }
+ ],
+ }
+ yield f"data: {_json.dumps(chunk)}"
+
+ elif event_type == "message_stop":
+ if thinking_open:
+ yield _content_chunk("")
+ thinking_open = False
+ yield "data: [DONE]"
+ await (
+ response.aclose()
+ ) # set PoolByteStream._closed=True FIRST
+ break
+ except GeneratorExit:
+ await response.aclose() # set PoolByteStream._closed=True FIRST
+ await lines_gen.aclose() # now safe — aclose() is a no-op
+ raise
+ finally:
+ # Surface per-event-type counts so reports of "no
+ # reasoning panel content" can be triaged at a glance:
+ # zero `content_block_delta:thinking_delta` entries
+ # means Anthropic skipped thinking for this prompt
+ # (adaptive can choose to); non-zero means thinking
+ # arrived and we wrapped it — any visual gap is then
+ # on the frontend.
+ logger.info(
+ "Anthropic stream event counts (model=%s): %s",
+ model,
+ event_counts,
+ )
+ await response.aclose()
+ await lines_gen.aclose()
+
+ except httpx.ConnectError as exc:
+ logger.error("Connection error to %s: %s", self.provider_type, exc)
+ yield _error_sse_line(
+ 502,
+ f"Failed to connect to {self.provider_type}: {exc}",
+ self.provider_type,
+ )
+ except httpx.ReadTimeout as exc:
+ logger.error("Read timeout from %s: %s", self.provider_type, exc)
+ yield _error_sse_line(
+ 504,
+ f"Timeout waiting for {self.provider_type} response",
+ self.provider_type,
+ )
+ except httpx.HTTPError as exc:
+ logger.error("HTTP error from %s: %s", self.provider_type, exc)
+ yield _error_sse_line(
+ 502,
+ f"Error communicating with {self.provider_type}: {exc}",
+ self.provider_type,
+ )
+
+ async def _stream_openai_responses(
+ self,
+ messages: list[dict[str, Any]],
+ model: str,
+ temperature: float,
+ top_p: float,
+ max_tokens: Optional[int],
+ enable_thinking: Optional[bool],
+ reasoning_effort: Optional[str],
+ ) -> AsyncGenerator[str, None]:
+ """
+ Call OpenAI's /v1/responses endpoint and translate its SSE stream back
+ into OpenAI Chat Completions chunk format.
+
+ The Responses API uses a different request shape (``input`` instead of
+ ``messages``, ``instructions`` for system prompts, ``max_output_tokens``
+ for the budget) and emits event-typed SSE frames (e.g.
+ ``response.output_text.delta``) rather than chat-completion chunks.
+ ``presence_penalty`` / ``top_k`` are not part of the Responses contract
+ and are dropped here intentionally.
+ """
+ import json as _json
+
+ # Split system messages out into a single `instructions` string and
+ # translate user/assistant messages into the Responses input shape.
+ instructions_parts: list[str] = []
+ input_items: list[dict[str, Any]] = []
+ for msg in messages:
+ role = msg.get("role")
+ content = msg.get("content", "")
+
+ if role == "system":
+ if isinstance(content, str):
+ if content:
+ instructions_parts.append(content)
+ elif isinstance(content, list):
+ for part in content:
+ if part.get("type") == "text" and part.get("text"):
+ instructions_parts.append(part["text"])
+ continue
+
+ if isinstance(content, str):
+ input_items.append({"role": role, "content": content})
+ continue
+
+ if isinstance(content, list):
+ translated_parts: list[dict[str, Any]] = []
+ for part in content:
+ part_type = part.get("type")
+ if part_type == "text":
+ translated_parts.append(
+ {"type": "input_text", "text": part.get("text", "")}
+ )
+ elif part_type == "image_url":
+ url = part.get("image_url", {}).get("url", "")
+ if url:
+ # Responses takes image_url as a flat string (both
+ # https:// URLs and data: URLs are accepted).
+ translated_parts.append(
+ {"type": "input_image", "image_url": url}
+ )
+ if translated_parts:
+ input_items.append({"role": role, "content": translated_parts})
+
+ # NOTE: gpt-5.x / o3 / gpt-4.5 are reasoning-class models. They reject
+ # temperature and top_p with `Unsupported parameter` 400s on
+ # /v1/responses (and on /v1/chat/completions for the same families).
+ # The PROVIDER_REGISTRY['openai'] model_id_allowlist already scopes
+ # the picker to those families, so we never need to send sampling
+ # knobs here. ``reasoning.effort`` defaults to "medium" server-side
+ # if omitted — surface it in a future commit if a knob is wanted.
+ del temperature, top_p # explicit drop — params are accepted for
+ # API symmetry with the other stream methods but not forwarded.
+
+ body: dict[str, Any] = {
+ "model": model,
+ "input": input_items,
+ "stream": True,
+ }
+ # `summary: "auto"` is what makes /v1/responses emit reasoning
+ # summary events — without it OpenAI returns no thinking text on
+ # most reasoning models, the SSE handler has no …
+ # to wrap, and the chat reasoning panel stays blank. Always pair
+ # an explicit effort with summary except for the explicit "off"
+ # case (effort: "none"), where summaries are pointless.
+ if reasoning_effort in (
+ "minimal",
+ "low",
+ "medium",
+ "high",
+ "max",
+ "xhigh",
+ ):
+ body["reasoning"] = {"effort": reasoning_effort, "summary": "auto"}
+ elif reasoning_effort == "none" or enable_thinking is False:
+ body["reasoning"] = {"effort": "none"}
+ elif enable_thinking is True:
+ body["reasoning"] = {"effort": "medium", "summary": "auto"}
+ if instructions_parts:
+ body["instructions"] = "\n\n".join(instructions_parts)
+ if max_tokens is not None:
+ body["max_output_tokens"] = max_tokens
+
+ url = f"{self.base_url}/responses"
+ completion_id = f"chatcmpl-openai-{model.replace('/', '-')}"
+
+ logger.info("Proxying OpenAI Responses API to %s (model=%s)", url, model)
+
+ try:
+ async with _http_client.stream(
+ "POST",
+ url,
+ json = body,
+ headers = self._auth_headers(),
+ timeout = self._stream_timeout,
+ ) as response:
+ if response.status_code != 200:
+ error_body = await response.aread()
+ error_text = error_body.decode("utf-8", errors = "replace")
+ logger.error(
+ "OpenAI Responses returned %d: %s",
+ response.status_code,
+ error_text[:500],
+ )
+ yield _error_sse_line(
+ response.status_code, error_text, self.provider_type
+ )
+ return
+
+ # NOTE: same manual __anext__ loop as stream_chat_completion —
+ # see comment there for the GeneratorExit / aclose ordering.
+ lines_gen = response.aiter_lines().__aiter__()
+ done_emitted = False
+ reasoning_open = False
+ reasoning_emitted = False
+
+ def _extract_reasoning_text(payload: Any) -> str:
+ if payload is None:
+ return ""
+ if isinstance(payload, str):
+ return payload
+ if isinstance(payload, list):
+ out: list[str] = []
+ for item in payload:
+ text = _extract_reasoning_text(item)
+ if text:
+ out.append(text)
+ return "".join(out)
+ if isinstance(payload, dict):
+ # OpenAI responses may carry reasoning summaries in
+ # different envelope fields across event variants.
+ for key in ("text", "delta", "content", "summary"):
+ if key in payload:
+ text = _extract_reasoning_text(payload.get(key))
+ if text:
+ return text
+ if payload.get("type") == "summary_text":
+ return _extract_reasoning_text(payload.get("text"))
+ return ""
+
+ def _chunk_with_text(text: str) -> str:
+ chunk = {
+ "id": completion_id,
+ "object": "chat.completion.chunk",
+ "choices": [
+ {
+ "index": 0,
+ "delta": {"content": text},
+ "finish_reason": None,
+ }
+ ],
+ }
+ return f"data: {_json.dumps(chunk)}"
+
+ try:
+ while True:
+ try:
+ line = await lines_gen.__anext__()
+ except StopAsyncIteration:
+ break
+ if not line or line.startswith("event:"):
+ continue
+ if not line.startswith("data:"):
+ continue
+
+ data_str = line[len("data:") :].strip()
+ if not data_str:
+ continue
+ if data_str == "[DONE]":
+ if not done_emitted:
+ yield "data: [DONE]"
+ done_emitted = True
+ break
+
+ try:
+ event = _json.loads(data_str)
+ except _json.JSONDecodeError:
+ continue
+
+ event_type = event.get("type")
+
+ if event_type == "response.output_text.delta":
+ delta_text = event.get("delta", "")
+ if delta_text:
+ if reasoning_open:
+ yield _chunk_with_text("")
+ reasoning_open = False
+ yield _chunk_with_text(delta_text)
+
+ elif event_type == "response.output_item.done":
+ item = event.get("item", {})
+ if (
+ isinstance(item, dict)
+ and item.get("type") == "reasoning"
+ ):
+ summary_text = _extract_reasoning_text(
+ item.get("summary")
+ )
+ if summary_text and not reasoning_emitted:
+ if not reasoning_open:
+ summary_text = f"{summary_text}"
+ reasoning_open = True
+ yield _chunk_with_text(summary_text)
+ reasoning_emitted = True
+
+ elif isinstance(event_type, str) and "reasoning" in event_type:
+ reasoning_delta = _extract_reasoning_text(event)
+ if reasoning_delta:
+ if not reasoning_open:
+ reasoning_delta = f"{reasoning_delta}"
+ reasoning_open = True
+ yield _chunk_with_text(reasoning_delta)
+ reasoning_emitted = True
+
+ elif event_type == "response.completed":
+ if reasoning_open:
+ yield _chunk_with_text("")
+ reasoning_open = False
+ chunk = {
+ "id": completion_id,
+ "object": "chat.completion.chunk",
+ "choices": [
+ {
+ "index": 0,
+ "delta": {},
+ "finish_reason": "stop",
+ }
+ ],
+ }
+ yield f"data: {_json.dumps(chunk)}"
+
+ elif event_type == "response.incomplete":
+ if reasoning_open:
+ yield _chunk_with_text("")
+ reasoning_open = False
+ chunk = {
+ "id": completion_id,
+ "object": "chat.completion.chunk",
+ "choices": [
+ {
+ "index": 0,
+ "delta": {},
+ "finish_reason": "length",
+ }
+ ],
+ }
+ yield f"data: {_json.dumps(chunk)}"
+
+ elif event_type in ("response.failed", "error"):
+ # Surface the failure to the client; let the
+ # outer route emit [DONE] as part of its cleanup.
+ error_payload = event.get("response", {}).get(
+ "error", {}
+ ) or {
+ "message": event.get("message", "Unknown error"),
+ "code": event.get("code"),
+ }
+ yield _error_sse_line(
+ 502,
+ _json.dumps(error_payload),
+ self.provider_type,
+ )
+ break
+ except GeneratorExit:
+ await response.aclose()
+ await lines_gen.aclose()
+ raise
+ finally:
+ await response.aclose()
+ await lines_gen.aclose()
+
+ except httpx.ConnectError as exc:
+ logger.error("Connection error to %s: %s", self.provider_type, exc)
+ yield _error_sse_line(
+ 502,
+ f"Failed to connect to {self.provider_type}: {exc}",
+ self.provider_type,
+ )
+ except httpx.ReadTimeout as exc:
+ logger.error("Read timeout from %s: %s", self.provider_type, exc)
+ yield _error_sse_line(
+ 504,
+ f"Timeout waiting for {self.provider_type} response",
+ self.provider_type,
+ )
+ except httpx.HTTPError as exc:
+ logger.error("HTTP error from %s: %s", self.provider_type, exc)
+ yield _error_sse_line(
+ 502,
+ f"Error communicating with {self.provider_type}: {exc}",
+ self.provider_type,
+ )
+
+ async def chat_completion(
+ self,
+ messages: list[dict[str, Any]],
+ model: str,
+ temperature: float = 0.7,
+ top_p: float = 0.95,
+ max_tokens: Optional[int] = None,
+ presence_penalty: float = 0.0,
+ ) -> dict[str, Any]:
+ """Non-streaming chat completion. Returns the full response dict.
+
+ Note: only valid for OpenAI-compatible providers. Anthropic requires its
+ own Messages API; use stream_chat_completion (with stream=False) instead
+ if a non-streaming Anthropic path is needed in the future.
+ """
+ body: dict[str, Any] = {
+ "model": model,
+ "messages": messages,
+ "stream": False,
+ "temperature": temperature,
+ "top_p": top_p,
+ "presence_penalty": presence_penalty,
+ }
+ if max_tokens is not None:
+ if self.provider_type == "openai":
+ body["max_completion_tokens"] = max_tokens
+ else:
+ body["max_tokens"] = max_tokens
+
+ response = await _http_client.post(
+ f"{self.base_url}/chat/completions",
+ json = body,
+ headers = self._auth_headers(),
+ timeout = self._timeout,
+ )
+ response.raise_for_status()
+ return response.json()
+
+ async def list_models(self) -> list[dict[str, Any]]:
+ """
+ Call GET /models on the provider to discover available models.
+
+ Returns a list of model dicts with at least 'id' and optionally
+ 'created', 'owned_by', etc.
+
+ All supported providers expose a /models endpoint:
+ - OpenAI-compatible: standard {"data": [...]} response
+ - Anthropic: https://api.anthropic.com/v1/models — same {"data": [...]} shape
+ """
+ try:
+ response = await _http_client.get(
+ f"{self.base_url}/models",
+ headers = self._auth_headers(),
+ timeout = self._timeout,
+ )
+ response.raise_for_status()
+ data = response.json()
+ # OpenAI format: {"data": [{"id": "...", ...}, ...]}
+ models = data.get("data", [])
+ return models
+ except httpx.HTTPError as exc:
+ logger.error("Failed to list models from %s: %s", self.provider_type, exc)
+ raise
+
+ async def verify_models_endpoint_lightweight(self) -> None:
+ """
+ Confirm GET /models returns 200 without buffering the full response body.
+
+ Used for providers with enormous catalogs (e.g. OpenRouter, Hugging Face router)
+ where downloading the full JSON would be prohibitive.
+ """
+ url = f"{self.base_url}/models"
+ try:
+ async with _http_client.stream(
+ "GET",
+ url,
+ headers = self._auth_headers(),
+ timeout = self._timeout,
+ ) as response:
+ if response.status_code != 200:
+ response.raise_for_status()
+ async for _chunk in response.aiter_bytes(chunk_size = 2048):
+ break
+ except httpx.HTTPError as exc:
+ logger.error(
+ "Lightweight /models check failed for %s: %s",
+ self.provider_type,
+ exc,
+ )
+ raise
+
+ async def close(self) -> None:
+ """No-op — the underlying client is shared across requests."""
+
+
+def _error_sse_line(status_code: int, message: str, provider_type: str) -> str:
+ """Format an error as an SSE data line in OpenAI error format."""
+ import json
+
+ error_obj = {
+ "error": {
+ "message": message,
+ "type": "provider_error",
+ "code": str(status_code),
+ "provider": provider_type,
+ }
+ }
+ return f"data: {json.dumps(error_obj)}"
diff --git a/studio/backend/core/inference/key_exchange.py b/studio/backend/core/inference/key_exchange.py
new file mode 100644
index 0000000000..f43bb16cf6
--- /dev/null
+++ b/studio/backend/core/inference/key_exchange.py
@@ -0,0 +1,127 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""
+RSA key pair for encrypting API keys in transit.
+
+The frontend encrypts API keys with the server's public key before
+including them in requests. The backend decrypts with its private key
+before forwarding to external providers.
+
+The key pair is generated at server startup and lives only in memory —
+it is regenerated on each restart. The frontend fetches the public key
+via GET /api/providers/public-key on load.
+"""
+
+import base64
+import hashlib
+import logging
+
+from cryptography.hazmat.primitives.asymmetric import rsa, padding
+from cryptography.hazmat.primitives import serialization, hashes
+
+logger = logging.getLogger(__name__)
+
+_private_key: rsa.RSAPrivateKey | None = None
+_public_key_pem: str | None = None
+_public_key_fingerprint: str | None = None
+
+
+def _compute_fingerprint(pem: str) -> str:
+ """SHA256 of the PEM bytes, truncated for log compactness."""
+ return hashlib.sha256(pem.encode("utf-8")).hexdigest()[:16]
+
+
+def init_key_pair() -> None:
+ """Generate an RSA-2048 key pair. Called once at server startup."""
+ global _private_key, _public_key_pem, _public_key_fingerprint
+ if _private_key is not None:
+ # Re-entry is suspicious — every fresh keypair invalidates all
+ # in-flight ciphertext encrypted against the previous public key.
+ # Log loudly so a regression that calls init twice is visible.
+ logger.warning(
+ "init_key_pair called again — replacing existing RSA keypair "
+ "(previous fingerprint=%s). Any frontend that cached the old "
+ "public key will start hitting decryption failures.",
+ _public_key_fingerprint,
+ )
+ _private_key = rsa.generate_private_key(
+ public_exponent = 65537,
+ key_size = 2048,
+ )
+ _public_key_pem = (
+ _private_key.public_key()
+ .public_bytes(
+ serialization.Encoding.PEM,
+ serialization.PublicFormat.SubjectPublicKeyInfo,
+ )
+ .decode("utf-8")
+ )
+ _public_key_fingerprint = _compute_fingerprint(_public_key_pem)
+ logger.info(
+ "RSA key pair generated for API key encryption (fingerprint=%s)",
+ _public_key_fingerprint,
+ )
+
+
+def get_public_key_fingerprint() -> str | None:
+ """Short SHA256 of the current public key PEM; None before init."""
+ return _public_key_fingerprint
+
+
+def get_public_key_pem() -> str:
+ """Return the PEM-encoded public key for the frontend."""
+ if _public_key_pem is None:
+ raise RuntimeError("Key pair not initialized. Call init_key_pair() first.")
+ return _public_key_pem
+
+
+def decrypt_api_key(encrypted_b64: str) -> str:
+ """
+ Decrypt an API key that was encrypted with the public key.
+
+ Args:
+ encrypted_b64: Base64-encoded RSA-OAEP ciphertext.
+
+ Returns:
+ The plaintext API key string.
+ """
+ if _private_key is None:
+ raise RuntimeError("Key pair not initialized. Call init_key_pair() first.")
+
+ try:
+ ciphertext = base64.b64decode(encrypted_b64)
+ except Exception as exc:
+ logger.warning(
+ "decrypt_api_key: base64 decode failed (input_len=%d, fingerprint=%s): %s: %s",
+ len(encrypted_b64),
+ _public_key_fingerprint,
+ type(exc).__name__,
+ exc,
+ )
+ raise
+
+ try:
+ plaintext = _private_key.decrypt(
+ ciphertext,
+ padding.OAEP(
+ mgf = padding.MGF1(algorithm = hashes.SHA256()),
+ algorithm = hashes.SHA256(),
+ label = None,
+ ),
+ )
+ except Exception as exc:
+ # Surface enough state to distinguish key mismatch (wrong public key
+ # used on encrypt) from a padding/algo mismatch or corrupted bytes.
+ # Expected ciphertext length for RSA-2048 is exactly 256 bytes.
+ logger.warning(
+ "decrypt_api_key: RSA decrypt failed (ciphertext_len=%d, expected=256, "
+ "fingerprint=%s, exc=%s): %s",
+ len(ciphertext),
+ _public_key_fingerprint,
+ type(exc).__name__,
+ exc,
+ )
+ raise
+
+ return plaintext.decode("utf-8")
diff --git a/studio/backend/core/inference/providers.py b/studio/backend/core/inference/providers.py
new file mode 100644
index 0000000000..4b6d7d6b17
--- /dev/null
+++ b/studio/backend/core/inference/providers.py
@@ -0,0 +1,287 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""
+Static registry of supported external LLM providers.
+
+All providers expose OpenAI-compatible /v1/chat/completions endpoints
+with Bearer token authentication and SSE streaming support.
+"""
+
+import re
+from typing import Any
+
+PROVIDER_REGISTRY: dict[str, dict[str, Any]] = {
+ "openai": {
+ "display_name": "OpenAI",
+ "base_url": "https://api.openai.com/v1",
+ "default_models": [
+ "gpt-5.5",
+ "gpt-5.4",
+ "gpt-5.4-mini",
+ "o3",
+ ],
+ "supports_streaming": True,
+ "supports_vision": True,
+ "supports_tool_calling": True,
+ "auth_header": "Authorization",
+ "auth_prefix": "Bearer ",
+ # Keep the model picker scoped to the current generation. The remote
+ # /v1/models listing returns dozens of historical snapshots, fine-tunes
+ # and non-chat models (embeddings, TTS, image, moderation) that we
+ # never want to surface in the chat UI. Filtering here so backend
+ # is the single source of truth.
+ "model_id_allowlist": re.compile(r"^(gpt-5\.[345]|gpt-4\.5|o3)(?:[-.]|$)"),
+ # Hide dated snapshots and the retired plain gpt-5.3 id.
+ "model_id_denylist": re.compile(r"^(gpt-5\.3)$|-\d{4}-\d{2}-\d{2}$"),
+ },
+ "anthropic": {
+ "display_name": "Anthropic",
+ "base_url": "https://api.anthropic.com/v1",
+ "default_models": [
+ "claude-opus-4-7",
+ "claude-opus-4-6",
+ "claude-sonnet-4-6",
+ "claude-opus-4-5",
+ "claude-sonnet-4-5",
+ "claude-haiku-4-5",
+ ],
+ # Anthropic /v1/models returns dated snapshot ids alongside the
+ # canonical names (e.g. claude-3-5-sonnet-20241022). Hide the
+ # YYYYMMDD-suffixed variants from the picker — same intent as the
+ # OpenAI denylist, just a different date format (no dashes between
+ # year/month/day).
+ "model_id_denylist": re.compile(r"-\d{8}$"),
+ "supports_streaming": True,
+ "supports_vision": True,
+ "supports_tool_calling": False,
+ "auth_header": "x-api-key",
+ "auth_prefix": "",
+ "extra_headers": {
+ "anthropic-version": "2023-06-01",
+ },
+ "openai_compatible": False,
+ "notes": "Native Anthropic Messages API. Uses x-api-key header and /v1/messages endpoint with SSE translation.",
+ },
+ "gemini": {
+ "display_name": "Google Gemini",
+ "base_url": "https://generativelanguage.googleapis.com/v1beta/openai",
+ # Curated lineup — Google's /v1beta/openai/models returns dozens
+ # of historical / experimental / embedding ids. Cap to the current
+ # 3.x family plus the rolling `*-latest` aliases.
+ "default_models": [
+ "gemini-3.1-pro-preview",
+ "gemini-3.1-flash-lite",
+ "gemini-3-flash-preview",
+ "gemini-pro-latest",
+ "gemini-flash-latest",
+ "gemini-flash-lite-latest",
+ ],
+ "supports_streaming": True,
+ "supports_vision": True,
+ "supports_tool_calling": True,
+ "auth_header": "Authorization",
+ "auth_prefix": "Bearer ",
+ "notes": "OpenAI-compatible endpoint. API key from https://aistudio.google.com/apikey.",
+ "model_id_allowlist": re.compile(
+ r"^(gemini-3\.1-flash-lite|gemini-3-flash-preview|"
+ r"gemini-3\.1-pro-preview|gemini-pro-latest|"
+ r"gemini-flash-latest|gemini-flash-lite-latest)$"
+ ),
+ },
+ "deepseek": {
+ "display_name": "DeepSeek",
+ "base_url": "https://api.deepseek.com/v1",
+ "default_models": [
+ "deepseek-chat",
+ "deepseek-reasoner",
+ ],
+ "supports_streaming": True,
+ "supports_vision": False,
+ "supports_tool_calling": True,
+ "auth_header": "Authorization",
+ "auth_prefix": "Bearer ",
+ "notes": "OpenAI-compatible API. deepseek-chat = V3, deepseek-reasoner = R1 thinking mode.",
+ },
+ "mistral": {
+ "display_name": "Mistral AI",
+ "base_url": "https://api.mistral.ai/v1",
+ "default_models": [
+ "codestral-latest",
+ "devstral-latest",
+ "devstral-medium-latest",
+ "magistral-medium-latest",
+ "ministral-14b-latest",
+ "ministral-3b-latest",
+ "ministral-8b-latest",
+ "mistral-large-latest",
+ "mistral-medium-latest",
+ "mistral-small-latest",
+ "mistral-tiny-latest",
+ "mistral-vibe-cli-latest",
+ ],
+ "supports_streaming": True,
+ "supports_vision": True,
+ "supports_tool_calling": True,
+ "auth_header": "Authorization",
+ "auth_prefix": "Bearer ",
+ "model_id_allowlist": re.compile(
+ r"^(codestral-latest|devstral-latest|devstral-medium-latest|"
+ r"magistral-medium-latest|ministral-(?:14b|3b|8b)-latest|"
+ r"mistral-(?:large|medium|small|tiny)-latest|"
+ r"mistral-vibe-cli-latest)$"
+ ),
+ },
+ "kimi": {
+ "display_name": "Kimi",
+ "base_url": "https://api.moonshot.ai/v1",
+ # Current Kimi model lineup per the official docs:
+ # https://platform.kimi.ai/docs/models
+ # Listing/overview endpoints used to enumerate them:
+ # https://platform.kimi.ai/docs/api/list-models
+ # https://platform.kimi.ai/docs/api/overview
+ # kimi-k2.6 and kimi-k2.5 are the two SoTA multimodal models we
+ # surface in the picker; everything else (moonshot-v1-*, dated
+ # k2 previews) is filtered out by model_id_allowlist below.
+ "default_models": [
+ "kimi-k2.6",
+ "kimi-k2.5",
+ ],
+ "supports_streaming": True,
+ "supports_vision": True,
+ "supports_tool_calling": True,
+ "auth_header": "Authorization",
+ "auth_prefix": "Bearer ",
+ "notes": "Moonshot API key. China: use base URL https://api.moonshot.cn/v1",
+ "model_id_allowlist": re.compile(r"^kimi-k2\.[56]$"),
+ # Both k2.6 and k2.5 are reasoning-class. The API rejects custom
+ # sampling: "invalid temperature: only 1 is allowed for this model"
+ # (and the same shape for top_p). Strip both fields from the
+ # outbound body so the server falls back to its required defaults.
+ "body_omit": ("temperature", "top_p"),
+ },
+ "qwen": {
+ "display_name": "Qwen",
+ "base_url": "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
+ "default_models": [
+ "qwen-plus",
+ "qwen-turbo",
+ "qwen-max",
+ "qwen2.5-72b-instruct",
+ ],
+ "supports_streaming": True,
+ "supports_vision": True,
+ "supports_tool_calling": True,
+ "auth_header": "Authorization",
+ "auth_prefix": "Bearer ",
+ "notes": "DashScope API key. China mainland: override base URL to https://dashscope.aliyuncs.com/compatible-mode/v1",
+ },
+ "huggingface": {
+ "display_name": "Hugging Face",
+ "base_url": "https://router.huggingface.co/v1",
+ # Seed the picker with a few popular ids so something is selectable
+ # before the live /v1/models call resolves. The remote listing is
+ # the source of truth — see model_list_mode below.
+ "default_models": [
+ "openai/gpt-oss-120b",
+ "deepseek-ai/DeepSeek-V3",
+ "meta-llama/Llama-3.3-70B-Instruct",
+ "Qwen/Qwen2.5-72B-Instruct",
+ ],
+ "supports_streaming": True,
+ "supports_vision": True,
+ "supports_tool_calling": True,
+ "auth_header": "Authorization",
+ "auth_prefix": "Bearer ",
+ "notes": (
+ "HF token from huggingface.co/settings/tokens. Uses the "
+ "OpenAI-compatible router at /v1/chat/completions; /v1/models "
+ "returns the cross-provider chat catalog. See "
+ "https://huggingface.co/docs/inference-providers/index."
+ ),
+ # /v1/models works on the HF router and returns the full chat-model
+ # catalog (state.org/model[:policy] ids). Switch to remote so users
+ # see live availability — the picker has a search box, and
+ # loadModels() merges defaults so default_models entries remain
+ # visible if the remote call fails.
+ "model_list_mode": "remote",
+ # Scope the catalog to first-party org repos we trust as primary
+ # sources. The HF /v1/models response is otherwise hundreds of
+ # ids long (community fine-tunes, mirrors, fp8 variants, etc.).
+ "model_id_allowlist": re.compile(
+ r"^(openai|deepseek-ai|google|meta-llama|Qwen|moonshotai|"
+ r"mistralai|zai-org)/"
+ ),
+ # Cap the post-filter list. /v1/models has no server-side limit
+ # or popularity sort, so this is just "first N matches" — pair it
+ # with the default_models seed so the most useful flagship ids
+ # are always among the top regardless of the API's order.
+ "model_id_limit": 15,
+ },
+ "openrouter": {
+ "display_name": "OpenRouter",
+ "base_url": "https://openrouter.ai/api/v1",
+ # Curated list for Studio's picker (explicitly locked, not live /models).
+ "default_models": [
+ "openrouter/free",
+ "openai/gpt-4o",
+ "anthropic/claude-sonnet-4-5",
+ "google/gemini-2.5-flash",
+ "mistralai/mistral-large-2411",
+ "deepseek/deepseek-r1",
+ "mistralai/mistral-small-3.1-24b-instruct",
+ "perceptron/perceptron-mk1",
+ "inclusionai/ring-2.6-1t:free",
+ "google/gemini-3.1-flash-lite",
+ "baidu/cobuddy:free",
+ "openai/gpt-chat-latest",
+ "x-ai/grok-4.3",
+ "ibm-granite/granite-4.1-8b",
+ "openrouter/owl-alpha",
+ "poolside/laguna-xs.2:free",
+ "~google/gemini-pro-latest",
+ "~moonshotai/kimi-latest",
+ ],
+ "supports_streaming": True,
+ "supports_vision": True,
+ "supports_tool_calling": True,
+ "auth_header": "Authorization",
+ "auth_prefix": "Bearer ",
+ "extra_headers": {
+ "HTTP-Referer": "https://unsloth.ai",
+ "X-Title": "Unsloth Studio",
+ },
+ "notes": "Unified gateway to 300+ models across all major providers. HTTP-Referer and X-Title headers sent for attribution.",
+ "model_list_mode": "curated",
+ },
+}
+
+
+def get_provider_info(provider_type: str) -> dict[str, Any] | None:
+ """Return the registry entry for a provider type, or None if unknown."""
+ return PROVIDER_REGISTRY.get(provider_type)
+
+
+def get_base_url(provider_type: str) -> str | None:
+ """Return the default base URL for a provider type."""
+ info = PROVIDER_REGISTRY.get(provider_type)
+ return info["base_url"] if info else None
+
+
+def list_available_providers() -> list[dict[str, Any]]:
+ """Return all registered providers (for the /registry endpoint)."""
+ result = []
+ for provider_type, info in PROVIDER_REGISTRY.items():
+ result.append(
+ {
+ "provider_type": provider_type,
+ "display_name": info["display_name"],
+ "base_url": info["base_url"],
+ "default_models": info["default_models"],
+ "supports_streaming": info["supports_streaming"],
+ "supports_vision": info.get("supports_vision", False),
+ "supports_tool_calling": info.get("supports_tool_calling", False),
+ "model_list_mode": info.get("model_list_mode", "remote"),
+ }
+ )
+ return result
diff --git a/studio/backend/main.py b/studio/backend/main.py
index 4955e988e6..c1c9ed1d90 100644
--- a/studio/backend/main.py
+++ b/studio/backend/main.py
@@ -120,6 +120,7 @@ from routes import (
inference_router,
inference_studio_router,
models_router,
+ providers_router,
training_history_router,
training_router,
)
@@ -222,6 +223,11 @@ async def lifespan(app: FastAPI):
threading.Thread(target = _precache, daemon = True).start()
+ # Initialize RSA key pair for API key encryption (external providers)
+ from core.inference.key_exchange import init_key_pair
+
+ init_key_pair()
+
if storage.ensure_default_admin():
bootstrap_pw = storage.get_bootstrap_password()
app.state.bootstrap_password = bootstrap_pw
@@ -474,6 +480,7 @@ app.include_router(inference_studio_router, prefix = "/api/inference", tags = ["
# so external tools (Open WebUI, SillyTavern, etc.) can use the
# standard /v1/chat/completions path.
app.include_router(inference_router, prefix = "/v1", tags = ["openai-compat"])
+app.include_router(providers_router, prefix = "/api/providers", tags = ["providers"])
app.include_router(datasets_router, prefix = "/api/datasets", tags = ["datasets"])
app.include_router(data_recipe_router, prefix = "/api/data-recipe", tags = ["data-recipe"])
app.include_router(export_router, prefix = "/api/export", tags = ["export"])
diff --git a/studio/backend/models/inference.py b/studio/backend/models/inference.py
index 746ac8bbc2..013328f6c6 100644
--- a/studio/backend/models/inference.py
+++ b/studio/backend/models/inference.py
@@ -531,9 +531,11 @@ class ChatCompletionRequest(BaseModel):
None,
description = "[x-unsloth] Enable/disable thinking/reasoning mode for supported models",
)
- reasoning_effort: Optional[Literal["low", "medium", "high"]] = Field(
+ reasoning_effort: Optional[
+ Literal["none", "minimal", "low", "medium", "high", "max", "xhigh"]
+ ] = Field(
None,
- description = "[x-unsloth] Reasoning effort level ('low'|'medium'|'high') for Harmony-style reasoning models (e.g. gpt-oss). Overrides enable_thinking when the active model uses reasoning_effort style.",
+ description = "[x-unsloth] Reasoning effort level ('none'|'minimal'|'low'|'medium'|'high'|'max'|'xhigh'). OpenAI `/v1/responses` accepts model-dependent subsets; Anthropic adaptive thinking uses `max` as the top tier on Claude 4.6 Opus/Sonnet (inbound `xhigh` is mapped to `max`) and `xhigh` on Claude 4.7 Opus; local Harmony/gpt-oss templates support low|medium|high.",
)
preserve_thinking: Optional[bool] = Field(
None,
@@ -570,6 +572,28 @@ class ChatCompletionRequest(BaseModel):
description = "[x-unsloth] Per-request cancellation token. Frontend sends a fresh UUID per run so /inference/cancel matches one specific generation.",
)
+ # ── External provider routing (x-unsloth extensions) ──────────
+ provider_id: Optional[str] = Field(
+ None,
+ description = "[x-unsloth] Saved provider config ID. If set with encrypted_api_key, routes to external LLM.",
+ )
+ provider_type: Optional[str] = Field(
+ None,
+ description = "[x-unsloth] Provider type (e.g. 'openai', 'mistral'). Used if provider_id is not set.",
+ )
+ external_model: Optional[str] = Field(
+ None,
+ description = "[x-unsloth] Model ID at the external provider.",
+ )
+ encrypted_api_key: Optional[str] = Field(
+ None,
+ description = "[x-unsloth] RSA-encrypted, base64-encoded API key for the external provider.",
+ )
+ provider_base_url: Optional[str] = Field(
+ None,
+ description = "[x-unsloth] Override base URL for the external provider.",
+ )
+
# ── Streaming response chunks ────────────────────────────────────
diff --git a/studio/backend/models/providers.py b/studio/backend/models/providers.py
new file mode 100644
index 0000000000..5678e69f62
--- /dev/null
+++ b/studio/backend/models/providers.py
@@ -0,0 +1,128 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""
+Pydantic schemas for the external LLM providers API.
+"""
+
+from typing import Literal, Optional
+
+from pydantic import BaseModel, Field
+
+
+# ── Registry (static provider info) ───────────────────────────────
+
+
+class ProviderRegistryEntry(BaseModel):
+ """A supported provider type with its default configuration."""
+
+ provider_type: str = Field(
+ ..., description = "Provider identifier (e.g. 'openai', 'mistral')"
+ )
+ display_name: str = Field(..., description = "Human-readable provider name")
+ base_url: str = Field(..., description = "Default API base URL")
+ default_models: list[str] = Field(
+ default_factory = list, description = "Well-known model IDs for this provider"
+ )
+ supports_streaming: bool = Field(
+ True, description = "Whether this provider supports SSE streaming"
+ )
+ supports_vision: bool = Field(
+ False, description = "Whether this provider supports vision/image input"
+ )
+ supports_tool_calling: bool = Field(
+ False, description = "Whether this provider supports tool/function calling"
+ )
+ model_list_mode: Literal["remote", "curated"] = Field(
+ "remote",
+ description = "remote = fetch /models; curated = huge catalogs — UI uses defaults + manual IDs only",
+ )
+
+
+# ── Provider config CRUD ──────────────────────────────────────────
+
+
+class ProviderCreate(BaseModel):
+ """Request to create a saved provider configuration."""
+
+ provider_type: str = Field(..., description = "Provider type from the registry")
+ display_name: str = Field(
+ ..., description = "User-chosen label (e.g. 'My OpenAI Key')"
+ )
+ base_url: Optional[str] = Field(
+ None,
+ description = "Custom base URL (overrides registry default). Omit to use the default.",
+ )
+
+
+class ProviderUpdate(BaseModel):
+ """Request to update a saved provider configuration."""
+
+ display_name: Optional[str] = Field(None, description = "New display name")
+ base_url: Optional[str] = Field(None, description = "New base URL")
+ is_enabled: Optional[bool] = Field(
+ None, description = "Enable or disable this provider"
+ )
+
+
+class ProviderResponse(BaseModel):
+ """A saved provider configuration (returned by list/get endpoints)."""
+
+ id: str = Field(..., description = "Unique provider config ID")
+ provider_type: str = Field(..., description = "Provider type (e.g. 'openai')")
+ display_name: str = Field(..., description = "User-chosen label")
+ base_url: str = Field(..., description = "API base URL")
+ is_enabled: bool = Field(True, description = "Whether this provider is enabled")
+ created_at: str = Field(..., description = "ISO 8601 creation timestamp")
+ updated_at: str = Field(..., description = "ISO 8601 last-update timestamp")
+
+
+# ── Model listing ─────────────────────────────────────────────────
+
+
+class ProviderModelInfo(BaseModel):
+ """A model available from an external provider."""
+
+ id: str = Field(..., description = "Model ID as expected by the provider API")
+ display_name: str = Field("", description = "Human-readable model name")
+ context_length: Optional[int] = Field(
+ None, description = "Maximum context length in tokens"
+ )
+ owned_by: Optional[str] = Field(None, description = "Model owner/organization")
+
+
+class ProviderModelsRequest(BaseModel):
+ """Request to list models from an external provider."""
+
+ provider_type: str = Field(..., description = "Provider type from the registry")
+ encrypted_api_key: str = Field(
+ ..., description = "RSA-encrypted, base64-encoded API key"
+ )
+ base_url: Optional[str] = Field(
+ None, description = "Custom base URL (overrides registry default)"
+ )
+
+
+# ── Connection testing ────────────────────────────────────────────
+
+
+class ProviderTestRequest(BaseModel):
+ """Request to test connectivity to an external provider."""
+
+ provider_type: str = Field(..., description = "Provider type from the registry")
+ encrypted_api_key: str = Field(
+ ..., description = "RSA-encrypted, base64-encoded API key"
+ )
+ base_url: Optional[str] = Field(
+ None, description = "Custom base URL (overrides registry default)"
+ )
+
+
+class ProviderTestResult(BaseModel):
+ """Result of a provider connectivity test."""
+
+ success: bool = Field(..., description = "Whether the test succeeded")
+ message: str = Field(..., description = "Human-readable result message")
+ models_count: Optional[int] = Field(
+ None, description = "Number of models found (if test succeeded)"
+ )
diff --git a/studio/backend/requirements/studio.txt b/studio/backend/requirements/studio.txt
index 1bf751c368..96f8816b57 100644
--- a/studio/backend/requirements/studio.txt
+++ b/studio/backend/requirements/studio.txt
@@ -16,3 +16,5 @@ huggingface-hub==0.36.2
structlog>=24.1.0
diceware
ddgs
+cryptography>=42.0.0
+httpx>=0.27.0
diff --git a/studio/backend/routes/__init__.py b/studio/backend/routes/__init__.py
index cf4586281b..62320b9084 100644
--- a/studio/backend/routes/__init__.py
+++ b/studio/backend/routes/__init__.py
@@ -14,6 +14,7 @@ from routes.auth import router as auth_router
from routes.data_recipe import router as data_recipe_router
from routes.export import router as export_router
from routes.training_history import router as training_history_router
+from routes.providers import router as providers_router
__all__ = [
"training_router",
@@ -25,4 +26,5 @@ __all__ = [
"data_recipe_router",
"export_router",
"training_history_router",
+ "providers_router",
]
diff --git a/studio/backend/routes/inference.py b/studio/backend/routes/inference.py
index 7102e12bf8..59928be3cf 100644
--- a/studio/backend/routes/inference.py
+++ b/studio/backend/routes/inference.py
@@ -204,6 +204,11 @@ from core.inference.anthropic_compat import (
)
from auth.authentication import get_current_subject
+from core.inference.key_exchange import decrypt_api_key
+from core.inference.providers import get_provider_info, get_base_url
+from core.inference.external_provider import ExternalProviderClient
+from storage import providers_db
+
import io
import wave
import base64
@@ -1464,6 +1469,161 @@ def _extract_content_parts(
return system_prompt, chat_messages, first_image_b64
+# ── External provider proxy ──────────────────────────────────────
+
+
+def _build_external_messages(
+ messages: list,
+ supports_vision: bool,
+) -> list[dict]:
+ """
+ Convert ChatMessage list to OpenAI-compatible dicts for external providers.
+
+ - Vision providers: preserve multimodal content arrays (image_url parts intact).
+ - Non-vision providers: flatten to text-only (images silently dropped).
+ """
+ result = []
+ for msg in messages:
+ if isinstance(msg.content, str):
+ # Skip assistant messages with empty content (some providers reject them)
+ if msg.role == "assistant" and not msg.content.strip():
+ continue
+ result.append({"role": msg.role, "content": msg.content})
+ elif isinstance(msg.content, list):
+ if supports_vision:
+ parts = []
+ for part in msg.content:
+ if part.type == "text":
+ parts.append({"type": "text", "text": part.text})
+ elif part.type == "image_url":
+ parts.append(
+ {
+ "type": "image_url",
+ "image_url": {"url": part.image_url.url},
+ }
+ )
+ result.append({"role": msg.role, "content": parts})
+ else:
+ # Non-vision provider — strip images, keep text only
+ text = "\n".join(p.text for p in msg.content if p.type == "text")
+ result.append({"role": msg.role, "content": text})
+ return result
+
+
+async def _proxy_to_external_provider(
+ payload: ChatCompletionRequest,
+ request: Request,
+) -> StreamingResponse:
+ """
+ Proxy a chat completion request to an external LLM provider.
+
+ Resolves provider config (from DB or registry), decrypts the API key,
+ and streams the response back in OpenAI SSE format.
+ """
+ # Resolve provider type and base URL
+ provider_type = payload.provider_type
+ base_url = payload.provider_base_url
+
+ if payload.provider_id:
+ config = providers_db.get_provider(payload.provider_id)
+ if config is None:
+ raise HTTPException(
+ status_code = 404,
+ detail = f"Provider config not found: {payload.provider_id}",
+ )
+ if not config["is_enabled"]:
+ raise HTTPException(
+ status_code = 400,
+ detail = f"Provider '{config['display_name']}' is disabled.",
+ )
+ provider_type = provider_type or config["provider_type"]
+ base_url = base_url or config["base_url"]
+
+ if not provider_type:
+ raise HTTPException(
+ status_code = 400,
+ detail = "Either provider_id or provider_type is required for external provider routing.",
+ )
+
+ # Fall back to registry default base URL
+ if not base_url:
+ base_url = get_base_url(provider_type)
+ if not base_url:
+ raise HTTPException(
+ status_code = 400,
+ detail = f"Unknown provider type: {provider_type}",
+ )
+
+ # Decrypt the API key
+ try:
+ api_key = decrypt_api_key(payload.encrypted_api_key)
+ except Exception as exc:
+ logger.warning("external_provider.decrypt_failed", error = str(exc))
+ raise HTTPException(
+ status_code = 400,
+ detail = "Failed to decrypt API key. The server key may have changed — try refreshing the page.",
+ )
+
+ model = payload.external_model or payload.model
+ if model == "default":
+ raise HTTPException(
+ status_code = 400,
+ detail = "external_model is required when using an external provider.",
+ )
+
+ # Build messages preserving multimodal content for vision-capable providers
+ from core.inference.providers import get_provider_info as _get_provider_info
+
+ _pinfo = _get_provider_info(provider_type) or {}
+ _supports_vision = _pinfo.get("supports_vision", False)
+ chat_messages = _build_external_messages(payload.messages, _supports_vision)
+
+ client = ExternalProviderClient(
+ provider_type = provider_type,
+ base_url = base_url,
+ api_key = api_key,
+ )
+
+ async def _stream():
+ gen = client.stream_chat_completion(
+ messages = chat_messages,
+ model = model,
+ temperature = payload.temperature,
+ top_p = payload.top_p,
+ max_tokens = payload.max_tokens,
+ presence_penalty = payload.presence_penalty,
+ top_k = payload.top_k,
+ enable_thinking = payload.enable_thinking,
+ reasoning_effort = payload.reasoning_effort,
+ stream = payload.stream,
+ )
+ try:
+ sent_done = False
+ async for line in gen:
+ yield f"{line}\n\n"
+ if "[DONE]" in line:
+ sent_done = True
+ if not sent_done:
+ yield "data: [DONE]\n\n"
+ except Exception as exc:
+ logger.error("external_provider.stream_error", error = str(exc))
+ finally:
+ try:
+ await gen.aclose()
+ except RuntimeError:
+ pass # suppress httpcore asyncgen cleanup error (Python 3.13 + httpcore 1.0.x)
+ await client.close()
+
+ return StreamingResponse(
+ _stream(),
+ media_type = "text/event-stream",
+ headers = {
+ "Cache-Control": "no-cache",
+ "X-Accel-Buffering": "no",
+ },
+ )
+
+
@router.post("/chat/completions")
async def openai_chat_completions(
payload: ChatCompletionRequest,
@@ -1483,6 +1643,10 @@ async def openai_chat_completions(
- GGUF models → llama-server via LlamaCppBackend
- Other models → Unsloth/transformers via InferenceBackend
"""
+ # ── External provider routing ────────────────────────────────
+ if payload.encrypted_api_key and (payload.provider_id or payload.provider_type):
+ return await _proxy_to_external_provider(payload, request)
+
llama_backend = get_llama_cpp_backend()
using_gguf = llama_backend.is_loaded
diff --git a/studio/backend/routes/providers.py b/studio/backend/routes/providers.py
new file mode 100644
index 0000000000..e21985d60b
--- /dev/null
+++ b/studio/backend/routes/providers.py
@@ -0,0 +1,338 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""
+API routes for external LLM provider management.
+
+Provides endpoints for:
+ - Discovering available provider types (registry)
+ - CRUD for saved provider configurations (no API keys stored)
+ - Fetching the RSA public key for API key encryption
+ - Testing provider connectivity
+ - Listing models from a provider
+"""
+
+import uuid
+import structlog
+from fastapi import APIRouter, Depends, HTTPException
+
+from auth.authentication import get_current_subject
+from core.inference.key_exchange import (
+ decrypt_api_key,
+ get_public_key_fingerprint,
+ get_public_key_pem,
+)
+from core.inference.providers import (
+ get_base_url,
+ get_provider_info,
+ list_available_providers,
+)
+from core.inference.external_provider import ExternalProviderClient
+from models.providers import (
+ ProviderCreate,
+ ProviderModelsRequest,
+ ProviderModelInfo,
+ ProviderResponse,
+ ProviderRegistryEntry,
+ ProviderTestRequest,
+ ProviderTestResult,
+ ProviderUpdate,
+)
+from storage import providers_db
+
+logger = structlog.get_logger(__name__)
+
+router = APIRouter()
+
+
+# ── Public key for API key encryption ─────────────────────────────
+
+
+@router.get("/public-key")
+async def get_public_key(
+ current_subject: str = Depends(get_current_subject),
+):
+ """Return the RSA public key PEM for client-side API key encryption.
+
+ The ``fingerprint`` field is a short SHA256 of the PEM and is meant
+ purely for diagnostics — a mismatch between what the frontend
+ captured at encrypt time and what the server reports here is a
+ clear signal that the keypair rotated mid-flight (e.g. the server
+ re-ran ``init_key_pair`` for any reason).
+ """
+ return {
+ "public_key": get_public_key_pem(),
+ "fingerprint": get_public_key_fingerprint(),
+ }
+
+
+# ── Provider registry (static) ───────────────────────────────────
+
+
+@router.get("/registry", response_model = list[ProviderRegistryEntry])
+async def list_registry(
+ current_subject: str = Depends(get_current_subject),
+):
+ """List all supported provider types with their default configurations."""
+ return list_available_providers()
+
+
+# ── Provider config CRUD ──────────────────────────────────────────
+
+
+@router.get("/", response_model = list[ProviderResponse])
+async def list_provider_configs(
+ current_subject: str = Depends(get_current_subject),
+):
+ """List all saved provider configurations."""
+ rows = providers_db.list_providers()
+ return [
+ ProviderResponse(
+ id = row["id"],
+ provider_type = row["provider_type"],
+ display_name = row["display_name"],
+ base_url = row["base_url"],
+ is_enabled = bool(row["is_enabled"]),
+ created_at = row["created_at"],
+ updated_at = row["updated_at"],
+ )
+ for row in rows
+ ]
+
+
+@router.post("/", response_model = ProviderResponse, status_code = 201)
+async def create_provider_config(
+ payload: ProviderCreate,
+ current_subject: str = Depends(get_current_subject),
+):
+ """Create a new saved provider configuration (no API key stored)."""
+ info = get_provider_info(payload.provider_type)
+ if info is None:
+ raise HTTPException(
+ status_code = 400,
+ detail = f"Unknown provider type: {payload.provider_type}. "
+ f"Use GET /api/providers/registry to see available types.",
+ )
+
+ provider_id = uuid.uuid4().hex[:16]
+ base_url = payload.base_url or info["base_url"]
+
+ providers_db.create_provider(
+ id = provider_id,
+ provider_type = payload.provider_type,
+ display_name = payload.display_name,
+ base_url = base_url,
+ )
+
+ row = providers_db.get_provider(provider_id)
+ return ProviderResponse(
+ id = row["id"],
+ provider_type = row["provider_type"],
+ display_name = row["display_name"],
+ base_url = row["base_url"],
+ is_enabled = bool(row["is_enabled"]),
+ created_at = row["created_at"],
+ updated_at = row["updated_at"],
+ )
+
+
+@router.put("/{provider_id}", response_model = ProviderResponse)
+async def update_provider_config(
+ provider_id: str,
+ payload: ProviderUpdate,
+ current_subject: str = Depends(get_current_subject),
+):
+ """Update a saved provider configuration."""
+ existing = providers_db.get_provider(provider_id)
+ if not existing:
+ raise HTTPException(status_code = 404, detail = "Provider not found")
+
+ updated = providers_db.update_provider(
+ id = provider_id,
+ display_name = payload.display_name,
+ base_url = payload.base_url,
+ is_enabled = payload.is_enabled,
+ )
+ if not updated:
+ raise HTTPException(status_code = 400, detail = "No fields to update")
+
+ row = providers_db.get_provider(provider_id)
+ return ProviderResponse(
+ id = row["id"],
+ provider_type = row["provider_type"],
+ display_name = row["display_name"],
+ base_url = row["base_url"],
+ is_enabled = bool(row["is_enabled"]),
+ created_at = row["created_at"],
+ updated_at = row["updated_at"],
+ )
+
+
+@router.delete("/{provider_id}", status_code = 204)
+async def delete_provider_config(
+ provider_id: str,
+ current_subject: str = Depends(get_current_subject),
+):
+ """Delete a saved provider configuration."""
+ deleted = providers_db.delete_provider(provider_id)
+ if not deleted:
+ raise HTTPException(status_code = 404, detail = "Provider not found")
+
+
+# ── Test connectivity ─────────────────────────────────────────────
+
+
+@router.post("/test", response_model = ProviderTestResult)
+async def test_provider(
+ payload: ProviderTestRequest,
+ current_subject: str = Depends(get_current_subject),
+):
+ """
+ Test connectivity to an external provider.
+
+ Makes a lightweight GET /models call to verify the API key works.
+ The encrypted_api_key is decrypted server-side and never stored.
+ """
+ info = get_provider_info(payload.provider_type)
+ if info is None:
+ raise HTTPException(
+ status_code = 400,
+ detail = f"Unknown provider type: {payload.provider_type}",
+ )
+
+ try:
+ api_key = decrypt_api_key(payload.encrypted_api_key)
+ except Exception as exc:
+ logger.warning("Failed to decrypt API key (%s): %s", type(exc).__name__, exc)
+ raise HTTPException(
+ status_code = 400,
+ detail = "Failed to decrypt API key. The public key may have changed — try refreshing the page.",
+ )
+
+ base_url = payload.base_url or info["base_url"]
+ client = ExternalProviderClient(
+ provider_type = payload.provider_type,
+ base_url = base_url,
+ api_key = api_key,
+ timeout = 15.0,
+ )
+
+ try:
+ if info.get("model_list_mode") == "curated":
+ await client.verify_models_endpoint_lightweight()
+ return ProviderTestResult(
+ success = True,
+ message = (
+ "Connected successfully. Full model list is not fetched for this provider — "
+ "use suggestions and manual model IDs in the dialog."
+ ),
+ models_count = None,
+ )
+ models = await client.list_models()
+ return ProviderTestResult(
+ success = True,
+ message = f"Connected successfully. Found {len(models)} model(s).",
+ models_count = len(models),
+ )
+ except Exception as exc:
+ logger.warning("Provider test failed for %s: %s", payload.provider_type, exc)
+ return ProviderTestResult(
+ success = False,
+ message = f"Connection failed: {exc}",
+ models_count = None,
+ )
+ finally:
+ await client.close()
+
+
+# ── List models from provider ─────────────────────────────────────
+
+
+@router.post("/models", response_model = list[ProviderModelInfo])
+async def list_provider_models(
+ payload: ProviderModelsRequest,
+ current_subject: str = Depends(get_current_subject),
+):
+ """
+ List models available from an external provider.
+
+ The encrypted_api_key is decrypted server-side and never stored.
+ """
+ info = get_provider_info(payload.provider_type)
+ if info is None:
+ raise HTTPException(
+ status_code = 400,
+ detail = f"Unknown provider type: {payload.provider_type}",
+ )
+
+ try:
+ api_key = decrypt_api_key(payload.encrypted_api_key)
+ except Exception as exc:
+ logger.warning("Failed to decrypt API key (%s): %s", type(exc).__name__, exc)
+ raise HTTPException(
+ status_code = 400,
+ detail = "Failed to decrypt API key. The public key may have changed — try refreshing the page.",
+ )
+
+ if info.get("model_list_mode") == "curated":
+ return [
+ ProviderModelInfo(
+ id = m,
+ display_name = m,
+ context_length = None,
+ owned_by = None,
+ )
+ for m in info.get("default_models", [])
+ ]
+
+ base_url = payload.base_url or info["base_url"]
+ client = ExternalProviderClient(
+ provider_type = payload.provider_type,
+ base_url = base_url,
+ api_key = api_key,
+ timeout = 15.0,
+ )
+
+ try:
+ models = await client.list_models()
+ allow_prefixes = info.get("model_id_allow_prefixes")
+ if allow_prefixes is not None:
+ prefix_tuple = tuple(str(p) for p in allow_prefixes if str(p))
+ if prefix_tuple:
+ models = [m for m in models if m.get("id", "").startswith(prefix_tuple)]
+ allowlist = info.get("model_id_allowlist")
+ if allowlist is not None:
+ models = [m for m in models if allowlist.match(m.get("id", ""))]
+ deny_exact = info.get("model_id_deny_exact")
+ if deny_exact is not None:
+ deny_ids = {str(m) for m in deny_exact if str(m)}
+ if deny_ids:
+ models = [m for m in models if m.get("id", "") not in deny_ids]
+ denylist = info.get("model_id_denylist")
+ if denylist is not None:
+ models = [m for m in models if not denylist.search(m.get("id", ""))]
+ # Apply an optional cap after filtering so registry entries with a
+ # large remote catalog (e.g. HF Inference Providers) can stay
+ # picker-sized. No popularity sort happens server-side, so this is
+ # "first N matches" — pair with default_models for any must-have
+ # flagship ids.
+ limit = info.get("model_id_limit")
+ if isinstance(limit, int) and limit > 0:
+ models = models[:limit]
+ return [
+ ProviderModelInfo(
+ id = m.get("id", ""),
+ display_name = m.get("id", ""),
+ context_length = m.get("context_length") or m.get("context_window"),
+ owned_by = m.get("owned_by"),
+ )
+ for m in models
+ ]
+ except Exception as exc:
+ logger.error("Failed to list models from %s: %s", payload.provider_type, exc)
+ raise HTTPException(
+ status_code = 502,
+ detail = f"Failed to list models from {payload.provider_type}: {exc}",
+ )
+ finally:
+ await client.close()
diff --git a/studio/backend/storage/providers_db.py b/studio/backend/storage/providers_db.py
new file mode 100644
index 0000000000..ca47fcbd80
--- /dev/null
+++ b/studio/backend/storage/providers_db.py
@@ -0,0 +1,153 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""
+SQLite storage for external LLM provider configurations.
+
+Follows the same pattern as studio_db.py — module-level functions,
+raw sqlite3, WAL mode, per-function connections.
+
+NOTE: API keys are NOT stored here. They live only in the browser
+(localStorage) and are sent encrypted per-request.
+"""
+
+import logging
+import sqlite3
+import threading
+from datetime import datetime, timezone
+from typing import Optional
+
+logger = logging.getLogger(__name__)
+
+from utils.paths import studio_db_path, ensure_dir
+
+_schema_lock = threading.Lock()
+_schema_ready = False
+
+
+def _ensure_schema(conn: sqlite3.Connection) -> None:
+ """Create the llm_providers table if it doesn't exist. Called once per process."""
+ conn.execute("PRAGMA journal_mode=WAL")
+ conn.execute(
+ """
+ CREATE TABLE IF NOT EXISTS llm_providers (
+ id TEXT NOT NULL PRIMARY KEY,
+ provider_type TEXT NOT NULL,
+ display_name TEXT NOT NULL,
+ base_url TEXT NOT NULL,
+ is_enabled INTEGER NOT NULL DEFAULT 1,
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+ )
+ """
+ )
+
+
+def get_connection() -> sqlite3.Connection:
+ """Open studio.db with WAL mode, create table once per process."""
+ global _schema_ready
+ db_path = studio_db_path()
+ ensure_dir(db_path.parent)
+ conn = sqlite3.connect(str(db_path))
+ conn.row_factory = sqlite3.Row
+ if not _schema_ready:
+ with _schema_lock:
+ if not _schema_ready:
+ try:
+ _ensure_schema(conn)
+ _schema_ready = True
+ except Exception:
+ conn.close()
+ raise
+ return conn
+
+
+def create_provider(
+ id: str,
+ provider_type: str,
+ display_name: str,
+ base_url: str,
+) -> None:
+ """Insert a new provider configuration."""
+ now = datetime.now(timezone.utc).isoformat()
+ conn = get_connection()
+ try:
+ conn.execute(
+ """
+ INSERT INTO llm_providers (id, provider_type, display_name, base_url, created_at, updated_at)
+ VALUES (?, ?, ?, ?, ?, ?)
+ """,
+ (id, provider_type, display_name, base_url, now, now),
+ )
+ conn.commit()
+ finally:
+ conn.close()
+
+
+def update_provider(
+ id: str,
+ display_name: Optional[str] = None,
+ base_url: Optional[str] = None,
+ is_enabled: Optional[bool] = None,
+) -> bool:
+ """Update fields on an existing provider. Returns True if a row was updated."""
+ updates = []
+ params = []
+ if display_name is not None:
+ updates.append("display_name = ?")
+ params.append(display_name)
+ if base_url is not None:
+ updates.append("base_url = ?")
+ params.append(base_url)
+ if is_enabled is not None:
+ updates.append("is_enabled = ?")
+ params.append(1 if is_enabled else 0)
+ if not updates:
+ return False
+ updates.append("updated_at = ?")
+ params.append(datetime.now(timezone.utc).isoformat())
+ params.append(id)
+
+ conn = get_connection()
+ try:
+ cursor = conn.execute(
+ f"UPDATE llm_providers SET {', '.join(updates)} WHERE id = ?",
+ params,
+ )
+ conn.commit()
+ return cursor.rowcount > 0
+ finally:
+ conn.close()
+
+
+def delete_provider(id: str) -> bool:
+ """Delete a provider by ID. Returns True if a row was deleted."""
+ conn = get_connection()
+ try:
+ cursor = conn.execute("DELETE FROM llm_providers WHERE id = ?", (id,))
+ conn.commit()
+ return cursor.rowcount > 0
+ finally:
+ conn.close()
+
+
+def get_provider(id: str) -> Optional[dict]:
+ """Fetch a single provider by ID."""
+ conn = get_connection()
+ try:
+ row = conn.execute("SELECT * FROM llm_providers WHERE id = ?", (id,)).fetchone()
+ return dict(row) if row else None
+ finally:
+ conn.close()
+
+
+def list_providers() -> list[dict]:
+ """List all provider configurations, ordered by creation time."""
+ conn = get_connection()
+ try:
+ rows = conn.execute(
+ "SELECT * FROM llm_providers ORDER BY created_at"
+ ).fetchall()
+ return [dict(row) for row in rows]
+ finally:
+ conn.close()
diff --git a/studio/backend/tests/test_anthropic_thinking_translation.py b/studio/backend/tests/test_anthropic_thinking_translation.py
new file mode 100644
index 0000000000..14f261ae6b
--- /dev/null
+++ b/studio/backend/tests/test_anthropic_thinking_translation.py
@@ -0,0 +1,404 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""
+Unit tests for the Anthropic extended-thinking translation in
+external_provider.
+
+Covers:
+- Adaptive-mode request body nests effort under
+ ``output_config: {effort: ""}`` per the Messages API
+ reference (a top-level ``effort`` field 400s with
+ "effort: Extra inputs are not permitted").
+- Streaming SSE: ``content_block_delta`` with
+ ``delta.type == "thinking_delta"`` is translated into inline
+ ``...`` chat-completion chunks so the frontend's
+ reasoning-panel pipeline lifts it correctly.
+- The ```` tag closes when the first ``text_delta`` arrives,
+ on ``content_block_stop``, on ``message_delta``, or on
+ ``message_stop``.
+- Thinking is paired with ``temperature=1`` and no ``top_p`` /
+ ``top_k`` on the wire (Anthropic extended-thinking contract).
+"""
+
+import asyncio
+import json
+
+import httpx
+
+from core.inference import external_provider as ep_mod
+from core.inference.external_provider import ExternalProviderClient
+
+
+def _drive(coro):
+ return asyncio.new_event_loop().run_until_complete(coro)
+
+
+async def _collect(agen):
+ out = []
+ async for line in agen:
+ out.append(line)
+ return out
+
+
+def _mock_http_client(monkeypatch, handler):
+ transport = httpx.MockTransport(handler)
+ monkeypatch.setattr(ep_mod, "_http_client", httpx.AsyncClient(transport = transport))
+
+
+def _make_client() -> ExternalProviderClient:
+ return ExternalProviderClient(
+ provider_type = "anthropic",
+ base_url = "https://api.anthropic.com/v1",
+ api_key = "sk-ant-test",
+ )
+
+
+def _anthropic_sse(events: list[dict]) -> bytes:
+ """Serialize a list of Messages-API event dicts as an SSE byte stream."""
+ chunks: list[str] = []
+ for event in events:
+ chunks.append(f"event: {event['type']}")
+ chunks.append(f"data: {json.dumps(event)}")
+ chunks.append("")
+ return ("\n".join(chunks) + "\n").encode("utf-8")
+
+
+def _payloads_from_lines(lines: list[str]) -> list:
+ out = []
+ for line in lines:
+ if not line.startswith("data:"):
+ continue
+ raw = line[len("data:") :].strip()
+ if not raw:
+ continue
+ if raw == "[DONE]":
+ out.append("[DONE]")
+ else:
+ out.append(json.loads(raw))
+ return out
+
+
+def test_adaptive_thinking_body_uses_output_config_effort_shape(monkeypatch):
+ captured: dict = {}
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ captured["body"] = json.loads(request.content.decode("utf-8"))
+ return httpx.Response(
+ 200,
+ content = _anthropic_sse([{"type": "message_stop"}]),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ async for _ in client._stream_anthropic(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "claude-opus-4-6",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = 4096,
+ top_k = None,
+ enable_thinking = None,
+ reasoning_effort = "medium",
+ ):
+ pass
+ await client.close()
+
+ _drive(run())
+
+ body = captured["body"]
+ # display=summarized is set explicitly so Opus 4.7 (which defaults to
+ # "omitted") still emits thinking_delta events for the reasoning panel.
+ assert body["thinking"] == {"type": "adaptive", "display": "summarized"}
+ # Documented shape: effort is nested under output_config.
+ # A top-level `effort` field produces a 400:
+ # "effort: Extra inputs are not permitted".
+ assert body["output_config"] == {"effort": "medium"}
+ assert "effort" not in body
+ # Extended-thinking contract: temperature=1, no top_p / top_k.
+ assert body["temperature"] == 1
+ assert "top_p" not in body
+ assert "top_k" not in body
+
+
+def test_adaptive_thinking_maps_xhigh_to_max_on_claude_4_6(monkeypatch):
+ captured: dict = {}
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ captured["body"] = json.loads(request.content.decode("utf-8"))
+ return httpx.Response(
+ 200,
+ content = _anthropic_sse([{"type": "message_stop"}]),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ async for _ in client._stream_anthropic(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "claude-sonnet-4-6",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = 4096,
+ top_k = None,
+ enable_thinking = None,
+ reasoning_effort = "xhigh",
+ ):
+ pass
+ await client.close()
+
+ _drive(run())
+
+ assert captured["body"]["output_config"] == {"effort": "max"}
+
+
+def test_adaptive_thinking_keeps_max_on_claude_4_6(monkeypatch):
+ captured: dict = {}
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ captured["body"] = json.loads(request.content.decode("utf-8"))
+ return httpx.Response(
+ 200,
+ content = _anthropic_sse([{"type": "message_stop"}]),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ async for _ in client._stream_anthropic(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "claude-opus-4-6",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = 4096,
+ top_k = None,
+ enable_thinking = None,
+ reasoning_effort = "max",
+ ):
+ pass
+ await client.close()
+
+ _drive(run())
+
+ assert captured["body"]["output_config"] == {"effort": "max"}
+
+
+def test_adaptive_thinking_keeps_xhigh_on_claude_4_7(monkeypatch):
+ captured: dict = {}
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ captured["body"] = json.loads(request.content.decode("utf-8"))
+ return httpx.Response(
+ 200,
+ content = _anthropic_sse([{"type": "message_stop"}]),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ async for _ in client._stream_anthropic(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "claude-opus-4-7",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = 4096,
+ top_k = None,
+ enable_thinking = None,
+ reasoning_effort = "xhigh",
+ ):
+ pass
+ await client.close()
+
+ _drive(run())
+
+ body = captured["body"]
+ assert body["output_config"] == {"effort": "xhigh"}
+ assert "effort" not in body
+
+
+def test_manual_thinking_body_uses_budget_tokens_on_4_5(monkeypatch):
+ captured: dict = {}
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ captured["body"] = json.loads(request.content.decode("utf-8"))
+ return httpx.Response(
+ 200,
+ content = _anthropic_sse([{"type": "message_stop"}]),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ async for _ in client._stream_anthropic(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "claude-opus-4-5",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = 1024,
+ top_k = None,
+ enable_thinking = None,
+ reasoning_effort = "high",
+ ):
+ pass
+ await client.close()
+
+ _drive(run())
+
+ body = captured["body"]
+ assert body["thinking"] == {"type": "enabled", "budget_tokens": 4096}
+ # max_tokens must be strictly greater than budget_tokens; we shipped 1024
+ # and budget is 4096, so the wrapper should bump max_tokens.
+ assert body["max_tokens"] > body["thinking"]["budget_tokens"]
+ # Manual-thinking path does not use output_config / effort — those are
+ # the adaptive-mode controls (Claude 4.6 / 4.7).
+ assert "effort" not in body
+ assert "output_config" not in body
+
+
+def test_thinking_delta_wrapped_in_think_tags(monkeypatch):
+ def handler(request: httpx.Request) -> httpx.Response:
+ events = [
+ {
+ "type": "content_block_start",
+ "index": 0,
+ "content_block": {"type": "thinking", "thinking": "", "signature": ""},
+ },
+ {
+ "type": "content_block_delta",
+ "index": 0,
+ "delta": {"type": "thinking_delta", "thinking": "First "},
+ },
+ {
+ "type": "content_block_delta",
+ "index": 0,
+ "delta": {"type": "thinking_delta", "thinking": "I plan."},
+ },
+ {
+ "type": "content_block_delta",
+ "index": 0,
+ "delta": {"type": "signature_delta", "signature": "abc123"},
+ },
+ {"type": "content_block_stop", "index": 0},
+ {
+ "type": "content_block_start",
+ "index": 1,
+ "content_block": {"type": "text", "text": ""},
+ },
+ {
+ "type": "content_block_delta",
+ "index": 1,
+ "delta": {"type": "text_delta", "text": "Answer."},
+ },
+ {"type": "content_block_stop", "index": 1},
+ {"type": "message_delta", "delta": {"stop_reason": "end_turn"}},
+ {"type": "message_stop"},
+ ]
+ return httpx.Response(
+ 200,
+ content = _anthropic_sse(events),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ lines = await _collect(
+ client._stream_anthropic(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "claude-opus-4-6",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = 4096,
+ top_k = None,
+ enable_thinking = True,
+ reasoning_effort = None,
+ )
+ )
+ await client.close()
+ return lines
+
+ lines = _drive(run())
+ payloads = _payloads_from_lines(lines)
+
+ combined = "".join(
+ p["choices"][0]["delta"].get("content", "")
+ for p in payloads
+ if isinstance(p, dict) and p["choices"][0]["delta"]
+ )
+
+ # Reasoning text should be wrapped in ..., followed by the
+ # answer text, and the stream should terminate with [DONE].
+ assert "First I plan." in combined
+ assert combined.endswith("Answer.")
+ # signature_delta is intentionally dropped — no leaked signature text.
+ assert "abc123" not in combined
+ assert "[DONE]" in payloads
+
+
+def test_thinking_only_turn_closes_tag_without_text_delta(monkeypatch):
+ """display=omitted on Claude 4.7 emits a signature_delta and no text.
+
+ The open is still triggered by the (synthetic) thinking_delta;
+ we want content_block_stop to close it cleanly so the tag never leaks
+ into the next chunk."""
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ events = [
+ {
+ "type": "content_block_start",
+ "index": 0,
+ "content_block": {"type": "thinking", "thinking": "", "signature": ""},
+ },
+ {
+ "type": "content_block_delta",
+ "index": 0,
+ "delta": {"type": "thinking_delta", "thinking": "internal"},
+ },
+ {"type": "content_block_stop", "index": 0},
+ {"type": "message_delta", "delta": {"stop_reason": "end_turn"}},
+ {"type": "message_stop"},
+ ]
+ return httpx.Response(
+ 200,
+ content = _anthropic_sse(events),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ lines = await _collect(
+ client._stream_anthropic(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "claude-opus-4-7",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = 4096,
+ top_k = None,
+ enable_thinking = True,
+ reasoning_effort = None,
+ )
+ )
+ await client.close()
+ return lines
+
+ payloads = _payloads_from_lines(_drive(run()))
+ combined = "".join(
+ p["choices"][0]["delta"].get("content", "")
+ for p in payloads
+ if isinstance(p, dict) and p["choices"][0]["delta"]
+ )
+ assert combined == "internal"
diff --git a/studio/backend/tests/test_desktop_auth.py b/studio/backend/tests/test_desktop_auth.py
index a7201ac433..913c3cc355 100644
--- a/studio/backend/tests/test_desktop_auth.py
+++ b/studio/backend/tests/test_desktop_auth.py
@@ -437,6 +437,7 @@ def test_health_response_reports_desktop_capability_fields(monkeypatch):
inference_router = APIRouter(),
inference_studio_router = APIRouter(),
models_router = APIRouter(),
+ providers_router = APIRouter(),
training_history_router = APIRouter(),
training_router = APIRouter(),
)
diff --git a/studio/backend/tests/test_openai_responses_translation.py b/studio/backend/tests/test_openai_responses_translation.py
new file mode 100644
index 0000000000..4ad6a19ea9
--- /dev/null
+++ b/studio/backend/tests/test_openai_responses_translation.py
@@ -0,0 +1,432 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""
+Unit tests for the OpenAI `/v1/responses` translation in external_provider.
+
+Covers:
+- Request body shape: system messages collapse into `instructions`, user/
+ assistant messages go into `input`, sampling knobs Responses does not
+ support (presence_penalty, top_k) are not forwarded.
+- SSE translation: `response.output_text.delta` events become OpenAI Chat
+ Completions chunks, `response.completed` emits a `finish_reason: stop`
+ chunk, the stream terminates with `data: [DONE]`.
+- Image parts in user content are rewritten from Chat Completions
+ `{type: image_url, image_url: {url}}` into Responses
+ `{type: input_image, image_url: }`.
+"""
+
+import asyncio
+import json
+
+import httpx
+
+from core.inference import external_provider as ep_mod
+from core.inference.external_provider import ExternalProviderClient
+
+
+def _drive(coro):
+ return asyncio.new_event_loop().run_until_complete(coro)
+
+
+async def _collect(agen):
+ out = []
+ async for line in agen:
+ out.append(line)
+ return out
+
+
+def _mock_http_client(monkeypatch, handler):
+ transport = httpx.MockTransport(handler)
+ monkeypatch.setattr(ep_mod, "_http_client", httpx.AsyncClient(transport = transport))
+
+
+def _make_client() -> ExternalProviderClient:
+ return ExternalProviderClient(
+ provider_type = "openai",
+ base_url = "https://api.openai.com/v1",
+ api_key = "sk-test",
+ )
+
+
+def _responses_sse(events: list[dict]) -> bytes:
+ """Serialize a list of Responses-API event dicts as an SSE byte stream."""
+ chunks: list[str] = []
+ for event in events:
+ chunks.append(f"event: {event['type']}")
+ chunks.append(f"data: {json.dumps(event)}")
+ chunks.append("")
+ chunks.append("data: [DONE]")
+ chunks.append("")
+ return ("\n".join(chunks) + "\n").encode("utf-8")
+
+
+def test_responses_request_body_uses_input_and_instructions(monkeypatch):
+ captured: dict = {}
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ captured["url"] = str(request.url)
+ captured["body"] = json.loads(request.content.decode("utf-8"))
+ return httpx.Response(
+ 200,
+ content = _responses_sse([{"type": "response.completed", "response": {}}]),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ async for _ in client._stream_openai_responses(
+ messages = [
+ {"role": "system", "content": "You are concise."},
+ {"role": "user", "content": "Hi"},
+ ],
+ model = "gpt-5.5",
+ temperature = 0.5,
+ top_p = 0.9,
+ max_tokens = 512,
+ enable_thinking = None,
+ reasoning_effort = None,
+ ):
+ pass
+ await client.close()
+
+ _drive(run())
+
+ assert captured["url"] == "https://api.openai.com/v1/responses"
+ body = captured["body"]
+ assert body["model"] == "gpt-5.5"
+ assert body["instructions"] == "You are concise."
+ assert body["input"] == [{"role": "user", "content": "Hi"}]
+ assert body["max_output_tokens"] == 512
+ assert body["stream"] is True
+ # Responses API on reasoning-class models (gpt-5.x / o3 / gpt-4.5 — the
+ # only OpenAI ids the registry allowlist exposes) rejects these as
+ # `Unsupported parameter`. Make sure we never silently forward them.
+ assert "temperature" not in body
+ assert "top_p" not in body
+ assert "presence_penalty" not in body
+ assert "frequency_penalty" not in body
+ assert "top_k" not in body
+ assert "messages" not in body
+
+
+def test_responses_translates_image_parts(monkeypatch):
+ captured: dict = {}
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ captured["body"] = json.loads(request.content.decode("utf-8"))
+ return httpx.Response(
+ 200,
+ content = _responses_sse([{"type": "response.completed", "response": {}}]),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ async for _ in client._stream_openai_responses(
+ messages = [
+ {
+ "role": "user",
+ "content": [
+ {"type": "text", "text": "What is this?"},
+ {
+ "type": "image_url",
+ "image_url": {"url": "data:image/png;base64,AAA"},
+ },
+ ],
+ }
+ ],
+ model = "gpt-5.5",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = None,
+ enable_thinking = None,
+ reasoning_effort = None,
+ ):
+ pass
+ await client.close()
+
+ _drive(run())
+
+ parts = captured["body"]["input"][0]["content"]
+ assert parts[0] == {"type": "input_text", "text": "What is this?"}
+ assert parts[1] == {
+ "type": "input_image",
+ "image_url": "data:image/png;base64,AAA",
+ }
+ # No max_output_tokens key when caller passes max_tokens=None.
+ assert "max_output_tokens" not in captured["body"]
+
+
+def test_responses_sse_translates_to_chat_completions_chunks(monkeypatch):
+ def handler(request: httpx.Request) -> httpx.Response:
+ events = [
+ {"type": "response.created"},
+ {"type": "response.output_text.delta", "delta": "Hello"},
+ {"type": "response.output_text.delta", "delta": ", world"},
+ {"type": "response.completed", "response": {}},
+ ]
+ return httpx.Response(
+ 200,
+ content = _responses_sse(events),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ lines = await _collect(
+ client._stream_openai_responses(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "gpt-5.5",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = None,
+ enable_thinking = None,
+ reasoning_effort = None,
+ )
+ )
+ await client.close()
+ return lines
+
+ lines = _drive(run())
+
+ # Drop empty / non-data lines for assertion clarity.
+ data_lines = [line for line in lines if line.startswith("data:")]
+ payloads = []
+ for line in data_lines:
+ raw = line[len("data:") :].strip()
+ if raw == "[DONE]":
+ payloads.append("[DONE]")
+ else:
+ payloads.append(json.loads(raw))
+
+ # Two text deltas, one terminal chunk, then [DONE].
+ assert payloads[0]["choices"][0]["delta"]["content"] == "Hello"
+ assert payloads[0]["choices"][0]["finish_reason"] is None
+ assert payloads[1]["choices"][0]["delta"]["content"] == ", world"
+ assert payloads[2]["choices"][0]["delta"] == {}
+ assert payloads[2]["choices"][0]["finish_reason"] == "stop"
+ assert payloads[-1] == "[DONE]"
+
+
+def test_responses_response_incomplete_maps_to_length_finish_reason(monkeypatch):
+ def handler(request: httpx.Request) -> httpx.Response:
+ events = [
+ {"type": "response.output_text.delta", "delta": "partial"},
+ {"type": "response.incomplete", "response": {}},
+ ]
+ return httpx.Response(
+ 200,
+ content = _responses_sse(events),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ lines = await _collect(
+ client._stream_openai_responses(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "gpt-5.5",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = 4,
+ enable_thinking = None,
+ reasoning_effort = None,
+ )
+ )
+ await client.close()
+ return lines
+
+ lines = _drive(run())
+ finish_reasons = [
+ json.loads(line[len("data:") :].strip())["choices"][0]["finish_reason"]
+ for line in lines
+ if line.startswith("data:")
+ and line[len("data:") :].strip() not in ("", "[DONE]")
+ ]
+ assert "length" in finish_reasons
+
+
+def test_responses_reasoning_effort_included_when_requested(monkeypatch):
+ captured: dict = {}
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ captured["body"] = json.loads(request.content.decode("utf-8"))
+ return httpx.Response(
+ 200,
+ content = _responses_sse([{"type": "response.completed", "response": {}}]),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ async for _ in client._stream_openai_responses(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "gpt-5.5",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = None,
+ enable_thinking = None,
+ reasoning_effort = "high",
+ ):
+ pass
+ await client.close()
+
+ _drive(run())
+ assert captured["body"]["reasoning"] == {"effort": "high", "summary": "auto"}
+
+
+def test_responses_reasoning_effort_none_omits_summary(monkeypatch):
+ captured: dict = {}
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ captured["body"] = json.loads(request.content.decode("utf-8"))
+ return httpx.Response(
+ 200,
+ content = _responses_sse([{"type": "response.completed", "response": {}}]),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ async for _ in client._stream_openai_responses(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "gpt-5.5",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = None,
+ enable_thinking = None,
+ reasoning_effort = "none",
+ ):
+ pass
+ await client.close()
+
+ _drive(run())
+ assert captured["body"]["reasoning"] == {"effort": "none"}
+
+
+def test_responses_reasoning_effort_xhigh_passthrough(monkeypatch):
+ captured: dict = {}
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ captured["body"] = json.loads(request.content.decode("utf-8"))
+ return httpx.Response(
+ 200,
+ content = _responses_sse([{"type": "response.completed", "response": {}}]),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ async for _ in client._stream_openai_responses(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "gpt-5.5",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = None,
+ enable_thinking = None,
+ reasoning_effort = "xhigh",
+ ):
+ pass
+ await client.close()
+
+ _drive(run())
+ assert captured["body"]["reasoning"] == {"effort": "xhigh", "summary": "auto"}
+
+
+def test_responses_enable_thinking_false_maps_to_reasoning_none(monkeypatch):
+ captured: dict = {}
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ captured["body"] = json.loads(request.content.decode("utf-8"))
+ return httpx.Response(
+ 200,
+ content = _responses_sse([{"type": "response.completed", "response": {}}]),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ async for _ in client._stream_openai_responses(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "gpt-5.5",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = None,
+ enable_thinking = False,
+ reasoning_effort = None,
+ ):
+ pass
+ await client.close()
+
+ _drive(run())
+ assert captured["body"]["reasoning"] == {"effort": "none"}
+
+
+def test_responses_reasoning_summary_wrapped_in_think_tags(monkeypatch):
+ def handler(request: httpx.Request) -> httpx.Response:
+ events = [
+ {
+ "type": "response.output_item.done",
+ "item": {
+ "type": "reasoning",
+ "summary": [{"type": "summary_text", "text": "plan"}],
+ },
+ },
+ {"type": "response.output_text.delta", "delta": "answer"},
+ {"type": "response.completed", "response": {}},
+ ]
+ return httpx.Response(
+ 200,
+ content = _responses_sse(events),
+ headers = {"content-type": "text/event-stream"},
+ )
+
+ _mock_http_client(monkeypatch, handler)
+
+ async def run():
+ client = _make_client()
+ lines = await _collect(
+ client._stream_openai_responses(
+ messages = [{"role": "user", "content": "hi"}],
+ model = "gpt-5.5",
+ temperature = 0.7,
+ top_p = 0.95,
+ max_tokens = None,
+ enable_thinking = None,
+ reasoning_effort = None,
+ )
+ )
+ await client.close()
+ return lines
+
+ lines = _drive(run())
+ data_lines = [
+ line[len("data:") :].strip()
+ for line in lines
+ if line.startswith("data:")
+ and line[len("data:") :].strip() not in ("", "[DONE]")
+ ]
+ payloads = [json.loads(raw) for raw in data_lines]
+ combined = "".join(
+ payload["choices"][0]["delta"].get("content", "")
+ for payload in payloads
+ if payload["choices"][0]["delta"]
+ )
+ assert "plananswer" in combined
diff --git a/studio/backend/tests/test_providers_api.py b/studio/backend/tests/test_providers_api.py
new file mode 100644
index 0000000000..0e668944f4
--- /dev/null
+++ b/studio/backend/tests/test_providers_api.py
@@ -0,0 +1,609 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""
+Integration tests for the external providers API.
+
+Requires a running Unsloth Studio server. Configure via environment variables:
+
+ export STUDIO_TEST_URL="http://localhost:8888" # default
+ export STUDIO_TEST_USER="unsloth" # default
+ export STUDIO_TEST_PASSWORD="..." # required — see .bootstrap_password
+
+ # Provider API keys — any left unset will have their tests automatically skipped
+ export OPENAI_API_KEY="sk-..."
+ export MISTRAL_API_KEY="..."
+ export GOOGLE_API_KEY="..."
+ export TOGETHER_API_KEY="..."
+ export FIREWORKS_API_KEY="..."
+ export PERPLEXITY_API_KEY="..."
+
+Run:
+ cd studio/backend
+ pytest tests/test_providers_api.py -v -s
+"""
+
+import base64
+import json
+import os
+
+import pytest
+import requests
+from cryptography.hazmat.primitives import hashes, serialization
+from cryptography.hazmat.primitives.asymmetric import padding
+
+# ── Configuration ─────────────────────────────────────────────────
+
+BASE_URL = os.getenv("STUDIO_TEST_URL", "http://localhost:8000")
+USERNAME = os.getenv("STUDIO_TEST_USER", "unsloth")
+PASSWORD = os.getenv("STUDIO_TEST_PASSWORD", "")
+
+# These tests require a live Studio server reachable at BASE_URL with a known
+# bootstrap password. Skip the whole module when that environment is missing
+# (e.g. on CI runners) so pytest discovery does not error out.
+pytestmark = pytest.mark.skipif(
+ not PASSWORD,
+ reason = "Integration test requires a running Studio server; set STUDIO_TEST_PASSWORD to enable.",
+)
+
+# Map provider_type → (env var name, model to use for inference test)
+_PROVIDER_CONFIGS: dict[str, tuple[str, str]] = {
+ "openai": ("OPENAI_API_KEY", "gpt-4o-mini"),
+ "mistral": ("MISTRAL_API_KEY", "mistral-small-2506"),
+ "gemini": ("GEMINI_API_KEY", "gemini-3-flash-preview"),
+ "openrouter": ("OPENROUTER_API_KEY", "openai/gpt-4o-mini"),
+ "anthropic": ("ANTHROPIC_API_KEY", "claude-haiku-4-5"),
+ "deepseek": ("DEEPSEEK_API_KEY", "deepseek-chat"),
+ "huggingface": ("HUGGINGFACE_API_KEY", "meta-llama/Llama-3.3-70B-Instruct"),
+ "kimi": ("MOONSHOT_API_KEY", "moonshot-v1-8k"),
+ "qwen": ("DASHSCOPE_API_KEY", "qwen-turbo"),
+}
+
+PROVIDER_KEYS: dict[str, str] = {
+ ptype: os.getenv(env_var, "") for ptype, (env_var, _) in _PROVIDER_CONFIGS.items()
+}
+
+EXPECTED_PROVIDER_TYPES = set(_PROVIDER_CONFIGS.keys())
+
+# ── Helpers ────────────────────────────────────────────────────────
+
+
+def _url(path: str) -> str:
+ return f"{BASE_URL}/{path.lstrip('/')}"
+
+
+def _parse_sse_stream(response: requests.Response) -> tuple[str, bool]:
+ """
+ Read a streaming SSE response and return (assembled_text, saw_done).
+
+ Each chunk is a JSON object with choices[0].delta.content.
+ The stream ends with `data: [DONE]`.
+ """
+ reply_parts: list[str] = []
+ saw_done = False
+
+ for raw_line in response.iter_lines():
+ if isinstance(raw_line, bytes):
+ raw_line = raw_line.decode("utf-8")
+ if not raw_line.startswith("data:"):
+ continue
+ data = raw_line[len("data:") :].strip()
+ if data == "[DONE]":
+ saw_done = True
+ break
+ try:
+ chunk = json.loads(data)
+ # Handle both error payloads and normal chunks
+ if "error" in chunk:
+ raise RuntimeError(f"Provider error in stream: {chunk['error']}")
+ delta = chunk.get("choices", [{}])[0].get("delta", {})
+ content = delta.get("content") or ""
+ if content:
+ reply_parts.append(content)
+ except (json.JSONDecodeError, IndexError, KeyError):
+ pass # skip malformed lines
+
+ return "".join(reply_parts), saw_done
+
+
+# ── Session-scoped fixtures ────────────────────────────────────────
+
+
+@pytest.fixture(scope = "session")
+def auth_headers() -> dict[str, str]:
+ """
+ Log in once per session and return auth headers.
+
+ On a fresh Studio install the bootstrap password triggers a forced password
+ change (must_change_password=True). Any subsequent API call using that token
+ returns 403 "Password change required". This fixture detects that state,
+ automatically completes the change-password flow, and re-logs in so all other
+ tests get a fully usable token.
+
+ The new password used during auto-change is:
+ STUDIO_TEST_NEW_PASSWORD (env var, optional)
+ or PASSWORD + "-test" (derived default)
+
+ On the second run, set STUDIO_TEST_PASSWORD to the new password.
+ """
+ assert PASSWORD, (
+ "STUDIO_TEST_PASSWORD is not set.\n"
+ "Run: export STUDIO_TEST_PASSWORD=$(cat studio/backend/.bootstrap_password)"
+ )
+
+ resp = requests.post(
+ _url("/api/auth/login"),
+ json = {"username": USERNAME, "password": PASSWORD},
+ timeout = 10,
+ )
+ assert resp.status_code == 200, f"Login failed ({resp.status_code}): {resp.text}"
+ body = resp.json()
+ token = body["access_token"]
+ assert token, "access_token is empty"
+
+ if body.get("must_change_password"):
+ # Bootstrap token is restricted — only /api/auth/change-password works with it.
+ # Auto-complete the forced change so the rest of the tests get a full token.
+ new_password = os.getenv("STUDIO_TEST_NEW_PASSWORD") or f"{PASSWORD}-test"
+ change_resp = requests.post(
+ _url("/api/auth/change-password"),
+ headers = {"Authorization": f"Bearer {token}"},
+ json = {"current_password": PASSWORD, "new_password": new_password},
+ timeout = 10,
+ )
+ assert (
+ change_resp.status_code == 200
+ ), f"Auto password-change failed ({change_resp.status_code}): {change_resp.text}"
+ token = change_resp.json()["access_token"]
+
+ return {"Authorization": f"Bearer {token}"}
+
+
+@pytest.fixture(scope = "session")
+def public_key_pem(auth_headers: dict[str, str]) -> str:
+ """Fetch RSA public key PEM once per session."""
+ resp = requests.get(
+ _url("/api/providers/public-key"),
+ headers = auth_headers,
+ timeout = 10,
+ )
+ assert resp.status_code == 200, f"Public key fetch failed: {resp.text}"
+ pem = resp.json().get("public_key", "")
+ assert pem.startswith("-----BEGIN PUBLIC KEY-----"), "Not a valid PEM public key"
+ return pem
+
+
+@pytest.fixture(scope = "session")
+def vision_image_data_url() -> str:
+ """
+ Download the sloth image once per session and return it as a base64 data URI.
+
+ Using a data URI instead of a remote URL ensures every provider receives
+ the image inline — Gemini's OpenAI-compatible layer does not fetch external
+ HTTP URLs, so raw image_url links silently produce empty replies for Gemini.
+ """
+ resp = requests.get(_VISION_IMAGE_URL, timeout = 30)
+ resp.raise_for_status()
+ content_type = resp.headers.get("Content-Type", "image/jpeg").split(";")[0].strip()
+ b64 = base64.b64encode(resp.content).decode("utf-8")
+ return f"data:{content_type};base64,{b64}"
+
+
+@pytest.fixture(scope = "session")
+def encrypt_key(public_key_pem: str):
+ """
+ Return a callable encrypt_key(plaintext: str) -> str (base64 RSA-OAEP ciphertext).
+ Uses the backend's RSA public key — mirrors what the frontend does.
+ """
+ # Decode PEM → load RSA public key
+ pem_bytes = public_key_pem.encode("utf-8")
+ rsa_pub = serialization.load_pem_public_key(pem_bytes)
+
+ def _encrypt(plaintext: str) -> str:
+ ciphertext = rsa_pub.encrypt(
+ plaintext.encode("utf-8"),
+ padding.OAEP(
+ mgf = padding.MGF1(algorithm = hashes.SHA256()),
+ algorithm = hashes.SHA256(),
+ label = None,
+ ),
+ )
+ return base64.b64encode(ciphertext).decode("utf-8")
+
+ return _encrypt
+
+
+# ── TestAuth ────────────────────────────────────────────────────────
+
+
+class TestAuth:
+ def test_login_returns_token(self):
+ """POST /api/auth/login returns a non-empty access_token."""
+ assert PASSWORD, "STUDIO_TEST_PASSWORD not set"
+ resp = requests.post(
+ _url("/api/auth/login"),
+ json = {"username": USERNAME, "password": PASSWORD},
+ timeout = 10,
+ )
+ assert (
+ resp.status_code == 200
+ ), f"Login failed ({resp.status_code}): {resp.text}"
+ body = resp.json()
+ assert body.get("access_token"), "access_token is missing or empty"
+ assert body.get("token_type") == "bearer"
+
+
+# ── TestPublicKey ────────────────────────────────────────────────────
+
+
+class TestPublicKey:
+ def test_public_key_is_valid_pem(
+ self, auth_headers: dict[str, str], public_key_pem: str
+ ):
+ """GET /api/providers/public-key returns an importable RSA PEM key."""
+ pem_bytes = public_key_pem.encode("utf-8")
+ key = serialization.load_pem_public_key(pem_bytes)
+ key_size = key.key_size # type: ignore[attr-defined]
+ assert key_size >= 2048, f"Key size too small: {key_size}"
+ print(f"\n RSA-{key_size} public key OK")
+
+
+# ── TestRegistry ────────────────────────────────────────────────────
+
+
+class TestRegistry:
+ def test_registry_returns_all_providers(self, auth_headers: dict[str, str]):
+ """GET /api/providers/registry returns all supported providers."""
+ resp = requests.get(
+ _url("/api/providers/registry"),
+ headers = auth_headers,
+ timeout = 10,
+ )
+ assert resp.status_code == 200, f"Registry failed: {resp.text}"
+ providers = resp.json()
+ assert (
+ len(providers) == 9
+ ), f"Expected 9 providers, got {len(providers)}: {providers}"
+ print(f"\n {'Provider':<12} {'Base URL'}")
+ print(f" {'-'*12} {'-'*45}")
+ for p in providers:
+ print(f" {p['provider_type']:<12} {p['base_url']}")
+
+ def test_registry_has_expected_types(self, auth_headers: dict[str, str]):
+ """All expected provider_type values are present in the registry."""
+ resp = requests.get(
+ _url("/api/providers/registry"),
+ headers = auth_headers,
+ timeout = 10,
+ )
+ assert resp.status_code == 200
+ returned_types = {p["provider_type"] for p in resp.json()}
+ missing = EXPECTED_PROVIDER_TYPES - returned_types
+ assert not missing, f"Missing provider types: {missing}"
+
+ def test_registry_entries_have_required_fields(self, auth_headers: dict[str, str]):
+ """Each registry entry has provider_type, display_name, base_url, default_models."""
+ resp = requests.get(
+ _url("/api/providers/registry"), headers = auth_headers, timeout = 10
+ )
+ assert resp.status_code == 200
+ for entry in resp.json():
+ for field in (
+ "provider_type",
+ "display_name",
+ "base_url",
+ "default_models",
+ "model_list_mode",
+ ):
+ assert field in entry, f"Missing field '{field}' in entry: {entry}"
+ assert entry["model_list_mode"] in ("remote", "curated")
+ assert isinstance(entry["default_models"], list)
+ assert len(entry["default_models"]) > 0
+
+
+# ── TestProviderCRUD ────────────────────────────────────────────────
+
+
+class TestProviderCRUD:
+ """
+ These tests run sequentially within the class and share state via class variables.
+ They create, read, update, and delete a single test provider config.
+ """
+
+ _created_id: str = ""
+
+ def test_create_provider(self, auth_headers: dict[str, str]):
+ """POST /api/providers/ creates a provider config and returns 201."""
+ resp = requests.post(
+ _url("/api/providers/"),
+ headers = auth_headers,
+ json = {"provider_type": "openai", "display_name": "Test OpenAI (pytest)"},
+ timeout = 10,
+ )
+ assert (
+ resp.status_code == 201
+ ), f"Create failed ({resp.status_code}): {resp.text}"
+ body = resp.json()
+ assert body.get("id"), "No id in response"
+ assert body["provider_type"] == "openai"
+ assert body["display_name"] == "Test OpenAI (pytest)"
+ assert body["is_enabled"] is True
+ TestProviderCRUD._created_id = body["id"]
+ print(f"\n created id={body['id']}")
+
+ def test_list_includes_created(self, auth_headers: dict[str, str]):
+ """GET /api/providers/ includes the newly created config."""
+ assert (
+ TestProviderCRUD._created_id
+ ), "No created_id (run test_create_provider first)"
+ resp = requests.get(_url("/api/providers/"), headers = auth_headers, timeout = 10)
+ assert resp.status_code == 200
+ ids = [p["id"] for p in resp.json()]
+ assert (
+ TestProviderCRUD._created_id in ids
+ ), f"Created id {TestProviderCRUD._created_id!r} not found in list: {ids}"
+ print(f"\n found id={TestProviderCRUD._created_id} in list of {len(ids)}")
+
+ def test_update_display_name(self, auth_headers: dict[str, str]):
+ """PUT /api/providers/{id} updates the display_name."""
+ assert TestProviderCRUD._created_id, "No created_id"
+ new_name = "Test OpenAI (pytest updated)"
+ resp = requests.put(
+ _url(f"/api/providers/{TestProviderCRUD._created_id}"),
+ headers = auth_headers,
+ json = {"display_name": new_name},
+ timeout = 10,
+ )
+ assert (
+ resp.status_code == 200
+ ), f"Update failed ({resp.status_code}): {resp.text}"
+ assert resp.json()["display_name"] == new_name
+ print(f"\n updated display_name to '{new_name}'")
+
+ def test_delete_provider(self, auth_headers: dict[str, str]):
+ """DELETE /api/providers/{id} removes the config (204) and it's gone from list."""
+ assert TestProviderCRUD._created_id, "No created_id"
+ resp = requests.delete(
+ _url(f"/api/providers/{TestProviderCRUD._created_id}"),
+ headers = auth_headers,
+ timeout = 10,
+ )
+ assert (
+ resp.status_code == 204
+ ), f"Delete failed ({resp.status_code}): {resp.text}"
+
+ # Confirm gone from list
+ list_resp = requests.get(
+ _url("/api/providers/"), headers = auth_headers, timeout = 10
+ )
+ ids = [p["id"] for p in list_resp.json()]
+ assert TestProviderCRUD._created_id not in ids, "Deleted provider still in list"
+ print(f"\n deleted id={TestProviderCRUD._created_id} confirmed gone")
+
+
+# ── TestProviderInference ────────────────────────────────────────────
+
+
+# Build parametrize list: (provider_type, model, api_key) for configured providers only
+_INFERENCE_PARAMS = [
+ pytest.param(
+ ptype,
+ model,
+ PROVIDER_KEYS.get(ptype, ""),
+ id = ptype,
+ marks = pytest.mark.skipif(
+ not PROVIDER_KEYS.get(ptype, ""),
+ reason = f"no {env_var} set",
+ ),
+ )
+ for ptype, (env_var, model) in _PROVIDER_CONFIGS.items()
+]
+
+
+class TestProviderInference:
+ """
+ Live inference tests — one parametrized set per provider.
+ Each test is automatically skipped when the provider's API key env var is not set.
+ """
+
+ @pytest.mark.parametrize("provider_type,model,api_key", _INFERENCE_PARAMS)
+ def test_connection(
+ self,
+ auth_headers: dict[str, str],
+ encrypt_key,
+ provider_type: str,
+ model: str,
+ api_key: str,
+ ):
+ """POST /api/providers/test → success: true."""
+ encrypted = encrypt_key(api_key)
+ resp = requests.post(
+ _url("/api/providers/test"),
+ headers = auth_headers,
+ json = {"provider_type": provider_type, "encrypted_api_key": encrypted},
+ timeout = 30,
+ )
+ assert (
+ resp.status_code == 200
+ ), f"Request failed ({resp.status_code}): {resp.text}"
+ body = resp.json()
+ assert (
+ body["success"] is True
+ ), f"Connection test failed for {provider_type}: {body.get('message')}"
+ print(f"\n [{provider_type}] connection OK — {body['message']}")
+
+ @pytest.mark.parametrize("provider_type,model,api_key", _INFERENCE_PARAMS)
+ def test_list_models(
+ self,
+ auth_headers: dict[str, str],
+ encrypt_key,
+ provider_type: str,
+ model: str,
+ api_key: str,
+ ):
+ """POST /api/providers/models → non-empty list, print first 3."""
+ encrypted = encrypt_key(api_key)
+ resp = requests.post(
+ _url("/api/providers/models"),
+ headers = auth_headers,
+ json = {"provider_type": provider_type, "encrypted_api_key": encrypted},
+ timeout = 30,
+ )
+ assert (
+ resp.status_code == 200
+ ), f"Request failed ({resp.status_code}): {resp.text}"
+ models = resp.json()
+ assert isinstance(models, list), f"Expected list, got {type(models)}"
+ assert len(models) > 0, f"No models returned for {provider_type}"
+ preview = [m["id"] for m in models[:3]]
+ print(f"\n [{provider_type}] {len(models)} models — first 3: {preview}")
+
+ @pytest.mark.parametrize("provider_type,model,api_key", _INFERENCE_PARAMS)
+ def test_chat_inference(
+ self,
+ auth_headers: dict[str, str],
+ encrypt_key,
+ provider_type: str,
+ model: str,
+ api_key: str,
+ ):
+ """POST /v1/chat/completions with provider fields → streamed reply."""
+ encrypted = encrypt_key(api_key)
+ payload = {
+ "messages": [{"role": "user", "content": "Say hello in one sentence."}],
+ "stream": True,
+ "temperature": 0.7,
+ "max_tokens": 64,
+ "provider_type": provider_type,
+ "external_model": model,
+ "encrypted_api_key": encrypted,
+ }
+ with requests.post(
+ _url("/v1/chat/completions"),
+ headers = {**auth_headers, "Content-Type": "application/json"},
+ json = payload,
+ stream = True,
+ timeout = 60,
+ ) as resp:
+ assert (
+ resp.status_code == 200
+ ), f"Chat completions failed ({resp.status_code}): {resp.text[:500]}"
+ reply, saw_done = _parse_sse_stream(resp)
+
+ assert reply.strip(), f"Empty reply from {provider_type}/{model}"
+ assert saw_done, f"Stream did not end with [DONE] for {provider_type}/{model}"
+ print(f'\n [{provider_type}/{model}] reply: "{reply.strip()}"')
+
+
+# ── TestVisionInference ─────────────────────────────────────────────
+
+# Sloth photo — used to test vision routing across providers
+_VISION_IMAGE_URL = (
+ "https://www.travelexcellence.com/images/where-to-see-sloths-in-costa-rica.jpg"
+)
+
+_VISION_PARAMS = [
+ pytest.param(
+ ptype,
+ model,
+ PROVIDER_KEYS.get(ptype, ""),
+ id = ptype,
+ marks = pytest.mark.skipif(
+ not PROVIDER_KEYS.get(ptype, ""),
+ reason = f"no key for {ptype}",
+ ),
+ )
+ for ptype, (_, model) in _PROVIDER_CONFIGS.items()
+ if ptype in {"openai", "mistral", "gemini", "anthropic", "openrouter"}
+]
+
+
+class TestVisionInference:
+ """
+ Send a 1×1 white PNG alongside a text question to each vision-capable provider.
+ Verifies that image content parts survive the proxy and the provider replies.
+ """
+
+ @pytest.mark.parametrize("provider_type,model,api_key", _VISION_PARAMS)
+ def test_vision_chat_inference(
+ self,
+ auth_headers: dict[str, str],
+ encrypt_key,
+ vision_image_data_url: str,
+ provider_type: str,
+ model: str,
+ api_key: str,
+ ):
+ """Image URL + text message → non-empty streamed reply."""
+ encrypted = encrypt_key(api_key)
+ payload = {
+ "messages": [
+ {
+ "role": "user",
+ "content": [
+ {
+ "type": "text",
+ "text": "Which animal is in this image? Reply in one word.",
+ },
+ {
+ "type": "image_url",
+ "image_url": {"url": vision_image_data_url},
+ },
+ ],
+ }
+ ],
+ "stream": True,
+ "max_tokens": 215,
+ "provider_type": provider_type,
+ "external_model": model,
+ "encrypted_api_key": encrypted,
+ }
+ with requests.post(
+ _url("/v1/chat/completions"),
+ headers = {**auth_headers, "Content-Type": "application/json"},
+ json = payload,
+ stream = True,
+ timeout = 60,
+ ) as resp:
+ assert (
+ resp.status_code == 200
+ ), f"Vision request failed ({resp.status_code}): {resp.text[:300]}"
+ reply, saw_done = _parse_sse_stream(resp)
+
+ assert reply.strip(), f"Empty reply from {provider_type}/{model}"
+ assert saw_done, f"Stream did not end with [DONE] for {provider_type}/{model}"
+ print(f"\n [{provider_type}/{model}] vision reply: {reply.strip()!r}")
+
+
+# ── TestLocalInferenceUnaffected ────────────────────────────────────
+
+
+class TestLocalInferenceUnaffected:
+ def test_chat_without_provider(self, auth_headers: dict[str, str]):
+ """
+ POST /v1/chat/completions without provider fields must not return 422 or 500.
+
+ 200 = a local model is loaded and responded.
+ 503 = no model loaded (expected in test environment — that's fine).
+ Any other 4xx/5xx (except 503) = regression in request handling.
+ """
+ resp = requests.post(
+ _url("/v1/chat/completions"),
+ headers = {**auth_headers, "Content-Type": "application/json"},
+ json = {
+ "messages": [{"role": "user", "content": "Hello"}],
+ "stream": False,
+ },
+ timeout = 15,
+ )
+ allowed = {200, 400, 503}
+ assert resp.status_code in allowed, (
+ f"Unexpected status {resp.status_code} for local inference path: {resp.text[:300]}\n"
+ f"This likely means the provider fields broke the base request schema."
+ )
+ status_label = (
+ "local model responded"
+ if resp.status_code == 200
+ else "no model loaded (expected)"
+ )
+ print(f"\n status={resp.status_code} ({status_label}) — local path unaffected")
diff --git a/studio/frontend/package-lock.json b/studio/frontend/package-lock.json
index 21d31d81e6..464f47c09c 100644
--- a/studio/frontend/package-lock.json
+++ b/studio/frontend/package-lock.json
@@ -58,6 +58,7 @@
"motion": "^12.34.0",
"next": "^16.1.6",
"next-themes": "^0.4.6",
+ "node-forge": "^1.4.0",
"radix-ui": "^1.4.3",
"react": "^19.2.4",
"react-day-picker": "^9.13.2",
@@ -80,6 +81,7 @@
"@eslint/js": "^9.39.1",
"@types/js-yaml": "^4.0.9",
"@types/node": "^25.5.2",
+ "@types/node-forge": "^1.3.14",
"@types/react": "^19.2.5",
"@types/react-dom": "^19.2.3",
"@vitejs/plugin-react": "^6.0.1",
@@ -7377,6 +7379,16 @@
"undici-types": "~7.19.0"
}
},
+ "node_modules/@types/node-forge": {
+ "version": "1.3.14",
+ "resolved": "https://registry.npmjs.org/@types/node-forge/-/node-forge-1.3.14.tgz",
+ "integrity": "sha512-mhVF2BnD4BO+jtOp7z1CdzaK4mbuK0LLQYAvdOLqHTavxFNq4zA1EmYkpnFjP8HOUzedfQkRnp0E2ulSAYSzAw==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "@types/node": "*"
+ }
+ },
"node_modules/@types/react": {
"version": "19.2.14",
"resolved": "https://registry.npmjs.org/@types/react/-/react-19.2.14.tgz",
@@ -13285,6 +13297,15 @@
"url": "https://opencollective.com/node-fetch"
}
},
+ "node_modules/node-forge": {
+ "version": "1.4.0",
+ "resolved": "https://registry.npmjs.org/node-forge/-/node-forge-1.4.0.tgz",
+ "integrity": "sha512-LarFH0+6VfriEhqMMcLX2F7SwSXeWwnEAJEsYm5QKWchiVYVvJyV9v7UDvUv+w5HO23ZpQTXDv/GxdDdMyOuoQ==",
+ "license": "(BSD-3-Clause OR GPL-2.0)",
+ "engines": {
+ "node": ">= 6.13.0"
+ }
+ },
"node_modules/node-releases": {
"version": "2.0.38",
"resolved": "https://registry.npmjs.org/node-releases/-/node-releases-2.0.38.tgz",
diff --git a/studio/frontend/package.json b/studio/frontend/package.json
index 3a02bb926a..c69b2fdf3e 100644
--- a/studio/frontend/package.json
+++ b/studio/frontend/package.json
@@ -66,6 +66,7 @@
"motion": "^12.34.0",
"next": "^16.1.6",
"next-themes": "^0.4.6",
+ "node-forge": "^1.4.0",
"radix-ui": "^1.4.3",
"react": "^19.2.4",
"react-day-picker": "^9.13.2",
@@ -92,6 +93,7 @@
"@biomejs/biome": "^1.9.4",
"@eslint/js": "^9.39.1",
"@types/js-yaml": "^4.0.9",
+ "@types/node-forge": "^1.3.14",
"@types/node": "^25.5.2",
"@types/react": "^19.2.5",
"@types/react-dom": "^19.2.3",
diff --git a/studio/frontend/public/provider-logos/anthropic.svg b/studio/frontend/public/provider-logos/anthropic.svg
new file mode 100644
index 0000000000..7545cc8f3e
--- /dev/null
+++ b/studio/frontend/public/provider-logos/anthropic.svg
@@ -0,0 +1,6 @@
+
\ No newline at end of file
diff --git a/studio/frontend/public/provider-logos/deepseek.svg b/studio/frontend/public/provider-logos/deepseek.svg
new file mode 100644
index 0000000000..d1ba06b942
--- /dev/null
+++ b/studio/frontend/public/provider-logos/deepseek.svg
@@ -0,0 +1,14 @@
+
+
\ No newline at end of file
diff --git a/studio/frontend/public/provider-logos/gemini.svg b/studio/frontend/public/provider-logos/gemini.svg
new file mode 100644
index 0000000000..9090dfb68e
--- /dev/null
+++ b/studio/frontend/public/provider-logos/gemini.svg
@@ -0,0 +1,72 @@
+
diff --git a/studio/frontend/public/provider-logos/huggingface.svg b/studio/frontend/public/provider-logos/huggingface.svg
new file mode 100644
index 0000000000..ab959d165f
--- /dev/null
+++ b/studio/frontend/public/provider-logos/huggingface.svg
@@ -0,0 +1,8 @@
+
diff --git a/studio/frontend/public/provider-logos/kimi.jpg b/studio/frontend/public/provider-logos/kimi.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..956a5b58b102ea01c21cd7a9e8697f42c3af73a1
GIT binary patch
literal 15554
zcmV;zJUzowNk&GxJOBV!MM6+kP&gp2JOBXj{{Wo{Vfk&`QV>4dk5JA*BDt}SJ*}t8{t2}zvF-X|MTpF;$O>utpCyaOZt)T-yJo&;rNsKXY>pBKk$F#AB7*SKkB;_
zfDhrH#Xpt*Nc}VZpZ{d`)BYM`2YKUWgmC{MeG6mclu}hfA62S
zAM(HUeXf63{+Io4?7#H?a37ifW&bb!6ZUiaXaBFjAK|~of2{v={^9-K`$zwOtp8^p
zx?aX#t3U5Pzthcx<{9~haS;prFp_Y5Rrhlz$yj6P3F3W&_RO7Xy03=u7Xa&|@#YF$
ze=zGJ9Z*Z|4Y*nDlhiSPiM@e<{d|7(Uvy5Sm}ELQm7U-Rh;ZH*y=#gzT>6SNh5H`{
zSLIGs^6vBk12TRG5-wS1c>(tT47`|Z(Vv_8J{G)8>q$E&FRZh4maVZPK+XXHuntM&
z(a6?8>kmtPRa7iqJp7YN#UT_-x{~X_l-sX%B2C*ATh}|c3CfJcshHplrP$@y|DCejr(;R$
z8{dOjZ|e$CTg<-^tYI*pwC@`vecyvc;W*8~T(1@f*#*r!lVcI!fR0svIuPBn$_D67Cfr%Db=Tdp21
z%7$In*8E|e+@0q-BwGh%@}X!RiP^W&;NR2_Yo?3;NAPbN`UF`$H|WDBmd%MkdTpS2%H52l
zAsgh7C@UJbH9^}ve@QyQV>wV>#{9gkw18vPSVxd2w}`81(Ui@CyaYwbHxo2lf}R
z0hI_&iWzXPQfduUJukcw`^u1}(H;r(Lo@=xA^Td@)d1j+i#}3-u2EL!Y6eM@CN|R4
z#R2D~{e{)ey5juC)FYB~<#eMXGJ9rF5bkHwQgT|E8lN0!TRqztxMlFugojmFBmS6g
zcryg
zp$&WlStW3oVhX4PG~s>+RH%NuBsw95-a$Tcqh
zK9kX>f@{A45Jrvuq%i}ftzoZ{ML%x+!L%Y+Ge>S`bwK+|Y}n
zRY-pw^9gn-|A}7Wyxm7-I-_#4sWH=(jTt!8akaqQzzEyrOBp|K6_v-Fw4}S1-Ecj1
zp}8MG?K&>X8`-}vCz&SoQ#xtU>pf|_-KgeAFSfeMy9Mq}ghaeHbdoJ^I5W-STm8VN
zGW|EB+-BsD{s8nj0piO7!CBivo7Iil*L(#6?!dITO?{(rwB;X>+zr1iXWweu(YS%d
zIdVFD6p$a}gV~`p9y0(C=fNUbG;`s?33gd^z2*>wtk$BUun2#b=KZtn#1dqlEX|KvKLa-Y7sNH=t7{
zq!9HJFy*175&`4Xc#u;?gI2$2!$S!k_jtbeK!6V4O0fQR5Y){e4bi;7N`im6R?erZ
z7{pF_uB?-k0E142U)}_n&mU6!)v8qXDm)nbGq%UW
z^5){De@H_kSIbxG21?L))-njfm7Wqk9qa_1VlQmI7?em|_{2_CZxmv_HD(d5dc+_Y
za+c&-b3XO$5K#akBuClqf*Nm5GMZN&(sik3Rk$9r4>{vmP}0qF$2x*TBS_s)baR$x
z#@29|MQ6W{7@d<)eTUMpebpPwAZ#8_K!
zr+5GW{`;f#Kj&{c{CvNE_aBRXB?lGn0qq&eWnUbK@070=^G!fLt2z%`a=oS*>mSLs
z%h=flwfemD>3Vto?Ybt;SO&_`eh$r^h}>@&h6UPZcr?NW<^1KEQ6Ag%%CpidDvfY$6%Tyi_A!DE<0d`=jYvWdxw|HQ!mfZ%
zg1vr&_2hy|<7K3RAwpay~S(
zt+A}J3Yp}9%(H%%(b7-amq@U|>DFh>_PjLb?|BWDko!f?V=1Sj_f?5h}NJ&@uMlM~I
zpY?AdD$!0Sp(P*R1te0^+K&<`VDYD#)s_;Me%=u4tS+(77;S69q;BQVNOFTNO;jC-
z#Ahi=fjJn7SB_#pz#fKg$wR?fuu
zXzg)wYH>Td?Swa#LlwV8%Y++`Do${43zjRI`e}+?3N(INziE?3`;ff+=p~|xB3Gyk
zyZiYw9@#9S5v0L*+e(sdg#Bs`YMjGW
zpTO^_$Mr`M{>xbazAlgzxhMG>-6+??~KEL-rX>
zk}~AT@}DAOm5~4|+&1SVzu$R%^UZKBXfkn?$9?qm4ZyKElw+YgUUq_)6X!v;7ag%M
zs~N3jas()Ski-1-wy%p%>fa<~U}H~N9za{%3`2(iW&8t9;2uW%J_$-z={EjFHCcUQ
zk(u~KdYG7of%C#LA=Gs}eUc`Bw?}SMBxQ#;L&`LFbX0BWp;Ho)1!0tF+-K?fPrH^$
zx^*&3lVL!yK@r2(cG^QfQf?eIMZlrf=wkLyTvo^aQY~egsj?-vR>3n75f&i-0Q)?L|B3bKp(eh7%p)jBpH4aUO`?hGP
z)^RVvaR;MaKMs!?Ugpl`Y4;mb*CqrC843100}>W
z7fvDvuuL<;8L%?xw+HilX+p3j+&sl7FhT?{g`ksbbHSS|i
z1pn-)-DrRRjB(}$svyld=rv&Y;hQ!)0!{$0ee+KMX15s*`iSA4DeQ5=EQ`F)+Bj2j
zdFm%;+OXTE+bJhN+~}HkD|a8{?~fykFaVH!YBcXw%1Q?6^n^zp#SGnDp2X!ct*P@_
z8H`yw$U*#<|1v_%A_4&_xv49%IfH44S|$#gK_b+?;{DuT^(p<42ka~#cwrP2(ac4m
zxAsvr+t1s*^`~HV`t)t`#|G6Qu96}@gUe7>In)pku*CxcT*==m41-?la^DNqe4k5}
zC$2>c>SwFQM?b)rBXTomWsTuQ)c|pd(sgN4T_zL+-T2G~;v{HofW-i#65ONbROld+
z<{2fm?chGzT!;qQ{71vNy1mM38el2f*Bb!FVzdV)W-!L?&e@=xhTHf+{&((68rIKa
z(tnKA*(Ac-E3ba5^o=i>!(vEF1K~8T-j&V2DjuHJQ9G8Q-GBlqf
zM?j+TLh*E0;N2?fAN?5242Jb)*+M1;dP@z#KWEy4N@
znjVv`aE{B4(PRAS5E^M44#&+!!oSujK>{SePRjI4Na`gBac~>)hq*~2sBWRnGSSLJRJrlnJ=|+(VvuJ~EG=Uc7%t{N`e}SXo8o`|hlq|Nt
zzCTAVsyCwqvw$!u+LHjW*V_qRcVS>z?Zc$NAz9RTA-(R1DIjbzdJ&#@O^US3K+VCz
zyMacCkWaaMunDP_u=L5vfF1FH_9;Lor=luj6G4b+l^3caWCQcxN882W{B+;ob+C*Qo8uEc=)nUZ2_`us;Ejb#RntTEw+(*cC
zIj#nbvZ=AWm(3S>m`AX>c7`T^ZmCR9MnG0f$pZVX>3%!YwF9dm(^A)5Dnwbi=7@nX
znHpZpnfog4%|IB?n)e6g)R{xirfg~{H5Q6|XYkjs$<3;O8tFY0WaGfPQDtG7pyX8sOm=HIJ`YK
zOm~Cn1w!8O7&b)JcB*?g*KeU*jLCrfs-`;~fd=C)eImRT52ph-31{5sN-uyvySy1v
z>*pY6IFI*bvglET++Nffa_*dyG;X|t1=fISv5lF4tmNu?zwhec0tZYMXTbIhXgIN8
z-dO5rFa%hPE}v+~GNZ(Tcl+o6ZAwajW0MN#w3%B(&1tGg+KE1aOVMN1u-e+4y^jUx
zF_gxs0w=G!f#zJ|>o0D#DKQ`p0#O**$8Z2Tne!zP{folc9eMb%jWSrxTM~|%c1}sI8C}8M
zDGI)hK<8linh1RFeHHMw_Z3LymBBChPqD~vn^ggpTp&*WKJ`e8(SZ~{5C-4(!dK
zS+MQw&TEgZKW)-^a|Ul;8+U=Bo94L%#*s76UCmG1$~=1p@aBiB1}_eBMf^aYf*4#^
zYU;VnsN=u|LzUVFVVbTlrSx-XRTf`#Nr*LG=JJ7_Uq~#5J^%msAiBV@&fb*5kWGJ?
zC7yhi!oedTTV#_O8m~f+0`DTT_o2lyj&$H1NM|wMniW2j$-)EP7cIiCq-zxB3o9d#2f)r8zvaHrskt9RC!-ZQJO%Zjque>)93V
z*JtTAmPbY{n%oPLDbd$3vO5QxW(8hYVZ0$>9IAMF_bLzh1KRVqSORgDJvB@a>{+8X
zH6e!fMQl{)Q*1@0s4zv+b7y#;WMtihoTntS%|D~9ygs4cBS0L&s0ag*FN|E1g?Z|p
z8`Hix&?o=?)qZ(kAoB9%I;WR~)_MfYVCQLJ6pmv@P9^^r9$x$1z}NV4=PuI9q<5+}
z8hyAs&;^$b0_OHDzrVn036tHw4)gyf`_nS(FV5~_>V6(mAK6p#cZjv5m;eC;b4na%
zm(2{`0xJ~-Wnes@Zwah5fsUy&Gk_)l6t%2;nxxgWu%0l2I|R^!FTukxHZ9u-FUAD_WRj#cbeqcgi1gsI)Olzt<>&qyssk%1-(NoyjA%vTtc_v9MQI0LfkC1H|(M=RJLKHD!rrsQiu
z{o$Vcqdl6~oLn~?I)mLOw|!^I0Jus;K5Z%qXQue(gRn{Jqo=|aRFoP>d8=#8`O>g#
z^7a#EZ*RweiHh;{;pFn?qnG9rHynf2*C>Uz@VM$cu=38yg`@o&fQV+AWO-eCmqN(q
zh;K7O3ga23en)spA&-`sY`#}#y%4MWcDuHjmkm;W!yqM3e@4Cgc=NC!MEq1jDV#Wi
zBFi0<af3NN$j;pDb+1MKSHSRZUC!yWN9T?~~En@ke
z!IL28z?)`8_<4y!LUxotzR6u6MRt-Gi^V(aQaSHobm6X=ciTyc(3r1^
z$L&9YjBjcT9u8JK_Yx=u1D-DiyfCUd{pS|
z9}TH5>NjC(byO0hZI_E{
z%-FVqzZYy)i}IXEr~Jds9CuN6h-djJPhf
zqBGiB5E;oXT1br<gY>B~=?JO~vZYB<`zv;}lAuVCOh
zk*!?%!gIii>I29bE`{yFBX6-5^%jh|rxQOJ;t
z6V(kBwXabBWniXi-0>?h_y^8x&&z4tc6z4L3&!@7u7<}D{IR`TitUz)x`3c2?>r!_#7T3=G(GlEsf@J|n
zfuQ#_z-w&X@*rlnkuuVWfgOXU8teLF%LAjm>&3BDWWHA{|7%IH((I01l{|i18T|cm
z7=X*`GE7G+8Cca>E^qI0mh=ZC*(S_zbAY;X_N)?X9Ed70&AXNvp&x{jw=bBJhJY>D
zgKc|D9ZH^4DKS@P%hV#Bs|EX8j6w(Fanj#tG$D#g8tN1KBXJ!PRj$6QAPw<~NjV>q
z{$x+o+Ul*2zd8UTLeLp5qdoUkC`p&pz&ZJmDMj1`uL*~KH>2*;oIX#|Ck^kXQ0FHC
zQ+J7s`WZTEf7kp%fB5JSHa^q4(H_0v?&p(o506B!>W0kuwhgjh+k#s;hgssArJELt
zU=Y3D(&E4qo@wC^_Db4NcF7K@M!UvwwyCB1TlXE8{!=Gp%yPqm%^g0<0X}qN0Yzz0
z0(j+ZY|u8Eq{9>CRNXt~o#Al=RD6L(Hk*3hOp^=p-fJsR1~z`3<)*Wc;yMh(UFsV*
z>Id>eN3oen_SS+(afO~tKRE~mje*B;!D=VjO2?&j^xj={7TWqRs8x*Fh_x1XjPPqn
zVyQd7R2E^M=p*D3q!2_bOe)$@%F}M@kT?xIsiwvM^j0YdzEi)*=SAJ(-{VVKPiL$4
zO1q-K=ATSiFs-0g(=V%?X!$SMk@ypCSjbN2{&bnC~onuKTPIEZRg;76;;>_`O
z=^CzRwFZ!{bLHo?_)Xw56v_N4;HUt4zb>#)Feb2id-D(Xx6F|kC`d#iR{7gbN2PM=
zB}Ic2-u3tVd29Ihx!njmnAxh?aSb?KFG&@cmNycCN|`}=OyaW_j_U>y=&xWJ1uUE#
zu#C|l!|0;cEYbv@U%etD*e-z`@04X+r{23*4EX2%pc(P7bCQpyisJpPlA5r>T$5J*
z?BSUW6@OLU;af|JnKQlSt9!am+9WOYND-|^nBV@Twvkq1AJDVrb2Y1L{^w*@kuZ6N
z-3Z|`fjZKv3{T&NK~)sWh}Thfo~pOTMh%;?3`{uyk&&ty!H*GjZ|Bx-{Pw&sXMX6h
z5K%-5ZK^y?vW8j7CdtSZ4eWx-ZG3O_KB_8}!(1vuh@spwZ4oGn+|oV(3nG6N{eNR7
z$JsL`m~ZUF*hR5O-(x!WKOY{ysN#+N2wN@M
zsK0O5S=BSn$K2#xcEKefZGB~)1zlB%3i|nutSw<VhDO!d)h63@OM%A5~mUa9As~DK!C`-
z3fB|q&U{RN0*onG07o9CL(NC7qTj>cPqU=DS^hp(K5jbY#PJh38-&P%X!UB1zj-)d
zgiU9>_pKsPRlymx
zy`Gd8?(x(4BS1IwJQ$dq>G5+Fe!1yWajIh1iW+G6@|_dTuD?L5k)9KIx5CT=ABzGv
z0{tB;z*WAjHUTZsD&EmY#3F4;%i?Rvcl
z`e#`VGht0qxx@;fmIp<PcTE+A*U#5q;qH1kvxeVKi8-Ju~2w+lC%|VX*sxhm@iy
z9SW-u^GG}PZNc}p+Kwb(j7@~7AE{3Zp)#acdM#NG&59$8ewG~VWa=PBEZrQJn^D*2
z;1-V6t0t#z(!3w_-4$wW5!1O1>uFTtH#<}c)nw&Q7yeJo_26P{b#L8d<-e26gv@~UeL=w|L
zSZ+|hgxEv>5z@hs{zpZ~nyg!+S9hLP4$)#qxl@FV-ZPP+Ja1xZE-ZJRnst4GNw#kc
zl~-u~L-y=8(AkBUtnIQRwB`l(0k&};$z>A1`MMw?hw1G1k?^z_9R-GMdMV|(0yH{K
z)u}TbDX@T)6U;gAZK^-%WUDi@y+Yz?^~)-xjY~5FNj68Qclz(e;|(RLo250i77>0@>Gz~5xZLXrqXMzSX;s)Q5cPJ;rU78=YPrZp
z?xq*qQf@ICX1vCh-wRC0)-yA@G$*8G^d&<{moi>fmNRnS9BXW)yRdvz=bAnsfgmXB8Kcj2AHtHOta=f&JU)_FHw!n~deZ2LY
z-d-1ucW53n*1K);e^tJwe$6cO?p(*!B^u?y6S{cF!=K-jZew$5u$Ihp48RQYT~YVn
z2ln}OhLVzHw)+8re31Xj7l2d$HMfw`Ll$6R(_)9d5sStVc>?Y&2AcVH5+5QiE+~U}
z8YiWN>QVD3sGRxDKtg~|aJVWOmelB9rkZY$L*pS@Z~dU}0M3&5W1RHErD2jyI5gl@
z+P(|i(SZbPv+QN~TE>!r=1!!1haf~%xte|C*{IzWW~;Y@FU
zwd|q`TWm4f0r6*LCxkI$>D1ZX*}C1@Y&jB39zrO;Soa7eFI8Gk9Bu9btGt*
zq-Iq5w0_EK9yYt)JEMSn(6;ds^DCY-E=)f_j1tq_DTyiI8&kNh3&TW-z&B?IrhX%w
zrf4y1I`8q5V*>P02*G_7PWLoXsb-X(Pn{=luaK>gy>mBHPB~mD^Z5gMCn5<&+@L0U
zl+L8$w}>O6C_4q5`IJ}7v%_fg(TV8(y-Iz|LR*Go#STVr`oZz`kvC6~JPjLbEm;ik
z-qrSI#>^dpr>1~t0yP5W$uCJvI&8;gG40M!KR``5aKW7!YAa-@Mfz1ZJUpjt^l!#>
z@Ywtv`?)3`VTwwZnlE}{|AIYm7)iO5h4ckcq#=sCO=6IVd-yDg4|hiL2RfLJu7zxE9~{0^i9wh
zVvIS7I8-lLC?bhWd?Fh=1@`4F`_dcL6_u;$K*W)&lRDQ3zGoT>H8p@>+p
zda@|GAZ|$GvisIqjeS`DV#3Y9legm#%W+oqLBCcT_bBsg+*oW+BdRNu@QCgn`yZ++
zH~`lrdEq+of}OkxIr|KR7BuaW<*t%ZoyYef%4#k&3e^v$C$?E-9M>)3v`PTe2k{9v
z(M^Ee3d>b*qbk)wL}O~K<6`N+3#j+ND4Ad`zvwyDv8~%LwGYZg7L33~M6)6nU)+9Y
z%J^F4aLD~CEOHc9(`*OXJXi73JiFF;boZ!wZRxvvGSq_^2ma&zE2T?>F$mu5zyw)Z
z85Pl+OdBESyDOcc^^kuTC{|<>U+2nK6nVf{NIYPbyizhrho}v@?pWbp+EaX{&M`r|
zu|f_zD8Ia~NEqTrMEJQm{}oST7cEIkMP&knD2lL~rM1dIEGMLQ9s8RU^{TvS+|q_m
z9||Q(>w-#I@tmh@wYM2|&?-m#f?ii4)OfVX?f3I5y&VtE4G7j*w#M2VnD*=I?Ur6g
z{Sg67x$H@~q(vMyU*Q>_Vt>ix6J|kp$+KDldQ;uRAEtA_8u9p3u32Iyl~ApF=vcjM
zz>a&+7p|M_`$l?Qi|CbXeM7Rywn~ezg{~BBVd5vgaj0=-Amg-
zDzboRxTE{}(^udh)oTBr4x_epkj|X8&|+1m{m&yhW2cS$Zb;t>B>xWoIW@yLT(~pt
z?7$q&m&^=3im}0W`T%T7rbdJR|4d0i#OrP8HKLhd0KV1{Sp)Dg5qCo@ayF$sG|*dE
zD|6)^Ud09b&y02W>V8^b
z)?lhs0Ee?oLRh+huwqHn|kT1qIQPFC$r
zGu1ahatLG{eWhzm=A{|^X+B51Gv*i?MLhmTQ8;85qw4i@X&Fc@gj*VW-27h6%{b=?
z`qPL)o#tkuXC+@{$%U_aA_g|^5d$s3<1mQ+tRjfuMw{0g)Q7tX+zr1rOdyMPmQ~C)c*#+~kcqkJ|_f2r64lOmtqz>JZ>Fp6d9Yn2tQ~b<=tN
zG;uBTKYrTt@*~t5nw1oQ0eL>R?58^t{-`vzQu~%>WKJT%P-l7wdgPbN`>cgAlgnTy
zI&t_kh1alFasMq6-UD|NO=gnE3$kiXjcOMGyqWKnd~xG-FK&?Gb;yulh%}6qi@&Rc
zQ}&0OBJ2DJuSz;X=hXsBCTTZ0qPhaBD%Ln&L=6FRk>(QV9+JLv`+pmcgzhvxN$yO1
zu>_MFG9dxVoiiuY@boF4{5DTgKQjxy^VF|sp2sK(5+pqoxDhl9{{C=_%L=0N9ZH`y
zJCY(Ox5&nA)xe;ZEekDC#W7l(-9Jrjue>ERY>~A_fI+4qaBeYF)w$?pjrB3qJI4?!zk4&;H4I&nqY3uqjSuwc%4Y)Bljb;VO0yvo
z+sk)-Td|VK(8O?78b_J4xU%ixK(G8f3rIj=HqzKq_LZ;m8erDq$}OusLNosCcuf?g
zu8-;Rq#>ya6JmsMoOIr8;U~yN)b1xlAW*3qGMr+-#6vzVR;%vMaWYjg=0^Q6u(mtx
zVKR@-%%P4rN(|+eDhYVds>&u*=APw;K)cub-1zMA?5^Q)lNuhT{5*H^;Uh|Il93@Z
zy2x-172tK~-A0{=JRLDm1mES?S0o~jwBW>e%Lk=i%NxUN5pIFvj;L@m(5V2DmouFzJkxo!4rj=^=HGJJ6MVh2s`0*(
z=JXGbohpn@P^%Pv=gs8P3$G2urP;Jy9g=wVRG-TS5FYxQ)2{*J3EM<8o
zsfP=LcGK4pD{21#vbGxGbf$C!bK}uo+`%`n{JOtAMGWTdI@DY(Tzr
zKy5dIBpm%`VO|*|iKAC)lX1U^*m2vo`L>DBKPmFw=qG6Nc7$G{v8{8J`K*fg_ZW+C
zP_LVQi@e!!MLi_GcIePamxHH-h}bGuGXzbC|4F-$
zvA~@ac1q#ffz?q`pKSlOaY^C}k_4v5>ZOxf?{1GMayW)>2)Z%}!9m15N9}5Kq
z87@(wao3DQ4i#aa_H1vCtw6EV=MkKAc5vSbu(MwwJ)UqslXb+O?=66#@k
zDy^Ub4$N=eS`jWp1k_Y{bUhYl3JO%ah?__v*46a`pU;T=YkiY;{%N>Mt;UuZcyWlE
zsx~6GXNYLAVvJ;(*!pL=$-gb2tZyL^4}Nh)L!-J~h?T8(I25?sIb!nA?qssftb!lt
zx?duO6VN93rju=g!(a`r0|fleBvM$JW)>c3pMXR@6a!J9br@Ulm98SlqIMA-3Z-9$
zhNQ|KE?JXT+8FrPRY(bds5bdtDgJV_4IvC4bPP*gB-r!1VZ5*|>HwMJ^*1-&j5S~`))|hi{OYbw#`sBv
zBzgvVGi%Z`KD9tBvqNLNZA55gjwUw%w4lW-xsNTV-PksM#QmJz5W?C%=)i!_ckvbi
z6c`4S=-`RoQ+Er=-gfD)rIN)H7F{4fA7X&(7Fd=D)Npe%ec=PS3rGSKyP~DktfF5V
zg=4r_XV>fh_HK!cT&)v$98vFrb4*a0(79o%hN<8=8vFJ`{@AqoF@x1^Ryn5Uy?Wub
z`iFz5V>*m96d94C0cIL!-Ez;tKK?s65YhLFcsgE~LI#EM76s!q5^IU@Y7f{pTO~Gq
z>PXf=%(*$@J!1$qHaPof4eLl_r~n_aA~?@3UUrp8WV
zWDoVu#YZVU`Eu|$b(<8y1tNFoqpN>_wjpBEya4^3g5K_u-ETVqI&2}&IFTVS_&Q7M
zup1Yz6L>klKQdNW*~s5qkj%2(_q&XO&%*;hzm#h5EE7!^Xiw#5PnkJ&dj9Te9&_x|
z=+1AW*^|Y8k}v4#aK`h=bo&Y!lK|2lFIz&j8b!@MWSS|j&~|6i3OWDollFa<=4K+d
z)-jMqJBAd=(1aYJep0VXTeIV%DclsId2VM&LMebtFWHJ_*5iDuvG8%+fsSdG@2mq-
zhX#C8YHP5%_tq}cT581!`=pfV6XDX#Jy};zJ37Uls+7s^|lLe)TN9w)=_KkV}1!2%TMhhY>kx^Qo#_KrGl|sxlUAY?iWLmp7ldhb+Fi_1KRTuu`TF}N?(Ayyf`kj&SffthTLE!G$w`RJV1WEk{8U{ZaRMA1A>?MfV~h7}
zfUAzv@$r7@*XeWZ0g~d(jvYNx>ceW1YtZc^6oMtoCQ|ZO@{q*z*wV$*h42MFwa-KL
zh?vAUWq70yqQL?-;k4`amH#_~_tHwD!_bU=NIVsN31;2VP&bSdOLRzRSurV^sf1(G
zJXJf(=Z3!^APT-~aOB(#bR1N@eOlA2kS#%y!~{xq5F=%;tx*r)k$$K~hiE~R)r#z-
zh}`fVDmV{EI^Ty@Qjh0GN6hvbVoGP2zU91_%-eU^b3D<%+!&K8IWCeF2!ri7xVq;wW
z0&piGTzc?p3yer6%ber5_9R|hp1d`MH0OXm6zmF%E~Sj%B(BJ)xJJDP%YayXlB~{KLZ}
zIt-2WJJ~QR_~Leg>M|*ofD)+`+WmXjQXhg{3Ehc(3$q>b^2|-xHPR;g$a
zT$&xJu+_6f6H4W{son${I`Jul$+=na8z6%cPP*3sQ{B({q2Uq
z_(B(X;IS0GOLi}0W|8UW$V$rJo{Kn@6f`+g2UORhCIZ1%CPXk9)KC#DFnh}ILW5x1
zt8`H{5ua16rE`$efbOc)JX$Q%LahsA@*;Ml%{3k-G<*dvB%u65Q(RT}0mX;;I+i0C{FItFw!%&fb1p%Q
zDj@UzRB~y&Gr7{st)Ku*Pb?Wid`7P%J1x1jUCrCjb7~lK
z)xeE7R+krgerfP-$7Jv)AaX(QjzW2HQD8%wpnoax7#W!~q<-3SNxNancq}&n#w)^C
zm=leGh&3KUh&MV@Icp5HxQDCD@ll=6Cy^AZN^LhND9qZj;r!THp}14Vy}NCv$VGtWWq$Mp|ZT@
QpM_5wk7l)q000000FI(=XaE2J
literal 0
HcmV?d00001
diff --git a/studio/frontend/public/provider-logos/misc/meta.svg b/studio/frontend/public/provider-logos/misc/meta.svg
new file mode 100644
index 0000000000..9fa656bd6b
--- /dev/null
+++ b/studio/frontend/public/provider-logos/misc/meta.svg
@@ -0,0 +1,19 @@
+
+
\ No newline at end of file
diff --git a/studio/frontend/public/provider-logos/misc/microsoft.svg b/studio/frontend/public/provider-logos/misc/microsoft.svg
new file mode 100644
index 0000000000..5334aa7ca6
--- /dev/null
+++ b/studio/frontend/public/provider-logos/misc/microsoft.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/studio/frontend/public/provider-logos/misc/minimax.png b/studio/frontend/public/provider-logos/misc/minimax.png
new file mode 100644
index 0000000000000000000000000000000000000000..e9472c676dc3af879f12d31ac0c66c0c0f211640
GIT binary patch
literal 8665
zcmb_?XIK;6x9^?|2`vGnhR}iqP^5Q3jfx726h(T8B3&dD1*rqRil`JDq9Vjbk?IS7
zL1}|1pmZrBAOs5_HFSi4ckrBV_rtxg
zjE@6=LpI(#Fh(&HG1!EC#aNnIA2jn07}uarYP$VWJU6$`W_G{r>>m4x-OfwL%-FkJ
zR*xU%IM7D61kCJl_$haON|!#eIdIwX$h@)j+&J!9Qi$~3WDV-bJ;k$I~
z*t*N^5&7U5Bd1jx<7IQRHK)BRCl7rmNBlln8vup+pN_=gJ?llUcUT9>ionIr}`Ul5^0@M+*xjZ
z$3yLk+TkIzYT=D%Z*83Zd(K*-R-n-y)L*6}as8fY3lhhnEb9f&hFS~?pmCbuyV=?9
zZ`0Q^=CbJ)?r5z;#M9Q^{~T6a`A95VU8!7cf2{mF(|7v&AJOgVtY#r?!R58mS_?WF
z>5=ztus@1>PG&u;4GHJ!bGceS7Zxvo_2+>m+cx{a-it<-p#th6Y^q}&~37SeO&d?iLArhkj-#ieDCkEr7HxbdS9~rm&X-u`dnlW0HG>+~1lu(uemiT2)e~G~;JUli
zM#gk`Qhoh?a9M|SO7h%=<>3U`*_^Ye{@K>ZOS6R#ydLIPBGfc%`xz09-Q7ArW5rAF
zqA;e>;WHvlay7H`7cGtZYZ1g!14
zrMI73;O6#H8X@i7(p+8Xynr%=
zQInIJsC5@6|30_4p67ZRxHiA2XC5Tf$yW5eVXWhz
zG0WW)cN9o>PsZ?*mep0OTfUdRL--T%l&}7^ci~arklWipRd?EcWvHLuQp?eLmpV7K
zcr^VE@*a#?XopvC1!3opqBz(m+SP3n(%;#hBwQp!`%wGy6&J1i9B;NRlWv+}3ZC&y
zs1Dx-wD!uSn}C)zTTFE)q=nzg>cxXZ7CO4PGs*z@_xB?;U|H~>-Kh`1_bNk{A4GWS
zwT#w~-w4r$uJ7Hv8?pK-YwV8xIn#543w*jv{_4*q0V|~)?+F9%`)XnUZAe4h5m5ES
z7M}2`g0@=@M*-cknAZkU;=DG4kdm55?o4h=&RD*QAX6<${t|YRj;MFS4fjzOvr~LP
zRB&2)H}d_#iPVdeV*1qtn6-psTXBNEgn`LnAlfPl??#BYa3YfDoRXYuQVD^J2L2vcwclu(P@aOm!2>n
z{9FI0-#6_)+jZ|zxd$`8+Y(u-mvVTKNN#m#@bJobYP{d#r^pNDkkX&x)}O0_qb|a-
zDc=CRC_wvc-W@6Dzhime7STL*fK|$|O*SU=5Ms$i8sPSF8LJ8|@H6!s@&gG*w6c#X)&6|&
z)Q)|njX#$ta9v{&FXD{z^0%=Ed(DH;q?4z4TNw8|imtose{gN4FftlobPAN%=AIlt
zxY;(Z@ntk5;Eldps
z#UL~n1b1;{x`1|7o$@-Dxg3LI7_Q0)PWntDP^q&1HaA>q4
zE<0t35zsflJaFYiCvr{lMjfK2_PtMlDU}O_%mB6x*k!;Ou*rFY^Eo)d1k!K`B-SaAo(=C$f_AX#1`
zUKM2%>E+jps?W9~t*r`R07ol#LPk{e=j@LEy3&A01&hB5ggUk(vy`GRMIv$@XzM*Z
z4i~8E&{veqzIgO+s!A~&`t`3fZ1eRNjP3$0y1l~c?&!`dM}%F4z5R~1IH~4J740W2
zbBqAm4bAJ?gTciEsS%{quU8EB-m|xaqoj#FE^?YDb*aZFMNe>05dkSq@ZjAeefdMi
zRn@Qjuk+A=3UPdo56SLi46u_y9|1i~gOTFw+z#`rV>h3@eUzM}743I2<~{Cg3(_UZ
zgLFY@es_*CeDE&Yk`81|Dcq7QlEJ@ed_myYenq+{r$8Q&`TC2OFZ$gob}NOtJRsVv`q{$d&PxEFe)n6|-C*)F9%b7n=Uirn?;(HCRAxQnI>
zHFADw*IvGICp5KrhW`pz{$oP2^ETuH%CLU#oo?hq)~x}B?~PfbgyQzsq$uSVP@mCl
z`nmUnCRZ>T7ro9yBID@S0YNGT5`6ADg`ORJ1m)TZA(
zpaJZ}&@YGl(20ONy`Rm#1JQ(9_Vyay
z6gooG?~)z41@!S~3JBH{hI@OtA6z4*-sspFJzztptv$V)&V}*-N1S>Oj}I+!LEMC>
z?_I^!NwK7V%>k&V1n7g>#L0|pWM^>#K9{Z^L4wFW#~2flgZeJyKnuckGoycvTS%Y3~+-UKE#C=QN&#?
z2Z;lSAF%VF{StVP#jVbT+5qA_;f{a>E|R409YM77L7#_y?S#n?O9e9PMvnX4$KY83
zxDtB*=s-4}EeRYUMxGwCET~P2(TPLcb(mxn0*n0U4{6JYTNj@h2$HjA5|RVqYkwx}
z!oV?b7N$A+(=?FVvm)&(5Vkbn6aWRR^8{00Y7t=9iqsz@!#H{zaB-*k&_IO%I_`@$
zktr~aMIwbh5yZx1z{Am44JtT}(bg!*s$@_ICmT?~1p=$X)7M*`R9)B13{!R
zL!o9ZdC=o5aRXbWZ)24Ws7sC;QMBRSfk|mbAG?`n4>*mgVoG)*pY*mp8bN%2sp-8(
z$VGt?(gjlCj*p+~(E%2W?u339%Bh@J`zxW{;2q)(JNQmcs=qB23v;Z5j)7+cPWIv(
zn;B}`z!=F3P@UP=CoXsOHSQ&d*|3hoS3rZbx*JN1-p8D{oL^((PvZv~^g1MSJ!N*G
zH}_NQ9WsLtgCGfy4RlcRaclV~vIh%GPoiBn!F9*JN!bF)b9ll$NH(B%Bd>-pq(+Km
zPaep_rp?N7f$4fAef^e8q@?~amhD+gb`BIo(AGYYIyqy_R2(Yo|0`^SXlKWjennl8
zhu`+p=OaVdOcCq`8zzkwE~w=X_{WP9ttx{4Jl3FJ7RD^zfD}}Lep$ITYm+u>)#@WW
zewdcloyeF@=tDhvTAM|*Ma3`8eHnF#0)|wXXjdY2F9!`Xr2Qp
z&!Zspa1i=c^#K+hvOCK}PNn_SJ?#1#ip1=A1Wu9p(3`1T({bD>Sr}gDcivb;s!hca
znIkw!wA=ciJ5+eUzzGSb7pULATorGTKPjr%y=mP7g{&Ey%08eUIVoPNQuA
z%MO==XbU?T54Bh;N;GFuD%Ne0i!fc?VqQN=ZrtVc0XYF<9+1S7?uK~<@)C^s}67%Iza;MUvv*;K<7HUr+tw(I#1-Ck6oR3gK
z#8mA3j9d$tzmTs>e@-bn1o0ptJ$R|Hr#VO9f?RT&77&8RMv(gY>;GEdUHwoL(1=Pp
z%%LmKgV4D{MJUoYmSqXCTYS+qF{!ar^S^0y9k~0aAees1Z@#Ilvf`yqv}k5Sj6K{v
zj1d_EsfDmeNNuMkosJnP&kJqRRtS=12dle&!pEgUSm25IqWm$A@6Anq-RInZ#+&~D
zc2zlR&QIzh?9qBAhh|1*j8U*`sN4dnsJ2hCE^Q6B7)#W=>v&upRZ}hE*{mkdPU2P8IOd^2k
zaDi0c_()~5PGmf7mGWE=XfVgZqhr;Z({)HrPiFvChv{%RecB@FQ{VFCX#)c|5Erk}BL*i`+{&
z)`#cDIxAYWH2V0II=6L*{51ns13+JZDN#Ed)02*V)6w^(Nr7B=bZrOJra|;Uh4Gob
zsGtZ&BI~`q3hk|T+}ZGEa1xnj6o
zn-#~6=s?;LVo0&i($;}Eh!kXP2`@gVu7bTnFYQFEx=Nq+$3bp5#T?b;cZ1oVISaCA
zS%oa;7$ASSTNqy%XTTbQQ@5wE;=KXuB$!fdxp2kG?38xVILEt60-jngGS?(XCtwaP
z#Iry4o03A)Of1a(*ey}7e6d~sKnt?EaC)LiA;80}W0?S<8SbdR1mJjYJVN-d5{wm4
zBkupYGQq4DLSp%E6fa1>6Su6&`QQ--6ENR(L84Q(7V}t^6avKqpI{{?eZ4TOK!9$*
zP*{nNO&Gv36^^cuD2v`4^$gNoF<|l|Aa&At0etLc-5fx+zyMM&4;+J>Ol0><`C~mF
z5hF;QprcF7TR@Cp1V%BuJ__wVVBsXQaud)e&_E{<*9EG^`+xdv1sfJGS_XU3QrL?5@nUUgkDf6T|7lG+^!VM2mI0
z*!7Jaxt$3mr*fa$Z`GAvj
zm9FB7?yr+&v5+DGEdEFwRSGeE{nY@uXL`ejH?dkbe)r8S5nV(wpzj4u6}iL9DU@Pg
zP=5qSaIXw>%s-;$eZmhWfp7ul33j?rp&MvDNpC^If!;lZG8fV)L~k9u^W!{`nWfE|
zQ8u{#r85VTfkF6sL0QA3INEXH>I+_j*M%1(i~^w|K|qKOUw+jz7d?^PgYdgc>@0-p
zF;sx2ws8=9@OBY0(3Poc#s#HL>IY4*dil
z#d?I1Rdc)7T>j!1lrYk$BQQ)_A!
zRuAC3*1K?CA%e_P%EVB8K|479X<6K$t`^(R+ASHecF%Gnwm7dFVmfpm
z!j=MP`qo}hA&wqZ4D25Cu&PJ2J_Fh|=Y9
zJ4S-3M#3uDkX~YVDMzwO94)kb6B1AlCmDTbpLfZYhM~4A(qTdwT
zjV-r>Iv(=@!4atS@TQjQ>_hIfk*OK28v$AcAzc)DR(4^sY5ynL)@T=}1)Ug%*F5YH4;)nds%b~i{AOJ83wwSgT7K)B{+
zULd?C;JW{DVRVhy|W*X
zo4kne0I%LT)S;CHSYK0Oj!6{_d{qRYrNyaNNOp6>7NDZ<&@=lDaJPH45C{K&H-
z%0^zD$bWvrX!*7_pEhLSH%#NH7(lwkGL*qUfLebtR2Vjg2b84D`$p`w$zrR2@g_ZfY^7nm$Qr`ut;zjhwxMSfgm(fC=sH8F$zYW&EL+2OmEbkwVNw2ylp
zMCtNS)c&1+uhX=jDO`
z#~}CRh=oieh4{9HTBUG)taRzyT~`O->fRq72I3w!rx)Ry!4(hn$Y
zW>Vn`wlP0=kZJfC_L{@4upb2u`WC5xWZR`%P7x3i+q)6gv+LGR(v5wAsFI~TY9J+V
ztk2U4kze2SRObVdu<$_w4!C;Wb)@4F?Hc1vhjA>=i|}~UHt^|^&_UcLr0|0KdLVcZ
zQ5^q-hqDc07cX1hFzjnz%)uuMCyR>32!p+GK9?>Y*cN#a$W(Q`v$VjS2T@a1d$t_h
zG$K2qmJtTvs;*~rl6WU#$X&BL$UY|CszfFaV#E!`psk<3ESF>qCB$WL
zFGU{lq3On&o|4Ls*0DehYLQQVz-N62DfC=loDCPttKmr|FL=bfKgNzCRrOeO
z|I=h}-6A-E58{|j{UbeABNI~?2>;8wzeHn8vv7n{grJ|vr^%<-20kG|pR1{6{e86=JG_
zJE_AOaTn14Qz07Te%ot#OP5v)lxof5ENJs}W*nlgZPN)KN&^-_qd}zQoP6k4biCEl
z?%vU-A*r8gBjZ!e*;2oK8%Y!S5iM
ztnPc@aOFhSsTLjjls-$bQO)~c>^;A-v!vx0x-6cxn+m6QxsO1u_372~0D#Vo{{pB9
zW@V8^E!S>l5;PCEla}xI@|j98>e!dH?fG^UVtwdxaD!2Z&fA!M>@Ce($dxbBc|S-o
zezb)PJx$Of8~!$+FFt_`;a8LXKmVWLAEDXixoDFinZ
\ No newline at end of file
diff --git a/studio/frontend/public/provider-logos/misc/perplexity.png b/studio/frontend/public/provider-logos/misc/perplexity.png
new file mode 100644
index 0000000000000000000000000000000000000000..9845765c7f72d4f14797bd7c7bc1457149716453
GIT binary patch
literal 6632
zcmd5=iC5Ftv!DC@W&y%l_Fb{cB1;eyiU@H*6i^~AtV$Hb9fB(sflF(lqC#yIDWb+#
z7X*R0unEf7Qbh|C+^J%TAc7zo7bF4+?_z)Fop;Xr2VPEcGIzdnXFfCc&dmMZeZs~5
zS{h~=0HC#S!MqRv3XlKPXu_GP|BW8`a}WkC^9$OLIPpr8O?C6cs{i$^Y@S%zY*X3z
zKTMmdX3J>@u8gbO9~=-pr4$5{{K`5gFP&+wphB
z-o)MUns*YbOW#e);hrq-OiJ-KLO%)?r8P{0oVPAM;){tj);#3xqgOcnQ^#ltORlIr
zJlfEaX7YsmT!Ne&9#ozl*1~5tR__KM9Z+p(kA|S3%4v*4dAlQ6b79P9
zC&a%Sbb(C|w=RXzKqZF>DR0+=v{wncF`Km*2uhD}@CDXyf#L
zRgO$q3ynFr=%#o%KI*1|!stdrcG720SkEf_9RGl&7$zjgt~&bi*Nk<)2jn_rkVp*>
z2sg#E@yLcWcu>~ehuXCgJfM41^;S5^g=8pFzrbmI@dc+V&Y|%&$QjXQwuC8pOvFv`
zWPGkK8Xh@h@K9}@5((=}N@@A#aTgL6nPR7L5SpPV>H0Xth*C9X1~Jl67Ea6lQ5ey}
z>n6@f{w;`Ef*&GEDW(T8uE5?{O^t`qA2k`<>9j`GA?Y%wbK0$|z7V$B@_XImUT2p#a1){mGqihC!G2KtKk9s6{%IHs~XNy%pk_A@!fi|^PezUgM?NF9zWRe*y0;iku9hNxbr@ptF8qj*WaD`?dxWB$W
z>c2okFiCK6O&Ah|E>Rau6_l(A16fHwNs4^5Q`yP>8;-%B2KCioS)oZu^<-x+M!OkG8muQmB-&_DzYkK*PK3>z
zHeckT)%t3-CLRF8jn(dW)e-}k2^tT!hQqU+8<3PMN_9HQYJIzdmhwbb84H}F_YkDR-z-#vUJN2XcpR;i2<1Om&nv)mNc$3~9kD(V@<)@N(
znp}nWpL`904&HrJ>@FB`NrtNsG&LiJEL!E|;#TOT`TY|*gm{vb0`;=^=S{L%p0O^P
za1CA>)Tcv(#XWU_2evPdvS4PHY2#E_RJY_IWDlouC6k!F5zZPisaqoSs`FFp;&hqB
z{dK5UgTJRPPLoNuw}lim-?)Av_JjnG-3PFN2d}w5P
zCT*b%Bl{3&o%fbn;v^}H(Sxpy)i)rzK4%U~ri*if-<;?1B~dJy30`tjF2s5rU0}iD
z_fF_I1an{Nfd!j?*2GGU|8k)Qu7?bR`Yu=%bLnfyrcIVKYSAX;%^G+T$~N}YkWDHI
zX32!OzHR{r9f(|ln2X?VTt64)m)g@Z5&nMvb3Jf5RJ;*+n#tOf`81eOL76FRu+SE-
zm-Oq%4QnoI;4YMXp}QT;f9u2+0h8Y0@DJ4QdQ9^vS=IM@j-xlvd-emLB{`ov5(H!&WZmhOGbVSWa!
zUq(l)c>GbJImE%ooNo%?<+BQ!5#l#Dr8>-G|0gLiaDd&ra(Wds)h6oU8*toriw@4H
z+taz
zTt6RY9Qm<}RMS3mg*7R8rr;p4_Z>tIZOVsL^ZGnMgWBw9>j>ZLbg)HEcuRQ!67{_a
zgg$!naE6zT@Eiec$K5^nMGj!Y10XI-nEgh@ML&WxuJknxV#PtjU?5%Gr7XY
z*6sB}!%p?x!{n0h_dljr5gyQxedve^S&Ldx)6RMz+c=AqnxV^v0(VXQWJ2%c01*Ki{~z(6
zZj7@vGjy5$y
zi_cvuY64XL!Q*W!vtx;lxXS;0WQif@+Ld7q0ZeJv*}DGOGMf3JtA|URiAdX%hIDb{
z@3Tt0hRP{%@vZgzg*BmXrufqq0EEEAhQPrrfXmbday^Xt9tM@4w)y`p`6NCE2nhbY(gm
zAVc5snW(ku!na&mJ~#!bo<$1g1gmN
z4K1F6bKMn-jM_EDSIO+TeL1_LXAQu*seWK$;#&g+7YvXnqs|#W3VFf9`tY&Pw;ldk
zMiyp7erjoP=S=iO+vR6xKj>YVRxo`1727j-Jn|P*a@Kn9+FmyEJ6Nk)3Y4(U-VZ##
z9aP%kU&vZ+zu^1VkhT@{*bzISX-Ra!e-S&P(brnyjQaf~hAmnmB_YVO1F$~o+!&jg
z1K0XHRcHXeyH`c28g~k;iVM2O;bVF&?FKH4!zF7#t4`<{`)MtS|bs
zrJc|H#;4w*9mBq`)F7x*FrJVMWbLGn_c+ugR>*G)8=WYBmZhHb9ck
z@9uWxrC)@pN50ta1f#2W(fG3KsY(Bu{&-xm4ay~DaA&4*2~VE?xzRvfSYr@wBy-ylZ>1sp_PDADtgBzma};!&TRvjt
zElqoY44nOPS$WISzNQ{rySv&CXAH+I^S9O(Rv3iqiXxPUG=zis{ev*&R!cQZ4?NTj
z+O^5EEhVw`m&}-zUHZ3CVOW+UNA56<7eLu1H4jF>Nr%8+Ee8-
z2Z*@Tc6Z@sD?xcktR?^ylz>
zl$ZKjeG@$F(^-iBejF97Ng7EjD=#Xmi6<}GXC
z5+qVgf2@xVm-z;Od#+6=1X%ejGZBgHr)e?NpuZlzc;>bvOnqRz95PmIo2AZtHRx}N
z(=&~mk#xRtiJFK+gw3q{93an!`bV!?KpLQ2q9IylKMgU{2K}Az>@y2e@Q%KmH4vli
zQlu>nQ7+LE-G1(O6Xh29{)rBF))tb(K|*x`7&!U`(2gt83|OA3`I&!0mzc`_Cb{_gp#KJJ8>j~wkFCqm2~w2`Eqsl+NoPq0s;Bv(6Qq8SX%D!H>m=L6
zN>{{GlB6DUi#LZ#1*xzQWT+;9vsS;6&P3^A(iZ^{#{X}!z*6?1!D9_~0_ZC(8w{~2
zL>oUX)R6a&Bzc^GAn5hlj#5)jD0Q&e$*`=Mwh-
ze@0sM$yBy=<#%wL3vRx(Bjl8^-7>Ume-(*9cGv1u!(gEJeAP%#5;;DcdbdB-f+i>1
z;0Bj5-6S*tCUqT6`@_`*DCyALnq9R97d<(65LV5MZqg>Fp{^Ct53^z*OjERU)W)I#
zq(;mwh2O3eBXY>uoE;t8WdbXxq?(#*Hb=<3AHZw)mwiN>D|yB^~a*Lu&{aKXHueR`^A@gfy;=7X}6VwD7aDAuouL60`<<&k>T==2W
z_{d=R^>Z=Ht-gcITP`-l)DHCKkm^#W3H(XbTHTEyVUO%Qn*4k)93HUN?~C?X<(pRa
z>EXcik&FK}^mCv8mC*!I!@qOj-w}+VVK%-95Wvat64Tct3qb
zk$j?dTHh(N1`uk?9$FO?R9$}UaFuwn5l(7#EMMo`eA!zw8j%YR|HjP&4@wSt+#yOs
z!0A|r>dLXM_VQD?>-xD6%3A+t;Ps}VM-{5Ks7j4_Syp)2*w!QU1|`dMkJ
z^!insY32^v>C)y#EsG$_u?H)C_pQKlNNDb(H%c9L9QGpgW%TVIHd?Tp1ZoPs{qvNK
z$nKI451)tQFV`%B>4Y6eef-vJ3teTu`6w;25mb7Y>PhC14dsnfdGjKX#viT_MEADc
znL#&oICB|Ce8P1Bmw7#~6@sWBkO>k!{a
zme+ZMEm};#EWp3TMfkwdakj2Zdee3>xQ)}=vZ|cNJkjsNj*x*Y6m$=V?~{#0226^~
zphHKLM5a!_bLVOY;~NB*2`zp-lr@!UBe)KbgH2pFkMB;zfqCAN`xnU@6UgRzs`w5G
zIGw~UlhzD-^GciVoQ-7bv7T13ui)v~baPmXGA_89BEB8rhhf)?Z*;Ni({AuIkWGr4
zGMyv3{Tg8!gGEbbC+&mv?CaHr3htF@eM*S6Ue(*lkCl`zOMmZbi3PMm1^MWkc}iROo4vt9LV063*5l++3l((`|-0AC}GvTzp4*Xo-%c791kwm8*EcD(Z`$^=wv-e>r1)4Je)=_YU3ThvnqAC4`#7|H3@}#L34drPvzBo
zUSuJ<|Hu&{AnU^Q5O&-9A#!oXylzf-59V=p;H@QeQQfE4r_HBj;doAQ*ov%3o28S`B5&3e-BD
zCE4(OmvJ~gOOhuZd|k|8fRxwckkR1sJ1inQcP04TQw?N&p)JA31L8}__HWY?cR;Tt
zDQn)TOSo<3Ux-88Hd8`u`VcuAr;w@TGwPPJ+6=!uI}cJ@;(5|&2qp3-E8P4r^dUr$
zypuA0#akhgo(&tfq!#m}u0(3*XsDVf@?R-KlwZ+^gvo=;7h)#@t-Px)0?bME%4%>a
zhbJV@2tia4Vq@v`B3^41ajNrO;4EB0Tq4BmFTN4N59gBFN{EFxgAn<}Wm&X$9g&VW
z)^Qdk&G;9dk!-vI6RDNBhmX^FhJYkJXQAtz&l`@lk8XRAk9uCv{udcI55lNYPJvjX_Q{
zp&Gv!Z*+$=y#_}edFoTB>Mm)Cu$1f>ouI-*$XgVPSitYC*tN&Qt<7TuZWhfC8jmf~k`H;tIpLE>FO7YAd-B%f6)R*Ra5CC_?xv`pZT5}Pn+Jb>)G#;W
Q1dz`{zs2)P=ZL=hZ!LJPX#fBK
literal 0
HcmV?d00001
diff --git a/studio/frontend/public/provider-logos/misc/xai.svg b/studio/frontend/public/provider-logos/misc/xai.svg
new file mode 100644
index 0000000000..0c83eb3d9b
--- /dev/null
+++ b/studio/frontend/public/provider-logos/misc/xai.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/studio/frontend/public/provider-logos/misc/z-ai.svg b/studio/frontend/public/provider-logos/misc/z-ai.svg
new file mode 100644
index 0000000000..28ca7280a1
--- /dev/null
+++ b/studio/frontend/public/provider-logos/misc/z-ai.svg
@@ -0,0 +1,215 @@
+
\ No newline at end of file
diff --git a/studio/frontend/public/provider-logos/mistral.svg b/studio/frontend/public/provider-logos/mistral.svg
new file mode 100644
index 0000000000..40c2591b31
--- /dev/null
+++ b/studio/frontend/public/provider-logos/mistral.svg
@@ -0,0 +1,19 @@
+
diff --git a/studio/frontend/public/provider-logos/openai.svg b/studio/frontend/public/provider-logos/openai.svg
new file mode 100644
index 0000000000..74d9b1b44b
--- /dev/null
+++ b/studio/frontend/public/provider-logos/openai.svg
@@ -0,0 +1,5 @@
+
+
\ No newline at end of file
diff --git a/studio/frontend/public/provider-logos/openrouter.svg b/studio/frontend/public/provider-logos/openrouter.svg
new file mode 100644
index 0000000000..4a4968b639
--- /dev/null
+++ b/studio/frontend/public/provider-logos/openrouter.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/studio/frontend/public/provider-logos/qwen.png b/studio/frontend/public/provider-logos/qwen.png
new file mode 100644
index 0000000000000000000000000000000000000000..67d2258f407d7b06323959c35f7c42a50c641136
GIT binary patch
literal 117172
zcmV)FK)=6INk&Gj$pHXYMM6+kP&iDV$pHW_N`uf48HjBgNs>^y|D^mA?w;Da3?lkJ
z0rRL;@B59Ff1*;w`i45T-FRL3q=lW{I<%}b&6|1P%=hn+S#uhwIP=}m4OlvaSH)`#Mm
z^$On06YyF-w!yLjj&@7`XY7yqkI8AXfgIbmX=}XqzPXye`SOqI+qQL3Cj_^#ZL7-q
z%EiAGa6>}+FNw%qMs6fYjwJcvqu%NZV~$NQfvL605lI*-dDkEkYp`5
zVJtC30$Up5?0pj`BO8FufVBeK0uTV1o^uF59yb98bbZThhYbOUH?3O;FK7^k
zcP5zYXsiH1lD6crd;!4q3_ywKiUI`);2u>9(?0;$5CAsO1vCKAXc7RoHO^54fUb40
zyMtb$4S?wzi-Qn=h!C_{0C`>k@ZI9TZJY=I3t+KW#l}5^0f;96r)2>k1ZCieek6cU
zq5wirXBAv>#AX;K5Q5w6-JRR)-F;QW^1?T_(gn;k@AOLV1z%aZjD24z!gwt(z&jOoO0T_nSg08vc0m=x&+`_ue
zjskF~zyXE>+8OCaid
zK!>8f7q3t<8_z-S;R<*H{s063zE{^28$&q*h|ORuUjabwK>%1_k7Ba{tk9tz28|?z
znLF@rx(`J{L`*;~k@sFF7V@@j%$xL5$myF~#Qgwk?|LBZXa5L7==zf{J4x1hLz31x
zr}IVJAn26Al#ryINKVk!kT%y)T9ia^kyZ)fa!_=1nM3VWPWd9}p8WmTYtbK(w)S4F
zLN1MahYr`BQ}sG23b(_gVt0K-d
z%dP92h#~W)r@ZD6>!y3nUxOtEOVN;RN|qwsS>o+3H{0ud%vWIY7BfqB-Du|S6Y_F9
zH?$*UEvqBWHD^R)M##qOILB;Hp)^%)nVs%?EXB4u?r}@3?VdkC*kFk|DTWbILYFzkl3QtJiH}>;O>xR>G({=&7)q?up0-=p+#%~Ot3$hFG29ZX$r5Xe
znVH^@ypoyXpQq^MZr8W9ZO1m*wsoG@^F>r!ZOyjb+LCRoX8tuI(QLQ2G28&#wr$(3
zWoBf&*M(b>ZB>#a*kybx*@->O%sg-NHXqK+%*Evy;Nfxo`t@_xwt#Kh@}}8I
zDGb+NaCexyySpQkOmKVW*s*-Ur%9&mtPOJwfw}U{y=1>@$lS{}mzGX|H~9b8?RG5X
z_xt~^>%Nz5OR^)Kw7Yw{le9}Zo$e%^WXHQtpO)i>yB-^#X_?b*vDSFfX^H!8No&%U
zPNzH8v8~JQ`@XLK@Au%o?(1IJ`~I)BYCU>G=E;6UaSTb-xX+|&g6H5HhmEuFyBfc6
z9l3GH_~5R)3WwmNkyQPWkq(|N9$h#robmb2Ob-&AuFBMuEu0L#a4wu1=N_LOe0LT0
z$K4iAH}dR^&-9V%Avi(1`bN5H;oJ}D;5Y8hk>Z}&jf_i*@i_};KY|XR+G
zV)pplxE-l#+{S0t`1Hm%&b6^Ff-laqg}Yu=qdmCvK|(E@@zmM)jgtqX7fx4Y9t7uP
zaED{q!b4SXa*#vi$w;4?vV~{i?D3i43uohQ>(JxV{fv{5uDJ#tv9T1X8vEjK<1Flf
z_JOxe+mfW;7<27??u(aMwryK&+y2xU8BvU={zv+gZQHhO+ui95WW4w8*=r2`+O{c@
zBguR0GjnqYj*N^1rKA#6RabR&LGv)rzsqz8dj`x$um`~SV`hfJDOO3P5>qNAB}N8#
zI2c&KwtxN7>PDyFF~I`lkF6~2TH4*+;!M)NT^sx`G5J%r0-J7fR=V!Vsdaacbq`Lh
zaRNBO|F_(dBjvt8j8GrPjOT%OI)S()jW
znVB&MmbEl1;x#mBXZ7+myUWb*m~Cx4qRdV+Q<{CuPP5Opv@El)_Z^gm!KIn2Hys|z
zg|49wQHCz2%T)h`)321f=%VOA#>HgnI8w{r<|L$hXPE@sH$
zWX_-jj|dM(GdD9yF#&q^|64BEQttbH)>>87)!nV6lfla%@>k%H
zjk_H8?fbazv!$(Ut@YghpnIP)vMc}L6z(?i4sh)W8;M+2#PwV_H}3M_P{dmBB~ULCdGAcoX%*Rs&t17r#GR?O6MYSXOnReah)CRw6Do_
z;cUFpA#@pwa8D=gbowIN2VZIJ5QoY}(i7rNx-X)hv~W)N5YCf!>RXZ|OOhnpVq#G>
zbC0OYYybcM-8(ZP+|5MTq9n5^nQ5vNw~?{7%{dnOa!U+M(N%<7kEJ^Lj4LbVOol3v(UMGp)dE(E?V$cmx0
z@K6=Zg@KO?;mODa(Oh_F(V|I@3o1`qE?im#i#8poplyZxXxea#=7k5&(o55mbzz4V
z9GXQN9@=X|db4yd6fJs_4ooT=+6GxM^jw_8?RN~4Gmx|-fahmfYyhjh8*h;gzac~c23|ApJoA>nolpLFP43toW+lB0Bq${hhKEa(N&_gsyMcFu
zU3fQMLwYxKfDr*Z0XnK00&vKrjS0V*Y}f=~9o7*r9>7((NY9y=k2;b(of&oL-#f3^
zJNSfupJ>|KAI)$~eob0<_h=~)78nGc$bkVClu89Gz<(Y88h;yb1gfrq#Hd*SobWsj
ztOH%(@xY-Q%Hc$YhkcbYUZPscPG~S`nobkO59-)9y?uvf*5y6}C+7)!yPy0&@5rz5
zI@2QB%knZjK>$pHJqT>T092ts7qEnY83IipReA`x;aLNA;&CQm5lDdt14rxxtYaOJ
zSP4AA%BX=UT9)k|sk7cUj0###{rArE^?0B0|0H?c6CG?=3wFgI
zM(HoqSwTT6n8IU%*+fbi74d?_Q33+9RiBXR=x03-*JFS7&Cv$V86?_7Q$nu^=1|UES
zD6p<|2pq8n%)%IQz|Ob=C=HiAicsRfL`qiTR;aaC_bvLkEHVu&o>$8tCoDc;S4ah6
zHc&7zj2(!8PC0aH`{Tj)2JW9X>+O8vf4(5t&DfUvIeb$G+8CqSuYeSz1*AgnaluSe
zgp}6`aIDsbUHgX(q|KX^Uz2=;p(Y7YCB}z~=)4XBaK~wPZL_CYlDp>NKl8j;kNxRi
zZO^Z1Vt`^TemSC+7SgHCI)(u!5FpSZGD^Y)9@`YS0SM3n$CWMthjf4u908wJDFF&(
zqURyXCB9^;2wKE27>Rf7m0IH_&GvhKmw_+ly?PtJVw8|y!}V54bXdn)eBl0ml~jmp
zM?`0f(#1L$ULTIpzEdH21GCgY16lK4dF*j78*ODcX95Mmc6Ler~9A2yCZd
zRtJPB+5Px4124>L^)`Rl^ft&i5EMxfBn8k;Qb<52CR)cY3i%Qf%n?K<3Ozi0v
z+^?hk{b_#+kP_EQBB@g$a(9n`jq_F+rNu(Af&j`0LDzP*)WphQ=Xz3WvnM()Fn9*t
z%2MYWSUNA&J6ISf1uNF}Wx
zW6}!QGYnMF0^Si|(SRXg-L0&HV0(T-1F!U?frax<<(O7F(r6L9L_fu^3{g5VbNFb1
z6EC|m&IP1CJPcwb3e0KS=tCc96o!Qy)qY9yNp}-JEi~;Lx3uRpz{gQrc-vefvNLKxhBO+LFuwzp+6N3+KR4M9=ap|*Xp$4{UCU`MvyVI)$b!sPPGsfV{qZ~8x~jvutNJ=uzhOyL+!Emq2}Cjw
z*#f74BLwUuuqFeFB>)?$y8+k$0;p#g@B#uPybj>)02U3nBY24*31`p77aCYKPt@bQ
zc9z)DQBG8AS)HQSqBB&RL~7FpnJgV$*G<`=#n^HOkHRHqQ)QdgF-0Bpl#c-^Up_HUHmK$jj>_z!JQ5{^`lV>y5Hi1wt2Ku*Dhf6KLPaUSw!&bCbF#a3-CO_0$*V*gf23{Q=nn5
zI{=jFdL6^8qe!&~4VVdeQElLXd7UCgH+zg-4MM2{C<=-&qJPc@_O1ur4PIYH4B7`I
z+S)x`-h=ry&pT^5cfDfW7s~sI9)Go?4EfP#<}I8W0_Ui%@@r@P)N#w5zV>o3>C)YcXf9~!BhB1j`*ld(<
zeo;EoGm9q53aC655=
zIR!cdxCPuKbLzQ70yU{N#fB}bO;F+e4uNKtL>wQ0R0`kF%&WE%A|-jGhfhdA)Qi~C^g!m~Er
zN|XA*EW=tMt|h-MnoH~guIoP!KAnG`nN@%EGemjR-{l3@Dn5kC(K;yb?QV;
zO?-}gyo>`<5eh0s%DVFLwBeoR6Vs20EF$krDk^{?v6BF0ljGgHXO4PN!8G7sc@XKb
zUcZ%)V|$q+gP_SCDg`_dKuF}f5N76Be#h0wNHL18N$~;LeB;>urDxQ)I=T?70gx0?
z01sj&oDp%Fe3mWdt|*bP_wu&ycH
zD4PopiFUYVE_I|#on&}?e-Z?S$A>|?SYIGOnMH<#0E!_19bg9pP}UVPpd{c;z{?1r
zbil^b0u2fjNo2^Tqup&!cRTH3;QfO~$A9frkBI&Rj~1$ufG7gg1*eF8H19@!zq0B5
z*yFDEOxg&F@->C7wLliQ!Ex8yo~U04T5>ZWBp?98%fLG7B37fpB%7|4yI5yn(!nE-
zRH`Ks02xAR``e1I4&wkNgV>bRL+Yw~*tpu#T_NoX1qO_Qz<7WQ_s_QHpF+^eEr*R9
zOcSY~#BB;hK*Ufd-p7~c8YmnzI`M1Y%rnqEQSAiO00sgHPgM14WZKGP;Y8YHHkSS|
zg%q?40eoP#v!?yPV|vfG`Q!efpiB7~T62dKQj(a+w?ZH~0s{GN$>og(dK@&07#7pC
zG?Fg71Fl`S69pE=@Oa+vw5(>1!p|csq*l7hz9%FK+mD8~
z$!`0v2vQtGD6W;&lNwCO$`Om~g=J<(BP!hkK_}BXO7|TByzClsvIJCO%83!L;#eQK
z%dpGu`ryzBf9SiBzQy2*so-@86fx9=Q)EUPaoY)``Bp-m`!?U+cc3fEPRib53XB9`
zj6x+St8iPH+9-1*-rhR$PO~SshDQA;ej>C{pwY$3F3JWVA%NmcfB;>5H@;d(v%h@j?T(9|yYg$y<8dGi!@&q8
zl+Nn`ivSLwfRHL^h3tuda0n=HBhUgy@hBj4fC?BBiKD<{_zR)mxT?9}mM7bO>i>Nm
z1Um5#e%RAsW&lZ|DoLHfF5brM1a8kwSPPH5rltgChvVdhXk})-7l3dpM^oYylWSi)
z90_3;
z4Ienqn|*BSIR@4p{MqUUpU#aK4)$aY)_{gm&^Kb=IxGmRnNf0&OOqcONByyz{!gtJ
z$T*cIs8DcV7L996l>_0n$dqTkcOAJ^`l*Vqu$YpyEf
z8dw>=ft5hmS6d!5pOpvr>Wl8{OW^|$Kwx<%yilMF3^4$dgQA2pmfq|A+{5g~na+zc
z$3@K@XiS-b`qdSNsRDu)xC5*aK?&m-j_FlkO$H)EAklD0^0>hxxEbye
zur}@zchlJF-Q;V5D%x
zEfYz}20?KY;gDIeT?k@qrqnQ+cnRXYC%awDbEE(Fb+G4nKX5xCqQ7Y@Or;(uGKDQ{
zMW?zTbt^-3^=f||IquJT)0Urzi3h69QNc?m>m
zLF{yF#WTzZMFEBQoHcq_R@WPtaBRWzs-}x3w##5|MI~!;^
z$aDN3__n7(7S=)10@Se*zQMTac%oCDDv$Ha-j~^!^aP~if){|4@~i_w6eeE5aU`BV
z*35I8p?2N<$E%~7Lc-Na*avVVp!EoAp!3)z;Il9$pa&8PWKR@0L_IeE0T$R8@Qu>~
zw*)>w>uCU2;D5;@lpA2E9d~)%Xkf>KJRW;o=&o7`ZJO3*zWvg}e8;wX{d|~2?ax}R
zr!me>vCKG8#QpWq11osG>1dw`?~XK0fnnCOfSAsZOV&G_&-QEscOTq2;SYQ^$5HfE
zLGX
zKMW56OQ1sl^(=sd0Lo$Dh-2W93btSXoS?0AO+aZ7%D`AGv~76w`mFX({{iRCIq@g|
zs|15A@26<5L;{HH+I}5++uF3rMyARxyQpdKr(jCW{T}34gH!QZ%l*R2E>7qB2CaU{
z$}uqw9mQ?~Yv;}JGKYiKjsBR<1uv`9dLTfI?(=G4D=MRJVrkuHqKC=S#|Wq%%9ObM
z2KuyW$L&3PF4n=I)vqUrubC6_>Yabtyf`QP)c=owR
zV`(h7w3!tX4G>{!T?5T--eSF(n7hmM|(Q;4Oe?l>Pg69bP|thY49YVtWoPhaVd-+K=XdDOgSfF0jYs#)fLI=*2v|6rLj)ylf$N+W$FRlDRrq%&d
zqm1>z4X>w(taw&Dqi1Gm<;7u)m}DgeGUvT9NM3(GC`IZW6q$tOGjp77+w6`zI^#M3
zUP_)GgpTD-PYzDQmp7QkGQuDX5$>YQCD9;G>aoqO%3TIB=C#@C`zbd_9Nq0#S7dB_
z;%yX}07~Y2fR5
zYmWDQ%LGSv)&{g2X^XpdWFPB44+ouOcYj6DGh7I2O5aw3l70^s<6?VkL?
zDfCKwR)5s1R4SDqAVn`r%$YN%spS;F7J+r33nX9xtPw!z0x9rd;0Q>-3RqcG9Y}y0
z7%9OhQFXMFJ^j_rTa#z=w4)odf$n~g9jRG~;apelt$096Ou`su0ce>ib`7`{mx&+^
zuYq>h^s($}===c#glO%>z_lT>#s(glm*)82H=FC|4|Npk^lJe@J7uM0u$HuSA&d9;
zzG(7Y@j|{qMh0aNcCxMuz#x3w#qat?>)lC-)ZChZ0TX(j2ZW)_D=`Uv8Wl%T^0%3n
zCSs7Y0Z@wHV@a-S6I(eF+wQTA)<>bYU*{3*9K-z{P!5kVaXQ{$bG;6;9~%pTG^Ye=
z-p=3B62=>d%sX@J@9X47jJr4Y8ZprTDIh4|a7^a1_vcQ+imlisXk(=z;f-0*0cirR
zF(V_4nN5u~PMWCcYd))e)-eTq1PJavI-m*kftLdvkbs3pSOW@lffP6dZUm%2QbfG~
zh$eVUU?7-h<~ZM%#^>114yfW3rJJH2XOjev$_1P5=sI8;?a$8AU}=oRl);Vp8=r{J
zhSOjGzjA=@zn)ZMJ_?;xJT$l0!65_l=9MwH{E%gw!y2&?*a)ORN;yRZh?d7z(RdoQnVT)o<%6Q3xSib|!O0{H#?iq0Vggbf
z)%P_$Ja(4KhIJI%SSF3UqSjsV_ven!wuzTQnFZ`@+g7F2C=|nM(c6i+{@s+ofC1q9
zCicTHVbMC-Vs%98h)x422&lnEM|AfY$euSQV&jCPAM@55@Qv^T+(eXsMeJ_|8;6}5
z6SVLUX+30DIwU^w7Fm?B6|*EwC=3@Y=<3P{7!PU($9EjZ54bnaV@R5NQ)#GcXsBrT
z61hh5*h>K0fD?exq>kn~QL1!6lsdj%fjW=?OwrT3wgnFnOMs<9uH_
zJEGrrgP;^Ik+i6lzf;HkxySWo*{$#iv9lhrO82Q_gX`j{$f_^rau&pILqZ#{5&sPy*f)U$R>&%
zE)ADZhD{yY6|+wqsRO?oXY31dh
zVF#=Y`V~v7C5?Ly9#Mil=>Z*9)e5Y^u@gs}zOFY`<|2~@e9U{54US^|%zM#u9+%_1
z8hYf}Ox>VOx_<||UwTZ9Y?feWM?e8U$lMZ*y2iEh*2L^AWyP33C0&X(I?w7PfI&{p
z2Ns7{4ClKtI7v4!;v)OYcvZC~mE|c{e
z$YJW{&HL-0=yxYDbZMR;4A%u4%_NETq{rr2F@)0@PGXuHB1Rk;0hKPS#FBs@Xl+84
z4j1nswZxvJSQ$M6;JP#v!OJ|(#1U`tIgh2aX!0%fGULEB&UDUA!b=iFbV@;oYg5M8&%%gIgSDO)@Y^W;T*8gCb
zg>awv*3Cuc5?_2qLg+c!FduqzZlzv}U7yaJv!vz^0Vu-Uu9w*$Bx6Jj{yL9J#IPvO
zh^RO$L6J@h$gV-xKJQ1=p1W`Be{ABF@Iuc4f*h&d!(Vx|xj!MbPUYhiZE=Dp(K#dx
zCy-F5*|fCX%>H4{JSoS1)nqz4DX&onK`Bzag#J0hoH@FJ1M}5d2q@9BF1Cj^>Q^2=
zuIW70lMU@Lb5yjeBI_G=cXT+qhvrF%805VMh^Uf)7D78nb9{ON-q@=Vi_F#pxeOv<
zvQ5cN^~%dQ*Wxc*)-JCM>49hwDd13mYXcqy*aB_|Ws5;5mw?X!QlJ5BF_sgaU`_OV
z`!yctUoNiZc^Rceu>yI*b?ptgWYf#-e{cA151t(B)yL(IB5wCF08J>YF-+iOX)Eu@
zgQHb)yKNmHvr3q3GohsnM39Eai}}Z^mtLCj24~A4=&Xfe*{paX%abMJ>G$#N%LcwU
zXtLF-|Nm8vZ05B#1XL_eeT=T92j7fwQ*Gm8$g9D&qy`xz3>MKBHk4f9Ri2bD>=n(}
zE5D^CC^4Ul)=sa@Wut+~2TdZz1s>ao1QLq1#U#CQ#4U_j@2FgXkLw8S9>yV%b
z0$3uFvP$GLUN(-Or!Gu>7jrg;-b|7u^|nHhaJ=IU@3j9ntHvM$gB~y3k&|MjrLcGw
zlnWj;NJ2dVbb&VkMguN;p76xLf+svytJMYUQeX`f1i;vJMS0FLAND;yUnIQ_*+4`$
zTCGR`w%7u=9{8f5Yqr4Fa}%_Ocs*be1YAI)u#F>Ipk+(XP0tD#C=?(A8Mvv%0re%M
z5wQc*#7G`AI`W`G%T}+mg$O1AFfkgGt;eSDY$;4^i8Ns78d|mpK^2@mF)=*Uv=2Yz
z8ZH>Zb_3OO*K%y@jt@xBfi9jbx(<0BJkm2zq
ztT-eMjP?u9nvNsT!-%0c)TTug?+9B>7dcdmmPpG_-1Gn+DvdH
zaBnSK*ykp!9RY@w;e@otIN49M~k_Aj=dcqwxkX!>U1t%k^
z1C*XGOUnQp1-=L_!ZTzPi=gBY$^%FtdnHR44o`TQ&kzS_XldwiH*hkNSi_qDdx2BL
z3mFhS3Q7sCVaXqn5u{qO7nC;(XgFdC^b9nt+0qdgkuTE0z+019U`sb(0R>BjQY8Fk_XOgm$@R%E>*PYuDdVcJ4!;bmh+gF)3i!6=`
zg;f$j6n)v2500akY6_I_1`U1>88@4UU7(}UAz$D)y5Pg(3sFlM^<-?lVI-LCCB(Ym
zkf=c91o8&gz~_wQ$mq~hbpZOIXS_MV0Q3*rbzHfPx3L3FD|944p^U^H@6ZD*0fjiy
zt2vnu4Dd0pOPFUt68!1?Ad0tXfm48imC@`(S@6F6R*ssMrbQFr2Dy2PJbh6y`eUC97STO1MrQ?qu_kpv)Y6c4d2uN?0kb%I@=RzwPK+(=VuT+bp#v;n
zG4z1}`hkAvhkiT^&^NS0-?WT2&<^xXi+_N=0oFi2z?$(5uKLJm1