ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'main' into studio/hardening-followup

Browse source 
Resolves the conflict in `studio/backend/main.py` `_csp_header()`. Both
sides widened the Content-Security-Policy for Hugging Face endpoints:

* This branch (9d693b35) extended `img-src` to cover
  `https://huggingface.co` and `https://cdn-avatars.huggingface.co`
  (so the model picker can render owner avatars) and broadened
  `connect-src` to cover `*.huggingface.co`, `cdn-lfs.huggingface.co`,
  `cdn-lfs.hf.co`, `hf.co`, and `*.hf.co` (so the picker can resolve
  LFS file metadata).
* `main` added `https://datasets-server.huggingface.co` to
  `connect-src` for the dataset picker.

Kept both: the new `img-src` and `connect-src` allowlist now includes
the union of every HF origin the frontend touches.

Verified post-merge:

- `pytest studio/backend/tests --deselect test_studio_api.py`:
  1119 passed, 46 skipped, 0 failed.
- CSP string in `main.py` parses cleanly and contains all three
  additions (`t3.gstatic.com https://huggingface.co`,
  `cdn-avatars.huggingface.co`, `datasets-server.huggingface.co`).
This commit is contained in:
Daniel Han 2026-05-16 07:01:58 +00:00
commit 6a7dad86d1
133 changed files with 16984 additions and 1430 deletions
Download patch file Download diff file Expand all files Collapse all files

107
.github/scripts/hf-download-with-retry.sh vendored Executable file
View file

@ -0,0 +1,107 @@
#!/usr/bin/env bash
#
# Download a single file from a Hugging Face repo with a stall-retry
# watchdog. Used by the Studio CI workflows so a hung hf-xet transfer
# kills + retries instead of silently consuming the job's timeout.
#
# Usage: hf-download-with-retry.sh REPO FILE LOCAL_DIR
#
# Why this exists
# ---------------
# huggingface_hub 1.15+ deprecated `hf_transfer` and routes every
# transfer through the `hf-xet` binary package. In CI we observed
# `hf download` on a 3 GB GGUF (gemma-4-E2B-it-UD-Q4_K_XL) progress
# to ~46% via Xet, then go completely silent for the remainder of
# the 30-min job timeout -- no progress bytes, no error, no exit.
# A sibling 940 MB mmproj on the same step downloaded in ~21s
# moments earlier, so the hang is per-file inside hf-xet rather
# than a network outage. The Xet env-vars below put hf-xet into
# its highest-throughput mode and force a 500 s client-read
# timeout; the watchdog loop ensures a stall does not eat the
# whole job: if the hf process has not exited after STALL_S
# seconds (default 180 = 3 min), we SIGTERM, then SIGKILL, then
# start a fresh attempt. Retries are unbounded -- the enclosing
# GitHub Actions job's `timeout-minutes` is the real bound.
#
# See https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables
# for the HF_XET_* documentation, and npm/cli#7308's pattern (silent
# CI hang with no error) for prior art on this class of failure.
set -uo pipefail
REPO="${1:?usage: hf-download-with-retry.sh REPO FILE [LOCAL_DIR]}"
FILE="${2:?usage: hf-download-with-retry.sh REPO FILE [LOCAL_DIR]}"
# LOCAL_DIR is optional. If empty, hf falls back to HF_HUB_CACHE
# (~/.cache/huggingface/hub) which is the desired path for callers
# that populate HF_HOME for a downstream Studio model load.
LOCAL_DIR="${3:-}"
# Stall threshold per attempt, in seconds. Override with
# HF_DOWNLOAD_STALL_SECONDS in the workflow env if 3 min is too tight
# for a specific runner / file. The script keeps retrying past this
# until the job timeout fires.
STALL_S="${HF_DOWNLOAD_STALL_SECONDS:-180}"
# hf-xet tuning. HF_HUB_ENABLE_HF_TRANSFER is deliberately NOT set --
# it is a no-op on huggingface_hub>=1.15 and only emits a deprecation
# FutureWarning. The five HF_XET_* knobs below mirror the settings
# Daniel asked for: max bandwidth + 64 parallel range gets, no chunk
# cache (download-once usage pattern), parallel disk writes (SSD/NVMe
# runners), and a generous 500 s read timeout so individual chunk
# requests fail loudly instead of stalling forever.
export HF_XET_HIGH_PERFORMANCE=1
export HF_XET_CHUNK_CACHE_SIZE_BYTES=0
export HF_XET_NUM_CONCURRENT_RANGE_GETS=64
export HF_XET_RECONSTRUCT_WRITE_SEQUENTIALLY=0
export HF_XET_CLIENT_READ_TIMEOUT=500
if [ -n "$LOCAL_DIR" ]; then
mkdir -p "$LOCAL_DIR"
fi
attempt=1
while : ; do
log="$(mktemp -t hf-download.XXXXXX)"
echo "[hf-download] $FILE attempt $attempt (stall threshold ${STALL_S}s, log=$log)"
if [ -n "$LOCAL_DIR" ]; then
hf download "$REPO" "$FILE" --local-dir "$LOCAL_DIR" > "$log" 2>&1 &
else
hf download "$REPO" "$FILE" > "$log" 2>&1 &
fi
pid=$!
elapsed=0
while kill -0 "$pid" 2>/dev/null && [ "$elapsed" -lt "$STALL_S" ]; do
sleep 5
elapsed=$((elapsed + 5))
done
if kill -0 "$pid" 2>/dev/null; then
echo "[hf-download] $FILE attempt $attempt exceeded ${STALL_S}s -- killing PID $pid and retrying"
kill -TERM "$pid" 2>/dev/null || true
sleep 2
kill -KILL "$pid" 2>/dev/null || true
wait "$pid" 2>/dev/null || true
echo "[hf-download] $FILE attempt $attempt log tail (last 40 lines):"
tail -40 "$log" || true
attempt=$((attempt + 1))
continue
fi
if wait "$pid"; then
rc=0
else
rc=$?
fi
if [ "$rc" -eq 0 ]; then
echo "[hf-download] $FILE attempt $attempt succeeded"
tail -20 "$log" || true
exit 0
fi
echo "[hf-download] $FILE attempt $attempt failed (exit $rc) -- retrying"
tail -40 "$log" || true
attempt=$((attempt + 1))
done

173
.github/workflows/consolidated-tests-ci.yml vendored
View file

@ -121,6 +121,8 @@ jobs:
UNSLOTH_IS_PRESENT: '1'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -204,7 +206,8 @@ jobs:
'numpy<3' pytest==9.0.3 pytest-asyncio httpx \
protobuf sentencepiece triton \
psutil packaging tqdm safetensors datasets \
'peft>=0.18,<0.20' 'accelerate>=0.34,<2'
'peft>=0.18,<0.20' 'accelerate>=0.34,<2' \
ipython
# torchvision: unsloth_zoo.vision_utils imports it at module scope.
pip install --index-url https://download.pytorch.org/whl/cpu \
'torch>=2.4,<2.11' 'torchvision<0.26'
@ -269,6 +272,50 @@ jobs:
tests/utils/test_trunc_normal_patch.py
python -m pytest --collect-only -q "$RUNNER_TEMP/unsloth-zoo/tests/"
- name: import_fixes drift detectors (18 tests, HARD GATE)
# One drift detector per fix_* / patch_* function in
# unsloth/import_fixes.py. The detectors assert the *healthy*
# upstream shape that the fix expects ABSENT the regression;
# ANY DRIFT DETECTED -> pytest.fail (NEVER skip) so the
# matrix cell goes red and the maintainer triages on the
# next PR, not in a downstream user's crash report.
#
# Pathologies covered by the suite (each maps to one fix
# function with the line range cited in the test docstring):
# * protobuf MessageFactory GetPrototype / GetMessageClass
# * datasets 4.4.x recursion range
# * TRL tuple-vs-bool _*_available caching
# * transformers PreTrainedModel.enable_input_require_grads
# source pattern flip
# * transformers torchcodec / causal_conv1d availability
# flags
# * transformers + accelerate is_wandb_available
# * peft.utils.transformers_weight_conversion importability
# + build_peft_weight_mapping signature
# * triton 3.6+ CompiledKernel num_ctas / cluster_dims
# * torch / torchvision pinned compatibility table
# * vllm guided_decoding_params / structured_outputs +
# aimv2 ovis config version
# * huggingface_hub is_offline_mode / HF_HUB_OFFLINE
# * torch.nn.init.trunc_normal_ presence (patch site for
# patch_trunc_normal_precision_issue)
# * xformers post-num_splits-key fix version
# HARD GATE: a red cell here is a real upstream regression
# without a corresponding zoo / unsloth-side workaround.
run: |
python -m pytest -v --tb=short tests/test_import_fixes_drift.py
- name: public-api surface drift detectors (9 tests, HARD GATE)
# Companion to test_import_fixes_drift.py: that file catches
# third-party drift; this one catches drift in unsloth's OWN
# public surface (FastLanguageModel / FastVisionModel /
# FastModel + their classmethods + is_bf16_supported). A
# rename here would silently break the unslothai/notebooks tree
# one PR cycle later -- this gate catches it BEFORE the
# breakage reaches users.
run: |
python -m pytest -v --tb=short tests/test_public_api_surface.py
- name: unsloth Bucket-A — CPU tests not in Repo tests (CPU)
# 16 tests across 5 files. They live inside tests/saving/ and
# tests/utils/, both of which Repo tests (CPU) excludes via --ignore
@ -840,14 +887,23 @@ jobs:
import _zoo_aggressive_cuda_spoof as _spoof
_spoof.apply()
# Hermetic cache dir + force compile path BEFORE importing
# unsloth_zoo.compiler (its globals capture env at module load).
# Hermetic cache dir + force compile path. The compiler's
# globals (UNSLOTH_COMPILE_LOCATION, UNSLOTH_COMPILE_USE_TEMP)
# are captured at module load; an earlier conftest `import
# unsloth` may have already imported unsloth_zoo.compiler with
# the default "unsloth_compiled_cache" path. Mutate the live
# module globals after import so this shim is robust to that
# ordering. Otherwise the compiler silently writes to the
# default cache and the per-model file assertion fails.
_CACHE = pathlib.Path(tempfile.mkdtemp(prefix="unsloth_cache_"))
os.environ["UNSLOTH_COMPILE_LOCATION"] = str(_CACHE)
os.environ["UNSLOTH_COMPILE_OVERWRITE"] = "1"
os.environ.pop("UNSLOTH_COMPILE_DISABLE", None)
import pytest
import unsloth_zoo.compiler as _zoo_compiler
_zoo_compiler.UNSLOTH_COMPILE_LOCATION = str(_CACHE)
_zoo_compiler.UNSLOTH_COMPILE_USE_TEMP = False
from unsloth_zoo.compiler import unsloth_compile_transformers
@ -906,6 +962,12 @@ jobs:
# Category E: undefined name in emitted file.
"perceiver": "name 'AbstractPreprocessor' is not defined",
"sam3_lite_text": "name 'Sam3LiteTextLayerScaledResidual' is not defined",
# Category F: compile exceeds 60s budget on the runner.
# First seen on transformers >=5,<6; each represents a slow
# or recursive source-rewriter path the zoo can address.
"beit": "TimeoutError: compile exceeds per-model budget",
"sam": "TimeoutError: compile exceeds per-model budget",
"sam_hq": "TimeoutError: compile exceeds per-model budget",
}
@ -921,40 +983,59 @@ jobs:
skipped -> no `modeling_<x>.py` file (expected for some
umbrella packages like `auto`, `deprecated`)
known -> in KNOWN_BROKEN_COMPILE; tracked for follow-up.
Any uncaught failure fails the cell."""
Any uncaught failure fails the cell.
Per-model SIGALRM cap so one infinite-looping model_type
cannot wedge the whole sweep + nuke the job timeout
(observed on transformers >=5,<6 -- 30+ min hang before
this guard landed)."""
import importlib as _il
import signal
ok = 0
skipped = []
known = []
new_failures = []
for model_type in _all_model_types():
modeling_path = f"transformers.models.{model_type}.modeling_{model_type}"
try:
_il.import_module(modeling_path)
except (ModuleNotFoundError, ImportError):
skipped.append((model_type, "no modeling file"))
continue
try:
unsloth_compile_transformers(
model_type=model_type, fast_lora_forwards=False,
)
except Exception as e:
msg = f"{type(e).__name__}: {str(e)[:200]}"
models = _all_model_types()
def _on_timeout(signum, frame):
raise TimeoutError("compile exceeded per-model budget")
prev_handler = signal.signal(signal.SIGALRM, _on_timeout)
try:
for i, model_type in enumerate(models):
if i % 25 == 0:
print(f" sweep progress: {i}/{len(models)} -> {model_type}", flush=True)
modeling_path = f"transformers.models.{model_type}.modeling_{model_type}"
try:
_il.import_module(modeling_path)
except (ModuleNotFoundError, ImportError):
skipped.append((model_type, "no modeling file"))
continue
signal.alarm(60)
try:
unsloth_compile_transformers(
model_type=model_type, fast_lora_forwards=False,
)
except Exception as e:
signal.alarm(0)
msg = f"{type(e).__name__}: {str(e)[:200]}"
if model_type in KNOWN_BROKEN_COMPILE:
known.append((model_type, msg))
else:
new_failures.append((model_type, msg))
continue
signal.alarm(0)
if model_type in KNOWN_BROKEN_COMPILE:
known.append((model_type, msg))
else:
new_failures.append((model_type, msg))
continue
if model_type in KNOWN_BROKEN_COMPILE:
# Came back green unexpectedly -- that's GOOD news,
# the bug was fixed. Surface it so we can drop the
# entry from KNOWN_BROKEN_COMPILE.
print(
f" UNEXPECTED-OK {model_type}: was in "
"KNOWN_BROKEN_COMPILE, now compiles cleanly. "
"Drop the entry."
)
ok += 1
# Came back green unexpectedly -- that's GOOD news,
# the bug was fixed. Surface it so we can drop the
# entry from KNOWN_BROKEN_COMPILE.
print(
f" UNEXPECTED-OK {model_type}: was in "
"KNOWN_BROKEN_COMPILE, now compiles cleanly. "
"Drop the entry."
)
ok += 1
finally:
signal.alarm(0)
signal.signal(signal.SIGALRM, prev_handler)
print(f"\nCompile sweep: ok={ok} skipped={len(skipped)} "
f"known-broken={len(known)} new-failures={len(new_failures)}")
for m, r in known:
@ -985,24 +1066,34 @@ jobs:
"""Spot-check on the three production-relevant families that
the compile_every sweep also covers; this case verifies the
emitted cache file has the model-specific RMSNorm class
attribute, not just that the file parses + imports."""
attribute, not just that the file parses + imports.
``unsloth_compile_transformers`` is not idempotent in-
process: calling it twice on the same modeling module
after rewriting class attributes corrupts the inspect
source/line cache and the second emitted file is malformed
Python. The sweep above already produced a valid cache
file for every non-KNOWN_BROKEN model_type, so just verify
that artefact here. Trigger a compile only when running
this test in isolation (no sweep preceded)."""
import importlib as _il
try:
_il.import_module(
modeling = _il.import_module(
f"transformers.models.{model_type}.modeling_{model_type}"
)
except ModuleNotFoundError:
pytest.skip(
f"transformers build lacks model_type={model_type}"
)
unsloth_compile_transformers(
model_type=model_type, fast_lora_forwards=False,
)
modeling = _il.import_module(
f"transformers.models.{model_type}.modeling_{model_type}"
)
assert getattr(modeling, "__UNSLOTH_PATCHED__", False) is True
combined = _CACHE / f"unsloth_compiled_module_{model_type}.py"
if not combined.exists():
unsloth_compile_transformers(
model_type=model_type, fast_lora_forwards=False,
)
modeling = _il.import_module(
f"transformers.models.{model_type}.modeling_{model_type}"
)
assert getattr(modeling, "__UNSLOTH_PATCHED__", False) is True
_verify_file(combined, must_expose=[rms_class])
@ -2013,6 +2104,8 @@ jobs:
UNSLOTH_IS_PRESENT: '1'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:

2
.github/workflows/lint-ci.yml vendored
View file

@ -44,6 +44,8 @@ jobs:
timeout-minutes: 5
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:

15
.github/workflows/mlx-ci.yml vendored
View file

@ -100,6 +100,8 @@ jobs:
egress-policy: audit
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -300,15 +302,10 @@ jobs:
"$LLAMA_QUANT" --help >/dev/null && echo " llama-quantize loads OK"
mkdir -p /tmp/ggufs
python -c "
from huggingface_hub import hf_hub_download
p = hf_hub_download(
'unsloth/gemma-3-270m-it-GGUF',
'gemma-3-270m-it-Q4_K_M.gguf',
local_dir = '/tmp/ggufs',
)
print('downloaded:', p)
"
bash .github/scripts/hf-download-with-retry.sh \
'unsloth/gemma-3-270m-it-GGUF' \
'gemma-3-270m-it-Q4_K_M.gguf' \
/tmp/ggufs
PORT=18080
echo "=== starting llama-server on 127.0.0.1:$PORT ==="

17
.github/workflows/notebooks-ci.yml vendored
View file

@ -88,6 +88,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: unsloth
persist-credentials: false
- name: Checkout unslothai/notebooks @ ${{ env.NOTEBOOKS_REF }}
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@ -96,6 +97,7 @@ jobs:
ref: ${{ env.NOTEBOOKS_REF }}
path: notebooks
fetch-depth: 0 # drift check needs git status / diff
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -196,12 +198,15 @@ jobs:
exit 1
fi
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with: { path: unsloth }
with:
persist-credentials: false
path: unsloth
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: unslothai/notebooks
ref: ${{ env.NOTEBOOKS_REF }}
path: notebooks
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: { python-version: '3.12', cache: 'pip' }
- name: Install
@ -239,12 +244,15 @@ jobs:
exit 1
fi
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with: { path: unsloth }
with:
persist-credentials: false
path: unsloth
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: unslothai/notebooks
ref: ${{ env.NOTEBOOKS_REF }}
path: notebooks
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: { python-version: '3.12', cache: 'pip' }
@ -342,12 +350,15 @@ jobs:
exit 1
fi
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with: { path: unsloth }
with:
persist-credentials: false
path: unsloth
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: unslothai/notebooks
ref: ${{ env.NOTEBOOKS_REF }}
path: notebooks
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: { python-version: '3.12' }

4
.github/workflows/release-desktop.yml vendored
View file

@ -36,6 +36,8 @@ jobs:
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
with:
persist-credentials: false
- name: Validate release versions
id: prepare
@ -343,6 +345,8 @@ jobs:
egress-policy: audit
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
with:
persist-credentials: false
# ── Linux dependencies ──
- name: Install Linux dependencies

14
.github/workflows/security-audit.yml vendored
View file

@ -127,6 +127,7 @@ jobs:
# Full history so TruffleHog can diff base..head; without
# this it sees only the latest commit and reports nothing.
fetch-depth: 0
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -136,8 +137,6 @@ jobs:
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable @ 2026-03-27
@ -722,6 +721,8 @@ jobs:
files.pythonhosted.org:443
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -893,6 +894,8 @@ jobs:
registry.npmjs.org:443
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -963,6 +966,8 @@ jobs:
files.pythonhosted.org:443
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -998,6 +1003,8 @@ jobs:
files.pythonhosted.org:443
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -1052,12 +1059,11 @@ jobs:
# Need the base commit accessible for `git show
# <base-sha>:studio/frontend/package-lock.json` below.
fetch-depth: 0
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:

9
.github/workflows/studio-api-smoke.yml vendored
View file

@ -51,6 +51,8 @@ jobs:
HF_HOME: ${{ github.workspace }}/hf-cache
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Linux deps
run: |
@ -61,8 +63,6 @@ jobs:
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -85,10 +85,9 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE"
- name: Save HF_HOME for ${{ env.GGUF_REPO }}
if: always() && steps.prime-hf.outcome == 'success'

4
.github/workflows/studio-backend-ci.yml vendored
View file

@ -53,6 +53,8 @@ jobs:
python: ['3.10', '3.11', '3.12', '3.13']
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -106,6 +108,8 @@ jobs:
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:

4
.github/workflows/studio-frontend-ci.yml vendored
View file

@ -36,6 +36,8 @@ jobs:
working-directory: studio/frontend
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
# FIXME: drop this step once @assistant-ui/* and assistant-stream
# leave 0.x -- on 1.x, caret ranges are conventional. Until then,
@ -55,8 +57,6 @@ jobs:
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
# Run the structural lockfile scan BEFORE npm ci. A compromised
# tarball runs its `prepare` / `postinstall` during `npm ci`,

30
.github/workflows/studio-inference-smoke.yml vendored
View file

@ -67,6 +67,8 @@ jobs:
HF_HOME: ${{ github.workspace }}/hf-cache
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Linux deps for llama.cpp prebuilt
run: |
@ -77,8 +79,6 @@ jobs:
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -99,10 +99,9 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE"
- name: Save HF_HOME for ${{ env.GGUF_REPO }}
if: always() && steps.prime-hf.outcome == 'success'
@ -315,6 +314,8 @@ jobs:
STUDIO_PORT: '18889'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Linux deps for llama.cpp prebuilt
run: |
@ -325,8 +326,6 @@ jobs:
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -347,10 +346,9 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p gguf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE" --local-dir gguf-cache
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE" gguf-cache
- name: Save GGUF model file
if: always() && steps.download-gguf.outcome == 'success'
@ -632,6 +630,8 @@ jobs:
HF_HOME: ${{ github.workspace }}/hf-cache
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Linux deps for llama.cpp prebuilt
run: |
@ -642,8 +642,6 @@ jobs:
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -664,12 +662,10 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$MMPROJ_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$MMPROJ_FILE"
- name: Save HF_HOME for ${{ env.GGUF_REPO }} (model + mmproj)
if: always() && steps.prime-hf.outcome == 'success'

9
.github/workflows/studio-mac-api-smoke.yml vendored
View file

@ -44,12 +44,12 @@ jobs:
HF_HOME: ${{ github.workspace }}/hf-cache
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -70,10 +70,9 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE"
- name: Save HF_HOME for ${{ env.GGUF_REPO }}
if: always() && steps.prime-hf.outcome == 'success'

101
.github/workflows/studio-mac-inference-smoke.yml vendored
View file

@ -4,8 +4,8 @@
# Three end-to-end smoke jobs that boot a freshly-installed Studio and
# exercise the surfaces real users hit through the OpenAI / Anthropic
# SDKs and curl. Each job picks the smallest model that exercises the
# behaviour under test, primes HF_HOME via actions/cache, and shares
# the install.sh --local --no-torch bootstrap.
# behaviour under test, primes a model cache via actions/cache, and
# shares the install.sh --local --no-torch bootstrap.
#
# 1. OpenAI, Anthropic API tests
# gemma-3-270m-it UD-Q4_K_XL (~254 MiB).
@ -40,7 +40,7 @@ on:
- '.github/workflows/studio-mac-inference-smoke.yml'
push:
branches: [main, pip]
# Manual trigger for pre-warming HF_HOME caches on main, or re-running
# Manual trigger for pre-warming model caches on main, or re-running
# against an arbitrary branch without pushing a no-op commit.
workflow_dispatch:
@ -67,12 +67,12 @@ jobs:
HF_HOME: ${{ github.workspace }}/hf-cache
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -93,13 +93,14 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE"
# Save partial caches on cancel/timeout -- hf download resumes by
# content hash. `outcome != skipped` keeps cache-hit a no-op.
- name: Save HF_HOME for ${{ env.GGUF_REPO }}
if: always() && steps.prime-hf.outcome == 'success'
if: always() && steps.prime-hf.outcome != 'skipped' && hashFiles('hf-cache/**/*.gguf') != ''
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: hf-cache
@ -315,12 +316,12 @@ jobs:
STUDIO_PORT: '18898'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -341,13 +342,13 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p gguf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE" --local-dir gguf-cache
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE" gguf-cache
# Save partial caches on cancel; next run resumes via content hash.
- name: Save GGUF model file
if: always() && steps.download-gguf.outcome == 'success'
if: always() && steps.download-gguf.outcome != 'skipped' && hashFiles('gguf-cache/**/*.gguf') != ''
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: gguf-cache
@ -677,57 +678,72 @@ jobs:
GGUF_FILE: gemma-4-E2B-it-UD-Q4_K_XL.gguf
MMPROJ_FILE: mmproj-F16.gguf
STUDIO_PORT: '18899'
HF_HOME: ${{ github.workspace }}/hf-cache
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
cache: 'pip'
- name: Restore HF_HOME for ${{ env.GGUF_REPO }} (model + mmproj)
id: cache-hf
# Cache flat .gguf + mmproj (Job 2's pattern). HF_HOME inflates
# ~3.6x via xet/blobs/snapshots, which made macOS saves never land.
# mmproj is auto-detected as a sibling via detect_mmproj_file
# (studio/backend/utils/models/model_config.py).
- name: Restore GGUF + mmproj files
id: cache-gguf
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
continue-on-error: true
with:
path: hf-cache
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-${{ env.MMPROJ_FILE }}-v1
path: gguf-cache
key: ${{ runner.os }}-gguf-${{ env.GGUF_REPO }}-${{ env.GGUF_FILE }}-${{ env.MMPROJ_FILE }}-v2
- name: Prime HF_HOME with the GGUF + mmproj
id: prime-hf
if: steps.cache-hf.outputs.cache-hit != 'true' || steps.cache-hf.outcome != 'success'
- name: Verify cache contains BOTH gguf + mmproj
id: verify-cache
if: steps.cache-gguf.outputs.cache-hit == 'true'
run: |
if [[ -f "gguf-cache/$GGUF_FILE" && -f "gguf-cache/$MMPROJ_FILE" ]]; then
echo "ok=true" >> "$GITHUB_OUTPUT"
else
echo "Partial cache hit -- forcing re-download."
echo "ok=false" >> "$GITHUB_OUTPUT"
fi
- name: Download GGUF + mmproj if cache miss or partial
id: download-gguf
if: steps.cache-gguf.outputs.cache-hit != 'true' || steps.verify-cache.outputs.ok != 'true'
# Authenticated + parallel: shared macos-14 NAT egress stalls
# multi-GB anonymous downloads.
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE" &
python -m pip install --upgrade huggingface_hub
mkdir -p gguf-cache
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE" gguf-cache &
MODEL_PID=$!
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$MMPROJ_FILE" &
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$MMPROJ_FILE" gguf-cache &
MMPROJ_PID=$!
wait "$MODEL_PID"
wait "$MMPROJ_PID"
# Fail loud on a partial download instead of in the next step.
find hf-cache -name "$GGUF_FILE" -o -name "$MMPROJ_FILE" \
| xargs -I{} ls -lhL {}
ls -lh "gguf-cache/$GGUF_FILE" "gguf-cache/$MMPROJ_FILE"
- name: Save HF_HOME for ${{ env.GGUF_REPO }} (model + mmproj)
if: always() && steps.prime-hf.outcome == 'success'
# Save partial caches on cancel. hashFiles guard avoids a hard
# save failure when the download step exits with no files. The
# additional mmproj-presence check stops a partial save from
# poisoning the cache for the next run.
- name: Save GGUF + mmproj files
if: always() && steps.download-gguf.outcome != 'skipped' && hashFiles('gguf-cache/**/*.gguf') != '' && hashFiles(format('gguf-cache/{0}', env.MMPROJ_FILE)) != ''
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: hf-cache
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-${{ env.MMPROJ_FILE }}-v1
path: gguf-cache
key: ${{ runner.os }}-gguf-${{ env.GGUF_REPO }}-${{ env.GGUF_FILE }}-${{ env.MMPROJ_FILE }}-v2
- name: Install Studio (--local, --no-torch)
env:
@ -782,12 +798,17 @@ jobs:
-H 'content-type: application/json' \
-d "{\"username\":\"unsloth\",\"password\":\"$NEW\"}" | jq -r .access_token)
echo "API_KEY=$TOKEN" >> "$GITHUB_ENV"
# Load the GGUF (mmproj is auto-detected via the HF repo
# lookup, the cached file is pulled out of HF_HOME).
# Load via local file path; mmproj sibling auto-detected by
# detect_mmproj_file (model_config.py). gguf_variant omitted
# -- it routes through _find_local_gguf_by_variant which
# expects a directory, not a file path.
GGUF_PATH="$GITHUB_WORKSPACE/gguf-cache/${GGUF_FILE}"
MMPROJ_PATH="$GITHUB_WORKSPACE/gguf-cache/${MMPROJ_FILE}"
ls -lh "$GGUF_PATH" "$MMPROJ_PATH"
curl -fs -X POST "http://127.0.0.1:${STUDIO_PORT}/api/inference/load" \
-H "Authorization: Bearer $TOKEN" -H 'content-type: application/json' \
--max-time 900 \
-d "{\"model_path\":\"$GGUF_REPO\",\"gguf_variant\":\"$GGUF_VARIANT\",\"is_lora\":false,\"max_seq_length\":2048}" \
-d "{\"model_path\":\"$GGUF_PATH\",\"is_lora\":false,\"max_seq_length\":2048}" \
| jq '{status, display_name, is_vision}'
- name: JSON schema decoding + image input

9
.github/workflows/studio-mac-ui-smoke.yml vendored
View file

@ -44,12 +44,12 @@ jobs:
HF_HOME: ${{ github.workspace }}/hf-cache
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -70,10 +70,9 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE"
- name: Save HF_HOME for ${{ env.GGUF_REPO }}
if: always() && steps.prime-hf.outcome == 'success'

4
.github/workflows/studio-mac-update-smoke.yml vendored
View file

@ -46,12 +46,12 @@ jobs:
timeout-minutes: 30
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:

4
.github/workflows/studio-tauri-smoke.yml vendored
View file

@ -40,6 +40,8 @@ jobs:
timeout-minutes: 25
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Linux native deps for Tauri / WebKit2GTK
run: |
@ -51,8 +53,6 @@ jobs:
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '24'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable @ 2026-03-27

9
.github/workflows/studio-ui-smoke.yml vendored
View file

@ -52,6 +52,8 @@ jobs:
HF_HOME: ${{ github.workspace }}/hf-cache
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Linux deps
run: |
@ -62,8 +64,6 @@ jobs:
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -84,10 +84,9 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE"
- name: Save HF_HOME for ${{ env.GGUF_REPO }}
if: always() && steps.prime-hf.outcome == 'success'

4
.github/workflows/studio-update-smoke.yml vendored
View file

@ -40,6 +40,8 @@ jobs:
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Linux deps for llama.cpp prebuilt
run: |
@ -50,8 +52,6 @@ jobs:
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:

9
.github/workflows/studio-windows-api-smoke.yml vendored
View file

@ -52,12 +52,12 @@ jobs:
PYTHONUTF8: '1'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -77,10 +77,9 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE"
- name: Save HF_HOME for ${{ env.GGUF_REPO }}
if: always() && steps.prime-hf.outcome == 'success'

60
.github/workflows/studio-windows-inference-smoke.yml vendored
View file

@ -62,12 +62,12 @@ jobs:
PYTHONUTF8: '1'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -99,10 +99,9 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE"
- name: Save HF_HOME cache for ${{ env.GGUF_REPO }}
# Only write a fresh cache entry when we actually rebuilt the
@ -343,9 +342,13 @@ jobs:
- name: Stop Studio
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true
sleep 2
# Run as cmd so we are not running through the Git Bash shell;
# Git Bash on windows-latest has been observed to exit 143
# (SIGTERM) from any inline kill/sleep block, masking a green
# test run. The runner reclaims the Studio child process at
# job end either way, so just emit a marker and exit 0.
shell: cmd
run: echo Stop Studio (no-op; runner reclaims STUDIO_PID=%STUDIO_PID% at job end)
- name: Upload logs
if: always()
@ -388,12 +391,12 @@ jobs:
PYTHONUTF8: '1'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -416,10 +419,9 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p gguf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE" --local-dir gguf-cache
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE" gguf-cache
- name: Save GGUF model cache
if: always() && steps.download-gguf.outcome == 'success'
@ -758,9 +760,13 @@ jobs:
- name: Stop Studio
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true
sleep 2
# Run as cmd so we are not running through the Git Bash shell;
# Git Bash on windows-latest has been observed to exit 143
# (SIGTERM) from any inline kill/sleep block, masking a green
# test run. The runner reclaims the Studio child process at
# job end either way, so just emit a marker and exit 0.
shell: cmd
run: echo Stop Studio (no-op; runner reclaims STUDIO_PID=%STUDIO_PID% at job end)
- name: Upload logs
if: always()
@ -796,12 +802,12 @@ jobs:
PYTHONUTF8: '1'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -827,12 +833,10 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$MMPROJ_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$MMPROJ_FILE"
- name: Save HF_HOME cache for ${{ env.GGUF_REPO }} (model + mmproj)
if: always() && steps.prime-hf.outcome == 'success'
@ -1144,9 +1148,13 @@ jobs:
- name: Stop Studio
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true
sleep 2
# Run as cmd so we are not running through the Git Bash shell;
# Git Bash on windows-latest has been observed to exit 143
# (SIGTERM) from any inline kill/sleep block, masking a green
# test run. The runner reclaims the Studio child process at
# job end either way, so just emit a marker and exit 0.
shell: cmd
run: echo Stop Studio (no-op; runner reclaims STUDIO_PID=%STUDIO_PID% at job end)
- name: Upload logs
if: always()

16
.github/workflows/studio-windows-ui-smoke.yml vendored
View file

@ -57,12 +57,19 @@ jobs:
PYTHONUTF8: '1'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
# No `cache: 'npm'`. setup-node's npm cache restore silently
# aborts the entire job on Windows runners when the npm cache
# path (`C:\npm\cache` per `npm config get cache`) doesn't yet
# exist on a fresh runner -- the step exits without an error
# message and every following step gets skipped. See
# npm/cli#7308. The frontend `npm ci` is fast enough without
# the cache that the reliability gain is worth the ~30s.
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@ -86,10 +93,9 @@ jobs:
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub hf_transfer
python -m pip install --upgrade huggingface_hub
mkdir -p hf-cache
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download "$GGUF_REPO" "$GGUF_FILE"
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE"
- name: Save HF_HOME for ${{ env.GGUF_REPO }}
if: always() && steps.prime-hf.outcome == 'success'

4
.github/workflows/studio-windows-update-smoke.yml vendored
View file

@ -58,12 +58,12 @@ jobs:
PYTHONUTF8: '1'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:

18
.github/workflows/version-compat-ci.yml vendored
View file

@ -58,6 +58,8 @@ jobs:
timeout-minutes: 12
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
@ -83,6 +85,8 @@ jobs:
timeout-minutes: 10
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
@ -107,6 +111,8 @@ jobs:
timeout-minutes: 8
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
@ -129,6 +135,8 @@ jobs:
timeout-minutes: 8
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
@ -151,6 +159,8 @@ jobs:
timeout-minutes: 8
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
@ -173,6 +183,8 @@ jobs:
timeout-minutes: 12
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
@ -200,7 +212,9 @@ jobs:
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with: { path: unsloth }
with:
persist-credentials: false
path: unsloth
- name: Clone unsloth-zoo @ main
run: |
# github.com occasionally 500s on the git fetch; retry so a
@ -279,6 +293,8 @@ jobs:
timeout-minutes: 20
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'

4
.github/workflows/wheel-smoke.yml vendored
View file

@ -42,12 +42,12 @@ jobs:
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:

247
scripts/verify_comment_only_diff.py Normal file
View file

@ -0,0 +1,247 @@
# Unsloth - 2x faster, 60% less VRAM LLM training and finetuning
# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Lesser General Public License for more details.
"""Deterministic comment / docstring-only verifier.
Compares a list of changed files between two git refs and reports whether
each diff is strictly comments / docstrings (Python) or comments
(YAML / GitHub Actions). Useful for gating a "comment trim" /
"docstring refactor" PR against accidental code drift.
Per .py file: parse both revs into AST, strip module / class / function
docstrings, then compare ast.unparse output. Pure Python comments are
discarded by the parser by construction, so any post-strip diff is real
code. Per .yml file: yaml.safe_load both sides and compare the parsed
Python object; if scalar values differ, also strip shell comments inside
``run: |`` block bodies before comparing. Exit code 0 = all OK, 1 = at
least one file has a real (non-comment) diff or an error.
Usage:
python scripts/verify_comment_only_diff.py [--base REF] [--head REF] path ...
Defaults: --base origin/main, --head HEAD. Paths are repo-relative.
Example:
git diff --name-only origin/main..HEAD \\
| xargs python scripts/verify_comment_only_diff.py --base origin/main
"""
from __future__ import annotations
import argparse
import ast
import difflib
import subprocess
import sys
from typing import Any
import yaml
def _git_show(rev: str, path: str) -> str:
return subprocess.check_output(
["git", "show", f"{rev}:{path}"],
text = True,
stderr = subprocess.DEVNULL,
)
def _strip_docstrings(tree: ast.AST) -> ast.AST:
"""Remove every string-literal docstring (Module / FunctionDef /
AsyncFunctionDef / ClassDef). Empty body becomes ``pass`` so
ast.unparse stays valid."""
for node in ast.walk(tree):
if isinstance(
node,
(ast.Module, ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef),
):
body = getattr(node, "body", None)
if not body:
continue
first = body[0]
if (
isinstance(first, ast.Expr)
and isinstance(first.value, ast.Constant)
and isinstance(first.value.value, str)
):
node.body = body[1:]
if not node.body:
node.body = [ast.Pass()]
return tree
def _normalize_py(src: str) -> str:
tree = ast.parse(src)
tree = _strip_docstrings(tree)
return ast.unparse(tree)
def _strip_shell_comments(s: str) -> str:
"""Strip pure-comment lines and inline trailing comments from a shell
snippet, then collapse runs of blank lines. Heuristic only: leaves a
line untouched if it has an odd quote count (open string)."""
out = []
for line in s.splitlines():
stripped = line.lstrip()
if stripped.startswith("#"):
continue
has_single = line.count("'") % 2 == 0
has_double = line.count('"') % 2 == 0
if has_single and has_double:
idx = line.find(" #")
if idx >= 0:
line = line[:idx].rstrip()
out.append(line)
norm = []
prev_blank = False
for line in out:
if line.strip() == "":
if prev_blank:
continue
prev_blank = True
else:
prev_blank = False
norm.append(line)
return "\n".join(norm).strip()
def _normalize_yaml_run_strings(obj: Any) -> Any:
"""Walk the parsed YAML object; for any multi-line string (i.e. a
``run: |`` script body), strip shell comments. Returns a normalised
copy."""
if isinstance(obj, dict):
return {k: _normalize_yaml_run_strings(v) for k, v in obj.items()}
if isinstance(obj, list):
return [_normalize_yaml_run_strings(x) for x in obj]
if isinstance(obj, str) and "\n" in obj:
return _strip_shell_comments(obj)
return obj
def _walk_yaml_diff(b: Any, a: Any, prefix: str = "") -> None:
"""Print a path-keyed summary of the first structural / scalar diff."""
if type(b) is not type(a):
print(
f" type-diff at {prefix or '/'}: "
f"{type(b).__name__} -> {type(a).__name__}",
)
return
if isinstance(b, dict):
keys = sorted((set(b.keys()) | set(a.keys())), key = lambda x: str(x))
for k in keys:
if k not in b:
print(f" added key {prefix}/{k}")
elif k not in a:
print(f" removed key {prefix}/{k}")
else:
_walk_yaml_diff(b[k], a[k], f"{prefix}/{k}")
elif isinstance(b, list):
if len(b) != len(a):
print(
f" list len at {prefix or '/'}: " f"{len(b)} -> {len(a)}",
)
for i, (bi, ai) in enumerate(zip(b, a)):
_walk_yaml_diff(bi, ai, f"{prefix}[{i}]")
elif b != a:
bs = repr(b)[:300]
as_ = repr(a)[:300]
print(f" scalar at {prefix or '/'}:")
print(f" before: {bs}")
print(f" after: {as_}")
def _verify_python(path: str, before: str, after: str) -> bool:
try:
norm_before = _normalize_py(before)
norm_after = _normalize_py(after)
except SyntaxError as exc:
print(f"FAIL {path}: SyntaxError parsing -- {exc}")
return False
if norm_before == norm_after:
print(f"OK {path} (AST identical after docstring strip)")
return True
diff = list(
difflib.unified_diff(
norm_before.splitlines(),
norm_after.splitlines(),
fromfile = f"{path}@before",
tofile = f"{path}@after",
n = 2,
)
)
print(f"FAIL {path}: AST differs after docstring strip:")
for line in diff[:40]:
print(f" {line}")
return False
def _verify_yaml(path: str, before: str, after: str) -> bool:
try:
raw_before = yaml.safe_load(before)
raw_after = yaml.safe_load(after)
except yaml.YAMLError as exc:
print(f"FAIL {path}: YAML parse error -- {exc}")
return False
if raw_before == raw_after:
print(f"OK {path} (YAML parsed object identical)")
return True
norm_before = _normalize_yaml_run_strings(raw_before)
norm_after = _normalize_yaml_run_strings(raw_after)
if norm_before == norm_after:
print(
f"OK {path} (YAML parsed object identical after "
f"stripping shell comments from run: bodies)",
)
return True
print(
f"FAIL {path}: YAML parsed objects still differ after stripping "
f"shell comments from `run:` bodies.",
)
_walk_yaml_diff(norm_before, norm_after)
return False
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
description = "Verify each path's diff between BASE and HEAD is "
"strictly comments / docstrings.",
)
parser.add_argument("--base", default = "origin/main", help = "base git ref")
parser.add_argument("--head", default = "HEAD", help = "head git ref")
parser.add_argument("paths", nargs = "+", help = "repo-relative paths")
args = parser.parse_args(argv)
rc = 0
print(f"Comparing {len(args.paths)} files: {args.base} vs {args.head}\n")
for path in args.paths:
try:
before = _git_show(args.base, path)
after = _git_show(args.head, path)
except subprocess.CalledProcessError as exc:
print(f"SKIP {path}: {exc}")
continue
if path.endswith(".py"):
if not _verify_python(path, before, after):
rc = 1
elif path.endswith((".yml", ".yaml")):
if not _verify_yaml(path, before, after):
rc = 1
else:
print(f"NOTE {path}: not .py or .yaml -- skipped automated check.")
return rc
if __name__ == "__main__":
sys.exit(main())

2973
studio/backend/core/inference/external_provider.py Normal file
View file

File diff suppressed because it is too large Load diff

127
studio/backend/core/inference/key_exchange.py Normal file
View file

@ -0,0 +1,127 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
RSA key pair for encrypting API keys in transit.
The frontend encrypts API keys with the server's public key before
including them in requests. The backend decrypts with its private key
before forwarding to external providers.
The key pair is generated at server startup and lives only in memory —
it is regenerated on each restart. The frontend fetches the public key
via GET /api/providers/public-key on load.
"""
import base64
import hashlib
import logging
from cryptography.hazmat.primitives.asymmetric import rsa, padding
from cryptography.hazmat.primitives import serialization, hashes
logger = logging.getLogger(__name__)
_private_key: rsa.RSAPrivateKey | None = None
_public_key_pem: str | None = None
_public_key_fingerprint: str | None = None
def _compute_fingerprint(pem: str) -> str:
"""SHA256 of the PEM bytes, truncated for log compactness."""
return hashlib.sha256(pem.encode("utf-8")).hexdigest()[:16]
def init_key_pair() -> None:
"""Generate an RSA-2048 key pair. Called once at server startup."""
global _private_key, _public_key_pem, _public_key_fingerprint
if _private_key is not None:
# Re-entry is suspicious — every fresh keypair invalidates all
# in-flight ciphertext encrypted against the previous public key.
# Log loudly so a regression that calls init twice is visible.
logger.warning(
"init_key_pair called again — replacing existing RSA keypair "
"(previous fingerprint=%s). Any frontend that cached the old "
"public key will start hitting decryption failures.",
_public_key_fingerprint,
)
_private_key = rsa.generate_private_key(
public_exponent = 65537,
key_size = 2048,
)
_public_key_pem = (
_private_key.public_key()
.public_bytes(
serialization.Encoding.PEM,
serialization.PublicFormat.SubjectPublicKeyInfo,
)
.decode("utf-8")
)
_public_key_fingerprint = _compute_fingerprint(_public_key_pem)
logger.info(
"RSA key pair generated for API key encryption (fingerprint=%s)",
_public_key_fingerprint,
)
def get_public_key_fingerprint() -> str | None:
"""Short SHA256 of the current public key PEM; None before init."""
return _public_key_fingerprint
def get_public_key_pem() -> str:
"""Return the PEM-encoded public key for the frontend."""
if _public_key_pem is None:
raise RuntimeError("Key pair not initialized. Call init_key_pair() first.")
return _public_key_pem
def decrypt_api_key(encrypted_b64: str) -> str:
"""
Decrypt an API key that was encrypted with the public key.
Args:
encrypted_b64: Base64-encoded RSA-OAEP ciphertext.
Returns:
The plaintext API key string.
"""
if _private_key is None:
raise RuntimeError("Key pair not initialized. Call init_key_pair() first.")
try:
ciphertext = base64.b64decode(encrypted_b64)
except Exception as exc:
logger.warning(
"decrypt_api_key: base64 decode failed (input_len=%d, fingerprint=%s): %s: %s",
len(encrypted_b64),
_public_key_fingerprint,
type(exc).__name__,
exc,
)
raise
try:
plaintext = _private_key.decrypt(
ciphertext,
padding.OAEP(
mgf = padding.MGF1(algorithm = hashes.SHA256()),
algorithm = hashes.SHA256(),
label = None,
),
)
except Exception as exc:
# Surface enough state to distinguish key mismatch (wrong public key
# used on encrypt) from a padding/algo mismatch or corrupted bytes.
# Expected ciphertext length for RSA-2048 is exactly 256 bytes.
logger.warning(
"decrypt_api_key: RSA decrypt failed (ciphertext_len=%d, expected=256, "
"fingerprint=%s, exc=%s): %s",
len(ciphertext),
_public_key_fingerprint,
type(exc).__name__,
exc,
)
raise
return plaintext.decode("utf-8")

18
studio/backend/core/inference/llama_cpp.py
View file

@ -2367,8 +2367,20 @@ class LlamaCppBackend:
if not Path(mmproj_path).is_file():
logger.warning(f"mmproj file not found: {mmproj_path}")
else:
cmd.extend(["--mmproj", mmproj_path])
logger.info(f"Using mmproj for vision: {mmproj_path}")
# #5347 guard for paths that bypass detect_mmproj_file.
from utils.models.model_config import (
mmproj_matches_model_family,
)
if not mmproj_matches_model_family(model_path, mmproj_path):
logger.warning(
f"Skipping mmproj with mismatched family: "
f"model={Path(model_path).name}, "
f"mmproj={Path(mmproj_path).name}"
)
else:
cmd.extend(["--mmproj", mmproj_path])
logger.info(f"Using mmproj for vision: {mmproj_path}")
# Option C: add --api-key for direct client access when enabled
import os as _os
@ -3747,7 +3759,7 @@ class LlamaCppBackend:
except json.JSONDecodeError:
logger.debug(
f"Skipping malformed SSE line: " f"{line[:100]}"
f"Skipping malformed SSE line: {line[:100]}"
)
if _stream_done:
break # exit outer for

4
studio/backend/core/inference/mlx_inference.py
View file

@ -116,11 +116,11 @@ class MLXInferenceBackend:
)
try:
from unsloth_zoo.mlx_loader import FastMLXModel
from unsloth_zoo.mlx.loader import FastMLXModel
except ImportError as e:
raise ImportError(
"Unsloth: MLX inference requires unsloth-zoo with the MLX modules "
"(unsloth_zoo.mlx_loader). Reinstall via install.sh on Apple Silicon."
"(unsloth_zoo.mlx.loader). Reinstall via install.sh on Apple Silicon."
) from e
model, tokenizer_or_processor = FastMLXModel.from_pretrained(

317
studio/backend/core/inference/providers.py Normal file
View file

@ -0,0 +1,317 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Static registry of supported external LLM providers.
All providers expose OpenAI-compatible /v1/chat/completions endpoints
with Bearer token authentication and SSE streaming support.
"""
import re
from typing import Any
PROVIDER_REGISTRY: dict[str, dict[str, Any]] = {
"openai": {
"display_name": "OpenAI",
"base_url": "https://api.openai.com/v1",
"default_models": [
"gpt-5.5",
"gpt-5.4",
"gpt-5.4-mini",
"o3",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
# Keep the model picker scoped to the current generation. The remote
# /v1/models listing returns dozens of historical snapshots, fine-tunes
# and non-chat models (embeddings, TTS, image, moderation) that we
# never want to surface in the chat UI. Filtering here so backend
# is the single source of truth.
"model_id_allowlist": re.compile(r"^(gpt-5\.[345]|gpt-4\.5|o3)(?:[-.]|$)"),
# Hide dated snapshots and the retired plain gpt-5.3 id.
"model_id_denylist": re.compile(r"^(gpt-5\.3)$|-\d{4}-\d{2}-\d{2}$"),
},
"anthropic": {
"display_name": "Anthropic",
"base_url": "https://api.anthropic.com/v1",
"default_models": [
"claude-opus-4-7",
"claude-opus-4-6",
"claude-sonnet-4-6",
"claude-opus-4-5",
"claude-sonnet-4-5",
"claude-haiku-4-5",
],
# Anthropic /v1/models returns dated snapshot ids alongside the
# canonical names (e.g. claude-3-5-sonnet-20241022). Hide the
# YYYYMMDD-suffixed variants from the picker — same intent as the
# OpenAI denylist, just a different date format (no dashes between
# year/month/day).
"model_id_denylist": re.compile(r"-\d{8}$"),
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": False,
"auth_header": "x-api-key",
"auth_prefix": "",
"extra_headers": {
"anthropic-version": "2023-06-01",
},
"openai_compatible": False,
"notes": "Native Anthropic Messages API. Uses x-api-key header and /v1/messages endpoint with SSE translation.",
},
"gemini": {
"display_name": "Google Gemini",
"base_url": "https://generativelanguage.googleapis.com/v1beta/openai",
# Curated lineup — Google's /v1beta/openai/models returns dozens
# of historical / experimental / embedding ids. Cap to the current
# 3.x family plus the rolling `*-latest` aliases.
"default_models": [
"gemini-3.1-pro-preview",
"gemini-3.1-flash-lite",
"gemini-3-flash-preview",
"gemini-pro-latest",
"gemini-flash-latest",
"gemini-flash-lite-latest",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"notes": "OpenAI-compatible endpoint. API key from https://aistudio.google.com/apikey.",
"model_id_allowlist": re.compile(
r"^(gemini-3\.1-flash-lite|gemini-3-flash-preview|"
r"gemini-3\.1-pro-preview|gemini-pro-latest|"
r"gemini-flash-latest|gemini-flash-lite-latest)$"
),
},
"deepseek": {
"display_name": "DeepSeek",
"base_url": "https://api.deepseek.com/v1",
"default_models": [
"deepseek-chat",
"deepseek-reasoner",
],
"supports_streaming": True,
"supports_vision": False,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"notes": "OpenAI-compatible API. deepseek-chat = V3, deepseek-reasoner = R1 thinking mode.",
},
"mistral": {
"display_name": "Mistral AI",
"base_url": "https://api.mistral.ai/v1",
"default_models": [
"codestral-latest",
"devstral-latest",
"devstral-medium-latest",
"magistral-medium-latest",
"ministral-14b-latest",
"ministral-3b-latest",
"ministral-8b-latest",
"mistral-large-latest",
"mistral-medium-latest",
"mistral-small-latest",
"mistral-tiny-latest",
"mistral-vibe-cli-latest",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"model_id_allowlist": re.compile(
r"^(codestral-latest|devstral-latest|devstral-medium-latest|"
r"magistral-medium-latest|ministral-(?:14b|3b|8b)-latest|"
r"mistral-(?:large|medium|small|tiny)-latest|"
r"mistral-vibe-cli-latest)$"
),
},
"kimi": {
"display_name": "Kimi",
"base_url": "https://api.moonshot.ai/v1",
# Current Kimi model lineup per the official docs:
# https://platform.kimi.ai/docs/models
# Listing/overview endpoints used to enumerate them:
# https://platform.kimi.ai/docs/api/list-models
# https://platform.kimi.ai/docs/api/overview
# kimi-k2.6 and kimi-k2.5 are the two SoTA multimodal models we
# surface in the picker; everything else (moonshot-v1-*, dated
# k2 previews) is filtered out by model_id_allowlist below.
"default_models": [
"kimi-k2.6",
"kimi-k2.5",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"notes": "Moonshot API key. China: use base URL https://api.moonshot.cn/v1",
"model_id_allowlist": re.compile(r"^kimi-k2\.[56]$"),
# Both k2.6 and k2.5 are reasoning-class. The API rejects custom
# sampling: "invalid temperature: only 1 is allowed for this model"
# (and the same shape for top_p). Strip both fields from the
# outbound body so the server falls back to its required defaults.
"body_omit": ("temperature", "top_p"),
},
"qwen": {
"display_name": "Qwen",
"base_url": "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
"default_models": [
"qwen-plus",
"qwen-turbo",
"qwen-max",
"qwen2.5-72b-instruct",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"notes": "DashScope API key. China mainland: override base URL to https://dashscope.aliyuncs.com/compatible-mode/v1",
},
"huggingface": {
"display_name": "Hugging Face",
"base_url": "https://router.huggingface.co/v1",
# Seed the picker with a few popular ids so something is selectable
# before the live /v1/models call resolves. The remote listing is
# the source of truth — see model_list_mode below.
"default_models": [
"openai/gpt-oss-120b",
"deepseek-ai/DeepSeek-V3",
"meta-llama/Llama-3.3-70B-Instruct",
"Qwen/Qwen2.5-72B-Instruct",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"notes": (
"HF token from huggingface.co/settings/tokens. Uses the "
"OpenAI-compatible router at /v1/chat/completions; /v1/models "
"returns the cross-provider chat catalog. See "
"https://huggingface.co/docs/inference-providers/index."
),
# /v1/models works on the HF router and returns the full chat-model
# catalog (state.org/model[:policy] ids). Switch to remote so users
# see live availability — the picker has a search box, and
# loadModels() merges defaults so default_models entries remain
# visible if the remote call fails.
"model_list_mode": "remote",
# Scope the catalog to first-party org repos we trust as primary
# sources. The HF /v1/models response is otherwise hundreds of
# ids long (community fine-tunes, mirrors, fp8 variants, etc.).
"model_id_allowlist": re.compile(
r"^(openai|deepseek-ai|google|meta-llama|Qwen|moonshotai|"
r"mistralai|zai-org)/"
),
# Cap the post-filter list. /v1/models has no server-side limit
# or popularity sort, so this is just "first N matches" — pair it
# with the default_models seed so the most useful flagship ids
# are always among the top regardless of the API's order.
"model_id_limit": 15,
},
"vllm": {
"display_name": "vLLM",
# User-supplied via provider_base_url; the route layer already falls
# back to the payload's base_url when the registry entry has none.
"base_url": "",
"default_models": [],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
# Force /v1/chat/completions in stream_chat_completion — vLLM's
# /v1/responses rebuilds messages and runs them through the loaded
# model's chat template, which 400s on strict-alternation templates
# (Gemma 3 raises "Conversation roles must alternate user/assistant
# /user/assistant/..."). The chat-completions path takes messages
# verbatim and avoids that template gauntlet.
"notes": "Self-hosted vLLM server. Always routed to /v1/chat/completions.",
# Surfaced through the frontend's CUSTOM_PROVIDER_PRESETS, not the
# /api/providers/registry dropdown — see list_available_providers.
"hidden": True,
},
"openrouter": {
"display_name": "OpenRouter",
"base_url": "https://openrouter.ai/api/v1",
# Curated list for Studio's picker (explicitly locked, not live /models).
"default_models": [
"openrouter/free",
"openai/gpt-4o",
"anthropic/claude-sonnet-4-5",
"google/gemini-2.5-flash",
"mistralai/mistral-large-2411",
"deepseek/deepseek-r1",
"mistralai/mistral-small-3.1-24b-instruct",
"perceptron/perceptron-mk1",
"inclusionai/ring-2.6-1t:free",
"google/gemini-3.1-flash-lite",
"baidu/cobuddy:free",
"openai/gpt-chat-latest",
"x-ai/grok-4.3",
"ibm-granite/granite-4.1-8b",
"openrouter/owl-alpha",
"poolside/laguna-xs.2:free",
"~google/gemini-pro-latest",
"~moonshotai/kimi-latest",
],
"supports_streaming": True,
"supports_vision": True,
"supports_tool_calling": True,
"auth_header": "Authorization",
"auth_prefix": "Bearer ",
"extra_headers": {
"HTTP-Referer": "https://unsloth.ai",
"X-Title": "Unsloth Studio",
},
"notes": "Unified gateway to 300+ models across all major providers. HTTP-Referer and X-Title headers sent for attribution.",
"model_list_mode": "curated",
},
}
def get_provider_info(provider_type: str) -> dict[str, Any] | None:
"""Return the registry entry for a provider type, or None if unknown."""
return PROVIDER_REGISTRY.get(provider_type)
def get_base_url(provider_type: str) -> str | None:
"""Return the default base URL for a provider type."""
info = PROVIDER_REGISTRY.get(provider_type)
return info["base_url"] if info else None
def list_available_providers() -> list[dict[str, Any]]:
"""Return all registered providers (for the /registry endpoint).
Hidden entries (``"hidden": True``) are filtered out — they exist in the
registry only for backend lookups (e.g. ``supports_vision`` for vLLM) and
are surfaced in the frontend via ``CUSTOM_PROVIDER_PRESETS`` instead of
the cloud-provider dropdown.
"""
result = []
for provider_type, info in PROVIDER_REGISTRY.items():
if info.get("hidden"):
continue
result.append(
{
"provider_type": provider_type,
"display_name": info["display_name"],
"base_url": info["base_url"],
"default_models": info["default_models"],
"supports_streaming": info["supports_streaming"],
"supports_vision": info.get("supports_vision", False),
"supports_tool_calling": info.get("supports_tool_calling", False),
"model_list_mode": info.get("model_list_mode", "remote"),
}
)
return result

3
studio/backend/core/training/trainer.py
View file

@ -3208,6 +3208,9 @@ class UnslothTrainer:
if eval_steps_val > 0:
config_args["eval_strategy"] = "steps"
config_args["eval_steps"] = eval_steps_val
config_args["per_device_eval_batch_size"] = config_args[
"per_device_train_batch_size"
]
logger.info(
f"✅ Evaluation enabled: eval_steps={eval_steps_val} (fraction of total steps)\n"
)

1
studio/backend/core/training/training.py
View file

@ -219,6 +219,7 @@ class TrainingBackend:
"max_steps": kwargs.get("max_steps", 0),
"save_steps": kwargs.get("save_steps", 0),
"weight_decay": kwargs.get("weight_decay", 0.001),
"max_grad_norm": kwargs.get("max_grad_norm", 0.0),
"random_seed": kwargs.get("random_seed", 3407),
"packing": kwargs.get("packing", False),
"optim": kwargs.get("optim", "adamw_8bit"),

96
studio/backend/core/training/worker.py
View file

@ -30,6 +30,7 @@ from utils.hardware import apply_gpu_ids
from utils.wheel_utils import (
direct_wheel_url,
flash_attn_wheel_url,
has_blackwell_gpu,
install_wheel,
probe_torch_wheel_env,
url_exists,
@ -313,6 +314,12 @@ def _should_try_runtime_flash_attn_install(max_seq_length: int) -> bool:
def _ensure_flash_attn_for_long_context(event_queue: Any, max_seq_length: int) -> None:
if not _should_try_runtime_flash_attn_install(max_seq_length):
return
if has_blackwell_gpu():
_send_status(
event_queue,
"Skipping flash-attn install: Blackwell GPU detected (sm_100+); no compatible prebuilt wheel",
)
return
installed = _install_package_wheel_first(
event_queue = event_queue,
@ -417,6 +424,55 @@ def _normalize_mlx_studio_scheduler(value):
return raw
def _resolve_mlx_local_dataset_files(file_paths: list) -> list[str]:
"""Resolve Studio local dataset uploads without importing the GPU trainer."""
from utils.paths import resolve_dataset_path
all_files: list[str] = []
for dataset_file in file_paths or []:
file_path = (
dataset_file
if os.path.isabs(dataset_file)
else str(resolve_dataset_path(dataset_file))
)
file_path_obj = Path(file_path)
if file_path_obj.is_dir():
parquet_dir = (
file_path_obj / "parquet-files"
if (file_path_obj / "parquet-files").exists()
else file_path_obj
)
parquet_files = sorted(parquet_dir.glob("*.parquet"))
if parquet_files:
all_files.extend(str(p) for p in parquet_files)
continue
candidates: list[Path] = []
for ext in (".json", ".jsonl", ".csv", ".parquet"):
candidates.extend(sorted(file_path_obj.glob(f"*{ext}")))
if candidates:
all_files.extend(str(c) for c in candidates)
continue
raise ValueError(f"No supported data files in directory: {file_path_obj}")
all_files.append(str(file_path_obj))
return all_files
def _mlx_local_dataset_loader_for_files(files: list[str]) -> str:
first_ext = Path(files[0]).suffix.lower()
if first_ext in (".json", ".jsonl"):
return "json"
if first_ext == ".csv":
return "csv"
if first_ext == ".parquet":
return "parquet"
raise ValueError(f"Unsupported dataset format: {files[0]}")
def _run_mlx_training(event_queue, stop_queue, config):
"""Self-contained MLX training path for Apple Silicon.
@ -442,8 +498,8 @@ def _run_mlx_training(event_queue, stop_queue, config):
import mlx.core as mx
try:
from unsloth_zoo.mlx_loader import FastMLXModel
from unsloth_zoo.mlx_trainer import (
from unsloth_zoo.mlx.loader import FastMLXModel
from unsloth_zoo.mlx.trainer import (
MLXTrainer,
MLXTrainingConfig,
train_on_responses_only,
@ -451,7 +507,7 @@ def _run_mlx_training(event_queue, stop_queue, config):
except ImportError as e:
raise ImportError(
"Unsloth: MLX training requires unsloth-zoo with the MLX modules "
"(unsloth_zoo.mlx_loader / unsloth_zoo.mlx_trainer). Reinstall via "
"(unsloth_zoo.mlx.loader / unsloth_zoo.mlx.trainer). Reinstall via "
"install.sh on Apple Silicon."
) from e
from datasets import load_dataset
@ -572,7 +628,6 @@ def _run_mlx_training(event_queue, stop_queue, config):
return ds
def _load_local(file_paths):
from core.training.trainer import UnslothTrainer
from datasets import load_from_disk
if len(file_paths) == 1:
@ -581,10 +636,10 @@ def _run_mlx_training(event_queue, stop_queue, config):
(p / "dataset_info.json").exists() or (p / "state.json").exists()
):
return load_from_disk(str(p))
all_files = UnslothTrainer._resolve_local_files(file_paths)
all_files = _resolve_mlx_local_dataset_files(file_paths)
if not all_files:
raise ValueError("No local dataset files found")
loader = UnslothTrainer._loader_for_files(all_files)
loader = _mlx_local_dataset_loader_for_files(all_files)
return load_dataset(loader, data_files = all_files, split = "train")
if hf_dataset:
@ -722,6 +777,12 @@ def _run_mlx_training(event_queue, stop_queue, config):
else:
eval_steps_val = int(eval_steps_val)
# MLX: per-element clip to [-1, 1]; norm clip disabled (it needs a
# global reduction that breaks MLX's eager pipeline). 1.0 (not 5.0):
# |g_i| > 5 rarely fires, so the historical 5.0 was effectively no-op.
max_grad_norm = 0.0
max_grad_value = 1.0 # TODO: expose MLX grad-clip in Studio UI for power users
trainer = MLXTrainer(
model = model,
tokenizer = tokenizer,
@ -736,6 +797,8 @@ def _run_mlx_training(event_queue, stop_queue, config):
lr_scheduler_type = lr_scheduler_type,
optim = optim_name,
weight_decay = float(config.get("weight_decay", 0.001) or 0.001),
max_grad_norm = max_grad_norm,
max_grad_value = max_grad_value,
logging_steps = 1,
max_seq_length = max_seq_length,
seed = config.get("random_seed", 3407),
@ -824,7 +887,17 @@ def _run_mlx_training(event_queue, stop_queue, config):
# ── 9. Real-time progress callback ──
_send("status", status_message = f"Training {model_name}...")
def _on_step(step, total, loss, lr, tok_s, peak_gb, elapsed, num_tokens):
def _on_step(
step,
total,
loss,
lr,
tok_s,
peak_gb,
elapsed,
num_tokens,
grad_norm = None,
):
eta = (elapsed / step * (total - step)) if step > 0 else 0
_send(
"progress",
@ -835,7 +908,7 @@ def _run_mlx_training(event_queue, stop_queue, config):
total_steps = total,
elapsed_seconds = elapsed,
eta_seconds = max(0, eta),
grad_norm = None,
grad_norm = grad_norm,
num_tokens = num_tokens,
eval_loss = None,
status_message = None,
@ -850,6 +923,11 @@ def _run_mlx_training(event_queue, stop_queue, config):
"train/tokens_per_sec": tok_s,
"train/peak_gb": peak_gb,
"train/num_tokens": num_tokens,
**(
{"train/grad_norm": grad_norm}
if grad_norm is not None
else {}
),
},
step = step,
)
@ -861,6 +939,8 @@ def _run_mlx_training(event_queue, stop_queue, config):
tb_writer.add_scalar("train/learning_rate", lr, step)
tb_writer.add_scalar("train/tokens_per_sec", tok_s, step)
tb_writer.add_scalar("train/peak_gb", peak_gb, step)
if grad_norm is not None:
tb_writer.add_scalar("train/grad_norm", grad_norm, step)
except Exception:
pass

10
studio/backend/main.py
View file

@ -120,6 +120,7 @@ from routes import (
inference_router,
inference_studio_router,
models_router,
providers_router,
training_history_router,
training_router,
)
@ -222,6 +223,11 @@ async def lifespan(app: FastAPI):
threading.Thread(target = _precache, daemon = True).start()
# Initialize RSA key pair for API key encryption (external providers)
from core.inference.key_exchange import init_key_pair
init_key_pair()
if storage.ensure_default_admin():
bootstrap_pw = storage.get_bootstrap_password()
app.state.bootstrap_password = bootstrap_pw
@ -293,7 +299,8 @@ def _build_csp(script_nonce: "str | None" = None) -> str:
"https://cdn-avatars.huggingface.co; "
"connect-src 'self' https://huggingface.co "
"https://*.huggingface.co https://cdn-lfs.huggingface.co "
"https://cdn-lfs.hf.co https://hf.co https://*.hf.co; "
"https://cdn-lfs.hf.co https://hf.co https://*.hf.co "
"https://datasets-server.huggingface.co; "
"style-src 'self' 'unsafe-inline'; "
f"{script_src}; "
"font-src 'self' data:; "
@ -489,6 +496,7 @@ app.include_router(inference_studio_router, prefix = "/api/inference", tags = ["
# so external tools (Open WebUI, SillyTavern, etc.) can use the
# standard /v1/chat/completions path.
app.include_router(inference_router, prefix = "/v1", tags = ["openai-compat"])
app.include_router(providers_router, prefix = "/api/providers", tags = ["providers"])
app.include_router(datasets_router, prefix = "/api/datasets", tags = ["datasets"])
app.include_router(data_recipe_router, prefix = "/api/data-recipe", tags = ["data-recipe"])
app.include_router(export_router, prefix = "/api/export", tags = ["export"])

114
studio/backend/models/inference.py
View file

@ -546,9 +546,11 @@ class ChatCompletionRequest(BaseModel):
None,
description = "[x-unsloth] Enable/disable thinking/reasoning mode for supported models",
)
reasoning_effort: Optional[Literal["low", "medium", "high"]] = Field(
reasoning_effort: Optional[
Literal["none", "minimal", "low", "medium", "high", "max", "xhigh"]
] = Field(
None,
description = "[x-unsloth] Reasoning effort level ('low'|'medium'|'high') for Harmony-style reasoning models (e.g. gpt-oss). Overrides enable_thinking when the active model uses reasoning_effort style.",
description = "[x-unsloth] Reasoning effort level ('none'|'minimal'|'low'|'medium'|'high'|'max'|'xhigh'). OpenAI `/v1/responses` accepts model-dependent subsets; Anthropic adaptive thinking uses `max` as the top tier on Claude 4.6 Opus/Sonnet (inbound `xhigh` is mapped to `max`) and `xhigh` on Claude 4.7 Opus; local Harmony/gpt-oss templates support low|medium|high.",
)
preserve_thinking: Optional[bool] = Field(
None,
@ -585,6 +587,114 @@ class ChatCompletionRequest(BaseModel):
description = "[x-unsloth] Per-request cancellation token. Frontend sends a fresh UUID per run so /inference/cancel matches one specific generation.",
)
# ── External provider routing (x-unsloth extensions) ──────────
provider_id: Optional[str] = Field(
None,
description = "[x-unsloth] Saved provider config ID. If set with encrypted_api_key, routes to external LLM.",
)
provider_type: Optional[str] = Field(
None,
description = "[x-unsloth] Provider type (e.g. 'openai', 'mistral'). Used if provider_id is not set.",
)
external_model: Optional[str] = Field(
None,
description = "[x-unsloth] Model ID at the external provider.",
)
encrypted_api_key: Optional[str] = Field(
None,
description = "[x-unsloth] RSA-encrypted, base64-encoded API key for the external provider.",
)
provider_base_url: Optional[str] = Field(
None,
description = "[x-unsloth] Override base URL for the external provider.",
)
enable_prompt_caching: Optional[bool] = Field(
None,
description = (
"[x-unsloth] Opt in to provider-side prompt caching. On Anthropic, "
"attaches cache_control={type:ephemeral} to the system block so the "
"static prefix is reused across turns. On OpenAI cloud, caching is "
"automatic for prompts >=1024 tokens and this flag is informational. "
"Ignored for every other provider (mistral, gemini, kimi, openrouter, "
"vllm, local, etc.). Treated as enabled when omitted."
),
)
openai_code_exec_container_id: Optional[str] = Field(
None,
description = (
"[x-unsloth] OpenAI shell-tool container id from the prior response "
"in the same chat thread. When set and `code_execution` is in "
"`enabled_tools`, the next /v1/responses call uses "
"environment.type='container_reference' so filesystem state "
"persists across turns. Unset → environment.type='container_auto' "
"and OpenAI creates a fresh container. Only meaningful for the "
"OpenAI cloud + gpt-5.5 family path; ignored otherwise."
),
)
# ── OpenAI shell-tool container management ─────────────────────
class OpenAIContainerRequest(BaseModel):
"""
Shared body for the three OpenAI container endpoints (list / create
/ delete). Carries the encrypted API key + base URL so the route
handler can decrypt it and proxy to the user's OpenAI account.
Same pattern as the inference proxy endpoints — keeps the key off
persistent storage on the backend.
"""
encrypted_api_key: str = Field(
...,
description = "[x-unsloth] RSA-encrypted, base64-encoded OpenAI API key.",
)
provider_base_url: Optional[str] = Field(
None,
description = "[x-unsloth] OpenAI base URL. Only api.openai.com is supported; non-cloud bases are rejected with 400.",
)
class CreateOpenAIContainerBody(OpenAIContainerRequest):
name: str = Field(
...,
min_length = 1,
max_length = 256,
description = "Human-readable container name. Surfaces in the picker UI.",
)
ttl_minutes: int = Field(
20,
ge = 1,
le = 20,
description = (
"Idle-timeout TTL the new container will inherit (anchor="
"last_active_at). OpenAI hard-caps this at 20 minutes and "
"rejects larger values with integer_above_max_value."
),
)
class DeleteOpenAIContainerBody(OpenAIContainerRequest):
container_id: str = Field(
...,
description = "OpenAI container id (cntr_...) to delete.",
)
class OpenAIContainerSummary(BaseModel):
"""One row from GET /v1/containers, reshaped for the UI."""
id: str
name: Optional[str] = None
created_at: Optional[int] = None
last_active_at: Optional[int] = None
expires_after_minutes: Optional[int] = None
status: Optional[str] = None
class ListOpenAIContainersResponse(BaseModel):
containers: list[OpenAIContainerSummary]
# ── Streaming response chunks ────────────────────────────────────

130
studio/backend/models/providers.py Normal file
View file

@ -0,0 +1,130 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Pydantic schemas for the external LLM providers API.
"""
from typing import Literal, Optional
from pydantic import BaseModel, Field
# ── Registry (static provider info) ───────────────────────────────
class ProviderRegistryEntry(BaseModel):
"""A supported provider type with its default configuration."""
provider_type: str = Field(
..., description = "Provider identifier (e.g. 'openai', 'mistral')"
)
display_name: str = Field(..., description = "Human-readable provider name")
base_url: str = Field(..., description = "Default API base URL")
default_models: list[str] = Field(
default_factory = list, description = "Well-known model IDs for this provider"
)
supports_streaming: bool = Field(
True, description = "Whether this provider supports SSE streaming"
)
supports_vision: bool = Field(
False, description = "Whether this provider supports vision/image input"
)
supports_tool_calling: bool = Field(
False, description = "Whether this provider supports tool/function calling"
)
model_list_mode: Literal["remote", "curated"] = Field(
"remote",
description = "remote = fetch /models; curated = huge catalogs — UI uses defaults + manual IDs only",
)
# ── Provider config CRUD ──────────────────────────────────────────
class ProviderCreate(BaseModel):
"""Request to create a saved provider configuration."""
provider_type: str = Field(..., description = "Provider type from the registry")
display_name: str = Field(
..., description = "User-chosen label (e.g. 'My OpenAI Key')"
)
base_url: Optional[str] = Field(
None,
description = "Custom base URL (overrides registry default). Omit to use the default.",
)
class ProviderUpdate(BaseModel):
"""Request to update a saved provider configuration."""
display_name: Optional[str] = Field(None, description = "New display name")
base_url: Optional[str] = Field(None, description = "New base URL")
is_enabled: Optional[bool] = Field(
None, description = "Enable or disable this provider"
)
class ProviderResponse(BaseModel):
"""A saved provider configuration (returned by list/get endpoints)."""
id: str = Field(..., description = "Unique provider config ID")
provider_type: str = Field(..., description = "Provider type (e.g. 'openai')")
display_name: str = Field(..., description = "User-chosen label")
base_url: str = Field(..., description = "API base URL")
is_enabled: bool = Field(True, description = "Whether this provider is enabled")
created_at: str = Field(..., description = "ISO 8601 creation timestamp")
updated_at: str = Field(..., description = "ISO 8601 last-update timestamp")
# ── Model listing ─────────────────────────────────────────────────
class ProviderModelInfo(BaseModel):
"""A model available from an external provider."""
id: str = Field(..., description = "Model ID as expected by the provider API")
display_name: str = Field("", description = "Human-readable model name")
context_length: Optional[int] = Field(
None, description = "Maximum context length in tokens"
)
owned_by: Optional[str] = Field(None, description = "Model owner/organization")
class ProviderModelsRequest(BaseModel):
"""Request to list models from an external provider."""
provider_type: str = Field(..., description = "Provider type from the registry")
encrypted_api_key: Optional[str] = Field(
None,
description = "RSA-encrypted, base64-encoded API key (optional for local providers)",
)
base_url: Optional[str] = Field(
None, description = "Custom base URL (overrides registry default)"
)
# ── Connection testing ────────────────────────────────────────────
class ProviderTestRequest(BaseModel):
"""Request to test connectivity to an external provider."""
provider_type: str = Field(..., description = "Provider type from the registry")
encrypted_api_key: Optional[str] = Field(
None,
description = "RSA-encrypted, base64-encoded API key (optional for local providers)",
)
base_url: Optional[str] = Field(
None, description = "Custom base URL (overrides registry default)"
)
class ProviderTestResult(BaseModel):
"""Result of a provider connectivity test."""
success: bool = Field(..., description = "Whether the test succeeded")
message: str = Field(..., description = "Human-readable result message")
models_count: Optional[int] = Field(
None, description = "Number of models found (if test succeeded)"
)

30
studio/backend/models/training.py
View file

@ -130,20 +130,23 @@ class TrainingStartRequest(BaseModel):
@field_validator("num_epochs")
@classmethod
def _check_num_epochs(cls, v: int) -> int:
# 0 is a sentinel meaning "use max_steps instead"; the frontend's
# steps-vs-epochs toggle sends it.
if v is None:
return 1
if v < 1 or v > _MAX_EPOCHS:
raise ValueError(f"num_epochs must be in [1, {_MAX_EPOCHS}] (got {v!r})")
if v < 0 or v > _MAX_EPOCHS:
raise ValueError(f"num_epochs must be in [0, {_MAX_EPOCHS}] (got {v!r})")
return v
@field_validator("max_steps")
@classmethod
def _check_max_steps(cls, v):
def _check_max_steps(cls, v: Optional[int]) -> Optional[int]:
# 0 is the frontend's sentinel for "use num_epochs instead".
if v is None:
return v
if not isinstance(v, int) or v < 1 or v > _MAX_STEPS:
if not isinstance(v, int) or v < 0 or v > _MAX_STEPS:
raise ValueError(
f"max_steps must be a positive int <= {_MAX_STEPS} (got {v!r})"
f"max_steps must be a non-negative int <= {_MAX_STEPS} (got {v!r})"
)
return v
@ -158,7 +161,7 @@ class TrainingStartRequest(BaseModel):
@field_validator("warmup_steps")
@classmethod
def _check_warmup_steps(cls, v):
def _check_warmup_steps(cls, v: Optional[int]) -> Optional[int]:
if v is None:
return v
if not isinstance(v, int) or v < 0 or v > _MAX_STEPS:
@ -259,6 +262,11 @@ class TrainingStartRequest(BaseModel):
max_steps: Optional[int] = Field(None, description = "Maximum training steps")
save_steps: int = Field(100, description = "Steps between checkpoints")
weight_decay: float = Field(0.001, description = "Weight decay")
max_grad_norm: float = Field(
0.0,
ge = 0,
description = "Global gradient norm clipping threshold. Set 0 to disable.",
)
random_seed: int = Field(42, description = "Random seed")
packing: bool = Field(False, description = "Enable sequence packing")
optim: str = Field("adamw_8bit", description = "Optimizer")
@ -321,6 +329,16 @@ class TrainingStartRequest(BaseModel):
description = "Physical GPU indices to use, for example [0, 1]. Omit or pass [] to use automatic selection. Explicit gpu_ids are unsupported when the parent CUDA_VISIBLE_DEVICES uses UUID/MIG entries.",
)
@model_validator(mode = "after")
def _check_steps_or_epochs(self) -> "TrainingStartRequest":
# num_epochs and max_steps each accept 0 as a "use the other one"
# sentinel. If both resolve to 0 there's nothing to train against.
if (self.max_steps is None or self.max_steps == 0) and self.num_epochs == 0:
raise ValueError(
"Either num_epochs or max_steps must be > 0; both cannot be 0."
)
return self
class TrainingJobResponse(BaseModel):
"""Immediate response when training is initiated"""

2
studio/backend/requirements/studio.txt
View file

@ -16,3 +16,5 @@ huggingface-hub==0.36.2
structlog>=24.1.0
diceware
ddgs
cryptography>=42.0.0
httpx>=0.27.0

2
studio/backend/routes/__init__.py
View file

@ -14,6 +14,7 @@ from routes.auth import router as auth_router
from routes.data_recipe import router as data_recipe_router
from routes.export import router as export_router
from routes.training_history import router as training_history_router
from routes.providers import router as providers_router
__all__ = [
"training_router",
@ -25,4 +26,5 @@ __all__ = [
"data_recipe_router",
"export_router",
"training_history_router",
"providers_router",
]

354
studio/backend/routes/inference.py
View file

@ -194,6 +194,11 @@ from models.inference import (
AnthropicResponseTextBlock,
AnthropicResponseToolUseBlock,
AnthropicUsage,
CreateOpenAIContainerBody,
DeleteOpenAIContainerBody,
ListOpenAIContainersResponse,
OpenAIContainerRequest,
OpenAIContainerSummary,
)
from core.inference.anthropic_compat import (
anthropic_messages_to_openai,
@ -204,6 +209,11 @@ from core.inference.anthropic_compat import (
)
from auth.authentication import get_current_subject
from core.inference.key_exchange import decrypt_api_key
from core.inference.providers import get_provider_info, get_base_url
from core.inference.external_provider import ExternalProviderClient
from storage import providers_db
import io
import wave
import base64
@ -1464,6 +1474,345 @@ def _extract_content_parts(
return system_prompt, chat_messages, first_image_b64
# ── External provider proxy ──────────────────────────────────────
def _build_external_messages(
messages: list,
supports_vision: bool,
) -> list[dict]:
"""
Convert ChatMessage list to OpenAI-compatible dicts for external providers.
- Vision providers: preserve multimodal content arrays (image_url parts intact).
- Non-vision providers: flatten to text-only (images silently dropped).
"""
result = []
for msg in messages:
if isinstance(msg.content, str):
# Skip assistant messages with empty content (some providers reject them)
if msg.role == "assistant" and not msg.content.strip():
continue
result.append({"role": msg.role, "content": msg.content})
elif isinstance(msg.content, list):
if supports_vision:
parts = []
for part in msg.content:
if part.type == "text":
parts.append({"type": "text", "text": part.text})
elif part.type == "image_url":
parts.append(
{
"type": "image_url",
"image_url": {"url": part.image_url.url},
}
)
result.append({"role": msg.role, "content": parts})
else:
# Non-vision provider — strip images, keep text only
text = "\n".join(p.text for p in msg.content if p.type == "text")
result.append({"role": msg.role, "content": text})
return result
async def _proxy_to_external_provider(
payload: ChatCompletionRequest,
request: Request,
) -> StreamingResponse:
"""
Proxy a chat completion request to an external LLM provider.
Resolves provider config (from DB or registry), decrypts the API key,
and streams the response back in OpenAI SSE format.
"""
# Resolve provider type and base URL
provider_type = payload.provider_type
base_url = payload.provider_base_url
if payload.provider_id:
config = providers_db.get_provider(payload.provider_id)
if config is None:
raise HTTPException(
status_code = 404,
detail = f"Provider config not found: {payload.provider_id}",
)
if not config["is_enabled"]:
raise HTTPException(
status_code = 400,
detail = f"Provider '{config['display_name']}' is disabled.",
)
provider_type = provider_type or config["provider_type"]
base_url = base_url or config["base_url"]
if not provider_type:
raise HTTPException(
status_code = 400,
detail = "Either provider_id or provider_type is required for external provider routing.",
)
# Fall back to registry default base URL
if not base_url:
base_url = get_base_url(provider_type)
if not base_url:
raise HTTPException(
status_code = 400,
detail = f"Unknown provider type: {provider_type}",
)
api_key = ""
if payload.encrypted_api_key:
try:
api_key = decrypt_api_key(payload.encrypted_api_key)
except Exception as exc:
logger.warning("external_provider.decrypt_failed", error = str(exc))
raise HTTPException(
status_code = 400,
detail = "Failed to decrypt API key. The server key may have changed — try refreshing the page.",
)
model = payload.external_model or payload.model
if model == "default":
raise HTTPException(
status_code = 400,
detail = "external_model is required when using an external provider.",
)
# Build messages preserving multimodal content for vision-capable providers
from core.inference.providers import get_provider_info as _get_provider_info
_pinfo = _get_provider_info(provider_type) or {}
_supports_vision = _pinfo.get("supports_vision", False)
chat_messages = _build_external_messages(payload.messages, _supports_vision)
client = ExternalProviderClient(
provider_type = provider_type,
base_url = base_url,
api_key = api_key,
)
async def _stream():
gen = client.stream_chat_completion(
messages = chat_messages,
model = model,
temperature = payload.temperature,
top_p = payload.top_p,
max_tokens = payload.max_tokens,
presence_penalty = payload.presence_penalty,
top_k = payload.top_k,
enable_thinking = payload.enable_thinking,
reasoning_effort = payload.reasoning_effort,
enabled_tools = payload.enabled_tools,
enable_prompt_caching = payload.enable_prompt_caching,
openai_code_exec_container_id = payload.openai_code_exec_container_id,
stream = payload.stream,
)
try:
sent_done = False
async for line in gen:
yield f"{line}\n\n"
if "[DONE]" in line:
sent_done = True
if not sent_done:
yield "data: [DONE]\n\n"
except Exception as exc:
logger.error("external_provider.stream_error", error = str(exc))
finally:
try:
await gen.aclose()
except RuntimeError:
pass # suppress httpcore asyncgen cleanup error (Python 3.13 + httpcore 1.0.x)
await client.close()
return StreamingResponse(
_stream(),
media_type = "text/event-stream",
headers = {
"Cache-Control": "no-cache",
"X-Accel-Buffering": "no",
},
)
# ── OpenAI shell-tool container management ───────────────────────
def _resolve_openai_cloud_client(
body: OpenAIContainerRequest,
) -> ExternalProviderClient:
"""
Decrypt the API key + validate the base URL points at OpenAI cloud,
then build an ExternalProviderClient for the three container CRUD
endpoints below. The shell tool only exists on api.openai.com, so
rejecting non-cloud bases up front prevents confusing 404s on
ollama / llama.cpp / vLLM / custom presets.
"""
base_url = body.provider_base_url or get_base_url("openai")
if not base_url or "api.openai.com" not in base_url:
raise HTTPException(
status_code = 400,
detail = (
"OpenAI container management is only available on the "
"managed cloud (api.openai.com). The provider's base URL "
f"points at {base_url!r}."
),
)
try:
api_key = decrypt_api_key(body.encrypted_api_key)
except Exception as exc:
logger.warning("external_provider.decrypt_failed", error = str(exc))
raise HTTPException(
status_code = 400,
detail = "Failed to decrypt API key. The server key may have changed — try refreshing the page.",
)
return ExternalProviderClient(
provider_type = "openai",
base_url = base_url,
api_key = api_key,
)
def _summarize_container(raw: dict) -> OpenAIContainerSummary:
expires = raw.get("expires_after")
expires_minutes: Optional[int] = None
if isinstance(expires, dict):
minutes = expires.get("minutes")
if isinstance(minutes, int):
expires_minutes = minutes
return OpenAIContainerSummary(
id = str(raw.get("id") or ""),
name = raw.get("name"),
created_at = raw.get("created_at")
if isinstance(raw.get("created_at"), int)
else None,
last_active_at = raw.get("last_active_at")
if isinstance(raw.get("last_active_at"), int)
else None,
expires_after_minutes = expires_minutes,
status = raw.get("status") if isinstance(raw.get("status"), str) else None,
)
@router.post(
"/external/openai/containers/list",
response_model = ListOpenAIContainersResponse,
)
async def list_openai_containers(
body: OpenAIContainerRequest,
current_subject: str = Depends(get_current_subject),
) -> ListOpenAIContainersResponse:
"""List the user's OpenAI shell-tool containers."""
client = _resolve_openai_cloud_client(body)
try:
try:
raw = await client.list_openai_containers()
except httpx.HTTPStatusError as exc:
detail = exc.response.text[:500] if exc.response is not None else str(exc)
raise HTTPException(
status_code = exc.response.status_code if exc.response else 502,
detail = f"OpenAI rejected /containers list: {detail}",
)
except httpx.HTTPError as exc:
raise HTTPException(
status_code = 502,
detail = f"Failed to reach OpenAI: {exc}",
)
# OpenAI keeps expired containers in /v1/containers indefinitely
# with status="expired" — they're effectively dead but still
# listed. Hide them so the picker only shows usable containers.
return ListOpenAIContainersResponse(
containers = [
_summarize_container(c)
for c in raw
if isinstance(c, dict) and c.get("status") != "expired"
],
)
finally:
await client.close()
@router.post(
"/external/openai/containers/create",
response_model = OpenAIContainerSummary,
)
async def create_openai_container(
body: CreateOpenAIContainerBody,
current_subject: str = Depends(get_current_subject),
) -> OpenAIContainerSummary:
"""Create a named container with the user-chosen idle TTL."""
client = _resolve_openai_cloud_client(body)
try:
try:
raw = await client.create_openai_container(
name = body.name,
ttl_minutes = body.ttl_minutes,
)
except httpx.HTTPStatusError as exc:
detail = exc.response.text[:500] if exc.response is not None else str(exc)
raise HTTPException(
status_code = exc.response.status_code if exc.response else 502,
detail = f"OpenAI rejected /containers create: {detail}",
)
except httpx.HTTPError as exc:
raise HTTPException(
status_code = 502,
detail = f"Failed to reach OpenAI: {exc}",
)
if not isinstance(raw, dict):
raise HTTPException(
status_code = 502,
detail = "OpenAI returned an unexpected container payload.",
)
return _summarize_container(raw)
finally:
await client.close()
@router.post("/external/openai/containers/delete", status_code = 204)
async def delete_openai_container(
body: DeleteOpenAIContainerBody,
current_subject: str = Depends(get_current_subject),
) -> None:
"""Delete a named container by id."""
logger.info(
"openai_container_delete.request subject=%s container_id=%s base_url=%s",
current_subject,
body.container_id,
body.provider_base_url,
)
client = _resolve_openai_cloud_client(body)
try:
try:
await client.delete_openai_container(body.container_id)
logger.info(
"openai_container_delete.success container_id=%s",
body.container_id,
)
except httpx.HTTPStatusError as exc:
detail = exc.response.text[:500] if exc.response is not None else str(exc)
logger.warning(
"openai_container_delete.openai_rejected container_id=%s status=%s body=%s",
body.container_id,
exc.response.status_code if exc.response else None,
detail,
)
raise HTTPException(
status_code = exc.response.status_code if exc.response else 502,
detail = f"OpenAI rejected /containers delete: {detail}",
)
except httpx.HTTPError as exc:
logger.warning(
"openai_container_delete.transport_error container_id=%s error=%s",
body.container_id,
exc,
)
raise HTTPException(
status_code = 502,
detail = f"Failed to reach OpenAI: {exc}",
)
finally:
await client.close()
@router.post("/chat/completions")
async def openai_chat_completions(
payload: ChatCompletionRequest,
@ -1483,6 +1832,11 @@ async def openai_chat_completions(
- GGUF models → llama-server via LlamaCppBackend
- Other models → Unsloth/transformers via InferenceBackend
"""
# ── External provider routing ────────────────────────────────
# encrypted_api_key is optional — local providers (llama.cpp / vLLM / Ollama) may run without auth.
if payload.provider_id or payload.provider_type:
return await _proxy_to_external_provider(payload, request)
llama_backend = get_llama_cpp_backend()
using_gguf = llama_backend.is_loaded

346
studio/backend/routes/providers.py Normal file
View file

@ -0,0 +1,346 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
API routes for external LLM provider management.
Provides endpoints for:
- Discovering available provider types (registry)
- CRUD for saved provider configurations (no API keys stored)
- Fetching the RSA public key for API key encryption
- Testing provider connectivity
- Listing models from a provider
"""
import uuid
import structlog
from fastapi import APIRouter, Depends, HTTPException
from auth.authentication import get_current_subject
from core.inference.key_exchange import (
decrypt_api_key,
get_public_key_fingerprint,
get_public_key_pem,
)
from core.inference.providers import (
get_base_url,
get_provider_info,
list_available_providers,
)
from core.inference.external_provider import ExternalProviderClient
from models.providers import (
ProviderCreate,
ProviderModelsRequest,
ProviderModelInfo,
ProviderResponse,
ProviderRegistryEntry,
ProviderTestRequest,
ProviderTestResult,
ProviderUpdate,
)
from storage import providers_db
logger = structlog.get_logger(__name__)
router = APIRouter()
# ── Public key for API key encryption ─────────────────────────────
@router.get("/public-key")
async def get_public_key(
current_subject: str = Depends(get_current_subject),
):
"""Return the RSA public key PEM for client-side API key encryption.
The ``fingerprint`` field is a short SHA256 of the PEM and is meant
purely for diagnostics — a mismatch between what the frontend
captured at encrypt time and what the server reports here is a
clear signal that the keypair rotated mid-flight (e.g. the server
re-ran ``init_key_pair`` for any reason).
"""
return {
"public_key": get_public_key_pem(),
"fingerprint": get_public_key_fingerprint(),
}
# ── Provider registry (static) ───────────────────────────────────
@router.get("/registry", response_model = list[ProviderRegistryEntry])
async def list_registry(
current_subject: str = Depends(get_current_subject),
):
"""List all supported provider types with their default configurations."""
return list_available_providers()
# ── Provider config CRUD ──────────────────────────────────────────
@router.get("/", response_model = list[ProviderResponse])
async def list_provider_configs(
current_subject: str = Depends(get_current_subject),
):
"""List all saved provider configurations."""
rows = providers_db.list_providers()
return [
ProviderResponse(
id = row["id"],
provider_type = row["provider_type"],
display_name = row["display_name"],
base_url = row["base_url"],
is_enabled = bool(row["is_enabled"]),
created_at = row["created_at"],
updated_at = row["updated_at"],
)
for row in rows
]
@router.post("/", response_model = ProviderResponse, status_code = 201)
async def create_provider_config(
payload: ProviderCreate,
current_subject: str = Depends(get_current_subject),
):
"""Create a new saved provider configuration (no API key stored)."""
info = get_provider_info(payload.provider_type)
if info is None:
raise HTTPException(
status_code = 400,
detail = f"Unknown provider type: {payload.provider_type}. "
f"Use GET /api/providers/registry to see available types.",
)
provider_id = uuid.uuid4().hex[:16]
base_url = payload.base_url or info["base_url"]
providers_db.create_provider(
id = provider_id,
provider_type = payload.provider_type,
display_name = payload.display_name,
base_url = base_url,
)
row = providers_db.get_provider(provider_id)
return ProviderResponse(
id = row["id"],
provider_type = row["provider_type"],
display_name = row["display_name"],
base_url = row["base_url"],
is_enabled = bool(row["is_enabled"]),
created_at = row["created_at"],
updated_at = row["updated_at"],
)
@router.put("/{provider_id}", response_model = ProviderResponse)
async def update_provider_config(
provider_id: str,
payload: ProviderUpdate,
current_subject: str = Depends(get_current_subject),
):
"""Update a saved provider configuration."""
existing = providers_db.get_provider(provider_id)
if not existing:
raise HTTPException(status_code = 404, detail = "Provider not found")
updated = providers_db.update_provider(
id = provider_id,
display_name = payload.display_name,
base_url = payload.base_url,
is_enabled = payload.is_enabled,
)
if not updated:
raise HTTPException(status_code = 400, detail = "No fields to update")
row = providers_db.get_provider(provider_id)
return ProviderResponse(
id = row["id"],
provider_type = row["provider_type"],
display_name = row["display_name"],
base_url = row["base_url"],
is_enabled = bool(row["is_enabled"]),
created_at = row["created_at"],
updated_at = row["updated_at"],
)
@router.delete("/{provider_id}", status_code = 204)
async def delete_provider_config(
provider_id: str,
current_subject: str = Depends(get_current_subject),
):
"""Delete a saved provider configuration."""
deleted = providers_db.delete_provider(provider_id)
if not deleted:
raise HTTPException(status_code = 404, detail = "Provider not found")
# ── Test connectivity ─────────────────────────────────────────────
@router.post("/test", response_model = ProviderTestResult)
async def test_provider(
payload: ProviderTestRequest,
current_subject: str = Depends(get_current_subject),
):
"""
Test connectivity to an external provider.
Makes a lightweight GET /models call to verify the API key works.
The encrypted_api_key is decrypted server-side and never stored.
"""
info = get_provider_info(payload.provider_type)
if info is None:
raise HTTPException(
status_code = 400,
detail = f"Unknown provider type: {payload.provider_type}",
)
api_key = ""
if payload.encrypted_api_key:
try:
api_key = decrypt_api_key(payload.encrypted_api_key)
except Exception as exc:
logger.warning(
"Failed to decrypt API key (%s): %s", type(exc).__name__, exc
)
raise HTTPException(
status_code = 400,
detail = "Failed to decrypt API key. The public key may have changed — try refreshing the page.",
)
base_url = payload.base_url or info["base_url"]
client = ExternalProviderClient(
provider_type = payload.provider_type,
base_url = base_url,
api_key = api_key,
timeout = 15.0,
)
try:
if info.get("model_list_mode") == "curated":
await client.verify_models_endpoint_lightweight()
return ProviderTestResult(
success = True,
message = (
"Connected successfully. Full model list is not fetched for this provider — "
"use suggestions and manual model IDs in the dialog."
),
models_count = None,
)
models = await client.list_models()
return ProviderTestResult(
success = True,
message = f"Connected successfully. Found {len(models)} model(s).",
models_count = len(models),
)
except Exception as exc:
logger.warning("Provider test failed for %s: %s", payload.provider_type, exc)
return ProviderTestResult(
success = False,
message = f"Connection failed: {exc}",
models_count = None,
)
finally:
await client.close()
# ── List models from provider ─────────────────────────────────────
@router.post("/models", response_model = list[ProviderModelInfo])
async def list_provider_models(
payload: ProviderModelsRequest,
current_subject: str = Depends(get_current_subject),
):
"""
List models available from an external provider.
The encrypted_api_key is decrypted server-side and never stored.
"""
info = get_provider_info(payload.provider_type)
if info is None:
raise HTTPException(
status_code = 400,
detail = f"Unknown provider type: {payload.provider_type}",
)
api_key = ""
if payload.encrypted_api_key:
try:
api_key = decrypt_api_key(payload.encrypted_api_key)
except Exception as exc:
logger.warning(
"Failed to decrypt API key (%s): %s", type(exc).__name__, exc
)
raise HTTPException(
status_code = 400,
detail = "Failed to decrypt API key. The public key may have changed — try refreshing the page.",
)
if info.get("model_list_mode") == "curated":
return [
ProviderModelInfo(
id = m,
display_name = m,
context_length = None,
owned_by = None,
)
for m in info.get("default_models", [])
]
base_url = payload.base_url or info["base_url"]
client = ExternalProviderClient(
provider_type = payload.provider_type,
base_url = base_url,
api_key = api_key,
timeout = 15.0,
)
try:
models = await client.list_models()
allow_prefixes = info.get("model_id_allow_prefixes")
if allow_prefixes is not None:
prefix_tuple = tuple(str(p) for p in allow_prefixes if str(p))
if prefix_tuple:
models = [m for m in models if m.get("id", "").startswith(prefix_tuple)]
allowlist = info.get("model_id_allowlist")
if allowlist is not None:
models = [m for m in models if allowlist.match(m.get("id", ""))]
deny_exact = info.get("model_id_deny_exact")
if deny_exact is not None:
deny_ids = {str(m) for m in deny_exact if str(m)}
if deny_ids:
models = [m for m in models if m.get("id", "") not in deny_ids]
denylist = info.get("model_id_denylist")
if denylist is not None:
models = [m for m in models if not denylist.search(m.get("id", ""))]
# Apply an optional cap after filtering so registry entries with a
# large remote catalog (e.g. HF Inference Providers) can stay
# picker-sized. No popularity sort happens server-side, so this is
# "first N matches" — pair with default_models for any must-have
# flagship ids.
limit = info.get("model_id_limit")
if isinstance(limit, int) and limit > 0:
models = models[:limit]
return [
ProviderModelInfo(
id = m.get("id", ""),
display_name = m.get("id", ""),
context_length = m.get("context_length") or m.get("context_window"),
owned_by = m.get("owned_by"),
)
for m in models
]
except Exception as exc:
logger.error("Failed to list models from %s: %s", payload.provider_type, exc)
raise HTTPException(
status_code = 502,
detail = f"Failed to list models from {payload.provider_type}: {exc}",
)
finally:
await client.close()

1
studio/backend/routes/training.py
View file

@ -215,6 +215,7 @@ async def start_training(
"max_steps": request.max_steps,
"save_steps": request.save_steps,
"weight_decay": request.weight_decay,
"max_grad_norm": request.max_grad_norm,
"random_seed": request.random_seed,
"packing": request.packing,
"optim": request.optim,

153
studio/backend/storage/providers_db.py Normal file
View file

@ -0,0 +1,153 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
SQLite storage for external LLM provider configurations.
Follows the same pattern as studio_db.py — module-level functions,
raw sqlite3, WAL mode, per-function connections.
NOTE: API keys are NOT stored here. They live only in the browser
(localStorage) and are sent encrypted per-request.
"""
import logging
import sqlite3
import threading
from datetime import datetime, timezone
from typing import Optional
logger = logging.getLogger(__name__)
from utils.paths import studio_db_path, ensure_dir
_schema_lock = threading.Lock()
_schema_ready = False
def _ensure_schema(conn: sqlite3.Connection) -> None:
"""Create the llm_providers table if it doesn't exist. Called once per process."""
conn.execute("PRAGMA journal_mode=WAL")
conn.execute(
"""
CREATE TABLE IF NOT EXISTS llm_providers (
id TEXT NOT NULL PRIMARY KEY,
provider_type TEXT NOT NULL,
display_name TEXT NOT NULL,
base_url TEXT NOT NULL,
is_enabled INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
)
"""
)
def get_connection() -> sqlite3.Connection:
"""Open studio.db with WAL mode, create table once per process."""
global _schema_ready
db_path = studio_db_path()
ensure_dir(db_path.parent)
conn = sqlite3.connect(str(db_path))
conn.row_factory = sqlite3.Row
if not _schema_ready:
with _schema_lock:
if not _schema_ready:
try:
_ensure_schema(conn)
_schema_ready = True
except Exception:
conn.close()
raise
return conn
def create_provider(
id: str,
provider_type: str,
display_name: str,
base_url: str,
) -> None:
"""Insert a new provider configuration."""
now = datetime.now(timezone.utc).isoformat()
conn = get_connection()
try:
conn.execute(
"""
INSERT INTO llm_providers (id, provider_type, display_name, base_url, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?)
""",
(id, provider_type, display_name, base_url, now, now),
)
conn.commit()
finally:
conn.close()
def update_provider(
id: str,
display_name: Optional[str] = None,
base_url: Optional[str] = None,
is_enabled: Optional[bool] = None,
) -> bool:
"""Update fields on an existing provider. Returns True if a row was updated."""
updates = []
params = []
if display_name is not None:
updates.append("display_name = ?")
params.append(display_name)
if base_url is not None:
updates.append("base_url = ?")
params.append(base_url)
if is_enabled is not None:
updates.append("is_enabled = ?")
params.append(1 if is_enabled else 0)
if not updates:
return False
updates.append("updated_at = ?")
params.append(datetime.now(timezone.utc).isoformat())
params.append(id)
conn = get_connection()
try:
cursor = conn.execute(
f"UPDATE llm_providers SET {', '.join(updates)} WHERE id = ?",
params,
)
conn.commit()
return cursor.rowcount > 0
finally:
conn.close()
def delete_provider(id: str) -> bool:
"""Delete a provider by ID. Returns True if a row was deleted."""
conn = get_connection()
try:
cursor = conn.execute("DELETE FROM llm_providers WHERE id = ?", (id,))
conn.commit()
return cursor.rowcount > 0
finally:
conn.close()
def get_provider(id: str) -> Optional[dict]:
"""Fetch a single provider by ID."""
conn = get_connection()
try:
row = conn.execute("SELECT * FROM llm_providers WHERE id = ?", (id,)).fetchone()
return dict(row) if row else None
finally:
conn.close()
def list_providers() -> list[dict]:
"""List all provider configurations, ordered by creation time."""
conn = get_connection()
try:
rows = conn.execute(
"SELECT * FROM llm_providers ORDER BY created_at"
).fetchall()
return [dict(row) for row in rows]
finally:
conn.close()

419
studio/backend/tests/test_anthropic_code_execution.py Normal file
View file

@ -0,0 +1,419 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Unit tests for Anthropic's server-side `code_execution_20250825` tool
translation in `_stream_anthropic`.
Covers:
- Request body: when ``enabled_tools=["code_execution"]``, the outbound
``tools`` array carries ``{"type": "code_execution_20250825", "name":
"code_execution"}`` and the ``anthropic-beta`` header includes
``code-execution-2025-08-25``.
- Combined request: ``enabled_tools=["web_search", "code_execution"]``
sends both tool entries; the beta header still merges the code-exec
flag onto whatever the registry contributed.
- SSE translation: a `bash_code_execution` server_tool_use +
`bash_code_execution_tool_result` pair emits one tool_start and one
tool_end ``_toolEvent`` chunk with the expected arguments and result.
- SSE translation: a `text_editor_code_execution` create + result emits
a tool_start with ``kind="text_editor"`` + parsed args, and tool_end
with ``"Created"`` (or ``"Updated"``) based on the ``is_file_update``
flag.
- Error path: a ``bash_code_execution_tool_result_error`` with
``error_code="container_expired"`` renders as ``"Error:
container_expired"`` in the tool_end ``result``.
"""
import asyncio
import json
import httpx
from core.inference import external_provider as ep_mod
from core.inference.external_provider import ExternalProviderClient
def _drive(coro):
return asyncio.new_event_loop().run_until_complete(coro)
async def _collect(agen):
out = []
async for line in agen:
out.append(line)
return out
def _mock_http_client(monkeypatch, handler):
transport = httpx.MockTransport(handler)
monkeypatch.setattr(ep_mod, "_http_client", httpx.AsyncClient(transport = transport))
def _make_client() -> ExternalProviderClient:
return ExternalProviderClient(
provider_type = "anthropic",
base_url = "https://api.anthropic.com/v1",
api_key = "sk-ant-test",
)
def _anthropic_sse(events: list[dict]) -> bytes:
chunks: list[str] = []
for event in events:
chunks.append(f"event: {event['type']}")
chunks.append(f"data: {json.dumps(event)}")
chunks.append("")
return ("\n".join(chunks) + "\n").encode("utf-8")
def _tool_events(lines: list[str]) -> list[dict]:
"""Extract `_toolEvent` payloads from emitted SSE data lines."""
out: list[dict] = []
for line in lines:
if not line.startswith("data:"):
continue
raw = line[len("data:") :].strip()
if not raw or raw == "[DONE]":
continue
try:
parsed = json.loads(raw)
except json.JSONDecodeError:
continue
if isinstance(parsed, dict) and "_toolEvent" in parsed:
out.append(parsed["_toolEvent"])
return out
def test_code_execution_tool_appended_to_request_body(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
captured["headers"] = dict(request.headers)
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "compute 2 + 2"}],
model = "claude-opus-4-7",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enabled_tools = ["code_execution"],
):
pass
await client.close()
_drive(run())
body = captured["body"]
tools = body.get("tools") or []
assert {
"type": "code_execution_20250825",
"name": "code_execution",
} in tools
# No web_search entry when only code_execution is enabled.
assert all(t.get("type") != "web_search_20250305" for t in tools)
# Beta header carries the documented flag.
beta_header = captured["headers"].get("anthropic-beta", "")
assert "code-execution-2025-08-25" in beta_header
def test_code_execution_with_web_search_sends_both_tools(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
captured["headers"] = dict(request.headers)
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "look it up and chart it"}],
model = "claude-opus-4-7",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enabled_tools = ["web_search", "code_execution"],
):
pass
await client.close()
_drive(run())
tools = captured["body"].get("tools") or []
tool_types = {t.get("type") for t in tools if isinstance(t, dict)}
assert "web_search_20250305" in tool_types
assert "code_execution_20250825" in tool_types
assert "code-execution-2025-08-25" in captured["headers"].get("anthropic-beta", "")
def test_no_code_execution_tool_when_pill_off(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
captured["headers"] = dict(request.headers)
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-7",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
):
pass
await client.close()
_drive(run())
tools = captured["body"].get("tools") or []
assert all(t.get("type") != "code_execution_20250825" for t in tools)
# Beta header must NOT mention code-execution when the tool isn't on
# — that flag is opt-in only.
assert "code-execution-2025-08-25" not in captured["headers"].get(
"anthropic-beta", ""
)
def test_bash_code_execution_emits_tool_start_and_end(monkeypatch):
sse_events = [
{"type": "message_start", "message": {"usage": {}}},
{
"type": "content_block_start",
"index": 0,
"content_block": {
"type": "server_tool_use",
"id": "srvtoolu_1",
"name": "bash_code_execution",
},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {
"type": "input_json_delta",
"partial_json": '{"command": "ls -la"}',
},
},
{"type": "content_block_stop", "index": 0},
{
"type": "content_block_start",
"index": 1,
"content_block": {
"type": "bash_code_execution_tool_result",
"tool_use_id": "srvtoolu_1",
"content": {
"type": "bash_code_execution_result",
"stdout": "total 24\ndrwxr-xr-x 2 user user 4096 Jan 1 12:00 .",
"stderr": "",
"return_code": 0,
},
},
},
{"type": "content_block_stop", "index": 1},
{"type": "message_stop"},
]
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = _anthropic_sse(sse_events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
return await _collect(
client._stream_anthropic(
messages = [{"role": "user", "content": "list files"}],
model = "claude-opus-4-7",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enabled_tools = ["code_execution"],
)
)
lines = _drive(run())
events = _tool_events(lines)
assert len(events) == 2
start, end = events
assert start["type"] == "tool_start"
assert start["tool_name"] == "code_execution"
assert start["tool_call_id"] == "srvtoolu_1"
assert start["arguments"] == {"kind": "bash", "command": "ls -la"}
assert end["type"] == "tool_end"
assert end["tool_call_id"] == "srvtoolu_1"
assert "total 24" in end["result"]
# Non-zero return_code not present, so no return_code line.
assert "return_code:" not in end["result"]
def test_text_editor_create_emits_kind_and_status(monkeypatch):
sse_events = [
{"type": "message_start", "message": {"usage": {}}},
{
"type": "content_block_start",
"index": 0,
"content_block": {
"type": "server_tool_use",
"id": "srvtoolu_2",
"name": "text_editor_code_execution",
},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {
"type": "input_json_delta",
"partial_json": (
'{"command": "create", "path": "new_file.txt", '
'"file_text": "hi"}'
),
},
},
{"type": "content_block_stop", "index": 0},
{
"type": "content_block_start",
"index": 1,
"content_block": {
"type": "text_editor_code_execution_tool_result",
"tool_use_id": "srvtoolu_2",
"content": {
"type": "text_editor_code_execution_result",
"is_file_update": False,
},
},
},
{"type": "content_block_stop", "index": 1},
{"type": "message_stop"},
]
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = _anthropic_sse(sse_events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
return await _collect(
client._stream_anthropic(
messages = [{"role": "user", "content": "write a file"}],
model = "claude-opus-4-7",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enabled_tools = ["code_execution"],
)
)
lines = _drive(run())
events = _tool_events(lines)
assert len(events) == 2
start, end = events
assert start["arguments"]["kind"] == "text_editor"
assert start["arguments"]["command"] == "create"
assert start["arguments"]["path"] == "new_file.txt"
assert end["result"] == "Created"
def test_code_execution_error_renders_error_code(monkeypatch):
sse_events = [
{"type": "message_start", "message": {"usage": {}}},
{
"type": "content_block_start",
"index": 0,
"content_block": {
"type": "server_tool_use",
"id": "srvtoolu_3",
"name": "bash_code_execution",
},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {
"type": "input_json_delta",
"partial_json": '{"command": "echo broken"}',
},
},
{"type": "content_block_stop", "index": 0},
{
"type": "content_block_start",
"index": 1,
"content_block": {
"type": "bash_code_execution_tool_result",
"tool_use_id": "srvtoolu_3",
"content": {
"type": "bash_code_execution_tool_result_error",
"error_code": "container_expired",
},
},
},
{"type": "content_block_stop", "index": 1},
{"type": "message_stop"},
]
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = _anthropic_sse(sse_events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
return await _collect(
client._stream_anthropic(
messages = [{"role": "user", "content": "run it"}],
model = "claude-opus-4-7",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enabled_tools = ["code_execution"],
)
)
lines = _drive(run())
events = _tool_events(lines)
assert len(events) == 2
end = events[1]
assert end["type"] == "tool_end"
assert end["result"] == "Error: container_expired"

404
studio/backend/tests/test_anthropic_thinking_translation.py Normal file
View file

@ -0,0 +1,404 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Unit tests for the Anthropic extended-thinking translation in
external_provider.
Covers:
- Adaptive-mode request body nests effort under
``output_config: {effort: "<level>"}`` per the Messages API
reference (a top-level ``effort`` field 400s with
"effort: Extra inputs are not permitted").
- Streaming SSE: ``content_block_delta`` with
``delta.type == "thinking_delta"`` is translated into inline
``<think>...</think>`` chat-completion chunks so the frontend's
reasoning-panel pipeline lifts it correctly.
- The ``<think>`` tag closes when the first ``text_delta`` arrives,
on ``content_block_stop``, on ``message_delta``, or on
``message_stop``.
- Thinking is paired with ``temperature=1`` and no ``top_p`` /
``top_k`` on the wire (Anthropic extended-thinking contract).
"""
import asyncio
import json
import httpx
from core.inference import external_provider as ep_mod
from core.inference.external_provider import ExternalProviderClient
def _drive(coro):
return asyncio.new_event_loop().run_until_complete(coro)
async def _collect(agen):
out = []
async for line in agen:
out.append(line)
return out
def _mock_http_client(monkeypatch, handler):
transport = httpx.MockTransport(handler)
monkeypatch.setattr(ep_mod, "_http_client", httpx.AsyncClient(transport = transport))
def _make_client() -> ExternalProviderClient:
return ExternalProviderClient(
provider_type = "anthropic",
base_url = "https://api.anthropic.com/v1",
api_key = "sk-ant-test",
)
def _anthropic_sse(events: list[dict]) -> bytes:
"""Serialize a list of Messages-API event dicts as an SSE byte stream."""
chunks: list[str] = []
for event in events:
chunks.append(f"event: {event['type']}")
chunks.append(f"data: {json.dumps(event)}")
chunks.append("")
return ("\n".join(chunks) + "\n").encode("utf-8")
def _payloads_from_lines(lines: list[str]) -> list:
out = []
for line in lines:
if not line.startswith("data:"):
continue
raw = line[len("data:") :].strip()
if not raw:
continue
if raw == "[DONE]":
out.append("[DONE]")
else:
out.append(json.loads(raw))
return out
def test_adaptive_thinking_body_uses_output_config_effort_shape(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-6",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = None,
reasoning_effort = "medium",
):
pass
await client.close()
_drive(run())
body = captured["body"]
# display=summarized is set explicitly so Opus 4.7 (which defaults to
# "omitted") still emits thinking_delta events for the reasoning panel.
assert body["thinking"] == {"type": "adaptive", "display": "summarized"}
# Documented shape: effort is nested under output_config.
# A top-level `effort` field produces a 400:
# "effort: Extra inputs are not permitted".
assert body["output_config"] == {"effort": "medium"}
assert "effort" not in body
# Extended-thinking contract: temperature=1, no top_p / top_k.
assert body["temperature"] == 1
assert "top_p" not in body
assert "top_k" not in body
def test_adaptive_thinking_maps_xhigh_to_max_on_claude_4_6(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-sonnet-4-6",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = None,
reasoning_effort = "xhigh",
):
pass
await client.close()
_drive(run())
assert captured["body"]["output_config"] == {"effort": "max"}
def test_adaptive_thinking_keeps_max_on_claude_4_6(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-6",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = None,
reasoning_effort = "max",
):
pass
await client.close()
_drive(run())
assert captured["body"]["output_config"] == {"effort": "max"}
def test_adaptive_thinking_keeps_xhigh_on_claude_4_7(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-7",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = None,
reasoning_effort = "xhigh",
):
pass
await client.close()
_drive(run())
body = captured["body"]
assert body["output_config"] == {"effort": "xhigh"}
assert "effort" not in body
def test_manual_thinking_body_uses_budget_tokens_on_4_5(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _anthropic_sse([{"type": "message_stop"}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 1024,
top_k = None,
enable_thinking = None,
reasoning_effort = "high",
):
pass
await client.close()
_drive(run())
body = captured["body"]
assert body["thinking"] == {"type": "enabled", "budget_tokens": 4096}
# max_tokens must be strictly greater than budget_tokens; we shipped 1024
# and budget is 4096, so the wrapper should bump max_tokens.
assert body["max_tokens"] > body["thinking"]["budget_tokens"]
# Manual-thinking path does not use output_config / effort — those are
# the adaptive-mode controls (Claude 4.6 / 4.7).
assert "effort" not in body
assert "output_config" not in body
def test_thinking_delta_wrapped_in_think_tags(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
events = [
{
"type": "content_block_start",
"index": 0,
"content_block": {"type": "thinking", "thinking": "", "signature": ""},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {"type": "thinking_delta", "thinking": "First "},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {"type": "thinking_delta", "thinking": "I plan."},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {"type": "signature_delta", "signature": "abc123"},
},
{"type": "content_block_stop", "index": 0},
{
"type": "content_block_start",
"index": 1,
"content_block": {"type": "text", "text": ""},
},
{
"type": "content_block_delta",
"index": 1,
"delta": {"type": "text_delta", "text": "Answer."},
},
{"type": "content_block_stop", "index": 1},
{"type": "message_delta", "delta": {"stop_reason": "end_turn"}},
{"type": "message_stop"},
]
return httpx.Response(
200,
content = _anthropic_sse(events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
lines = await _collect(
client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-6",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = True,
reasoning_effort = None,
)
)
await client.close()
return lines
lines = _drive(run())
payloads = _payloads_from_lines(lines)
combined = "".join(
p["choices"][0]["delta"].get("content", "")
for p in payloads
if isinstance(p, dict) and p["choices"][0]["delta"]
)
# Reasoning text should be wrapped in <think>...</think>, followed by the
# answer text, and the stream should terminate with [DONE].
assert "<think>First I plan.</think>" in combined
assert combined.endswith("Answer.")
# signature_delta is intentionally dropped — no leaked signature text.
assert "abc123" not in combined
assert "[DONE]" in payloads
def test_thinking_only_turn_closes_tag_without_text_delta(monkeypatch):
"""display=omitted on Claude 4.7 emits a signature_delta and no text.
The <think> open is still triggered by the (synthetic) thinking_delta;
we want content_block_stop to close it cleanly so the tag never leaks
into the next chunk."""
def handler(request: httpx.Request) -> httpx.Response:
events = [
{
"type": "content_block_start",
"index": 0,
"content_block": {"type": "thinking", "thinking": "", "signature": ""},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {"type": "thinking_delta", "thinking": "internal"},
},
{"type": "content_block_stop", "index": 0},
{"type": "message_delta", "delta": {"stop_reason": "end_turn"}},
{"type": "message_stop"},
]
return httpx.Response(
200,
content = _anthropic_sse(events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
lines = await _collect(
client._stream_anthropic(
messages = [{"role": "user", "content": "hi"}],
model = "claude-opus-4-7",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
top_k = None,
enable_thinking = True,
reasoning_effort = None,
)
)
await client.close()
return lines
payloads = _payloads_from_lines(_drive(run()))
combined = "".join(
p["choices"][0]["delta"].get("content", "")
for p in payloads
if isinstance(p, dict) and p["choices"][0]["delta"]
)
assert combined == "<think>internal</think>"

1
studio/backend/tests/test_desktop_auth.py
View file

@ -437,6 +437,7 @@ def test_health_response_reports_desktop_capability_fields(monkeypatch):
inference_router = APIRouter(),
inference_studio_router = APIRouter(),
models_router = APIRouter(),
providers_router = APIRouter(),
training_history_router = APIRouter(),
training_router = APIRouter(),
)

326
studio/backend/tests/test_detect_mmproj_file.py Normal file
View file

@ -0,0 +1,326 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Tests for :func:`utils.models.model_config.detect_mmproj_file` (#5347)."""
from __future__ import annotations
from pathlib import Path
import struct
from utils.models.model_config import (
_detect_family_token,
detect_mmproj_file,
mmproj_matches_model_family,
)
_GGUF_MAGIC = 0x46554747
def _gguf_with_general(path: Path, fields: dict) -> Path:
"""Write a minimal GGUF with only ``general.*`` string KVs."""
body = b""
for k, v in fields.items():
kb = k.encode("utf-8")
vb = v.encode("utf-8")
body += struct.pack("<Q", len(kb)) + kb
body += struct.pack("<I", 8) # STRING vtype
body += struct.pack("<Q", len(vb)) + vb
header = struct.pack("<IIQQ", _GGUF_MAGIC, 3, 0, len(fields))
path.parent.mkdir(parents = True, exist_ok = True)
path.write_bytes(header + body)
return path
def _touch(path: Path) -> Path:
path.parent.mkdir(parents = True, exist_ok = True)
path.write_bytes(b"")
return path
def test_returns_none_when_no_mmproj(tmp_path: Path):
model = _touch(tmp_path / "Qwen3.5-9B-Q4_K_M.gguf")
assert detect_mmproj_file(str(model)) is None
def test_single_matching_family_mmproj_picked(tmp_path: Path):
"""Single same-family projector: returned (historical behaviour)."""
model = _touch(tmp_path / "Qwen3.5-9B-Q4_K_M.gguf")
mmproj = _touch(tmp_path / "Qwen3.5-9B-BF16-mmproj.gguf")
assert detect_mmproj_file(str(model)) == str(mmproj.resolve())
def test_hf_style_unprefixed_mmproj_still_works(tmp_path: Path):
"""HF convention: weight + ``mmproj-F16.gguf`` sibling."""
model = _touch(tmp_path / "model.gguf")
mmproj = _touch(tmp_path / "mmproj-F16.gguf")
assert detect_mmproj_file(str(model)) == str(mmproj.resolve())
def test_blocks_single_cross_family_projector(tmp_path: Path):
"""#5347 core: Qwen weight + lone Gemma mmproj returns None."""
model = _touch(tmp_path / "Qwen3.5-9B-Q4_K_M.gguf")
_touch(tmp_path / "gemma-4-26B-A4B-it.mmproj-q8_0.gguf")
assert detect_mmproj_file(str(model)) is None
def test_picks_matching_family_among_mixed_candidates(tmp_path: Path):
"""Mixed Qwen + Gemma projectors: pick Qwen, drop Gemma."""
model = _touch(tmp_path / "Qwen3.5-9B-Q4_K_M.gguf")
qwen_mm = _touch(tmp_path / "Qwen3.5-9B-BF16-mmproj.gguf")
_touch(tmp_path / "gemma-4-26B-A4B-it.mmproj-q8_0.gguf")
assert detect_mmproj_file(str(model)) == str(qwen_mm.resolve())
def test_prefers_longest_prefix_within_same_family(tmp_path: Path):
"""Same family, different sizes: longest shared stem prefix wins."""
model = _touch(tmp_path / "Qwen3.5-35B-A3B-UD-Q4_K_L.gguf")
_touch(tmp_path / "Qwen3.5-9B-BF16-mmproj.gguf")
big_mm = _touch(tmp_path / "Qwen3.5-35B-A3B-BF16-mmproj.gguf")
assert detect_mmproj_file(str(model)) == str(big_mm.resolve())
def test_unrecognised_family_does_not_break_detection(tmp_path: Path):
"""Unknown model family must not return None on a sole candidate."""
model = _touch(tmp_path / "MyCustomBrand-7B-Q4_K_M.gguf")
mmproj = _touch(tmp_path / "MyCustomBrand-7B-BF16-mmproj.gguf")
assert detect_mmproj_file(str(model)) == str(mmproj.resolve())
def test_directory_path_returns_first_candidate(tmp_path: Path):
"""Directory path: no model stem to compare; legacy first-candidate."""
_touch(tmp_path / "Qwen3.5-9B-BF16-mmproj.gguf")
_touch(tmp_path / "gemma-4-26B-A4B-it.mmproj-q8_0.gguf")
result = detect_mmproj_file(str(tmp_path))
assert result is not None
assert "mmproj" in Path(result).name.lower()
def test_search_root_walk_still_works(tmp_path: Path):
"""Snapshot layout: weight in quant subdir, mmproj at snapshot root."""
snapshot = tmp_path / "snapshot"
weight = _touch(snapshot / "BF16" / "Qwen3.5-9B-BF16.gguf")
mmproj = _touch(snapshot / "Qwen3.5-9B-BF16-mmproj.gguf")
result = detect_mmproj_file(str(weight), search_root = str(snapshot))
assert result == str(mmproj.resolve())
# -- Family token detection: word-bounded matching ----------------------
def test_family_token_phi_does_not_match_sapphire():
"""``phi`` substring inside ``sapphire`` must not tag Phi."""
assert _detect_family_token("sapphire-7b-q4_k_m.gguf") is None
def test_family_token_yi_does_not_match_tinyish_names():
"""``yi`` must not cross letter boundaries (``yip``)."""
assert _detect_family_token("yip-7b.gguf") is None
assert _detect_family_token("yi-vl-6b.gguf") == "yi"
def test_family_token_mimo_does_not_match_mimosa():
"""``mimo`` must not tag ``mimosa``."""
assert _detect_family_token("mimosa-rosa-7b.gguf") is None
assert _detect_family_token("MiMo-VL-7B-RL-BF16.gguf") == "mimo"
def test_family_token_mistral_does_not_match_ministral():
"""Pin Mistral-derivative tagging."""
assert _detect_family_token("Ministral-3-8B-Instruct-2512-BF16.gguf") == "ministral"
assert _detect_family_token("Mistral-7B-Instruct-v0.3.gguf") == "mistral"
assert _detect_family_token("Magistral-Small-2506-BF16.gguf") == "magistral"
assert (
_detect_family_token("Devstral-Small-2-24B-Instruct-2512-BF16.gguf")
== "devstral"
)
def test_family_token_picks_leftmost_when_multiple_present():
"""Leftmost family token wins, not tuple order."""
assert _detect_family_token("llama-phi-merge.gguf") == "llama"
assert _detect_family_token("phi-llama-merge.gguf") == "phi"
assert _detect_family_token("llama3-3b-instruct.gguf") == "llama"
def test_family_token_new_families_recognised():
"""Catalogue-audit additions tag correctly."""
assert _detect_family_token("NVIDIA-Nemotron-3-Nano-Omni-30B.gguf") == "nemotron"
assert _detect_family_token("Kimi-K2.6-BF16.gguf") == "kimi"
assert _detect_family_token("Nanonets-OCR-s-BF16.gguf") == "nanonets"
assert _detect_family_token("Cosmos-Reason1-7B-BF16.gguf") == "cosmos"
assert _detect_family_token("Apriel-1.5-15b-Thinker-BF16.gguf") == "apriel"
assert _detect_family_token("LFM2.5-VL-1.6B-BF16.gguf") == "lfm"
# -- Cross-family rejection with the expanded token list ----------------
def test_blocks_cross_family_for_new_token_pair(tmp_path: Path):
"""Nemotron weight + lone Gemma projector returns None."""
model = _touch(
tmp_path / "NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-MXFP4_MOE.gguf"
)
_touch(tmp_path / "gemma-4-26B-A4B-it.mmproj-q8_0.gguf")
assert detect_mmproj_file(str(model)) is None
def test_picks_devstral_mmproj_in_mixed_dir(tmp_path: Path):
"""Devstral weight + Devstral mmproj + a Qwen mmproj: pick Devstral."""
model = _touch(tmp_path / "Devstral-Small-2-24B-Instruct-2512-BF16.gguf")
dev_mm = _touch(tmp_path / "Devstral-Small-2-mmproj-bf16.gguf")
_touch(tmp_path / "Qwen3.5-9B-BF16-mmproj.gguf")
assert detect_mmproj_file(str(model)) == str(dev_mm.resolve())
# -- Launcher-level family guard ----------------------------------------
def test_mmproj_family_guard_blocks_cross_family():
assert (
mmproj_matches_model_family(
"/models/Qwen3.5-9B-Q4_K_M.gguf",
"/models/gemma-4-26B-A4B-it.mmproj-q8_0.gguf",
)
is False
)
def test_mmproj_family_guard_allows_same_family():
assert (
mmproj_matches_model_family(
"/models/Qwen3.5-9B-Q4_K_M.gguf",
"/models/Qwen3.5-9B-BF16-mmproj.gguf",
)
is True
)
def test_mmproj_family_guard_allows_generic_hf_mmproj():
"""No family token on the projector: wildcard."""
assert (
mmproj_matches_model_family(
"/models/Qwen3.5-9B-Q4_K_M.gguf",
"/models/mmproj-F16.gguf",
)
is True
)
def test_mmproj_family_guard_allows_unrecognised_model_family():
"""No family token on the model: wildcard."""
assert (
mmproj_matches_model_family(
"/models/Apriel-1.5-15b-Thinker-BF16.gguf",
"/models/mmproj-F16.gguf",
)
is True
)
# -- Metadata-primary pairing in detect_mmproj_file ---------------------
def test_metadata_url_match_picked_over_filename_lookalike(tmp_path: Path):
"""URL match beats a longer-prefix sibling."""
weight = _gguf_with_general(
tmp_path / "Qwen3.5-9B-Q4_K_M.gguf",
{
"general.architecture": "qwen2vl",
"general.type": "model",
"general.basename": "Qwen3.5",
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
},
)
# Closer filename prefix, wrong upstream.
_gguf_with_general(
tmp_path / "Qwen3.5-9B-mmproj-bf16.gguf",
{
"general.architecture": "clip",
"general.type": "mmproj",
"general.basename": "Qwen3.5",
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-1.5B",
},
)
# Matching upstream.
correct = _gguf_with_general(
tmp_path / "mmproj-BF16.gguf",
{
"general.architecture": "clip",
"general.type": "mmproj",
"general.basename": "Qwen3.5",
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
},
)
assert detect_mmproj_file(str(weight)) == str(correct.resolve())
def test_metadata_url_mismatch_dropped(tmp_path: Path):
"""Filenames match family but metadata disagrees: returns None."""
weight = _gguf_with_general(
tmp_path / "qwen-9b.gguf",
{
"general.architecture": "qwen2vl",
"general.type": "model",
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
},
)
_gguf_with_general(
tmp_path / "qwen-9b-mmproj.gguf",
{
"general.architecture": "clip",
"general.type": "mmproj",
"general.base_model.0.repo_url": "https://huggingface.co/google/gemma-3-9B",
},
)
assert detect_mmproj_file(str(weight)) is None
def test_metadata_identifies_mmproj_without_filename_hint(tmp_path: Path):
"""Projector named ``vision-projector.gguf`` discovered via header."""
weight = _gguf_with_general(
tmp_path / "Qwen3.5-9B.gguf",
{
"general.architecture": "qwen2vl",
"general.type": "model",
"general.basename": "Qwen3.5",
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
},
)
projector = _gguf_with_general(
tmp_path / "vision-projector.gguf",
{
"general.architecture": "clip",
"general.type": "mmproj",
"general.basename": "Qwen3.5",
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
},
)
assert detect_mmproj_file(str(weight)) == str(projector.resolve())
def test_metadata_score_outranks_filename_prefix(tmp_path: Path):
"""Score 100 (URL match) beats score 0 (long filename prefix)."""
weight = _gguf_with_general(
tmp_path / "Qwen3.5-9B-Q4_K_M.gguf",
{
"general.architecture": "qwen2vl",
"general.type": "model",
"general.basename": "Qwen3.5",
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
},
)
# Headerless: long shared stem, score 0.
_touch(tmp_path / "Qwen3.5-9B-Q4_K_M-mmproj.gguf")
# Headered: generic name, score 100.
correct = _gguf_with_general(
tmp_path / "mmproj-BF16.gguf",
{
"general.architecture": "clip",
"general.type": "mmproj",
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
},
)
assert detect_mmproj_file(str(weight)) == str(correct.resolve())

216
studio/backend/tests/test_gguf_metadata.py Normal file
View file

@ -0,0 +1,216 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Tests for :mod:`utils.models.gguf_metadata`. Synthesise small GGUF
headers in tmp dirs so we never depend on real model files."""
from __future__ import annotations
import struct
from pathlib import Path
from typing import Iterable, Mapping
from utils.models.gguf_metadata import (
is_mmproj_by_metadata,
pairing_score,
read_gguf_general_metadata,
)
_GGUF_MAGIC = 0x46554747
_VTYPE_STRING = 8
_VTYPE_UINT32 = 4
_VTYPE_ARRAY = 9
def _enc_string(s: str) -> bytes:
b = s.encode("utf-8")
return struct.pack("<Q", len(b)) + b
def _enc_kv_string(key: str, value: str) -> bytes:
return _enc_string(key) + struct.pack("<I", _VTYPE_STRING) + _enc_string(value)
def _enc_kv_uint32(key: str, value: int) -> bytes:
return (
_enc_string(key) + struct.pack("<I", _VTYPE_UINT32) + struct.pack("<I", value)
)
def _enc_kv_string_array(key: str, values: Iterable[str]) -> bytes:
vals = list(values)
out = _enc_string(key) + struct.pack("<I", _VTYPE_ARRAY)
out += struct.pack("<I", _VTYPE_STRING) + struct.pack("<Q", len(vals))
for v in vals:
out += _enc_string(v)
return out
def _write_synthetic_gguf(
path: Path,
general_strings: Mapping[str, str],
*,
extra_uint32: Mapping[str, int] | None = None,
extra_string_arrays: Mapping[str, Iterable[str]] | None = None,
) -> Path:
"""Minimal GGUF: header + KV body, no tensors."""
extra_uint32 = extra_uint32 or {}
extra_string_arrays = extra_string_arrays or {}
kv_count = len(general_strings) + len(extra_uint32) + len(extra_string_arrays)
body = b""
for k, v in general_strings.items():
body += _enc_kv_string(k, v)
for k, v in extra_uint32.items():
body += _enc_kv_uint32(k, v)
for k, v in extra_string_arrays.items():
body += _enc_kv_string_array(k, v)
header = struct.pack(
"<IIQQ",
_GGUF_MAGIC,
3, # version
0, # tensor_count
kv_count,
)
path.parent.mkdir(parents = True, exist_ok = True)
path.write_bytes(header + body)
return path
# --- read_gguf_general_metadata ----------------------------------------
def test_returns_none_for_missing_file(tmp_path: Path):
assert read_gguf_general_metadata(str(tmp_path / "nope.gguf")) is None
def test_returns_none_for_non_gguf(tmp_path: Path):
p = tmp_path / "garbage.gguf"
p.write_bytes(b"not a gguf file at all, just bytes")
assert read_gguf_general_metadata(str(p)) is None
def test_extracts_general_string_fields(tmp_path: Path):
p = _write_synthetic_gguf(
tmp_path / "model.gguf",
{
"general.architecture": "qwen2vl",
"general.type": "model",
"general.basename": "Qwen3.5",
"general.organization": "Qwen",
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
"general.base_model.0.name": "Qwen3.5 9B",
"general.base_model.0.organization": "Qwen",
},
)
meta = read_gguf_general_metadata(str(p))
assert meta is not None
assert meta["general.architecture"] == "qwen2vl"
assert meta["general.basename"] == "Qwen3.5"
assert (
meta["general.base_model.0.repo_url"]
== "https://huggingface.co/Qwen/Qwen3.5-9B"
)
def test_skips_unrelated_fields_without_breaking(tmp_path: Path):
"""Skip unwanted arrays and uint32s without losing position."""
p = _write_synthetic_gguf(
tmp_path / "model.gguf",
{"general.basename": "Foo"},
extra_uint32 = {"qwen2vl.context_length": 32768},
extra_string_arrays = {"tokenizer.ggml.tokens": ["a", "bc", "def"]},
)
meta = read_gguf_general_metadata(str(p))
assert meta == {"general.basename": "Foo"}
def test_metadata_is_cached(tmp_path: Path):
"""Cache invalidates on size change."""
p = _write_synthetic_gguf(
tmp_path / "model.gguf",
{"general.basename": "First"},
)
first = read_gguf_general_metadata(str(p))
assert first == {"general.basename": "First"}
# Force size change so the (path, mtime, size) key invalidates.
_write_synthetic_gguf(
tmp_path / "model.gguf",
{"general.basename": "Second", "general.organization": "X"},
)
second = read_gguf_general_metadata(str(p))
assert second == {"general.basename": "Second", "general.organization": "X"}
# --- is_mmproj_by_metadata --------------------------------------------
def test_is_mmproj_by_metadata_signals():
assert is_mmproj_by_metadata({"general.type": "mmproj"}) is True
assert is_mmproj_by_metadata({"general.type": "MMProj"}) is True
assert is_mmproj_by_metadata({"general.type": "model"}) is False
assert is_mmproj_by_metadata({"general.basename": "foo"}) is None
assert is_mmproj_by_metadata({}) is None
assert is_mmproj_by_metadata(None) is None
# --- pairing_score -----------------------------------------------------
def test_pairing_score_base_model_url_match():
weight = {
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
}
mmproj = {
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
}
assert pairing_score(weight, mmproj) == 100
def test_pairing_score_base_model_url_mismatch():
weight = {
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
}
mmproj = {
"general.base_model.0.repo_url": "https://huggingface.co/google/gemma-3-9B",
}
assert pairing_score(weight, mmproj) == -1
def test_pairing_score_base_model_url_trailing_slash_normalised():
weight = {
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B/",
}
mmproj = {
"general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3.5-9B",
}
assert pairing_score(weight, mmproj) == 100
def test_pairing_score_basename_plus_org_fallback():
weight = {
"general.basename": "Nanonets-Ocr-S",
"general.base_model.0.organization": "Nanonets",
}
mmproj = {
"general.basename": "Nanonets-Ocr-S",
"general.base_model.0.organization": "Nanonets",
}
assert pairing_score(weight, mmproj) == 80
def test_pairing_score_basename_only_fallback():
assert (
pairing_score(
{"general.basename": "Nanonets-Ocr-S"},
{"general.basename": "Nanonets-Ocr-S"},
)
== 60
)
def test_pairing_score_no_overlap_returns_zero():
"""One side empty: scorer punts to filename fallback."""
assert pairing_score({"general.basename": "Foo"}, {}) == 0
assert pairing_score({}, {"general.basename": "Foo"}) == 0
assert pairing_score(None, {"general.basename": "Foo"}) == 0

9
studio/backend/tests/test_mlx_inference_backend.py
View file

@ -56,11 +56,14 @@ def _install_fake_fast_mlx(monkeypatch, calls):
return _DummyModel(), _DummyTokenizer()
unsloth_zoo_pkg = types.ModuleType("unsloth_zoo")
mlx_loader = types.ModuleType("unsloth_zoo.mlx_loader")
mlx_pkg = types.ModuleType("unsloth_zoo.mlx")
mlx_loader = types.ModuleType("unsloth_zoo.mlx.loader")
mlx_loader.FastMLXModel = _FastMLXModel
unsloth_zoo_pkg.mlx_loader = mlx_loader
unsloth_zoo_pkg.mlx = mlx_pkg
mlx_pkg.loader = mlx_loader
monkeypatch.setitem(sys.modules, "unsloth_zoo", unsloth_zoo_pkg)
monkeypatch.setitem(sys.modules, "unsloth_zoo.mlx_loader", mlx_loader)
monkeypatch.setitem(sys.modules, "unsloth_zoo.mlx", mlx_pkg)
monkeypatch.setitem(sys.modules, "unsloth_zoo.mlx.loader", mlx_loader)
def test_mlx_inference_text_load_forwards_studio_settings(monkeypatch):

1
studio/backend/tests/test_mlx_training_worker_config.py
View file

@ -37,6 +37,7 @@ def _load_worker_module():
for name in (
"direct_wheel_url",
"flash_attn_wheel_url",
"has_blackwell_gpu",
"install_wheel",
"probe_torch_wheel_env",
"url_exists",

391
studio/backend/tests/test_openai_code_execution.py Normal file
View file

@ -0,0 +1,391 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Unit tests for OpenAI's server-side `shell` tool translation in
`_stream_openai_responses`.
Covers:
- Request body: ``enabled_tools=["code_execution"]`` on the OpenAI
cloud base_url appends ``{"type": "shell", "environment": {"type":
"container_auto"}}`` to ``tools``.
- Container reuse: when ``openai_code_exec_container_id`` is provided,
the outgoing ``environment.type`` flips to ``"container_reference"``
and the id propagates.
- Cloud guard: code_execution on a non-cloud base_url (e.g. a local
OpenAI-compat preset / ollama / llama.cpp / vLLM) does NOT add the
shell tool, preventing a guaranteed 400 from those servers.
- SSE translation: a `shell_call` + `shell_call_output` pair emits one
``_toolEvent`` `tool_start` (`tool_name="code_execution"`,
`arguments.kind="bash"`) and one `tool_end` whose `result` contains
the joined stdout from the shell_call_output entries.
- Container surfacing: container_id captured from
`response.completed.container_id` is emitted as a synthetic
`container_ready` `_toolEvent` (only when it differs from the
inbound id).
- Stale-container handling: 400 with "container expired" body emits a
`container_invalidated` event before propagating the error.
"""
import asyncio
import json
import httpx
from core.inference import external_provider as ep_mod
from core.inference.external_provider import ExternalProviderClient
def _drive(coro):
return asyncio.new_event_loop().run_until_complete(coro)
async def _collect(agen):
out = []
async for line in agen:
out.append(line)
return out
def _mock_http_client(monkeypatch, handler):
transport = httpx.MockTransport(handler)
monkeypatch.setattr(ep_mod, "_http_client", httpx.AsyncClient(transport = transport))
def _make_client(base_url: str = "https://api.openai.com/v1") -> ExternalProviderClient:
return ExternalProviderClient(
provider_type = "openai",
base_url = base_url,
api_key = "sk-test",
)
def _openai_sse(events: list[dict]) -> bytes:
chunks: list[str] = []
for event in events:
chunks.append(f"event: {event['type']}")
chunks.append(f"data: {json.dumps(event)}")
chunks.append("")
return ("\n".join(chunks) + "\n").encode("utf-8")
def _tool_events(lines: list[str]) -> list[dict]:
out: list[dict] = []
for line in lines:
if not line.startswith("data:"):
continue
raw = line[len("data:") :].strip()
if not raw or raw == "[DONE]":
continue
try:
parsed = json.loads(raw)
except json.JSONDecodeError:
continue
if isinstance(parsed, dict) and "_toolEvent" in parsed:
out.append(parsed["_toolEvent"])
return out
def test_shell_tool_added_on_cloud_with_container_auto(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _openai_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "compute 2+2"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enable_thinking = None,
reasoning_effort = None,
enabled_tools = ["code_execution"],
):
pass
await client.close()
_drive(run())
tools = captured["body"].get("tools") or []
assert {
"type": "shell",
"environment": {"type": "container_auto"},
} in tools
def test_shell_tool_uses_container_reference_when_id_supplied(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _openai_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "what did i write earlier"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enable_thinking = None,
reasoning_effort = None,
enabled_tools = ["code_execution"],
openai_code_exec_container_id = "cntr_abc123",
):
pass
await client.close()
_drive(run())
tools = captured["body"].get("tools") or []
assert {
"type": "shell",
"environment": {
"type": "container_reference",
"container_id": "cntr_abc123",
},
} in tools
def test_shell_tool_refused_for_non_cloud_base_url(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _openai_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client(base_url = "http://localhost:11434/v1")
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enable_thinking = None,
reasoning_effort = None,
enabled_tools = ["code_execution"],
):
pass
await client.close()
_drive(run())
tools = captured["body"].get("tools") or []
# Shell tool must NOT leak to local OpenAI-compat servers — those
# 400 on the unknown tool type.
assert all(t.get("type") != "shell" for t in tools)
def test_shell_call_emits_tool_start_and_end(monkeypatch):
sse_events = [
{
"type": "response.output_item.added",
"item": {
"type": "shell_call",
"id": "scall_1",
"action": {"commands": ["ls -la"]},
},
},
{
"type": "response.output_item.done",
"item": {
"type": "shell_call",
"id": "scall_1",
"action": {"commands": ["ls -la"]},
"status": "completed",
},
},
{
"type": "response.output_item.done",
"item": {
"type": "shell_call_output",
"id": "scout_1",
"call_id": "scall_1",
"output": [
{
"stdout": "total 24\ndrwxr-xr-x .",
"stderr": "",
"outcome": {"type": "exit", "exit_code": 0},
}
],
},
},
{"type": "response.completed", "response": {}},
]
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = _openai_sse(sse_events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
return await _collect(
client._stream_openai_responses(
messages = [{"role": "user", "content": "list files"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enable_thinking = None,
reasoning_effort = None,
enabled_tools = ["code_execution"],
)
)
lines = _drive(run())
events = _tool_events(lines)
starts = [e for e in events if e["type"] == "tool_start"]
ends = [e for e in events if e["type"] == "tool_end"]
assert len(starts) == 1
assert len(ends) == 1
assert starts[0]["tool_name"] == "code_execution"
assert starts[0]["tool_call_id"] == "scall_1"
assert starts[0]["arguments"] == {"kind": "bash", "command": "ls -la"}
assert ends[0]["tool_call_id"] == "scall_1"
assert "total 24" in ends[0]["result"]
def test_container_ready_emitted_when_new_id_surfaces(monkeypatch):
sse_events = [
{
"type": "response.completed",
"response": {"container_id": "cntr_new_456"},
},
]
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = _openai_sse(sse_events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
return await _collect(
client._stream_openai_responses(
messages = [{"role": "user", "content": "do stuff"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enable_thinking = None,
reasoning_effort = None,
enabled_tools = ["code_execution"],
)
)
lines = _drive(run())
events = _tool_events(lines)
ready = [e for e in events if e["type"] == "container_ready"]
assert len(ready) == 1
assert ready[0]["container_id"] == "cntr_new_456"
def test_container_ready_not_emitted_when_id_unchanged(monkeypatch):
sse_events = [
{
"type": "response.completed",
"response": {"container_id": "cntr_same_789"},
},
]
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = _openai_sse(sse_events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
return await _collect(
client._stream_openai_responses(
messages = [{"role": "user", "content": "do stuff"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enable_thinking = None,
reasoning_effort = None,
enabled_tools = ["code_execution"],
openai_code_exec_container_id = "cntr_same_789",
)
)
lines = _drive(run())
events = _tool_events(lines)
# No churn — id matches the one already on the thread record.
assert not any(e["type"] == "container_ready" for e in events)
def test_stale_container_emits_invalidated(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
400,
content = json.dumps(
{
"error": {
"message": "container has expired",
"type": "invalid_request_error",
}
}
).encode("utf-8"),
headers = {"content-type": "application/json"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
return await _collect(
client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4096,
enable_thinking = None,
reasoning_effort = None,
enabled_tools = ["code_execution"],
openai_code_exec_container_id = "cntr_stale_999",
)
)
lines = _drive(run())
events = _tool_events(lines)
invalidated = [e for e in events if e["type"] == "container_invalidated"]
assert len(invalidated) == 1

201
studio/backend/tests/test_openai_container_crud.py Normal file
View file

@ -0,0 +1,201 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Unit tests for the /v1/containers CRUD client methods.
Covers:
- All three calls (list / create / delete) send
``OpenAI-Beta: containers=v1``. Without it, OpenAI silently no-ops
the DELETE while still returning 200 ``{"deleted": true}``.
- ``delete_openai_container`` raises when the response body does not
report ``{"deleted": true}``, even on a 2xx response.
"""
from __future__ import annotations
import asyncio
import json
import httpx
import pytest
from core.inference import external_provider as ep_mod
from core.inference.external_provider import ExternalProviderClient
def _drive(coro):
return asyncio.new_event_loop().run_until_complete(coro)
def _mock_http_client(monkeypatch, handler):
"""Wire `handler` for both the shared `_http_client` AND any
per-call `httpx.AsyncClient(...)` instances. delete_openai_container
intentionally creates a fresh AsyncClient (see comment in
external_provider.delete_openai_container) so the test must
also intercept that constructor."""
transport = httpx.MockTransport(handler)
monkeypatch.setattr(ep_mod, "_http_client", httpx.AsyncClient(transport = transport))
real_async_client = httpx.AsyncClient
def _patched_async_client(*args, **kwargs):
kwargs["transport"] = transport
return real_async_client(*args, **kwargs)
monkeypatch.setattr(ep_mod.httpx, "AsyncClient", _patched_async_client)
def _make_client() -> ExternalProviderClient:
return ExternalProviderClient(
provider_type = "openai",
base_url = "https://api.openai.com/v1",
api_key = "sk-test",
)
def test_list_sends_openai_beta_header(monkeypatch):
seen: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
seen["headers"] = dict(request.headers)
seen["url"] = str(request.url)
return httpx.Response(
200,
json = {"data": [{"id": "cntr_x", "name": "auto"}]},
)
_mock_http_client(monkeypatch, handler)
result = _drive(_make_client().list_openai_containers())
assert result == [{"id": "cntr_x", "name": "auto"}]
assert seen["headers"].get("openai-beta") == "containers=v1"
assert seen["url"] == "https://api.openai.com/v1/containers"
def test_create_sends_openai_beta_header(monkeypatch):
seen: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
seen["headers"] = dict(request.headers)
seen["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(200, json = {"id": "cntr_new", "name": "analysis"})
_mock_http_client(monkeypatch, handler)
result = _drive(
_make_client().create_openai_container(name = "analysis", ttl_minutes = 30)
)
assert result == {"id": "cntr_new", "name": "analysis"}
assert seen["headers"].get("openai-beta") == "containers=v1"
assert seen["body"]["name"] == "analysis"
assert seen["body"]["expires_after"] == {
"anchor": "last_active_at",
"minutes": 30,
}
def test_delete_sends_openai_beta_header_and_accepts_confirmation(monkeypatch):
seen: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
seen["headers"] = dict(request.headers)
seen["url"] = str(request.url)
seen["method"] = request.method
return httpx.Response(
200,
json = {"id": "cntr_x", "object": "container.deleted", "deleted": True},
)
_mock_http_client(monkeypatch, handler)
_drive(_make_client().delete_openai_container("cntr_x"))
assert seen["method"] == "DELETE"
assert seen["url"] == "https://api.openai.com/v1/containers/cntr_x"
assert seen["headers"].get("openai-beta") == "containers=v1"
def test_delete_raises_when_response_lacks_deleted_true(monkeypatch):
"""OpenAI returns 200 ``{"deleted": true}`` even when the request is
silently rejected (e.g. before we started sending OpenAI-Beta).
Defensive guard: when the body omits ``deleted: true``, surface it
as an error so the UI can report the failure instead of falsely
reporting success."""
def handler(request: httpx.Request) -> httpx.Response:
# 200 but no deleted flag — simulate an unexpected payload shape.
return httpx.Response(200, json = {"id": "cntr_x", "object": "container"})
_mock_http_client(monkeypatch, handler)
with pytest.raises(httpx.HTTPError, match = "did not confirm container deletion"):
_drive(_make_client().delete_openai_container("cntr_x"))
def test_delete_raises_when_deleted_is_false(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
json = {"id": "cntr_x", "object": "container.deleted", "deleted": False},
)
_mock_http_client(monkeypatch, handler)
with pytest.raises(httpx.HTTPError, match = "did not confirm container deletion"):
_drive(_make_client().delete_openai_container("cntr_x"))
def test_delete_raises_when_body_is_not_json(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(200, content = b"<html>OK</html>")
_mock_http_client(monkeypatch, handler)
with pytest.raises(httpx.HTTPError, match = "did not confirm container deletion"):
_drive(_make_client().delete_openai_container("cntr_x"))
def test_delete_propagates_openai_4xx(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(404, json = {"error": {"message": "not found"}})
_mock_http_client(monkeypatch, handler)
with pytest.raises(httpx.HTTPStatusError):
_drive(_make_client().delete_openai_container("cntr_missing"))
def test_list_route_filters_expired_containers(monkeypatch):
"""OpenAI keeps containers in /v1/containers indefinitely with
status="expired" after their idle TTL passes — they can't be
used but still show up. The list route must drop them so the
picker only surfaces usable containers."""
from routes import inference as inf_mod
from models.inference import OpenAIContainerRequest
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
json = {
"data": [
{"id": "cntr_active", "name": "live", "status": "running"},
{"id": "cntr_dead", "name": "old", "status": "expired"},
{"id": "cntr_unknown", "name": "no-status"},
],
},
)
_mock_http_client(monkeypatch, handler)
def fake_resolve(_body):
return _make_client()
monkeypatch.setattr(inf_mod, "_resolve_openai_cloud_client", fake_resolve)
body = OpenAIContainerRequest(
encrypted_api_key = "enc",
provider_base_url = "https://api.openai.com/v1",
)
response = _drive(inf_mod.list_openai_containers(body, current_subject = "u"))
ids = [c.id for c in response.containers]
assert "cntr_active" in ids
assert "cntr_unknown" in ids # missing status is treated as usable
assert "cntr_dead" not in ids

494
studio/backend/tests/test_openai_responses_translation.py Normal file
View file

@ -0,0 +1,494 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Unit tests for the OpenAI `/v1/responses` translation in external_provider.
Covers:
- Request body shape: system messages collapse into `instructions`, user/
assistant messages go into `input`, sampling knobs Responses does not
support (presence_penalty, top_k) are not forwarded.
- SSE translation: `response.output_text.delta` events become OpenAI Chat
Completions chunks, `response.completed` emits a `finish_reason: stop`
chunk, the stream terminates with `data: [DONE]`.
- Image parts in user content are rewritten from Chat Completions
`{type: image_url, image_url: {url}}` into Responses
`{type: input_image, image_url: <url>}`.
"""
import asyncio
import json
import httpx
from core.inference import external_provider as ep_mod
from core.inference.external_provider import ExternalProviderClient
def _drive(coro):
return asyncio.new_event_loop().run_until_complete(coro)
async def _collect(agen):
out = []
async for line in agen:
out.append(line)
return out
def _mock_http_client(monkeypatch, handler):
transport = httpx.MockTransport(handler)
monkeypatch.setattr(ep_mod, "_http_client", httpx.AsyncClient(transport = transport))
def _make_client() -> ExternalProviderClient:
return ExternalProviderClient(
provider_type = "openai",
base_url = "https://api.openai.com/v1",
api_key = "sk-test",
)
def _responses_sse(events: list[dict]) -> bytes:
"""Serialize a list of Responses-API event dicts as an SSE byte stream."""
chunks: list[str] = []
for event in events:
chunks.append(f"event: {event['type']}")
chunks.append(f"data: {json.dumps(event)}")
chunks.append("")
chunks.append("data: [DONE]")
chunks.append("")
return ("\n".join(chunks) + "\n").encode("utf-8")
def test_responses_request_body_uses_input_and_instructions(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["url"] = str(request.url)
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [
{"role": "system", "content": "You are concise."},
{"role": "user", "content": "Hi"},
],
model = "gpt-5.5",
temperature = 0.5,
top_p = 0.9,
max_tokens = 512,
enable_thinking = None,
reasoning_effort = None,
):
pass
await client.close()
_drive(run())
assert captured["url"] == "https://api.openai.com/v1/responses"
body = captured["body"]
assert body["model"] == "gpt-5.5"
assert body["instructions"] == "You are concise."
assert body["input"] == [{"role": "user", "content": "Hi"}]
assert body["max_output_tokens"] == 512
assert body["stream"] is True
# Responses API on reasoning-class models (gpt-5.x / o3 / gpt-4.5 — the
# only OpenAI ids the registry allowlist exposes) rejects these as
# `Unsupported parameter`. Make sure we never silently forward them.
assert "temperature" not in body
assert "top_p" not in body
assert "presence_penalty" not in body
assert "frequency_penalty" not in body
assert "top_k" not in body
assert "messages" not in body
def test_responses_translates_image_parts(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "What is this?"},
{
"type": "image_url",
"image_url": {"url": "data:image/png;base64,AAA"},
},
],
}
],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = None,
):
pass
await client.close()
_drive(run())
parts = captured["body"]["input"][0]["content"]
assert parts[0] == {"type": "input_text", "text": "What is this?"}
assert parts[1] == {
"type": "input_image",
"image_url": "data:image/png;base64,AAA",
}
# No max_output_tokens key when caller passes max_tokens=None.
assert "max_output_tokens" not in captured["body"]
def test_responses_sse_translates_to_chat_completions_chunks(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
events = [
{"type": "response.created"},
{"type": "response.output_text.delta", "delta": "Hello"},
{"type": "response.output_text.delta", "delta": ", world"},
{"type": "response.completed", "response": {}},
]
return httpx.Response(
200,
content = _responses_sse(events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
lines = await _collect(
client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = None,
)
)
await client.close()
return lines
lines = _drive(run())
# Drop empty / non-data lines for assertion clarity.
data_lines = [line for line in lines if line.startswith("data:")]
payloads = []
for line in data_lines:
raw = line[len("data:") :].strip()
if raw == "[DONE]":
payloads.append("[DONE]")
else:
payloads.append(json.loads(raw))
# Two text deltas, one terminal chunk, then [DONE].
assert payloads[0]["choices"][0]["delta"]["content"] == "Hello"
assert payloads[0]["choices"][0]["finish_reason"] is None
assert payloads[1]["choices"][0]["delta"]["content"] == ", world"
assert payloads[2]["choices"][0]["delta"] == {}
assert payloads[2]["choices"][0]["finish_reason"] == "stop"
assert payloads[-1] == "[DONE]"
def test_responses_response_incomplete_maps_to_length_finish_reason(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
events = [
{"type": "response.output_text.delta", "delta": "partial"},
{"type": "response.incomplete", "response": {}},
]
return httpx.Response(
200,
content = _responses_sse(events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
lines = await _collect(
client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 4,
enable_thinking = None,
reasoning_effort = None,
)
)
await client.close()
return lines
lines = _drive(run())
finish_reasons = [
json.loads(line[len("data:") :].strip())["choices"][0]["finish_reason"]
for line in lines
if line.startswith("data:")
and line[len("data:") :].strip() not in ("", "[DONE]")
]
assert "length" in finish_reasons
def test_responses_reasoning_effort_included_when_requested(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = "high",
):
pass
await client.close()
_drive(run())
assert captured["body"]["reasoning"] == {"effort": "high", "summary": "auto"}
def test_responses_reasoning_summary_omitted_for_o3(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "o3",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = "high",
):
pass
await client.close()
_drive(run())
assert captured["body"]["reasoning"] == {"effort": "high"}
def test_responses_reasoning_summary_omitted_for_o3_with_enable_thinking(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "o3",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = True,
reasoning_effort = None,
):
pass
await client.close()
_drive(run())
assert captured["body"]["reasoning"] == {"effort": "medium"}
def test_responses_reasoning_effort_none_omits_summary(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = "none",
):
pass
await client.close()
_drive(run())
assert captured["body"]["reasoning"] == {"effort": "none"}
def test_responses_reasoning_effort_xhigh_passthrough(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = "xhigh",
):
pass
await client.close()
_drive(run())
assert captured["body"]["reasoning"] == {"effort": "xhigh", "summary": "auto"}
def test_responses_enable_thinking_false_maps_to_reasoning_none(monkeypatch):
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _responses_sse([{"type": "response.completed", "response": {}}]),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
async for _ in client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = False,
reasoning_effort = None,
):
pass
await client.close()
_drive(run())
assert captured["body"]["reasoning"] == {"effort": "none"}
def test_responses_reasoning_summary_wrapped_in_think_tags(monkeypatch):
def handler(request: httpx.Request) -> httpx.Response:
events = [
{
"type": "response.output_item.done",
"item": {
"type": "reasoning",
"summary": [{"type": "summary_text", "text": "plan"}],
},
},
{"type": "response.output_text.delta", "delta": "answer"},
{"type": "response.completed", "response": {}},
]
return httpx.Response(
200,
content = _responses_sse(events),
headers = {"content-type": "text/event-stream"},
)
_mock_http_client(monkeypatch, handler)
async def run():
client = _make_client()
lines = await _collect(
client._stream_openai_responses(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = None,
enable_thinking = None,
reasoning_effort = None,
)
)
await client.close()
return lines
lines = _drive(run())
data_lines = [
line[len("data:") :].strip()
for line in lines
if line.startswith("data:")
and line[len("data:") :].strip() not in ("", "[DONE]")
]
payloads = [json.loads(raw) for raw in data_lines]
combined = "".join(
payload["choices"][0]["delta"].get("content", "")
for payload in payloads
if payload["choices"][0]["delta"]
)
assert "<think>plan</think>answer" in combined

609
studio/backend/tests/test_providers_api.py Normal file
View file

@ -0,0 +1,609 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""
Integration tests for the external providers API.
Requires a running Unsloth Studio server. Configure via environment variables:
export STUDIO_TEST_URL="http://localhost:8888" # default
export STUDIO_TEST_USER="unsloth" # default
export STUDIO_TEST_PASSWORD="..." # required — see .bootstrap_password
# Provider API keys — any left unset will have their tests automatically skipped
export OPENAI_API_KEY="sk-..."
export MISTRAL_API_KEY="..."
export GOOGLE_API_KEY="..."
export TOGETHER_API_KEY="..."
export FIREWORKS_API_KEY="..."
export PERPLEXITY_API_KEY="..."
Run:
cd studio/backend
pytest tests/test_providers_api.py -v -s
"""
import base64
import json
import os
import pytest
import requests
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import padding
# ── Configuration ─────────────────────────────────────────────────
BASE_URL = os.getenv("STUDIO_TEST_URL", "http://localhost:8000")
USERNAME = os.getenv("STUDIO_TEST_USER", "unsloth")
PASSWORD = os.getenv("STUDIO_TEST_PASSWORD", "")
# These tests require a live Studio server reachable at BASE_URL with a known
# bootstrap password. Skip the whole module when that environment is missing
# (e.g. on CI runners) so pytest discovery does not error out.
pytestmark = pytest.mark.skipif(
not PASSWORD,
reason = "Integration test requires a running Studio server; set STUDIO_TEST_PASSWORD to enable.",
)
# Map provider_type → (env var name, model to use for inference test)
_PROVIDER_CONFIGS: dict[str, tuple[str, str]] = {
"openai": ("OPENAI_API_KEY", "gpt-4o-mini"),
"mistral": ("MISTRAL_API_KEY", "mistral-small-2506"),
"gemini": ("GEMINI_API_KEY", "gemini-3-flash-preview"),
"openrouter": ("OPENROUTER_API_KEY", "openai/gpt-4o-mini"),
"anthropic": ("ANTHROPIC_API_KEY", "claude-haiku-4-5"),
"deepseek": ("DEEPSEEK_API_KEY", "deepseek-chat"),
"huggingface": ("HUGGINGFACE_API_KEY", "meta-llama/Llama-3.3-70B-Instruct"),
"kimi": ("MOONSHOT_API_KEY", "moonshot-v1-8k"),
"qwen": ("DASHSCOPE_API_KEY", "qwen-turbo"),
}
PROVIDER_KEYS: dict[str, str] = {
ptype: os.getenv(env_var, "") for ptype, (env_var, _) in _PROVIDER_CONFIGS.items()
}
EXPECTED_PROVIDER_TYPES = set(_PROVIDER_CONFIGS.keys())
# ── Helpers ────────────────────────────────────────────────────────
def _url(path: str) -> str:
return f"{BASE_URL}/{path.lstrip('/')}"
def _parse_sse_stream(response: requests.Response) -> tuple[str, bool]:
"""
Read a streaming SSE response and return (assembled_text, saw_done).
Each chunk is a JSON object with choices[0].delta.content.
The stream ends with `data: [DONE]`.
"""
reply_parts: list[str] = []
saw_done = False
for raw_line in response.iter_lines():
if isinstance(raw_line, bytes):
raw_line = raw_line.decode("utf-8")
if not raw_line.startswith("data:"):
continue
data = raw_line[len("data:") :].strip()
if data == "[DONE]":
saw_done = True
break
try:
chunk = json.loads(data)
# Handle both error payloads and normal chunks
if "error" in chunk:
raise RuntimeError(f"Provider error in stream: {chunk['error']}")
delta = chunk.get("choices", [{}])[0].get("delta", {})
content = delta.get("content") or ""
if content:
reply_parts.append(content)
except (json.JSONDecodeError, IndexError, KeyError):
pass # skip malformed lines
return "".join(reply_parts), saw_done
# ── Session-scoped fixtures ────────────────────────────────────────
@pytest.fixture(scope = "session")
def auth_headers() -> dict[str, str]:
"""
Log in once per session and return auth headers.
On a fresh Studio install the bootstrap password triggers a forced password
change (must_change_password=True). Any subsequent API call using that token
returns 403 "Password change required". This fixture detects that state,
automatically completes the change-password flow, and re-logs in so all other
tests get a fully usable token.
The new password used during auto-change is:
STUDIO_TEST_NEW_PASSWORD (env var, optional)
or PASSWORD + "-test" (derived default)
On the second run, set STUDIO_TEST_PASSWORD to the new password.
"""
assert PASSWORD, (
"STUDIO_TEST_PASSWORD is not set.\n"
"Run: export STUDIO_TEST_PASSWORD=$(cat studio/backend/.bootstrap_password)"
)
resp = requests.post(
_url("/api/auth/login"),
json = {"username": USERNAME, "password": PASSWORD},
timeout = 10,
)
assert resp.status_code == 200, f"Login failed ({resp.status_code}): {resp.text}"
body = resp.json()
token = body["access_token"]
assert token, "access_token is empty"
if body.get("must_change_password"):
# Bootstrap token is restricted — only /api/auth/change-password works with it.
# Auto-complete the forced change so the rest of the tests get a full token.
new_password = os.getenv("STUDIO_TEST_NEW_PASSWORD") or f"{PASSWORD}-test"
change_resp = requests.post(
_url("/api/auth/change-password"),
headers = {"Authorization": f"Bearer {token}"},
json = {"current_password": PASSWORD, "new_password": new_password},
timeout = 10,
)
assert (
change_resp.status_code == 200
), f"Auto password-change failed ({change_resp.status_code}): {change_resp.text}"
token = change_resp.json()["access_token"]
return {"Authorization": f"Bearer {token}"}
@pytest.fixture(scope = "session")
def public_key_pem(auth_headers: dict[str, str]) -> str:
"""Fetch RSA public key PEM once per session."""
resp = requests.get(
_url("/api/providers/public-key"),
headers = auth_headers,
timeout = 10,
)
assert resp.status_code == 200, f"Public key fetch failed: {resp.text}"
pem = resp.json().get("public_key", "")
assert pem.startswith("-----BEGIN PUBLIC KEY-----"), "Not a valid PEM public key"
return pem
@pytest.fixture(scope = "session")
def vision_image_data_url() -> str:
"""
Download the sloth image once per session and return it as a base64 data URI.
Using a data URI instead of a remote URL ensures every provider receives
the image inline — Gemini's OpenAI-compatible layer does not fetch external
HTTP URLs, so raw image_url links silently produce empty replies for Gemini.
"""
resp = requests.get(_VISION_IMAGE_URL, timeout = 30)
resp.raise_for_status()
content_type = resp.headers.get("Content-Type", "image/jpeg").split(";")[0].strip()
b64 = base64.b64encode(resp.content).decode("utf-8")
return f"data:{content_type};base64,{b64}"
@pytest.fixture(scope = "session")
def encrypt_key(public_key_pem: str):
"""
Return a callable encrypt_key(plaintext: str) -> str (base64 RSA-OAEP ciphertext).
Uses the backend's RSA public key — mirrors what the frontend does.
"""
# Decode PEM → load RSA public key
pem_bytes = public_key_pem.encode("utf-8")
rsa_pub = serialization.load_pem_public_key(pem_bytes)
def _encrypt(plaintext: str) -> str:
ciphertext = rsa_pub.encrypt(
plaintext.encode("utf-8"),
padding.OAEP(
mgf = padding.MGF1(algorithm = hashes.SHA256()),
algorithm = hashes.SHA256(),
label = None,
),
)
return base64.b64encode(ciphertext).decode("utf-8")
return _encrypt
# ── TestAuth ────────────────────────────────────────────────────────
class TestAuth:
def test_login_returns_token(self):
"""POST /api/auth/login returns a non-empty access_token."""
assert PASSWORD, "STUDIO_TEST_PASSWORD not set"
resp = requests.post(
_url("/api/auth/login"),
json = {"username": USERNAME, "password": PASSWORD},
timeout = 10,
)
assert (
resp.status_code == 200
), f"Login failed ({resp.status_code}): {resp.text}"
body = resp.json()
assert body.get("access_token"), "access_token is missing or empty"
assert body.get("token_type") == "bearer"
# ── TestPublicKey ────────────────────────────────────────────────────
class TestPublicKey:
def test_public_key_is_valid_pem(
self, auth_headers: dict[str, str], public_key_pem: str
):
"""GET /api/providers/public-key returns an importable RSA PEM key."""
pem_bytes = public_key_pem.encode("utf-8")
key = serialization.load_pem_public_key(pem_bytes)
key_size = key.key_size # type: ignore[attr-defined]
assert key_size >= 2048, f"Key size too small: {key_size}"
print(f"\n RSA-{key_size} public key OK")
# ── TestRegistry ────────────────────────────────────────────────────
class TestRegistry:
def test_registry_returns_all_providers(self, auth_headers: dict[str, str]):
"""GET /api/providers/registry returns all supported providers."""
resp = requests.get(
_url("/api/providers/registry"),
headers = auth_headers,
timeout = 10,
)
assert resp.status_code == 200, f"Registry failed: {resp.text}"
providers = resp.json()
assert (
len(providers) == 9
), f"Expected 9 providers, got {len(providers)}: {providers}"
print(f"\n {'Provider':<12} {'Base URL'}")
print(f" {'-'*12} {'-'*45}")
for p in providers:
print(f" {p['provider_type']:<12} {p['base_url']}")
def test_registry_has_expected_types(self, auth_headers: dict[str, str]):
"""All expected provider_type values are present in the registry."""
resp = requests.get(
_url("/api/providers/registry"),
headers = auth_headers,
timeout = 10,
)
assert resp.status_code == 200
returned_types = {p["provider_type"] for p in resp.json()}
missing = EXPECTED_PROVIDER_TYPES - returned_types
assert not missing, f"Missing provider types: {missing}"
def test_registry_entries_have_required_fields(self, auth_headers: dict[str, str]):
"""Each registry entry has provider_type, display_name, base_url, default_models."""
resp = requests.get(
_url("/api/providers/registry"), headers = auth_headers, timeout = 10
)
assert resp.status_code == 200
for entry in resp.json():
for field in (
"provider_type",
"display_name",
"base_url",
"default_models",
"model_list_mode",
):
assert field in entry, f"Missing field '{field}' in entry: {entry}"
assert entry["model_list_mode"] in ("remote", "curated")
assert isinstance(entry["default_models"], list)
assert len(entry["default_models"]) > 0
# ── TestProviderCRUD ────────────────────────────────────────────────
class TestProviderCRUD:
"""
These tests run sequentially within the class and share state via class variables.
They create, read, update, and delete a single test provider config.
"""
_created_id: str = ""
def test_create_provider(self, auth_headers: dict[str, str]):
"""POST /api/providers/ creates a provider config and returns 201."""
resp = requests.post(
_url("/api/providers/"),
headers = auth_headers,
json = {"provider_type": "openai", "display_name": "Test OpenAI (pytest)"},
timeout = 10,
)
assert (
resp.status_code == 201
), f"Create failed ({resp.status_code}): {resp.text}"
body = resp.json()
assert body.get("id"), "No id in response"
assert body["provider_type"] == "openai"
assert body["display_name"] == "Test OpenAI (pytest)"
assert body["is_enabled"] is True
TestProviderCRUD._created_id = body["id"]
print(f"\n created id={body['id']}")
def test_list_includes_created(self, auth_headers: dict[str, str]):
"""GET /api/providers/ includes the newly created config."""
assert (
TestProviderCRUD._created_id
), "No created_id (run test_create_provider first)"
resp = requests.get(_url("/api/providers/"), headers = auth_headers, timeout = 10)
assert resp.status_code == 200
ids = [p["id"] for p in resp.json()]
assert (
TestProviderCRUD._created_id in ids
), f"Created id {TestProviderCRUD._created_id!r} not found in list: {ids}"
print(f"\n found id={TestProviderCRUD._created_id} in list of {len(ids)}")
def test_update_display_name(self, auth_headers: dict[str, str]):
"""PUT /api/providers/{id} updates the display_name."""
assert TestProviderCRUD._created_id, "No created_id"
new_name = "Test OpenAI (pytest updated)"
resp = requests.put(
_url(f"/api/providers/{TestProviderCRUD._created_id}"),
headers = auth_headers,
json = {"display_name": new_name},
timeout = 10,
)
assert (
resp.status_code == 200
), f"Update failed ({resp.status_code}): {resp.text}"
assert resp.json()["display_name"] == new_name
print(f"\n updated display_name to '{new_name}'")
def test_delete_provider(self, auth_headers: dict[str, str]):
"""DELETE /api/providers/{id} removes the config (204) and it's gone from list."""
assert TestProviderCRUD._created_id, "No created_id"
resp = requests.delete(
_url(f"/api/providers/{TestProviderCRUD._created_id}"),
headers = auth_headers,
timeout = 10,
)
assert (
resp.status_code == 204
), f"Delete failed ({resp.status_code}): {resp.text}"
# Confirm gone from list
list_resp = requests.get(
_url("/api/providers/"), headers = auth_headers, timeout = 10
)
ids = [p["id"] for p in list_resp.json()]
assert TestProviderCRUD._created_id not in ids, "Deleted provider still in list"
print(f"\n deleted id={TestProviderCRUD._created_id} confirmed gone")
# ── TestProviderInference ────────────────────────────────────────────
# Build parametrize list: (provider_type, model, api_key) for configured providers only
_INFERENCE_PARAMS = [
pytest.param(
ptype,
model,
PROVIDER_KEYS.get(ptype, ""),
id = ptype,
marks = pytest.mark.skipif(
not PROVIDER_KEYS.get(ptype, ""),
reason = f"no {env_var} set",
),
)
for ptype, (env_var, model) in _PROVIDER_CONFIGS.items()
]
class TestProviderInference:
"""
Live inference tests — one parametrized set per provider.
Each test is automatically skipped when the provider's API key env var is not set.
"""
@pytest.mark.parametrize("provider_type,model,api_key", _INFERENCE_PARAMS)
def test_connection(
self,
auth_headers: dict[str, str],
encrypt_key,
provider_type: str,
model: str,
api_key: str,
):
"""POST /api/providers/test → success: true."""
encrypted = encrypt_key(api_key)
resp = requests.post(
_url("/api/providers/test"),
headers = auth_headers,
json = {"provider_type": provider_type, "encrypted_api_key": encrypted},
timeout = 30,
)
assert (
resp.status_code == 200
), f"Request failed ({resp.status_code}): {resp.text}"
body = resp.json()
assert (
body["success"] is True
), f"Connection test failed for {provider_type}: {body.get('message')}"
print(f"\n [{provider_type}] connection OK — {body['message']}")
@pytest.mark.parametrize("provider_type,model,api_key", _INFERENCE_PARAMS)
def test_list_models(
self,
auth_headers: dict[str, str],
encrypt_key,
provider_type: str,
model: str,
api_key: str,
):
"""POST /api/providers/models → non-empty list, print first 3."""
encrypted = encrypt_key(api_key)
resp = requests.post(
_url("/api/providers/models"),
headers = auth_headers,
json = {"provider_type": provider_type, "encrypted_api_key": encrypted},
timeout = 30,
)
assert (
resp.status_code == 200
), f"Request failed ({resp.status_code}): {resp.text}"
models = resp.json()
assert isinstance(models, list), f"Expected list, got {type(models)}"
assert len(models) > 0, f"No models returned for {provider_type}"
preview = [m["id"] for m in models[:3]]
print(f"\n [{provider_type}] {len(models)} models — first 3: {preview}")
@pytest.mark.parametrize("provider_type,model,api_key", _INFERENCE_PARAMS)
def test_chat_inference(
self,
auth_headers: dict[str, str],
encrypt_key,
provider_type: str,
model: str,
api_key: str,
):
"""POST /v1/chat/completions with provider fields → streamed reply."""
encrypted = encrypt_key(api_key)
payload = {
"messages": [{"role": "user", "content": "Say hello in one sentence."}],
"stream": True,
"temperature": 0.7,
"max_tokens": 64,
"provider_type": provider_type,
"external_model": model,
"encrypted_api_key": encrypted,
}
with requests.post(
_url("/v1/chat/completions"),
headers = {**auth_headers, "Content-Type": "application/json"},
json = payload,
stream = True,
timeout = 60,
) as resp:
assert (
resp.status_code == 200
), f"Chat completions failed ({resp.status_code}): {resp.text[:500]}"
reply, saw_done = _parse_sse_stream(resp)
assert reply.strip(), f"Empty reply from {provider_type}/{model}"
assert saw_done, f"Stream did not end with [DONE] for {provider_type}/{model}"
print(f'\n [{provider_type}/{model}] reply: "{reply.strip()}"')
# ── TestVisionInference ─────────────────────────────────────────────
# Sloth photo — used to test vision routing across providers
_VISION_IMAGE_URL = (
"https://www.travelexcellence.com/images/where-to-see-sloths-in-costa-rica.jpg"
)
_VISION_PARAMS = [
pytest.param(
ptype,
model,
PROVIDER_KEYS.get(ptype, ""),
id = ptype,
marks = pytest.mark.skipif(
not PROVIDER_KEYS.get(ptype, ""),
reason = f"no key for {ptype}",
),
)
for ptype, (_, model) in _PROVIDER_CONFIGS.items()
if ptype in {"openai", "mistral", "gemini", "anthropic", "openrouter"}
]
class TestVisionInference:
"""
Send a 1×1 white PNG alongside a text question to each vision-capable provider.
Verifies that image content parts survive the proxy and the provider replies.
"""
@pytest.mark.parametrize("provider_type,model,api_key", _VISION_PARAMS)
def test_vision_chat_inference(
self,
auth_headers: dict[str, str],
encrypt_key,
vision_image_data_url: str,
provider_type: str,
model: str,
api_key: str,
):
"""Image URL + text message → non-empty streamed reply."""
encrypted = encrypt_key(api_key)
payload = {
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Which animal is in this image? Reply in one word.",
},
{
"type": "image_url",
"image_url": {"url": vision_image_data_url},
},
],
}
],
"stream": True,
"max_tokens": 215,
"provider_type": provider_type,
"external_model": model,
"encrypted_api_key": encrypted,
}
with requests.post(
_url("/v1/chat/completions"),
headers = {**auth_headers, "Content-Type": "application/json"},
json = payload,
stream = True,
timeout = 60,
) as resp:
assert (
resp.status_code == 200
), f"Vision request failed ({resp.status_code}): {resp.text[:300]}"
reply, saw_done = _parse_sse_stream(resp)
assert reply.strip(), f"Empty reply from {provider_type}/{model}"
assert saw_done, f"Stream did not end with [DONE] for {provider_type}/{model}"
print(f"\n [{provider_type}/{model}] vision reply: {reply.strip()!r}")
# ── TestLocalInferenceUnaffected ────────────────────────────────────
class TestLocalInferenceUnaffected:
def test_chat_without_provider(self, auth_headers: dict[str, str]):
"""
POST /v1/chat/completions without provider fields must not return 422 or 500.
200 = a local model is loaded and responded.
503 = no model loaded (expected in test environment — that's fine).
Any other 4xx/5xx (except 503) = regression in request handling.
"""
resp = requests.post(
_url("/v1/chat/completions"),
headers = {**auth_headers, "Content-Type": "application/json"},
json = {
"messages": [{"role": "user", "content": "Hello"}],
"stream": False,
},
timeout = 15,
)
allowed = {200, 400, 503}
assert resp.status_code in allowed, (
f"Unexpected status {resp.status_code} for local inference path: {resp.text[:300]}\n"
f"This likely means the provider fields broke the base request schema."
)
status_label = (
"local model responded"
if resp.status_code == 200
else "no model loaded (expected)"
)
print(f"\n status={resp.status_code} ({status_label}) — local path unaffected")

42
studio/backend/tests/test_training_raw_support.py
View file

@ -70,6 +70,48 @@ class TestTrainingRawSupport(unittest.TestCase):
self.assertTrue(config["load_in_4bit"])
self.assertEqual(config["embedding_learning_rate"], 1e-5)
def test_training_backend_forwards_grad_clipping_controls(self):
backend = TrainingBackend()
class DummyProcess:
pid = 12345
def start(self):
return None
class DummyThread:
def start(self):
return None
dummy_queue = object()
with (
patch(
"core.training.training.prepare_gpu_selection",
return_value = ([0], {"selection_mode": "auto"}),
),
patch(
"core.training.training._CTX.Queue",
side_effect = [dummy_queue, dummy_queue],
),
patch(
"core.training.training._CTX.Process", return_value = DummyProcess()
) as mock_process,
patch(
"core.training.training.threading.Thread",
return_value = DummyThread(),
),
):
backend.start_training(
job_id = "test-grad-clip",
model_name = "unsloth/test",
training_type = "LoRA/QLoRA",
max_grad_norm = 0.7,
)
config = mock_process.call_args.kwargs["kwargs"]["config"]
self.assertEqual(config["max_grad_norm"], 0.7)
def test_training_route_forwards_embedding_learning_rate(self):
training_route = _load_route_module(
"training_route_module_raw_support",

25
studio/backend/tests/test_training_worker_flash_attn.py
View file

@ -37,6 +37,7 @@ def test_runtime_flash_attn_prefers_prebuilt_wheel(monkeypatch):
statuses: list[str] = []
monkeypatch.delenv(worker._FLASH_ATTN_SKIP_ENV, raising = False)
monkeypatch.setattr(worker, "has_blackwell_gpu", lambda: False)
monkeypatch.setattr(builtins, "__import__", _missing_flash_attn_import())
monkeypatch.setattr(
worker,
@ -65,6 +66,7 @@ def test_runtime_flash_attn_falls_back_to_pypi(monkeypatch):
statuses: list[str] = []
monkeypatch.delenv(worker._FLASH_ATTN_SKIP_ENV, raising = False)
monkeypatch.setattr(worker, "has_blackwell_gpu", lambda: False)
monkeypatch.setattr(builtins, "__import__", _missing_flash_attn_import())
monkeypatch.setattr(
worker,
@ -112,6 +114,29 @@ def test_runtime_flash_attn_skip_env_avoids_all_install_work(monkeypatch):
worker._sp.run.assert_not_called()
def test_runtime_flash_attn_skips_on_blackwell(monkeypatch):
statuses: list[str] = []
install_mock = mock.Mock()
monkeypatch.delenv(worker._FLASH_ATTN_SKIP_ENV, raising = False)
monkeypatch.setattr(
worker, "_should_try_runtime_flash_attn_install", lambda max_seq: True
)
monkeypatch.setattr(worker, "has_blackwell_gpu", lambda: True)
monkeypatch.setattr(worker, "_install_package_wheel_first", install_mock)
monkeypatch.setattr(
worker,
"_send_status",
lambda queue, message: statuses.append(message),
)
worker._ensure_flash_attn_for_long_context(event_queue = [], max_seq_length = 65536)
install_mock.assert_not_called()
assert len(statuses) == 1
assert "Blackwell" in statuses[0]
def test_causal_conv1d_fast_path_preserves_wheel_first_install_args(monkeypatch):
install_mock = mock.Mock(return_value = True)
monkeypatch.setattr(worker, "_install_package_wheel_first", install_mock)

25
studio/backend/utils/datasets/chat_templates.py
View file

@ -28,6 +28,23 @@ DEFAULT_ALPACA_TEMPLATE = """Below is an instruction that describes a task, pair
{}"""
def _is_mlx_runtime() -> bool:
try:
from unsloth_zoo.mlx import is_mlx_available
except ImportError:
return False
return is_mlx_available()
def _chat_template_kwargs() -> dict:
if not _is_mlx_runtime():
return {}
return {
"patch_saving": False,
"use_zoo_tokenizer_patch": True,
}
def get_tokenizer_chat_template(tokenizer, model_name):
"""
Gets appropriate chat template for tokenizer based on model.
@ -60,6 +77,7 @@ def get_tokenizer_chat_template(tokenizer, model_name):
tokenizer = get_chat_template(
tokenizer,
chat_template = matched_template,
**_chat_template_kwargs(),
)
except Exception as e:
logger.info(f"⚠️ Failed to apply Unsloth template '{matched_template}': {e}")
@ -79,6 +97,7 @@ def get_tokenizer_chat_template(tokenizer, model_name):
tokenizer = get_chat_template(
tokenizer,
chat_template = "chatml",
**_chat_template_kwargs(),
)
except Exception as e:
logger.info(f"⚠️ Failed to apply default ChatML template: {e}")
@ -255,7 +274,11 @@ def apply_chat_template_to_dataset(
if not (hasattr(tokenizer, 'chat_template') and tokenizer.chat_template):
try:
from unsloth.chat_templates import get_chat_template
tokenizer = get_chat_template(tokenizer, chat_template = "alpaca")
tokenizer = get_chat_template(
tokenizer,
chat_template = "alpaca",
**_chat_template_kwargs(),
)
logger.info(f"📝 Set alpaca chat template on tokenizer for model saving")
except Exception as e:
logger.info(f"⚠️ Could not set alpaca template on tokenizer: {e}")

236
studio/backend/utils/models/gguf_metadata.py Normal file
View file

@ -0,0 +1,236 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Free-function ``general.*`` reader for GGUF headers, used by
``detect_mmproj_file`` to pair weights and projectors via
``general.base_model.0.repo_url``. ~30 ms per file, cached by
(path, mtime, size)."""
from __future__ import annotations
import os
import struct
import threading
from pathlib import Path
from typing import Dict, Optional, Tuple
from loggers import get_logger
logger = get_logger(__name__)
_GGUF_MAGIC = 0x46554747 # b"GGUF" LE u32
_WANTED_GENERAL_KEYS: frozenset[str] = frozenset(
{
"general.architecture",
"general.type",
"general.name",
"general.basename",
"general.organization",
"general.size_label",
"general.finetune",
"general.base_model.0.name",
"general.base_model.0.organization",
"general.base_model.0.repo_url",
"general.repo_url",
"general.source.url",
"general.source.repo_url",
"general.source.huggingface.repository",
}
)
# Cache failed parses too so a broken file is not retried each scan.
_CacheKey = Tuple[str, int, int]
_METADATA_CACHE: Dict[_CacheKey, Optional[Dict[str, str]]] = {}
_CACHE_LOCK = threading.Lock()
_CACHE_MAX_ENTRIES = 4096
def _cache_key(path: str) -> Optional[_CacheKey]:
try:
st = os.stat(path)
except OSError:
return None
try:
resolved = str(Path(path).resolve())
except OSError:
resolved = str(path)
return (resolved, st.st_mtime_ns, st.st_size)
def read_gguf_general_metadata(path: str) -> Optional[Dict[str, str]]:
"""Return ``general.*`` strings from a GGUF header, or ``None`` if
the file is missing, unreadable, or not a GGUF. ``{}`` means the
file is valid but carries none of the wanted keys."""
key = _cache_key(path)
if key is None:
return None
with _CACHE_LOCK:
if key in _METADATA_CACHE:
return _METADATA_CACHE[key]
result = _parse_gguf_header(path)
with _CACHE_LOCK:
# Arbitrary eviction; header reads are cheap so true LRU is overkill.
while len(_METADATA_CACHE) >= _CACHE_MAX_ENTRIES:
try:
_METADATA_CACHE.pop(next(iter(_METADATA_CACHE)))
except StopIteration:
break
_METADATA_CACHE[key] = result
return result
def _parse_gguf_header(path: str) -> Optional[Dict[str, str]]:
out: Dict[str, str] = {}
try:
with open(path, "rb") as f:
head = f.read(24)
if len(head) < 24:
return None
magic, _version, _tcount, kv_count = struct.unpack("<IIQQ", head)
if magic != _GGUF_MAGIC:
return None
for _ in range(kv_count):
try:
klen_bytes = f.read(8)
if len(klen_bytes) < 8:
break
klen = struct.unpack("<Q", klen_bytes)[0]
if klen > 1 << 20: # 1 MB sanity bound
break
kbytes = f.read(klen)
if len(kbytes) < klen:
break
key = kbytes.decode("utf-8", "replace")
vt_bytes = f.read(4)
if len(vt_bytes) < 4:
break
vtype = struct.unpack("<I", vt_bytes)[0]
if vtype == 8 and key in _WANTED_GENERAL_KEYS:
slen_bytes = f.read(8)
if len(slen_bytes) < 8:
break
slen = struct.unpack("<Q", slen_bytes)[0]
if slen > 1 << 22: # 4 MB sanity bound
break
sbytes = f.read(slen)
if len(sbytes) < slen:
break
out[key] = sbytes.decode("utf-8", "replace")
else:
if not _skip_gguf_value(f, vtype):
break
except (struct.error, UnicodeDecodeError):
break
except OSError as e:
logger.debug(f"read_gguf_general_metadata: cannot open {path}: {e}")
return None
except Exception as e:
logger.debug(f"read_gguf_general_metadata: parse failure on {path}: {e}")
return None
return out
# Strings (8) and arrays (9) are handled inline.
_FIXED_VTYPE_SIZES: Dict[int, int] = {
0: 1, # uint8
1: 1, # int8
2: 2, # uint16
3: 2, # int16
4: 4, # uint32
5: 4, # int32
6: 4, # float32
7: 1, # bool
10: 8, # uint64
11: 8, # int64
12: 8, # float64
}
def _skip_gguf_value(f, vtype: int) -> bool:
"""Advance past one GGUF value. ``f.seek(.., 1)`` past EOF is legal
on a regular file so truncation is detected on the next read; we
only return False for unknown types or sanity-bound overflow."""
if vtype == 8: # STRING
slen_bytes = f.read(8)
if len(slen_bytes) < 8:
return False
slen = struct.unpack("<Q", slen_bytes)[0]
if slen > 1 << 30: # 1 GB sanity bound
return False
f.seek(slen, 1)
return True
if vtype == 9: # ARRAY
head = f.read(12)
if len(head) < 12:
return False
atype, alen = struct.unpack("<IQ", head)
if alen > 1 << 30:
return False
if atype == 8:
for _ in range(alen):
slen_bytes = f.read(8)
if len(slen_bytes) < 8:
return False
slen = struct.unpack("<Q", slen_bytes)[0]
if slen > 1 << 30:
return False
f.seek(slen, 1)
return True
sz = _FIXED_VTYPE_SIZES.get(atype)
if sz is None:
return False
f.seek(sz * alen, 1)
return True
sz = _FIXED_VTYPE_SIZES.get(vtype)
if sz is None:
return False
f.seek(sz, 1)
return True
def is_mmproj_by_metadata(meta: Optional[Dict[str, str]]) -> Optional[bool]:
"""True/False from ``general.type``; None means fall back to filename."""
if not meta:
return None
t = meta.get("general.type")
if t is None:
return None
return t.lower() == "mmproj"
def pairing_score(
weight_meta: Optional[Dict[str, str]],
mmproj_meta: Optional[Dict[str, str]],
) -> int:
"""Pairing confidence: 100 = base_model URL match, 80 = basename + org,
60 = basename, -1 = definitive mismatch, 0 = decide from filename."""
if not weight_meta or not mmproj_meta:
return 0
w_url = weight_meta.get("general.base_model.0.repo_url")
p_url = mmproj_meta.get("general.base_model.0.repo_url")
if w_url and p_url:
return 100 if w_url.strip().rstrip("/") == p_url.strip().rstrip("/") else -1
w_base = weight_meta.get("general.basename")
p_base = mmproj_meta.get("general.basename")
w_org = weight_meta.get("general.base_model.0.organization") or weight_meta.get(
"general.organization"
)
p_org = mmproj_meta.get("general.base_model.0.organization") or mmproj_meta.get(
"general.organization"
)
if w_base and p_base and w_org and p_org:
if w_base.lower() == p_base.lower() and w_org.lower() == p_org.lower():
return 80
return -1
if w_base and p_base:
return 60 if w_base.lower() == p_base.lower() else -1
return 0

256
studio/backend/utils/models/model_config.py
View file

@ -19,6 +19,11 @@ from utils.paths import (
resolve_export_dir,
)
from utils.utils import without_hf_auth
from utils.models.gguf_metadata import (
is_mmproj_by_metadata,
pairing_score,
read_gguf_general_metadata,
)
import structlog
from loggers import get_logger
import os
@ -801,12 +806,15 @@ _AUDIO_TOKEN_PATTERNS = {
"whisper": lambda tokens: "<|startoftranscript|>" in tokens,
"audio_vlm": lambda tokens: "<audio_soft_token>" in tokens,
"bicodec": lambda tokens: any(t.startswith("<|bicodec_") for t in tokens),
"dac": lambda tokens: "<|audio_start|>" in tokens
and "<|audio_end|>" in tokens
and "<|text_start|>" in tokens
and "<|text_end|>" in tokens,
"snac": lambda tokens: sum(1 for t in tokens if t.startswith("<custom_token_"))
> 10000,
"dac": lambda tokens: (
"<|audio_start|>" in tokens
and "<|audio_end|>" in tokens
and "<|text_start|>" in tokens
and "<|text_end|>" in tokens
),
"snac": lambda tokens: (
sum(1 for t in tokens if t.startswith("<custom_token_")) > 10000
),
}
@ -913,6 +921,85 @@ def _is_mmproj(filename: str) -> bool:
return "mmproj" in filename.lower()
# Family tokens for #5347's filename fallback. Lowercase. Order does not
# matter (see ``_detect_family_token``).
_MODEL_FAMILY_TOKENS: tuple[str, ...] = (
"qwen",
"gemma",
"llama",
"mistral",
"ministral",
"magistral",
"devstral",
"phi",
"deepseek",
"internvl",
"minicpm",
"llava",
"glm",
"yi",
"command-r",
"molmo",
"pixtral",
"smolvlm",
"moondream",
"granite",
"ovis",
"nemotron",
"kimi",
"nanonets",
"cosmos",
"mimo",
"apriel",
"lfm",
)
# Word-bounded match: any letter on either side disqualifies. Stops
# ``phi`` matching ``sapphire``, ``yi`` matching ``tiny``, etc.
_FAMILY_TOKEN_RE_CACHE: Dict[str, "_re.Pattern[str]"] = {}
def _family_token_re(token: str) -> "_re.Pattern[str]":
pat = _FAMILY_TOKEN_RE_CACHE.get(token)
if pat is None:
pat = _re.compile(rf"(?:^|[^a-z])({_re.escape(token)})(?:[^a-z]|$)")
_FAMILY_TOKEN_RE_CACHE[token] = pat
return pat
def _detect_family_token(filename: str) -> Optional[str]:
"""Leftmost-position match; ties prefer the longer token."""
name = filename.lower()
best: Optional[tuple[int, int, str]] = None # (start, -len, token)
for token in _MODEL_FAMILY_TOKENS:
m = _family_token_re(token).search(name)
if m is None:
continue
key = (m.start(1), -len(token), token)
if best is None or key < best:
best = key
return None if best is None else best[2]
def mmproj_matches_model_family(model_path: str, mmproj_path: str) -> bool:
"""Defense-in-depth guard for the launcher: True unless both filenames
carry recognised family tokens that disagree."""
model_fam = _detect_family_token(Path(model_path).name)
mmproj_fam = _detect_family_token(Path(mmproj_path).name)
if model_fam is None or mmproj_fam is None:
return True
return model_fam == mmproj_fam
def _shared_prefix_len(a: str, b: str) -> int:
n = min(len(a), len(b))
for i in range(n):
if a[i] != b[i]:
return i
return n
def _is_gguf_filename(filename: str) -> bool:
return filename.lower().endswith(".gguf")
@ -927,33 +1014,18 @@ def _iter_gguf_files(directory: Path, recursive: bool = False):
def detect_mmproj_file(path: str, search_root: Optional[str] = None) -> Optional[str]:
"""
Find the mmproj (vision projection) GGUF file for a given model.
"""Find the mmproj GGUF for a model.
Args:
path: Directory to search — or a .gguf file (uses its parent dir
as the starting point).
search_root: Optional outer directory that should also be scanned
(and any directory between it and ``path``). This handles
local layouts where the model weights live in a quant-named
subdir (``snapshot/BF16/foo.gguf``) but the mmproj sits at
the snapshot root (``snapshot/mmproj-BF16.gguf``). When
``None``, only the immediate parent dir is scanned, matching
the historical behavior.
Returns:
Full path to the mmproj .gguf file, or None if not found.
"""
``path``: directory or a .gguf file. ``search_root``: optional ancestor
to also walk (snapshot layouts where the weight is in ``snapshot/BF16/``
but the projector sits at ``snapshot/``). Returns the projector path or
``None``."""
p = Path(path)
start_dir = p.parent if p.is_file() else p
if not start_dir.is_dir():
return None
# Build the list of dirs to scan: immediate dir first, then walk up
# to (and including) ``search_root`` if it is an ancestor. We walk
# incrementally rather than recursing into ``search_root`` so we
# don't accidentally pick up an mmproj from a sibling subdir
# belonging to a different model variant.
# Walk incrementally so a sibling subdir's mmproj cannot leak in.
seen: set[Path] = set()
scan_order: list[Path] = []
@ -969,12 +1041,7 @@ def detect_mmproj_file(path: str, search_root: Optional[str] = None) -> Optional
_add(start_dir)
# When ``path`` is a symlink (e.g. Ollama's ``.studio_links/...gguf``
# -> ``blobs/sha256-...``), the symlink's parent directory rarely
# contains the mmproj sibling; the real mmproj file lives next to
# the symlink target. Add the target's parent to the scan so vision
# GGUFs that are surfaced via symlinks are still recognised as
# vision models.
# Ollama's .studio_links/foo.gguf -> blobs/sha256-...: also scan target dir.
try:
if p.is_symlink() and p.is_file():
target_parent = p.resolve().parent
@ -986,14 +1053,12 @@ def detect_mmproj_file(path: str, search_root: Optional[str] = None) -> Optional
try:
root_resolved = Path(search_root).resolve()
start_resolved = start_dir.resolve()
# Only walk if start_dir is inside (or equal to) search_root.
if root_resolved == start_resolved or (
start_resolved.is_relative_to(root_resolved)
if hasattr(start_resolved, "is_relative_to")
else str(start_resolved).startswith(str(root_resolved) + "/")
):
cur = start_resolved
# Walk up from start_dir to (and including) root_resolved.
while cur != root_resolved and cur.parent != cur:
cur = cur.parent
_add(cur)
@ -1002,11 +1067,66 @@ def detect_mmproj_file(path: str, search_root: Optional[str] = None) -> Optional
except OSError:
pass
candidates: list[Path] = []
seen_resolved: set[Path] = set()
for d in scan_order:
for f in _iter_gguf_files(d):
if _is_mmproj(f.name):
return str(f.resolve())
return None
try:
resolved = f.resolve()
except OSError:
continue
if resolved in seen_resolved:
continue
# Prefer ``general.type=='mmproj'``; fall back to filename.
meta = read_gguf_general_metadata(str(resolved))
by_meta = is_mmproj_by_metadata(meta)
if by_meta is True or (by_meta is None and _is_mmproj(f.name)):
seen_resolved.add(resolved)
candidates.append(resolved)
if not candidates:
return None
# Directory path: no model name to compare against; legacy behaviour.
if not p.is_file():
return str(candidates[0])
# Stage 1: GGUF metadata. Stage 2: filename family token (#5347).
model_stem = p.stem.lower()
model_family = _detect_family_token(p.name)
weight_meta = read_gguf_general_metadata(str(p))
scored: list[tuple[int, Path]] = []
for c in candidates:
cand_meta = read_gguf_general_metadata(str(c))
meta_score = pairing_score(weight_meta, cand_meta)
if meta_score == -1:
logger.info(f"detect_mmproj_file: dropped {c.name} (metadata mismatch)")
continue
if meta_score == 0 and model_family is not None:
# Unrecognised candidate family is a wildcard (``mmproj-F16.gguf``).
cand_family = _detect_family_token(c.name)
if cand_family is not None and cand_family != model_family:
logger.info(
f"detect_mmproj_file: dropped {c.name} "
f"(filename family {cand_family!r} vs model {model_family!r})"
)
continue
scored.append((meta_score, c))
if not scored:
return None
# Score first, then longest shared prefix, then shorter stem.
best = max(
scored,
key = lambda sc: (
sc[0],
_shared_prefix_len(model_stem, sc[1].stem.lower()),
-len(sc[1].stem),
),
)
return str(best[1])
def detect_gguf_model(path: str) -> Optional[str]:
@ -1360,7 +1480,7 @@ def detect_gguf_model_remote(
if attempt < 2:
time.sleep(2**attempt)
logger.warning(
f"Could not check GGUF files for '{repo_id}' after 3 attempts: " f"{last_err}"
f"Could not check GGUF files for '{repo_id}' after 3 attempts: {last_err}"
)
return None
@ -1696,20 +1816,21 @@ def get_base_model_from_checkpoint(checkpoint_path: str) -> Optional[str]:
)
return base_model
training_args_path = checkpoint_path_obj / "training_args.bin"
if training_args_path.exists():
try:
import torch
training_args = torch.load(training_args_path)
if hasattr(training_args, "model_name_or_path"):
base_model = training_args.model_name_or_path
logger.info(
"Detected base model from training_args.bin: %s", base_model
)
return base_model
except Exception as e:
logger.warning(f"Could not load training_args.bin: {e}")
# TODO: torch.load default weights_only=True (torch >= 2.6) rejects pickled TrainingArguments; re-enable via safe_globals or weights_only=False once threat model allows.
# training_args_path = checkpoint_path_obj / "training_args.bin"
# if training_args_path.exists():
# try:
# import torch
#
# training_args = torch.load(training_args_path)
# if hasattr(training_args, "model_name_or_path"):
# base_model = training_args.model_name_or_path
# logger.info(
# "Detected base model from training_args.bin: %s", base_model
# )
# return base_model
# except Exception as e:
# logger.warning(f"Could not load training_args.bin: {e}")
dir_name = checkpoint_path_obj.name
if dir_name.startswith("unsloth_"):
@ -1757,20 +1878,21 @@ def get_base_model_from_lora(lora_path: str) -> Optional[str]:
return base_model
# Fallback: try training_args.bin (requires torch)
training_args_path = lora_path_obj / "training_args.bin"
if training_args_path.exists():
try:
import torch
training_args = torch.load(training_args_path)
if hasattr(training_args, "model_name_or_path"):
base_model = training_args.model_name_or_path
logger.info(
f"Detected base model from training_args.bin: {base_model}"
)
return base_model
except Exception as e:
logger.warning(f"Could not load training_args.bin: {e}")
# TODO: torch.load default weights_only=True (torch >= 2.6) rejects pickled TrainingArguments; also an RCE sink for third-party LoRAs via this route, re-enable behind a trust check if needed.
# training_args_path = lora_path_obj / "training_args.bin"
# if training_args_path.exists():
# try:
# import torch
#
# training_args = torch.load(training_args_path)
# if hasattr(training_args, "model_name_or_path"):
# base_model = training_args.model_name_or_path
# logger.info(
# f"Detected base model from training_args.bin: {base_model}"
# )
# return base_model
# except Exception as e:
# logger.warning(f"Could not load training_args.bin: {e}")
# Last resort: parse from directory name
# Format: unsloth_Meta-Llama-3.1-8B-Instruct-bnb-4bit_timestamp

44
studio/backend/utils/wheel_utils.py
View file

@ -3,6 +3,7 @@
from __future__ import annotations
import functools
import json
import logging
import platform
@ -22,6 +23,49 @@ FLASH_ATTN_RELEASE_BASE_URL = (
)
@functools.lru_cache(maxsize = 1)
def has_blackwell_gpu() -> bool:
"""Return True if any visible NVIDIA GPU has compute capability >= 10.0
(Blackwell: sm_100, sm_120, sm_121, ...).
Dao-AILab does not publish prebuilt flash-attention wheels for these
architectures, and the older-arch wheels fail to load on Blackwell, so
callers use this gate to skip the flash-attn install/upgrade path.
Result is cached for the process lifetime since GPU hardware does not
change. Tests that mock subprocess/nvidia-smi must call
``has_blackwell_gpu.cache_clear()`` before each invocation.
"""
exe = shutil.which("nvidia-smi")
if not exe:
return False
try:
result = subprocess.run(
[exe, "--query-gpu=compute_cap", "--format=csv,noheader"],
stdout = subprocess.PIPE,
stderr = subprocess.DEVNULL,
text = True,
timeout = 10,
env = child_env_without_native_path_secret(),
)
except (OSError, subprocess.TimeoutExpired):
return False
if result.returncode != 0:
return False
for line in result.stdout.splitlines():
cap = line.strip()
if not cap:
continue
major_part = cap.split(".", 1)[0]
try:
major = int(major_part)
except ValueError:
continue
if major >= 10:
return True
return False
def linux_wheel_platform_tag() -> str | None:
machine = platform.machine().lower()
if sys.platform.startswith("linux"):

819
studio/frontend/package-lock.json generated
View file

@ -33,7 +33,7 @@
"@streamdown/math": "1.0.2",
"@streamdown/mermaid": "1.0.2",
"@tailwindcss/vite": "^4.2.2",
"@tanstack/react-router": "^1.159.10",
"@tanstack/react-router": "1.169.2",
"@tanstack/react-table": "^8.21.3",
"@tauri-apps/api": "^2.10.1",
"@tauri-apps/plugin-clipboard-manager": "^2.3.2",
@ -56,8 +56,8 @@
"lucide-react": "^1.7.0",
"mammoth": "^1.11.0",
"motion": "^12.34.0",
"next": "^16.1.6",
"next-themes": "^0.4.6",
"node-forge": "^1.4.0",
"radix-ui": "^1.4.3",
"react": "^19.2.4",
"react-day-picker": "^9.13.2",
@ -80,6 +80,7 @@
"@eslint/js": "^9.39.1",
"@types/js-yaml": "^4.0.9",
"@types/node": "^25.5.2",
"@types/node-forge": "^1.3.14",
"@types/react": "^19.2.5",
"@types/react-dom": "^19.2.3",
"@vitejs/plugin-react": "^6.0.1",
@ -1540,472 +1541,6 @@
"mlly": "^1.8.2"
}
},
"node_modules/@img/colour": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@img/colour/-/colour-1.1.0.tgz",
"integrity": "sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==",
"license": "MIT",
"optional": true,
"engines": {
"node": ">=18"
}
},
"node_modules/@img/sharp-darwin-arm64": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-darwin-arm64/-/sharp-darwin-arm64-0.34.5.tgz",
"integrity": "sha512-imtQ3WMJXbMY4fxb/Ndp6HBTNVtWCUI0WdobyheGf5+ad6xX8VIDO8u2xE4qc/fr08CKG/7dDseFtn6M6g/r3w==",
"cpu": [
"arm64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
},
"optionalDependencies": {
"@img/sharp-libvips-darwin-arm64": "1.2.4"
}
},
"node_modules/@img/sharp-darwin-x64": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-darwin-x64/-/sharp-darwin-x64-0.34.5.tgz",
"integrity": "sha512-YNEFAF/4KQ/PeW0N+r+aVVsoIY0/qxxikF2SWdp+NRkmMB7y9LBZAVqQ4yhGCm/H3H270OSykqmQMKLBhBJDEw==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
},
"optionalDependencies": {
"@img/sharp-libvips-darwin-x64": "1.2.4"
}
},
"node_modules/@img/sharp-libvips-darwin-arm64": {
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-arm64/-/sharp-libvips-darwin-arm64-1.2.4.tgz",
"integrity": "sha512-zqjjo7RatFfFoP0MkQ51jfuFZBnVE2pRiaydKJ1G/rHZvnsrHAOcQALIi9sA5co5xenQdTugCvtb1cuf78Vf4g==",
"cpu": [
"arm64"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
"darwin"
],
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-libvips-darwin-x64": {
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-x64/-/sharp-libvips-darwin-x64-1.2.4.tgz",
"integrity": "sha512-1IOd5xfVhlGwX+zXv2N93k0yMONvUlANylbJw1eTah8K/Jtpi15KC+WSiaX/nBmbm2HxRM1gZ0nSdjSsrZbGKg==",
"cpu": [
"x64"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
"darwin"
],
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-libvips-linux-arm": {
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm/-/sharp-libvips-linux-arm-1.2.4.tgz",
"integrity": "sha512-bFI7xcKFELdiNCVov8e44Ia4u2byA+l3XtsAj+Q8tfCwO6BQ8iDojYdvoPMqsKDkuoOo+X6HZA0s0q11ANMQ8A==",
"cpu": [
"arm"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-libvips-linux-arm64": {
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm64/-/sharp-libvips-linux-arm64-1.2.4.tgz",
"integrity": "sha512-excjX8DfsIcJ10x1Kzr4RcWe1edC9PquDRRPx3YVCvQv+U5p7Yin2s32ftzikXojb1PIFc/9Mt28/y+iRklkrw==",
"cpu": [
"arm64"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-libvips-linux-ppc64": {
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-ppc64/-/sharp-libvips-linux-ppc64-1.2.4.tgz",
"integrity": "sha512-FMuvGijLDYG6lW+b/UvyilUWu5Ayu+3r2d1S8notiGCIyYU/76eig1UfMmkZ7vwgOrzKzlQbFSuQfgm7GYUPpA==",
"cpu": [
"ppc64"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-libvips-linux-riscv64": {
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-riscv64/-/sharp-libvips-linux-riscv64-1.2.4.tgz",
"integrity": "sha512-oVDbcR4zUC0ce82teubSm+x6ETixtKZBh/qbREIOcI3cULzDyb18Sr/Wcyx7NRQeQzOiHTNbZFF1UwPS2scyGA==",
"cpu": [
"riscv64"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-libvips-linux-s390x": {
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-s390x/-/sharp-libvips-linux-s390x-1.2.4.tgz",
"integrity": "sha512-qmp9VrzgPgMoGZyPvrQHqk02uyjA0/QrTO26Tqk6l4ZV0MPWIW6LTkqOIov+J1yEu7MbFQaDpwdwJKhbJvuRxQ==",
"cpu": [
"s390x"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-libvips-linux-x64": {
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-x64/-/sharp-libvips-linux-x64-1.2.4.tgz",
"integrity": "sha512-tJxiiLsmHc9Ax1bz3oaOYBURTXGIRDODBqhveVHonrHJ9/+k89qbLl0bcJns+e4t4rvaNBxaEZsFtSfAdquPrw==",
"cpu": [
"x64"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-libvips-linuxmusl-arm64": {
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-arm64/-/sharp-libvips-linuxmusl-arm64-1.2.4.tgz",
"integrity": "sha512-FVQHuwx1IIuNow9QAbYUzJ+En8KcVm9Lk5+uGUQJHaZmMECZmOlix9HnH7n1TRkXMS0pGxIJokIVB9SuqZGGXw==",
"cpu": [
"arm64"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-libvips-linuxmusl-x64": {
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-x64/-/sharp-libvips-linuxmusl-x64-1.2.4.tgz",
"integrity": "sha512-+LpyBk7L44ZIXwz/VYfglaX/okxezESc6UxDSoyo2Ks6Jxc4Y7sGjpgU9s4PMgqgjj1gZCylTieNamqA1MF7Dg==",
"cpu": [
"x64"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
"linux"
],
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-linux-arm": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-linux-arm/-/sharp-linux-arm-0.34.5.tgz",
"integrity": "sha512-9dLqsvwtg1uuXBGZKsxem9595+ujv0sJ6Vi8wcTANSFpwV/GONat5eCkzQo/1O6zRIkh0m/8+5BjrRr7jDUSZw==",
"cpu": [
"arm"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
},
"optionalDependencies": {
"@img/sharp-libvips-linux-arm": "1.2.4"
}
},
"node_modules/@img/sharp-linux-arm64": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-linux-arm64/-/sharp-linux-arm64-0.34.5.tgz",
"integrity": "sha512-bKQzaJRY/bkPOXyKx5EVup7qkaojECG6NLYswgktOZjaXecSAeCWiZwwiFf3/Y+O1HrauiE3FVsGxFg8c24rZg==",
"cpu": [
"arm64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
},
"optionalDependencies": {
"@img/sharp-libvips-linux-arm64": "1.2.4"
}
},
"node_modules/@img/sharp-linux-ppc64": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-linux-ppc64/-/sharp-linux-ppc64-0.34.5.tgz",
"integrity": "sha512-7zznwNaqW6YtsfrGGDA6BRkISKAAE1Jo0QdpNYXNMHu2+0dTrPflTLNkpc8l7MUP5M16ZJcUvysVWWrMefZquA==",
"cpu": [
"ppc64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
},
"optionalDependencies": {
"@img/sharp-libvips-linux-ppc64": "1.2.4"
}
},
"node_modules/@img/sharp-linux-riscv64": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-linux-riscv64/-/sharp-linux-riscv64-0.34.5.tgz",
"integrity": "sha512-51gJuLPTKa7piYPaVs8GmByo7/U7/7TZOq+cnXJIHZKavIRHAP77e3N2HEl3dgiqdD/w0yUfiJnII77PuDDFdw==",
"cpu": [
"riscv64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
},
"optionalDependencies": {
"@img/sharp-libvips-linux-riscv64": "1.2.4"
}
},
"node_modules/@img/sharp-linux-s390x": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-linux-s390x/-/sharp-linux-s390x-0.34.5.tgz",
"integrity": "sha512-nQtCk0PdKfho3eC5MrbQoigJ2gd1CgddUMkabUj+rBevs8tZ2cULOx46E7oyX+04WGfABgIwmMC0VqieTiR4jg==",
"cpu": [
"s390x"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
},
"optionalDependencies": {
"@img/sharp-libvips-linux-s390x": "1.2.4"
}
},
"node_modules/@img/sharp-linux-x64": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-linux-x64/-/sharp-linux-x64-0.34.5.tgz",
"integrity": "sha512-MEzd8HPKxVxVenwAa+JRPwEC7QFjoPWuS5NZnBt6B3pu7EG2Ge0id1oLHZpPJdn3OQK+BQDiw9zStiHBTJQQQQ==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
},
"optionalDependencies": {
"@img/sharp-libvips-linux-x64": "1.2.4"
}
},
"node_modules/@img/sharp-linuxmusl-arm64": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-arm64/-/sharp-linuxmusl-arm64-0.34.5.tgz",
"integrity": "sha512-fprJR6GtRsMt6Kyfq44IsChVZeGN97gTD331weR1ex1c1rypDEABN6Tm2xa1wE6lYb5DdEnk03NZPqA7Id21yg==",
"cpu": [
"arm64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
},
"optionalDependencies": {
"@img/sharp-libvips-linuxmusl-arm64": "1.2.4"
}
},
"node_modules/@img/sharp-linuxmusl-x64": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-x64/-/sharp-linuxmusl-x64-0.34.5.tgz",
"integrity": "sha512-Jg8wNT1MUzIvhBFxViqrEhWDGzqymo3sV7z7ZsaWbZNDLXRJZoRGrjulp60YYtV4wfY8VIKcWidjojlLcWrd8Q==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
},
"optionalDependencies": {
"@img/sharp-libvips-linuxmusl-x64": "1.2.4"
}
},
"node_modules/@img/sharp-wasm32": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-wasm32/-/sharp-wasm32-0.34.5.tgz",
"integrity": "sha512-OdWTEiVkY2PHwqkbBI8frFxQQFekHaSSkUIJkwzclWZe64O1X4UlUjqqqLaPbUpMOQk6FBu/HtlGXNblIs0huw==",
"cpu": [
"wasm32"
],
"license": "Apache-2.0 AND LGPL-3.0-or-later AND MIT",
"optional": true,
"dependencies": {
"@emnapi/runtime": "^1.7.0"
},
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-win32-arm64": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-win32-arm64/-/sharp-win32-arm64-0.34.5.tgz",
"integrity": "sha512-WQ3AgWCWYSb2yt+IG8mnC6Jdk9Whs7O0gxphblsLvdhSpSTtmu69ZG1Gkb6NuvxsNACwiPV6cNSZNzt0KPsw7g==",
"cpu": [
"arm64"
],
"license": "Apache-2.0 AND LGPL-3.0-or-later",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-win32-ia32": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-win32-ia32/-/sharp-win32-ia32-0.34.5.tgz",
"integrity": "sha512-FV9m/7NmeCmSHDD5j4+4pNI8Cp3aW+JvLoXcTUo0IqyjSfAZJ8dIUmijx1qaJsIiU+Hosw6xM5KijAWRJCSgNg==",
"cpu": [
"ia32"
],
"license": "Apache-2.0 AND LGPL-3.0-or-later",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@img/sharp-win32-x64": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/@img/sharp-win32-x64/-/sharp-win32-x64-0.34.5.tgz",
"integrity": "sha512-+29YMsqY2/9eFEiW93eqWnuLcWcufowXewwSNIT6UwZdUUCrM3oFjMWH/Z6/TMmb4hlFenmfAVbpWeup2jryCw==",
"cpu": [
"x64"
],
"license": "Apache-2.0 AND LGPL-3.0-or-later",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@inquirer/ansi": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/@inquirer/ansi/-/ansi-2.0.5.tgz",
@ -2266,140 +1801,6 @@
"@emnapi/runtime": "^1.7.1"
}
},
"node_modules/@next/env": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/env/-/env-16.2.4.tgz",
"integrity": "sha512-dKkkOzOSwFYe5RX6y26fZgkSpVAlIOJKQHIiydQcrWH6y/97+RceSOAdjZ14Qa3zLduVUy0TXcn+EiM6t4rPgw==",
"license": "MIT"
},
"node_modules/@next/swc-darwin-arm64": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-16.2.4.tgz",
"integrity": "sha512-OXTFFox5EKN1Ym08vfrz+OXxmCcEjT4SFMbNRsWZE99dMqt2Kcusl5MqPXcW232RYkMLQTy0hqgAMEsfEd/l2A==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@next/swc-darwin-x64": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/swc-darwin-x64/-/swc-darwin-x64-16.2.4.tgz",
"integrity": "sha512-XhpVnUfmYWvD3YrXu55XdcAkQtOnvaI6wtQa8fuF5fGoKoxIUZ0kWPtcOfqJEWngFF/lOS9l3+O9CcownhiQxQ==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@next/swc-linux-arm64-gnu": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-gnu/-/swc-linux-arm64-gnu-16.2.4.tgz",
"integrity": "sha512-Mx/tjlNA3G8kg14QvuGAJ4xBwPk1tUHq56JxZ8CXnZwz1Etz714soCEzGQQzVMz4bEnGPowzkV6Xrp6wAkEWOQ==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@next/swc-linux-arm64-musl": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-musl/-/swc-linux-arm64-musl-16.2.4.tgz",
"integrity": "sha512-iVMMp14514u7Nup2umQS03nT/bN9HurK8ufylC3FZNykrwjtx7V1A7+4kvhbDSCeonTVqV3Txnv0Lu+m2oDXNg==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@next/swc-linux-x64-gnu": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/swc-linux-x64-gnu/-/swc-linux-x64-gnu-16.2.4.tgz",
"integrity": "sha512-EZOvm1aQWgnI/N/xcWOlnS3RQBk0VtVav5Zo7n4p0A7UKyTDx047k8opDbXgBpHl4CulRqRfbw3QrX2w5UOXMQ==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@next/swc-linux-x64-musl": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/swc-linux-x64-musl/-/swc-linux-x64-musl-16.2.4.tgz",
"integrity": "sha512-h9FxsngCm9cTBf71AR4fGznDEDx1hS7+kSEiIRjq5kO1oXWm07DxVGZjCvk0SGx7TSjlUqhI8oOyz7NfwAdPoA==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@next/swc-win32-arm64-msvc": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/swc-win32-arm64-msvc/-/swc-win32-arm64-msvc-16.2.4.tgz",
"integrity": "sha512-3NdJV5OXMSOeJYijX+bjaLge3mJBlh4ybydbT4GFoB/2hAojWHtMhl3CYlYoMrjPuodp0nzFVi4Tj2+WaMg+Ow==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@next/swc-win32-x64-msvc": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-16.2.4.tgz",
"integrity": "sha512-kMVGgsqhO5YTYODD9IPGGhA6iprWidQckK3LmPeW08PIFENRmgfb4MjXHO+p//d+ts2rpjvK5gXWzXSMrPl9cw==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@noble/ciphers": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/@noble/ciphers/-/ciphers-1.3.0.tgz",
@ -6450,15 +5851,6 @@
"react": "^18.0.0 || ^19.0.0"
}
},
"node_modules/@swc/helpers": {
"version": "0.5.15",
"resolved": "https://registry.npmjs.org/@swc/helpers/-/helpers-0.5.15.tgz",
"integrity": "sha512-JQ5TuMi45Owi4/BIMAJBoSQoOJu12oOk/gADqlcUL9JEdHB8vyjUSsxqeNXnmXHjYKMi2WcYtezGEEhqUI/E2g==",
"license": "Apache-2.0",
"dependencies": {
"tslib": "^2.8.0"
}
},
"node_modules/@tabby_ai/hijri-converter": {
"version": "1.0.5",
"resolved": "https://registry.npmjs.org/@tabby_ai/hijri-converter/-/hijri-converter-1.0.5.tgz",
@ -7377,6 +6769,16 @@
"undici-types": "~7.19.0"
}
},
"node_modules/@types/node-forge": {
"version": "1.3.14",
"resolved": "https://registry.npmjs.org/@types/node-forge/-/node-forge-1.3.14.tgz",
"integrity": "sha512-mhVF2BnD4BO+jtOp7z1CdzaK4mbuK0LLQYAvdOLqHTavxFNq4zA1EmYkpnFjP8HOUzedfQkRnp0E2ulSAYSzAw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/react": {
"version": "19.2.14",
"resolved": "https://registry.npmjs.org/@types/react/-/react-19.2.14.tgz",
@ -8455,12 +7857,6 @@
"node": ">= 12"
}
},
"node_modules/client-only": {
"version": "0.0.1",
"resolved": "https://registry.npmjs.org/client-only/-/client-only-0.0.1.tgz",
"integrity": "sha512-IV3Ou0jSMzZrd3pZ48nLkT9DA7Ag1pnPzaiQhpW7c3RbcqqzvzzVu+L8gfqMp/8IM2MQtSiqaCxrrcfu8I8rMA==",
"license": "MIT"
},
"node_modules/cliui": {
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/cliui/-/cliui-8.0.1.tgz",
@ -13184,59 +12580,6 @@
"node": ">= 0.6"
}
},
"node_modules/next": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/next/-/next-16.2.4.tgz",
"integrity": "sha512-kPvz56wF5frc+FxlHI5qnklCzbq53HTwORaWBGdT0vNoKh1Aya9XC8aPauH4NJxqtzbWsS5mAbctm4cr+EkQ2Q==",
"license": "MIT",
"dependencies": {
"@next/env": "16.2.4",
"@swc/helpers": "0.5.15",
"baseline-browser-mapping": "^2.9.19",
"caniuse-lite": "^1.0.30001579",
"postcss": "8.4.31",
"styled-jsx": "5.1.6"
},
"bin": {
"next": "dist/bin/next"
},
"engines": {
"node": ">=20.9.0"
},
"optionalDependencies": {
"@next/swc-darwin-arm64": "16.2.4",
"@next/swc-darwin-x64": "16.2.4",
"@next/swc-linux-arm64-gnu": "16.2.4",
"@next/swc-linux-arm64-musl": "16.2.4",
"@next/swc-linux-x64-gnu": "16.2.4",
"@next/swc-linux-x64-musl": "16.2.4",
"@next/swc-win32-arm64-msvc": "16.2.4",
"@next/swc-win32-x64-msvc": "16.2.4",
"sharp": "^0.34.5"
},
"peerDependencies": {
"@opentelemetry/api": "^1.1.0",
"@playwright/test": "^1.51.1",
"babel-plugin-react-compiler": "*",
"react": "^18.2.0 || 19.0.0-rc-de68d2f4-20241204 || ^19.0.0",
"react-dom": "^18.2.0 || 19.0.0-rc-de68d2f4-20241204 || ^19.0.0",
"sass": "^1.3.0"
},
"peerDependenciesMeta": {
"@opentelemetry/api": {
"optional": true
},
"@playwright/test": {
"optional": true
},
"babel-plugin-react-compiler": {
"optional": true
},
"sass": {
"optional": true
}
}
},
"node_modules/next-themes": {
"version": "0.4.6",
"resolved": "https://registry.npmjs.org/next-themes/-/next-themes-0.4.6.tgz",
@ -13285,6 +12628,15 @@
"url": "https://opencollective.com/node-fetch"
}
},
"node_modules/node-forge": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/node-forge/-/node-forge-1.4.0.tgz",
"integrity": "sha512-LarFH0+6VfriEhqMMcLX2F7SwSXeWwnEAJEsYm5QKWchiVYVvJyV9v7UDvUv+w5HO23ZpQTXDv/GxdDdMyOuoQ==",
"license": "(BSD-3-Clause OR GPL-2.0)",
"engines": {
"node": ">= 6.13.0"
}
},
"node_modules/node-releases": {
"version": "2.0.38",
"resolved": "https://registry.npmjs.org/node-releases/-/node-releases-2.0.38.tgz",
@ -13817,34 +13169,6 @@
"points-on-curve": "0.2.0"
}
},
"node_modules/postcss": {
"version": "8.4.31",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.4.31.tgz",
"integrity": "sha512-PS08Iboia9mts/2ygV3eLpY5ghnUcfLV/EXTOW1E2qYxJKGGBUtNjN76FYHnMs36RmARn41bC0AZmn+rR0OVpQ==",
"funding": [
{
"type": "opencollective",
"url": "https://opencollective.com/postcss/"
},
{
"type": "tidelift",
"url": "https://tidelift.com/funding/github/npm/postcss"
},
{
"type": "github",
"url": "https://github.com/sponsors/ai"
}
],
"license": "MIT",
"dependencies": {
"nanoid": "^3.3.6",
"picocolors": "^1.0.0",
"source-map-js": "^1.0.2"
},
"engines": {
"node": "^10 || ^12 || >=14"
}
},
"node_modules/postcss-selector-parser": {
"version": "7.1.1",
"resolved": "https://registry.npmjs.org/postcss-selector-parser/-/postcss-selector-parser-7.1.1.tgz",
@ -13858,24 +13182,6 @@
"node": ">=4"
}
},
"node_modules/postcss/node_modules/nanoid": {
"version": "3.3.12",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.12.tgz",
"integrity": "sha512-ZB9RH/39qpq5Vu6Y+NmUaFhQR6pp+M2Xt76XBnEwDaGcVAqhlvxrl3B2bKS5D3NH3QR76v3aSrKaF/Kiy7lEtQ==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/ai"
}
],
"license": "MIT",
"bin": {
"nanoid": "bin/nanoid.cjs"
},
"engines": {
"node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1"
}
},
"node_modules/powershell-utils": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/powershell-utils/-/powershell-utils-0.1.0.tgz",
@ -15142,64 +14448,6 @@
"url": "https://github.com/sponsors/colinhacks"
}
},
"node_modules/sharp": {
"version": "0.34.5",
"resolved": "https://registry.npmjs.org/sharp/-/sharp-0.34.5.tgz",
"integrity": "sha512-Ou9I5Ft9WNcCbXrU9cMgPBcCK8LiwLqcbywW3t4oDV37n1pzpuNLsYiAV8eODnjbtQlSDwZ2cUEeQz4E54Hltg==",
"hasInstallScript": true,
"license": "Apache-2.0",
"optional": true,
"dependencies": {
"@img/colour": "^1.0.0",
"detect-libc": "^2.1.2",
"semver": "^7.7.3"
},
"engines": {
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
},
"funding": {
"url": "https://opencollective.com/libvips"
},
"optionalDependencies": {
"@img/sharp-darwin-arm64": "0.34.5",
"@img/sharp-darwin-x64": "0.34.5",
"@img/sharp-libvips-darwin-arm64": "1.2.4",
"@img/sharp-libvips-darwin-x64": "1.2.4",
"@img/sharp-libvips-linux-arm": "1.2.4",
"@img/sharp-libvips-linux-arm64": "1.2.4",
"@img/sharp-libvips-linux-ppc64": "1.2.4",
"@img/sharp-libvips-linux-riscv64": "1.2.4",
"@img/sharp-libvips-linux-s390x": "1.2.4",
"@img/sharp-libvips-linux-x64": "1.2.4",
"@img/sharp-libvips-linuxmusl-arm64": "1.2.4",
"@img/sharp-libvips-linuxmusl-x64": "1.2.4",
"@img/sharp-linux-arm": "0.34.5",
"@img/sharp-linux-arm64": "0.34.5",
"@img/sharp-linux-ppc64": "0.34.5",
"@img/sharp-linux-riscv64": "0.34.5",
"@img/sharp-linux-s390x": "0.34.5",
"@img/sharp-linux-x64": "0.34.5",
"@img/sharp-linuxmusl-arm64": "0.34.5",
"@img/sharp-linuxmusl-x64": "0.34.5",
"@img/sharp-wasm32": "0.34.5",
"@img/sharp-win32-arm64": "0.34.5",
"@img/sharp-win32-ia32": "0.34.5",
"@img/sharp-win32-x64": "0.34.5"
}
},
"node_modules/sharp/node_modules/semver": {
"version": "7.7.4",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.7.4.tgz",
"integrity": "sha512-vFKC2IEtQnVhpT78h1Yp8wzwrf8CM+MzKMHGJZfBtzhZNycRFnXsHk6E5TxIkkMsgNS7mdX3AGB7x2QM2di4lA==",
"license": "ISC",
"optional": true,
"bin": {
"semver": "bin/semver.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/shebang-command": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz",
@ -15580,29 +14828,6 @@
"inline-style-parser": "0.2.7"
}
},
"node_modules/styled-jsx": {
"version": "5.1.6",
"resolved": "https://registry.npmjs.org/styled-jsx/-/styled-jsx-5.1.6.tgz",
"integrity": "sha512-qSVyDTeMotdvQYoHWLNGwRFJHC+i+ZvdBRYosOFgC+Wg1vx4frN2/RG/NA7SYqqvKNLf39P2LSRA2pu6n0XYZA==",
"license": "MIT",
"dependencies": {
"client-only": "0.0.1"
},
"engines": {
"node": ">= 12.0.0"
},
"peerDependencies": {
"react": ">= 16.8.0 || 17.x.x || ^18.0.0-0 || ^19.0.0-0"
},
"peerDependenciesMeta": {
"@babel/core": {
"optional": true
},
"babel-plugin-macros": {
"optional": true
}
}
},
"node_modules/stylis": {
"version": "4.4.0",
"resolved": "https://registry.npmjs.org/stylis/-/stylis-4.4.0.tgz",

3
studio/frontend/package.json
View file

@ -64,8 +64,8 @@
"lucide-react": "^1.7.0",
"mammoth": "^1.11.0",
"motion": "^12.34.0",
"next": "^16.1.6",
"next-themes": "^0.4.6",
"node-forge": "^1.4.0",
"radix-ui": "^1.4.3",
"react": "^19.2.4",
"react-day-picker": "^9.13.2",
@ -92,6 +92,7 @@
"@biomejs/biome": "^1.9.4",
"@eslint/js": "^9.39.1",
"@types/js-yaml": "^4.0.9",
"@types/node-forge": "^1.3.14",
"@types/node": "^25.5.2",
"@types/react": "^19.2.5",
"@types/react-dom": "^19.2.3",

6
studio/frontend/public/provider-logos/anthropic.svg Normal file
View file

@ -0,0 +1,6 @@
<svg xmlns="http://www.w3.org/2000/svg" width="900" height="900" viewBox="0 0 900 900">
<g>
<path d="M 222.16 664.50 L 212.35 691.50 L 206.42 691.53 C163.39,691.75 102.00,690.80 102.00,689.92 C102.00,689.34 103.32,685.63 104.94,681.68 C106.56,677.73 124.74,632.20 145.34,580.50 C165.94,528.80 205.07,430.70 232.29,362.50 C259.51,294.30 284.65,231.20 288.14,222.28 L 294.50 206.05 L 404.68 206.00 L 405.94 208.75 C407.23,211.56 413.90,227.98 437.50,286.50 C444.82,304.65 453.36,325.80 456.49,333.50 C459.61,341.20 475.70,381.02 492.24,422.00 C508.78,462.98 528.13,510.90 535.25,528.50 C542.36,546.10 553.72,574.22 560.48,591.00 C567.25,607.78 575.81,628.92 579.50,638.00 C585.05,651.65 598.92,686.11 600.73,690.76 C601.12,691.76 590.44,691.97 547.95,691.76 L 494.69 691.50 L 489.31 678.00 C486.35,670.58 481.94,659.33 479.52,653.00 C477.10,646.67 471.91,633.17 468.00,623.00 C464.08,612.83 459.67,601.24 458.19,597.25 L 455.51 590.00 L 249.31 590.00 L 245.56 600.25 C237.09,623.44 231.46,638.87 222.16,664.50 ZM 798.00 691.05 C798.00,691.64 777.31,692.00 743.50,692.00 C703.78,692.00 689.00,691.69 689.00,690.87 C689.00,690.26 685.87,681.82 682.04,672.12 C678.21,662.43 670.76,643.47 665.49,630.00 C660.21,616.53 652.36,596.50 648.03,585.50 C638.84,562.16 624.19,524.76 620.02,514.00 C615.63,502.69 600.65,464.57 593.49,446.50 C590.00,437.70 582.57,418.80 576.98,404.50 C562.77,368.16 549.76,334.97 543.27,318.50 C540.24,310.80 536.29,300.67 534.50,296.00 C532.71,291.33 526.45,275.35 520.59,260.50 C514.73,245.65 507.70,227.83 504.97,220.91 C502.23,213.98 500.00,207.85 500.00,207.29 C500.00,204.88 522.83,204.39 576.12,205.66 L 603.75 206.32 L 624.52 257.91 C635.94,286.28 646.96,313.77 649.01,319.00 C651.05,324.23 663.76,355.95 677.26,389.50 C699.90,445.80 737.77,540.07 780.62,646.80 C790.18,670.61 798.00,690.53 798.00,691.05 ZM 285.33 500.42 C285.62,501.18 305.07,501.42 351.82,501.24 L 417.90 500.97 L 415.37 494.24 C413.98,490.53 410.81,482.33 408.32,476.00 C405.83,469.67 403.42,463.38 402.98,462.00 C402.53,460.62 398.95,451.17 395.03,441.00 C391.11,430.83 384.57,413.73 380.50,403.00 C376.42,392.27 369.89,375.17 365.97,365.00 C362.05,354.83 357.46,342.77 355.77,338.20 C354.08,333.64 352.38,330.26 352.00,330.70 C351.61,331.14 347.21,341.85 342.21,354.50 C333.07,377.64 317.55,416.89 296.36,470.42 C290.06,486.32 285.10,499.82 285.33,500.42 Z" fill="rgb(37,37,36)"/>
<path d="M 0.00 450.00 L 0.00 0.00 L 450.00 0.00 L 900.00 0.00 L 900.00 450.00 L 900.00 900.00 L 450.00 900.00 L 0.00 900.00 L 0.00 450.00 ZM 222.16 664.50 C231.46,638.87 237.09,623.44 245.56,600.25 L 249.31 590.00 L 352.41 590.00 L 455.51 590.00 L 458.19 597.25 C459.67,601.24 464.08,612.83 468.00,623.00 C471.91,633.17 477.10,646.67 479.52,653.00 C481.94,659.33 486.35,670.58 489.31,678.00 L 494.69 691.50 L 547.95 691.76 C590.44,691.97 601.12,691.76 600.73,690.76 C598.92,686.11 585.05,651.65 579.50,638.00 C575.81,628.92 567.25,607.78 560.48,591.00 C553.72,574.22 542.36,546.10 535.25,528.50 C528.13,510.90 508.78,462.98 492.24,422.00 C475.70,381.02 459.61,341.20 456.49,333.50 C453.36,325.80 444.82,304.65 437.50,286.50 C413.90,227.98 407.23,211.56 405.94,208.75 L 404.68 206.00 L 349.59 206.03 L 294.50 206.05 L 288.14 222.28 C284.65,231.20 259.51,294.30 232.29,362.50 C205.07,430.70 165.94,528.80 145.34,580.50 C124.74,632.20 106.56,677.73 104.94,681.68 C103.32,685.63 102.00,689.34 102.00,689.92 C102.00,690.80 163.39,691.75 206.42,691.53 L 212.35 691.50 L 222.16 664.50 ZM 798.00 691.05 C798.00,690.53 790.18,670.61 780.62,646.80 C737.77,540.07 699.90,445.80 677.26,389.50 C663.76,355.95 651.05,324.23 649.01,319.00 C646.96,313.77 635.94,286.28 624.52,257.91 L 603.75 206.32 L 576.12 205.66 C522.83,204.39 500.00,204.88 500.00,207.29 C500.00,207.85 502.23,213.98 504.97,220.91 C507.70,227.83 514.73,245.65 520.59,260.50 C526.45,275.35 532.71,291.33 534.50,296.00 C536.29,300.67 540.24,310.80 543.27,318.50 C549.76,334.97 562.77,368.16 576.98,404.50 C582.57,418.80 590.00,437.70 593.49,446.50 C600.65,464.57 615.63,502.69 620.02,514.00 C624.19,524.76 638.84,562.16 648.03,585.50 C652.36,596.50 660.21,616.53 665.49,630.00 C670.76,643.47 678.21,662.43 682.04,672.12 C685.87,681.82 689.00,690.26 689.00,690.87 C689.00,691.69 703.78,692.00 743.50,692.00 C777.31,692.00 798.00,691.64 798.00,691.05 ZM 285.33 500.42 C285.10,499.82 290.06,486.32 296.36,470.42 C317.55,416.89 333.07,377.64 342.21,354.50 C347.21,341.85 351.61,331.14 352.00,330.70 C352.38,330.26 354.08,333.64 355.77,338.20 C357.46,342.77 362.05,354.83 365.97,365.00 C369.89,375.17 376.42,392.27 380.50,403.00 C384.57,413.73 391.11,430.83 395.03,441.00 C398.95,451.17 402.53,460.62 402.98,462.00 C403.42,463.38 405.83,469.67 408.32,476.00 C410.81,482.33 413.98,490.53 415.37,494.24 L 417.90 500.97 L 351.82 501.24 C305.07,501.42 285.62,501.18 285.33,500.42 Z" fill="rgb(209,155,118)"/>
</g>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 4.7 KiB

14
studio/frontend/public/provider-logos/deepseek.svg Normal file
View file

@ -0,0 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg id="Layer_2" data-name="Layer 2" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 377.1 277.86">
<defs>
<style>
.cls-1 {
fill: #4d6bfe;
stroke-width: 0px;
}
</style>
</defs>
<g id="Layer_1-2" data-name="Layer 1">
<path id="path" class="cls-1" d="M373.15,23.32c-4-1.95-5.72,1.77-8.06,3.66-.79.62-1.47,1.43-2.14,2.14-5.85,6.26-12.67,10.36-21.57,9.86-13.04-.71-24.16,3.38-33.99,13.37-2.09-12.31-9.04-19.66-19.6-24.38-5.54-2.45-11.13-4.9-14.99-10.23-2.71-3.78-3.44-8-4.81-12.16-.85-2.51-1.72-5.09-4.6-5.52-3.13-.5-4.36,2.14-5.58,4.34-4.93,8.99-6.82,18.92-6.65,28.97.43,22.58,9.97,40.56,28.89,53.37,2.16,1.46,2.71,2.95,2.03,5.09-1.29,4.4-2.82,8.68-4.19,13.09-.85,2.82-2.14,3.44-5.15,2.2-10.39-4.34-19.37-10.76-27.29-18.55-13.46-13.02-25.63-27.41-40.81-38.67-3.57-2.64-7.12-5.09-10.81-7.41-15.49-15.07,2.03-27.45,6.08-28.9,4.25-1.52,1.47-6.79-12.23-6.73-13.69.06-26.24,4.65-42.21,10.76-2.34.93-4.79,1.61-7.32,2.14-14.5-2.73-29.55-3.35-45.29-1.58-29.62,3.32-53.28,17.34-70.68,41.28C1.29,88.2-3.63,120.88,2.39,155c6.33,35.91,24.64,65.68,52.8,88.94,29.18,24.1,62.8,35.91,101.15,33.65,23.29-1.33,49.23-4.46,78.48-29.24,7.38,3.66,15.12,5.12,27.97,6.23,9.89.93,19.41-.5,26.79-2.02,11.55-2.45,10.75-13.15,6.58-15.13-33.87-15.78-26.44-9.36-33.2-14.54,17.21-20.41,43.15-41.59,53.3-110.19.79-5.46.11-8.87,0-13.3-.06-2.67.54-3.72,3.61-4.03,8.48-.96,16.72-3.29,24.28-7.47,21.94-12,30.78-31.69,32.87-55.33.31-3.6-.06-7.35-3.86-9.24ZM181.96,235.97c-32.83-25.83-48.74-34.33-55.31-33.96-6.14.34-5.04,7.38-3.69,11.97,1.41,4.53,3.26,7.66,5.85,11.63,1.78,2.64,3.01,6.57-1.78,9.49-10.57,6.58-28.95-2.2-29.82-2.64-21.38-12.59-39.26-29.24-51.87-52.01-12.16-21.92-19.23-45.43-20.39-70.52-.31-6.08,1.47-8.22,7.49-9.3,7.92-1.46,16.11-1.77,24.03-.62,33.49,4.9,62.01,19.91,85.9,43.63,13.65,13.55,23.97,29.71,34.61,45.49,11.3,16.78,23.48,32.75,38.97,45.84,5.46,4.59,9.83,8.09,14,10.67-12.59,1.4-33.62,1.71-47.99-9.68ZM197.69,134.65c0-2.7,2.15-4.84,4.87-4.84.6,0,1.16.12,1.66.31.67.25,1.29.62,1.77,1.18.87.84,1.36,2.08,1.36,3.35,0,2.7-2.15,4.84-4.85,4.84s-4.81-2.14-4.81-4.84ZM246.55,159.77c-3.13,1.27-6.26,2.39-9.27,2.51-4.67.22-9.77-1.68-12.55-4-4.3-3.6-7.36-5.61-8.67-11.94-.54-2.7-.23-6.85.25-9.24,1.12-5.15-.12-8.44-3.74-11.44-2.96-2.45-6.7-3.1-10.82-3.1-1.54,0-2.95-.68-4-1.24-1.72-.87-3.13-3.01-1.78-5.64.43-.84,2.53-2.92,3.02-3.29,5.58-3.19,12.03-2.14,18,.25,5.54,2.26,9.71,6.42,15.72,12.28,6.16,7.1,7.26,9.09,10.76,14.39,2.76,4.19,5.29,8.47,7.01,13.37,1.04,3.04-.31,5.55-3.94,7.1Z"/>
</g>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 2.5 KiB

72
studio/frontend/public/provider-logos/gemini.svg Normal file
View file

File diff suppressed because one or more lines are too long
Side by side

After

Width:  |  Height:  |  Size: 3 MiB

8
studio/frontend/public/provider-logos/huggingface.svg Normal file
View file

File diff suppressed because one or more lines are too long
Side by side

After

Width:  |  Height:  |  Size: 34 KiB

BIN
studio/frontend/public/provider-logos/kimi.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 15 KiB

1
studio/frontend/public/provider-logos/llama_cpp.svg Normal file
View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" viewBox="0 0 512 512"><path d="m356.4 201.3-32.8 58.3c-43.3-33.3-107.4-38.2-150.7-2.4-69.8 57.6-64.9 190.8 43.7 191.6 30.4 0 56.2-14.3 83.9-23.8l14.6 58.1c-24.6 11.4-49.6 23.1-76.6 26.7-246 33.5-231.9-321.6-9.5-340.1 46.7-3.9 87.8 8.3 127.6 31.6zm-169.9-55.9c-37.4 11.2-72.2 31.8-98.5 60.8-4.9-58.8 8.3-177.7 73.7-201 9.7-3.4 43-11.9 42.1 5.3-1 17.3-24.1 46.9-29.7 63-9.7 28.2-.7 47.6 12.6 72.2zm92.4 252.8h-36.5v-41.3h-41.3v-34h37.7l3.6-3.6v-40.1h36.5V323h38.9v34h-38.9zm133.7-41.3v41.3h-36.5v-41.3h-38.9v-34h38.9v-43.8h36.5v40.1l3.6 3.6h37.7v34h-41.3zM305.4 31.4c4.9 7.3-22.6 38.7-27 46.7-12.6 23.8-4.1 37.4 5.3 60-27.5-4.1-53-.7-80.2 2.4C209.6 88.3 239 12.2 305.4 31.4" style="fill:#ff8236"/></svg>
Side by side

After

Width:  |  Height:  |  Size: 763 B

19
studio/frontend/public/provider-logos/misc/meta.svg Normal file
View file

@ -0,0 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cc="http://creativecommons.org/ns#" width="287.56" height="191">
<desc>Logo of Meta Platforms -- Graphic created by Detmar Owen</desc>
<defs>
<linearGradient id="Grad_Logo1" x1="61" y1="117" x2="259" y2="127" gradientUnits="userSpaceOnUse">
<stop style="stop-color:#0064e1" offset="0"/>
<stop style="stop-color:#0064e1" offset="0.4"/>
<stop style="stop-color:#0073ee" offset="0.83"/>
<stop style="stop-color:#0082fb" offset="1"/>
</linearGradient>
<linearGradient id="Grad_Logo2" x1="45" y1="139" x2="45" y2="66" gradientUnits="userSpaceOnUse">
<stop style="stop-color:#0082fb" offset="0"/>
<stop style="stop-color:#0064e0" offset="1"/>
</linearGradient>
</defs>
<path id="Logo0" style="fill:#0081fb" d="m31.06,125.96c0,10.98 2.41,19.41 5.56,24.51 4.13,6.68 10.29,9.51 16.57,9.51 8.1,0 15.51-2.01 29.79-21.76 11.44-15.83 24.92-38.05 33.99-51.98l15.36-23.6c10.67-16.39 23.02-34.61 37.18-46.96 11.56-10.08 24.03-15.68 36.58-15.68 21.07,0 41.14,12.21 56.5,35.11 16.81,25.08 24.97,56.67 24.97,89.27 0,19.38-3.82,33.62-10.32,44.87-6.28,10.88-18.52,21.75-39.11,21.75l0-31.02c17.63,0 22.03-16.2 22.03-34.74 0-26.42-6.16-55.74-19.73-76.69-9.63-14.86-22.11-23.94-35.84-23.94-14.85,0-26.8,11.2-40.23,31.17-7.14,10.61-14.47,23.54-22.7,38.13l-9.06,16.05c-18.2,32.27-22.81,39.62-31.91,51.75-15.95,21.24-29.57,29.29-47.5,29.29-21.27,0-34.72-9.21-43.05-23.09-6.8-11.31-10.14-26.15-10.14-43.06z"/>
<path id="Logo1" style="fill:url(#Grad_Logo1)" d="m24.49,37.3c14.24-21.95 34.79-37.3 58.36-37.3 13.65,0 27.22,4.04 41.39,15.61 15.5,12.65 32.02,33.48 52.63,67.81l7.39,12.32c17.84,29.72 27.99,45.01 33.93,52.22 7.64,9.26 12.99,12.02 19.94,12.02 17.63,0 22.03-16.2 22.03-34.74l27.4-.86c0,19.38-3.82,33.62-10.32,44.87-6.28,10.88-18.52,21.75-39.11,21.75-12.8,0-24.14-2.78-36.68-14.61-9.64-9.08-20.91-25.21-29.58-39.71l-25.79-43.08c-12.94-21.62-24.81-37.74-31.68-45.04-7.39-7.85-16.89-17.33-32.05-17.33-12.27,0-22.69,8.61-31.41,21.78z"/>
<path id="Logo2" style="fill:url(#Grad_Logo2)" d="m82.35,31.23c-12.27,0-22.69,8.61-31.41,21.78-12.33,18.61-19.88,46.33-19.88,72.95 0,10.98 2.41,19.41 5.56,24.51l-26.48,17.44c-6.8-11.31-10.14-26.15-10.14-43.06 0-30.75 8.44-62.8 24.49-87.55 14.24-21.95 34.79-37.3 58.36-37.3z"/>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 2.3 KiB

1
studio/frontend/public/provider-logos/misc/microsoft.svg Normal file
View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 23 23"><path fill="#f3f3f3" d="M0 0h23v23H0z"/><path fill="#f35325" d="M1 1h10v10H1z"/><path fill="#81bc06" d="M12 1h10v10H12z"/><path fill="#05a6f0" d="M1 12h10v10H1z"/><path fill="#ffba08" d="M12 12h10v10H12z"/></svg>
Side by side

After

Width:  |  Height:  |  Size: 272 B

BIN
studio/frontend/public/provider-logos/misc/minimax.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 8.5 KiB

1
studio/frontend/public/provider-logos/misc/nvidia.svg Normal file
View file

@ -0,0 +1 @@
<svg viewBox="0 0 271.7 179.7" xmlns="http://www.w3.org/2000/svg" width="2500" height="1653"><path d="M101.3 53.6V37.4c1.6-.1 3.2-.2 4.8-.2 44.4-1.4 73.5 38.2 73.5 38.2S148.2 119 114.5 119c-4.5 0-8.9-.7-13.1-2.1V67.7c17.3 2.1 20.8 9.7 31.1 27l23.1-19.4s-16.9-22.1-45.3-22.1c-3-.1-6 .1-9 .4m0-53.6v24.2l4.8-.3c61.7-2.1 102 50.6 102 50.6s-46.2 56.2-94.3 56.2c-4.2 0-8.3-.4-12.4-1.1v15c3.4.4 6.9.7 10.3.7 44.8 0 77.2-22.9 108.6-49.9 5.2 4.2 26.5 14.3 30.9 18.7-29.8 25-99.3 45.1-138.7 45.1-3.8 0-7.4-.2-11-.6v21.1h170.2V0H101.3zm0 116.9v12.8c-41.4-7.4-52.9-50.5-52.9-50.5s19.9-22 52.9-25.6v14h-.1c-17.3-2.1-30.9 14.1-30.9 14.1s7.7 27.3 31 35.2M27.8 77.4s24.5-36.2 73.6-40V24.2C47 28.6 0 74.6 0 74.6s26.6 77 101.3 84v-14c-54.8-6.8-73.5-67.2-73.5-67.2z" fill="#76b900"/></svg>
Side by side

After

Width:  |  Height:  |  Size: 771 B

BIN
studio/frontend/public/provider-logos/misc/perplexity.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 6.5 KiB

1
studio/frontend/public/provider-logos/misc/xai.svg Normal file
View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 466.04 516.93"><polygon points="0.12 182.71 234.14 516.92 338.15 516.92 104.13 182.71 0.12 182.71"/><polygon points="0 516.92 104.08 516.92 156.08 442.67 104.04 368.34 0 516.92"/><polygon points="466.04 0 361.96 0 182.1 256.86 234.15 331.18 466.04 0"/><polygon points="380.78 516.92 466.04 516.92 466.04 37.16 380.78 158.92 380.78 516.92"/></svg>
Side by side

After

Width:  |  Height:  |  Size: 399 B

215
studio/frontend/public/provider-logos/misc/z-ai.svg Normal file
View file

@ -0,0 +1,215 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" x="0px" y="0px" viewBox="0.0 0.0 30.0 30.0" style="enable-background:new 0 0 30 30;" xml:space="preserve" width="316.22776601683796" height="316.22776601683796">
<style type="text/css">
.st0{opacity:0.3;fill:#E2E4E7;}
.st1{opacity:0.8;fill:#E2E4E7;stroke:#FFFFFF;stroke-width:5;stroke-miterlimit:10;}
.st2{fill:url(#SVGID_1_);}
.st3{fill:none;stroke:#E0E4E9;stroke-width:0.25;stroke-miterlimit:10;}
.st4{fill:none;}
.st5{fill:#9DA1A5;}
.st6{fill-rule:evenodd;clip-rule:evenodd;fill:none;}
.st7{fill-rule:evenodd;clip-rule:evenodd;fill:#DFE2E7;}
.st8{fill-rule:evenodd;clip-rule:evenodd;fill:#CDD4DA;}
.st9{fill-rule:evenodd;clip-rule:evenodd;fill:#B3BCC7;}
.st10{fill-rule:evenodd;clip-rule:evenodd;fill:#9DAAB7;}
.st11{fill-rule:evenodd;clip-rule:evenodd;fill:#8698A8;}
.st12{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_2_);}
.st13{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_3_);}
.st14{fill:#1F63EC;}
.st15{fill:#2D2D2D;}
.st16{fill:none;stroke:#E0E4E9;stroke-width:0.5;stroke-miterlimit:10;}
.st17{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_4_);}
.st18{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_5_);}
.st19{fill:none;stroke:#677380;stroke-width:0.5;stroke-miterlimit:10;}
.st20{fill:none;stroke:url(#SVGID_6_);stroke-width:2;stroke-miterlimit:10;}
.st21{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_7_);}
.st22{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_8_);}
.st23{fill:#FFFFFF;}
.st24{fill-rule:evenodd;clip-rule:evenodd;fill:#2D2D2D;}
.st25{clip-path:url(#SVGID_10_);}
.st26{clip-path:url(#SVGID_12_);}
.st27{fill:url(#SVGID_13_);}
.st28{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_14_);}
.st29{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_15_);}
.st30{clip-path:url(#SVGID_17_);}
.st31{clip-path:url(#SVGID_19_);}
.st32{fill:url(#SVGID_20_);}
.st33{fill:none;stroke:url(#SVGID_21_);stroke-width:2;stroke-miterlimit:10;}
.st34{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_22_);}
.st35{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_23_);}
.st36{clip-path:url(#SVGID_25_);}
.st37{clip-path:url(#SVGID_27_);}
.st38{fill:url(#SVGID_28_);}
.st39{clip-path:url(#SVGID_30_);}
.st40{clip-path:url(#SVGID_32_);}
.st41{fill:url(#SVGID_33_);}
.st42{fill-rule:evenodd;clip-rule:evenodd;fill:#126EF6;}
.st43{fill-rule:evenodd;clip-rule:evenodd;fill:#FFFFFF;}
.st44{clip-path:url(#SVGID_35_);}
.st45{clip-path:url(#SVGID_37_);}
.st46{fill:url(#SVGID_38_);}
.st47{fill-rule:evenodd;clip-rule:evenodd;fill:#9DA1A5;}
.st48{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_39_);}
.st49{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_40_);}
.st50{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_41_);}
.st51{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_42_);}
.st52{fill:none;stroke:url(#SVGID_43_);stroke-width:2;stroke-miterlimit:10;}
.st53{fill-rule:evenodd;clip-rule:evenodd;fill:none;stroke:#E0E4E9;stroke-width:0.5;stroke-miterlimit:10;}
.st54{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_44_);}
.st55{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_45_);}
.st56{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_46_);}
.st57{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_47_);}
.st58{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_48_);}
.st59{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_49_);}
.st60{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_50_);}
.st61{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_51_);}
.st62{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_52_);}
.st63{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_53_);}
.st64{clip-path:url(#SVGID_55_);}
.st65{clip-path:url(#SVGID_57_);}
.st66{fill:url(#SVGID_58_);}
.st67{clip-path:url(#SVGID_60_);}
.st68{clip-path:url(#SVGID_62_);}
.st69{fill:url(#SVGID_63_);}
.st70{fill:none;stroke:url(#SVGID_64_);stroke-width:2;stroke-miterlimit:10;}
.st71{clip-path:url(#SVGID_66_);}
.st72{clip-path:url(#SVGID_68_);}
.st73{fill:url(#SVGID_69_);}
.st74{clip-path:url(#SVGID_71_);}
.st75{clip-path:url(#SVGID_73_);}
.st76{fill:url(#SVGID_74_);}
.st77{clip-path:url(#SVGID_76_);}
.st78{clip-path:url(#SVGID_78_);}
.st79{fill:url(#SVGID_79_);}
.st80{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_80_);}
.st81{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_81_);}
.st82{clip-path:url(#SVGID_83_);}
.st83{clip-path:url(#SVGID_85_);}
.st84{fill:url(#SVGID_86_);}
.st85{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_87_);}
.st86{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_88_);}
.st87{clip-path:url(#SVGID_90_);}
.st88{clip-path:url(#SVGID_92_);}
.st89{fill:url(#SVGID_93_);}
.st90{fill:none;stroke:url(#SVGID_94_);stroke-width:2;stroke-miterlimit:10;}
.st91{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_95_);}
.st92{fill-rule:evenodd;clip-rule:evenodd;fill:url(#SVGID_96_);}
.st93{clip-path:url(#SVGID_98_);}
.st94{clip-path:url(#SVGID_100_);}
.st95{fill:url(#SVGID_101_);}
.st96{clip-path:url(#SVGID_103_);}
.st97{clip-path:url(#SVGID_105_);}
.st98{fill:url(#SVGID_106_);}
.st99{clip-path:url(#SVGID_108_);}
.st100{clip-path:url(#SVGID_110_);}
.st101{fill:url(#SVGID_111_);}
.st102{fill:#FFFFFF;stroke:#B3BCC7;stroke-width:0.275;stroke-miterlimit:10;}
.st103{clip-path:url(#SVGID_113_);}
.st104{fill:#FDD138;}
.st105{fill:#FCA62F;}
.st106{fill:#FB7927;}
.st107{fill:#F44B22;}
.st108{fill:#D81915;}
.st109{fill:#2D2D2D;stroke:#FFFFFF;stroke-width:0.3354;stroke-miterlimit:10;}
.st110{fill:none;stroke:#65727F;stroke-width:2;stroke-miterlimit:10;}
.st111{fill:none;stroke:#65727F;stroke-width:0.75;stroke-miterlimit:10;}
.st112{fill:url(#SVGID_114_);}
.st113{fill:#D06C50;}
.st114{fill:#2D2D2D;stroke:#B3BCC7;stroke-width:0.275;stroke-miterlimit:10;}
.st115{opacity:0.2;}
.st116{fill:none;stroke:#677380;stroke-width:0.3564;stroke-miterlimit:10;}
.st117{fill:none;stroke:#677380;stroke-width:0.3564;stroke-miterlimit:10;stroke-dasharray:1.0212,1.0212;}
.st118{fill:none;stroke:#677380;stroke-width:0.3564;stroke-miterlimit:10;stroke-dasharray:1.0205,1.0205;}
.st119{opacity:0.2;fill:none;}
.st120{fill:none;stroke:#677380;stroke-width:0.3689;stroke-miterlimit:10;}
.st121{fill:none;stroke:#677380;stroke-width:0.3689;stroke-miterlimit:10;stroke-dasharray:1.0509,1.0509;}
.st122{opacity:0.3;fill:#1F63EC;}
.st123{fill:#2D2D2D;stroke:#FFFFFF;stroke-width:0.3162;stroke-miterlimit:10;}
.st124{fill:#FFFFFF;stroke:#B3BCC7;stroke-width:0.3162;stroke-miterlimit:10;}
.st125{clip-path:url(#SVGID_118_);}
.st126{fill:url(#SVGID_119_);}
.st127{fill:none;stroke:#DFE2E7;stroke-width:0.75;stroke-miterlimit:10;}
.st128{fill:#9DA1A5;stroke:#FFFFFF;stroke-miterlimit:10;}
.st129{fill:url(#SVGID_120_);}
.st130{fill:none;stroke:#677380;stroke-width:0.75;stroke-miterlimit:10;}
.st131{opacity:0.4;}
.st132{clip-path:url(#SVGID_122_);}
.st133{clip-path:url(#SVGID_124_);}
.st134{fill:url(#SVGID_125_);}
.st135{fill:none;stroke:#8392A3;stroke-width:0.35;stroke-miterlimit:10;}
.st136{fill:none;stroke:#8392A3;stroke-width:0.35;stroke-miterlimit:10;stroke-dasharray:0.9951,0.9951;}
.st137{fill:none;stroke:#8392A3;stroke-width:0.35;stroke-miterlimit:10;stroke-dasharray:1.004,1.004;}
.st138{fill:none;stroke:url(#SVGID_126_);stroke-width:1.5;stroke-miterlimit:10;}
.st139{fill:url(#SVGID_127_);}
.st140{fill:none;stroke:#DDE0E4;stroke-width:0.35;stroke-miterlimit:10;}
.st141{fill:#2D2D2D;stroke:#A9B3BE;stroke-width:0.275;stroke-miterlimit:10;}
.st142{fill-rule:evenodd;clip-rule:evenodd;fill:#126EF4;}
.st143{fill:#FFFFFF;stroke:#B1BAC4;stroke-width:0.275;stroke-miterlimit:10;}
.st144{fill:#CE6C50;}
.st145{fill:#5B5B5B;}
.st146{fill:#8392A3;}
.st147{fill:none;stroke:url(#SVGID_128_);stroke-width:1.5;stroke-miterlimit:10;}
.st148{fill:url(#SVGID_129_);}
.st149{fill:none;stroke:#B5BDC4;stroke-width:0.7;stroke-miterlimit:10;}
.st150{opacity:0.6;fill:none;stroke:#78838E;stroke-width:0.35;stroke-miterlimit:10;}
.st151{opacity:0.2;fill:none;stroke:#8392A3;stroke-width:0.35;stroke-miterlimit:10;stroke-dasharray:1,1;}
.st152{fill:none;stroke:#DDE0E4;stroke-width:0.75;stroke-miterlimit:10;}
.st153{fill:none;stroke:#8392A3;stroke-width:0.5;stroke-miterlimit:10;}
.st154{opacity:0.2;fill:none;stroke:#677380;stroke-width:0.3564;stroke-miterlimit:10;stroke-dasharray:1.0182,1.0182;}
.st155{fill:none;stroke:#DDE0E4;stroke-width:0.765;stroke-miterlimit:10;}
.st156{fill:url(#SVGID_130_);}
.st157{fill:url(#SVGID_131_);}
.st158{fill:#B1BAC4;}
.st159{fill:#CBD1D8;}
.st160{fill:#0B1B2B;}
.st161{fill:#91D119;}
.st162{opacity:0.7;}
.st163{fill:#FFFFFF;stroke:#000000;stroke-width:0.4418;stroke-miterlimit:10;}
.st164{fill:none;stroke:#939CAA;stroke-width:0.2209;stroke-miterlimit:10;}
.st165{fill:none;stroke:#FFFFFF;stroke-width:3.0924;stroke-miterlimit:10;}
.st166{fill:url(#SVGID_132_);}
.st167{fill:none;stroke:url(#SVGID_133_);stroke-width:1.714;stroke-miterlimit:10;}
.st168{fill:url(#SVGID_134_);}
.st169{fill:url(#SVGID_135_);}
.st170{fill:url(#SVGID_136_);}
.st171{fill:url(#SVGID_137_);}
.st172{fill:url(#SVGID_138_);}
.st173{fill:url(#SVGID_139_);}
.st174{fill:url(#SVGID_140_);}
.st175{fill:url(#SVGID_141_);}
.st176{fill:url(#SVGID_142_);}
.st177{fill:url(#SVGID_143_);}
.st178{fill:url(#SVGID_144_);}
.st179{fill:none;stroke:#1F63EC;stroke-width:4;stroke-miterlimit:10;}
.st180{fill:none;stroke:#0B1B2B;stroke-width:4;stroke-miterlimit:10;}
.st181{fill:none;stroke:#677380;stroke-width:0.3989;stroke-miterlimit:10;}
.st182{fill:none;stroke:#677380;stroke-width:0.3989;stroke-miterlimit:10;stroke-dasharray:1.14,1.14;}
.st183{fill:#257AF1;}
.st184{opacity:0.3;fill:#FFFFFF;}
.st185{fill:none;stroke:#98A5B2;stroke-width:4;stroke-miterlimit:10;}
.st186{fill:none;stroke:#65727F;stroke-width:0.3989;stroke-miterlimit:10;}
.st187{fill:none;stroke:#65727F;stroke-width:0.3989;stroke-miterlimit:10;stroke-dasharray:1.14,1.14;}
.st188{fill:none;stroke:#DDDFE4;stroke-width:0.75;stroke-miterlimit:10;}
.st189{fill:#9A9EA2;}
.st190{fill-rule:evenodd;clip-rule:evenodd;fill:#3267AC;}
.st191{fill:#FFFFFF;stroke:#AFB8C3;stroke-width:0.275;stroke-miterlimit:10;}
.st192{fill:#C5694E;}
.st193{fill:#8192A2;}
.st194{fill:#2D2D2D;stroke:#FFFFFF;stroke-width:0.6317;stroke-miterlimit:10;}
</style>
<g id="图层_2">
</g>
<g id="图层_1">
<path class="st194" d="M24.51,28.51H5.49c-2.21,0-4-1.79-4-4V5.49c0-2.21,1.79-4,4-4h19.03c2.21,0,4,1.79,4,4v19.03 C28.51,26.72,26.72,28.51,24.51,28.51z"/>
<g>
<g>
<g>
<g>
<path class="st23" d="M15.47,7.1l-1.3,1.85c-0.2,0.29-0.54,0.47-0.9,0.47h-7.1V7.09C6.16,7.1,15.47,7.1,15.47,7.1z"/>
<polygon class="st23" points="24.3,7.1 13.14,22.91 5.7,22.91 16.86,7.1 "/>
<path class="st23" d="M14.53,22.91l1.31-1.86c0.2-0.29,0.54-0.47,0.9-0.47h7.09v2.33H14.53z"/>
</g>
</g>
</g>
</g>
</g>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 11 KiB

19
studio/frontend/public/provider-logos/mistral.svg Normal file
View file

@ -0,0 +1,19 @@
<svg width="191" height="135" viewBox="0 0 191 135" fill="none" xmlns="http://www.w3.org/2000/svg">
<g clip-path="url(#clip0_134_208)">
<path d="M54.3221 0H27.1531V27.0892H54.3221V0Z" fill="#FFD800"/>
<path d="M162.984 0H135.815V27.0892H162.984V0Z" fill="#FFD800"/>
<path d="M81.4823 27.0913H27.1531V54.1805H81.4823V27.0913Z" fill="#FFAF00"/>
<path d="M162.99 27.0913H108.661V54.1805H162.99V27.0913Z" fill="#FFAF00"/>
<path d="M162.972 54.168H27.1531V81.2572H162.972V54.168Z" fill="#FF8205"/>
<path d="M54.3221 81.2593H27.1531V108.349H54.3221V81.2593Z" fill="#FA500F"/>
<path d="M108.661 81.2593H81.4917V108.349H108.661V81.2593Z" fill="#FA500F"/>
<path d="M162.984 81.2593H135.815V108.349H162.984V81.2593Z" fill="#FA500F"/>
<path d="M81.4879 108.339H-0.00146484V135.429H81.4879V108.339Z" fill="#E10500"/>
<path d="M190.159 108.339H108.661V135.429H190.159V108.339Z" fill="#E10500"/>
</g>
<defs>
<clipPath id="clip0_134_208">
<rect width="190.141" height="135" fill="white"/>
</clipPath>
</defs>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 1,001 B

14
studio/frontend/public/provider-logos/ollama.svg Normal file
View file

File diff suppressed because one or more lines are too long
Side by side

After

Width:  |  Height:  |  Size: 8.6 KiB

5
studio/frontend/public/provider-logos/openai.svg Normal file
View file

@ -0,0 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg id="Layer_1" xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 158.7128 157.296">
<!-- Generator: Adobe Illustrator 29.2.1, SVG Export Plug-In . SVG Version: 2.1.0 Build 116) -->
<path d="M60.8734,57.2556v-14.9432c0-1.2586.4722-2.2029,1.5728-2.8314l30.0443-17.3023c4.0899-2.3593,8.9662-3.4599,13.9988-3.4599,18.8759,0,30.8307,14.6289,30.8307,30.2006,0,1.1007,0,2.3593-.158,3.6178l-31.1446-18.2467c-1.8872-1.1006-3.7754-1.1006-5.6629,0l-39.4812,22.9651ZM131.0276,115.4561v-35.7074c0-2.2028-.9446-3.7756-2.8318-4.8763l-39.481-22.9651,12.8982-7.3934c1.1007-.6285,2.0453-.6285,3.1458,0l30.0441,17.3024c8.6523,5.0341,14.4708,15.7296,14.4708,26.1107,0,11.9539-7.0769,22.965-18.2461,27.527v.0021ZM51.593,83.9964l-12.8982-7.5497c-1.1007-.6285-1.5728-1.5728-1.5728-2.8314v-34.6048c0-16.8303,12.8982-29.5722,30.3585-29.5722,6.607,0,12.7403,2.2029,17.9324,6.1349l-30.987,17.9324c-1.8871,1.1007-2.8314,2.6735-2.8314,4.8764v45.6159l-.0014-.0015ZM79.3562,100.0403l-18.4829-10.3811v-22.0209l18.4829-10.3811,18.4812,10.3811v22.0209l-18.4812,10.3811ZM91.2319,147.8591c-6.607,0-12.7403-2.2031-17.9324-6.1344l30.9866-17.9333c1.8872-1.1005,2.8318-2.6728,2.8318-4.8759v-45.616l13.0564,7.5498c1.1005.6285,1.5723,1.5728,1.5723,2.8314v34.6051c0,16.8297-13.0564,29.5723-30.5147,29.5723v.001ZM53.9522,112.7822l-30.0443-17.3024c-8.652-5.0343-14.471-15.7296-14.471-26.1107,0-12.1119,7.2356-22.9652,18.403-27.5272v35.8634c0,2.2028.9443,3.7756,2.8314,4.8763l39.3248,22.8068-12.8982,7.3938c-1.1007.6287-2.045.6287-3.1456,0ZM52.2229,138.5791c-17.7745,0-30.8306-13.3713-30.8306-29.8871,0-1.2585.1578-2.5169.3143-3.7754l30.987,17.9323c1.8871,1.1005,3.7757,1.1005,5.6628,0l39.4811-22.807v14.9435c0,1.2585-.4721,2.2021-1.5728,2.8308l-30.0443,17.3025c-4.0898,2.359-8.9662,3.4605-13.9989,3.4605h.0014ZM91.2319,157.296c19.0327,0,34.9188-13.5272,38.5383-31.4594,17.6164-4.562,28.9425-21.0779,28.9425-37.908,0-11.0112-4.719-21.7066-13.2133-29.4143.7867-3.3035,1.2595-6.607,1.2595-9.909,0-22.4929-18.2471-39.3247-39.3251-39.3247-4.2461,0-8.3363.6285-12.4262,2.045-7.0792-6.9213-16.8318-11.3254-27.5271-11.3254-19.0331,0-34.9191,13.5268-38.5384,31.4591C11.3255,36.0212,0,52.5373,0,69.3675c0,11.0112,4.7184,21.7065,13.2125,29.4142-.7865,3.3035-1.2586,6.6067-1.2586,9.9092,0,22.4923,18.2466,39.3241,39.3248,39.3241,4.2462,0,8.3362-.6277,12.426-2.0441,7.0776,6.921,16.8302,11.3251,27.5271,11.3251Z"/>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 2.4 KiB

1
studio/frontend/public/provider-logos/openrouter.svg Normal file
View file

@ -0,0 +1 @@
<svg fill="none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><g clip-path="url(#prefix__clip0_8_13)"><path fill-rule="evenodd" clip-rule="evenodd" d="M358.485 41.75l154.027 87.573v1.856l-155.605 86.634.362-45.162-17.514-.64c-22.592-.598-34.368.042-48.384 2.346-22.699 3.734-43.478 12.31-67.136 28.843l-46.208 32.107c-6.059 4.16-10.56 7.168-14.507 9.706l-10.987 6.87-8.469 4.992 8.213 4.906 11.307 7.211c10.155 6.699 24.96 16.981 57.621 39.808 23.68 16.533 44.438 25.109 67.136 28.843l6.4.96c14.806 1.941 29.334 2.005 60.267.704l.469-46.059 154.027 87.573v1.856l-155.605 86.656.298-39.722-13.546.469c-29.568.896-45.59.043-66.944-3.456-36.139-5.973-69.547-19.755-104.128-43.925l-46.038-32a467.072 467.072 0 00-16.106-10.624l-9.963-5.974c-5.38-3.1-10.785-6.157-16.213-9.173C62.037 314.24 12.01 301.141 0 301.141v-90.197l2.987.085c12.032-.149 62.08-13.269 81.258-23.978l21.675-12.374 9.344-5.845c9.131-5.973 22.869-15.488 57.301-39.531 34.582-24.17 67.968-37.973 104.128-43.925 24.576-4.053 42.112-4.544 81.366-2.944l.426-40.683z" fill="#000"/></g><defs><clipPath id="prefix__clip0_8_13"><path fill="#fff" d="M0 0h512v512H0z"/></clipPath></defs></svg>
Side by side

After

Width:  |  Height:  |  Size: 1.1 KiB

BIN
studio/frontend/public/provider-logos/qwen.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 114 KiB

1
studio/frontend/public/provider-logos/vllm.svg Normal file
View file

@ -0,0 +1 @@
<svg version="1.1" viewBox="0.0 0.0 96.0 96.0" fill="none" stroke="none" stroke-linecap="square" stroke-miterlimit="10" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"><clipPath id="g31e21232314_0_33.0"><path d="m0 0l96.0 0l0 96.0l-96.0 0l0 -96.0z" clip-rule="nonzero"/></clipPath><g clip-path="url(#g31e21232314_0_33.0)"><path fill="#d9d9d9" d="m41.04961 80.271324l1.8897629 0l0 2.3307114l-1.8897629 0z" fill-rule="evenodd"/><path fill="#d9d9d9" d="m42.221855 81.45145l1.8897629 0l0 2.3307037l-1.8897629 0z" fill-rule="evenodd"/><g filter="url(#shadowFilter-g31e21232314_0_33.1)"><use xlink:href="#g31e21232314_0_33.1" transform="matrix(1.0 0.0 0.0 1.0 0.0 2.0)"/></g><defs><filter id="shadowFilter-g31e21232314_0_33.1" filterUnits="userSpaceOnUse"><feGaussianBlur in="SourceAlpha" stdDeviation="2.0" result="blur"/><feComponentTransfer in="blur" color-interpolation-filters="sRGB"><feFuncR type="linear" slope="0" intercept="0.0"/><feFuncG type="linear" slope="0" intercept="0.0"/><feFuncB type="linear" slope="0" intercept="0.0"/><feFuncA type="linear" slope="0.0" intercept="0"/></feComponentTransfer></filter></defs><g id="g31e21232314_0_33.1"><path fill="#d9d9d9" d="m42.22417 28.470434l0 55.307083l-27.653543 -55.307083z" fill-rule="evenodd"/></g><g filter="url(#shadowFilter-g31e21232314_0_33.2)"><use xlink:href="#g31e21232314_0_33.2" transform="matrix(1.0 0.0 0.0 1.0 0.0 2.0)"/></g><defs><filter id="shadowFilter-g31e21232314_0_33.2" filterUnits="userSpaceOnUse"><feGaussianBlur in="SourceAlpha" stdDeviation="2.0" result="blur"/><feComponentTransfer in="blur" color-interpolation-filters="sRGB"><feFuncR type="linear" slope="0" intercept="0.0"/><feFuncG type="linear" slope="0" intercept="0.0"/><feFuncB type="linear" slope="0" intercept="0.0"/><feFuncA type="linear" slope="0.0" intercept="0"/></feComponentTransfer></filter></defs><g id="g31e21232314_0_33.2"><path fill="#d9d9d9" d="m42.223038 83.77752l21.729656 0l18.653545 -70.385826l-25.574802 13.461943z" fill-rule="evenodd"/></g><g filter="url(#shadowFilter-g31e21232314_0_33.3)"><use xlink:href="#g31e21232314_0_33.3" transform="matrix(1.0 0.0 0.0 1.0 0.0 2.0)"/></g><defs><filter id="shadowFilter-g31e21232314_0_33.3" filterUnits="userSpaceOnUse"><feGaussianBlur in="SourceAlpha" stdDeviation="2.0" result="blur"/><feComponentTransfer in="blur" color-interpolation-filters="sRGB"><feFuncR type="linear" slope="0" intercept="0.0"/><feFuncG type="linear" slope="0" intercept="0.0"/><feFuncB type="linear" slope="0" intercept="0.0"/><feFuncA type="linear" slope="0.0" intercept="0"/></feComponentTransfer></filter></defs><g id="g31e21232314_0_33.3"><path fill="#fdb515" d="m41.0477 27.293962l0 55.30709l-27.653542 -55.30709z" fill-rule="evenodd"/></g><g filter="url(#shadowFilter-g31e21232314_0_33.4)"><use xlink:href="#g31e21232314_0_33.4" transform="matrix(1.0 0.0 0.0 1.0 0.0 2.0)"/></g><defs><filter id="shadowFilter-g31e21232314_0_33.4" filterUnits="userSpaceOnUse"><feGaussianBlur in="SourceAlpha" stdDeviation="2.0" result="blur"/><feComponentTransfer in="blur" color-interpolation-filters="sRGB"><feFuncR type="linear" slope="0" intercept="0.0"/><feFuncG type="linear" slope="0" intercept="0.0"/><feFuncB type="linear" slope="0" intercept="0.0"/><feFuncA type="linear" slope="0.0" intercept="0"/></feComponentTransfer></filter></defs><g id="g31e21232314_0_33.4"><path fill="#30a2ff" d="m41.046566 82.60105l21.72966 0l18.653545 -70.385826l-25.574806 13.461943z" fill-rule="evenodd"/></g></g></svg>
Side by side

After

Width:  |  Height:  |  Size: 3.4 KiB

262
studio/frontend/src/components/assistant-ui/model-selector.tsx
View file

@ -10,24 +10,90 @@ import {
} from "@/components/ui/popover";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { usePlatformStore } from "@/config/env";
import { isCustomProviderType } from "@/features/chat/external-providers";
import { cn } from "@/lib/utils";
import {
ArrowDown01Icon,
CloudIcon,
DashboardSquare01Icon,
FolderSearchIcon,
Logout01Icon,
Search01Icon,
} from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
import { useMemo, useState } from "react";
import type {
DeletedModelRef,
ExternalModelOption,
LoraModelOption,
ModelOption,
ModelSelectorChangeMeta,
} from "./model-selector/types";
import { HubModelPicker, LoraModelPicker } from "./model-selector/pickers";
import { Input } from "../ui/input";
const PROVIDER_LOGO_EXT: Record<string, "svg" | "png" | "jpg"> = {
openai: "svg",
mistral: "svg",
gemini: "svg",
anthropic: "svg",
deepseek: "svg",
huggingface: "svg",
kimi: "jpg",
qwen: "png",
openrouter: "svg",
vllm: "svg",
ollama: "svg",
llama_cpp: "svg",
};
function providerLogoSrc(providerType: string | undefined): string | undefined {
if (!providerType) return undefined;
const ext = PROVIDER_LOGO_EXT[providerType];
if (!ext) return undefined;
return `${import.meta.env.BASE_URL}provider-logos/${providerType}.${ext}`;
}
function ExternalProviderLogo({
providerType,
className,
title,
}: {
providerType: string | undefined;
className?: string;
title?: string;
}) {
const src = providerLogoSrc(providerType);
if (!src && isCustomProviderType(providerType)) {
return (
<span title={title} aria-hidden={true} className="inline-flex shrink-0">
<HugeiconsIcon
icon={DashboardSquare01Icon}
className={cn("shrink-0", className)}
/>
</span>
);
}
if (!src) return null;
return (
<img
src={src}
alt=""
title={title}
aria-hidden={true}
className={cn(
"shrink-0 object-contain",
providerType === "openai" && "dark:invert",
className,
)}
/>
);
}
export type {
DeletedModelRef,
ExternalModelOption,
LoraModelOption,
ModelOption,
ModelSelectorChangeMeta,
@ -36,6 +102,7 @@ export type {
interface ModelSelectorProps {
models: ModelOption[];
loraModels?: LoraModelOption[];
externalModels?: ExternalModelOption[];
value?: string;
defaultValue?: string;
activeGgufVariant?: string | null;
@ -53,11 +120,13 @@ interface ModelSelectorProps {
onOpenChange?: (open: boolean) => void;
triggerDataTour?: string;
contentDataTour?: string;
showCloudIndicator?: boolean;
}
function ModelSelectorTrigger({
currentModel,
isLoaded,
showCloudIndicator = false,
variant = "outline",
size = "default",
className,
@ -65,6 +134,7 @@ function ModelSelectorTrigger({
}: {
currentModel?: ModelOption;
isLoaded: boolean;
showCloudIndicator?: boolean;
variant?: "outline" | "ghost" | "muted";
size?: "sm" | "default" | "lg";
className?: string;
@ -90,12 +160,27 @@ function ModelSelectorTrigger({
{isLoaded && (
<span className="size-2 shrink-0 rounded-full bg-emerald-500" />
)}
<span className="flex min-w-0 flex-1 items-baseline gap-2">
<span className="min-w-0 flex-1 truncate font-heading text-[16px] font-medium leading-tight text-black dark:text-white">
{currentModel?.icon ? (
<span className="flex shrink-0 items-center">{currentModel.icon}</span>
) : null}
<span className="flex min-w-0 flex-1 items-baseline">
<span className="min-w-0 flex flex-1 items-baseline truncate font-heading text-[16px] font-medium leading-tight text-black dark:text-white">
{currentModel?.name ?? "Select model"}
{showCloudIndicator ? (
<HugeiconsIcon
icon={CloudIcon}
strokeWidth={1.75}
className="relative top-[0.15625rem] ml-1.5 mr-[0.36rem] size-3.5 shrink-0 text-muted-foreground"
/>
) : null}
</span>
{currentModel?.description && (
<span className="shrink-0 text-xs leading-none text-muted-foreground">
<span
className={cn(
"shrink-0 text-xs leading-none text-muted-foreground",
showCloudIndicator ? "" : "ml-2",
)}
>
{currentModel.description}
</span>
)}
@ -115,6 +200,7 @@ function ModelSelectorTrigger({
function ModelSelectorContent({
models,
loraModels,
externalModels,
value,
onSelect,
onEject,
@ -127,6 +213,7 @@ function ModelSelectorContent({
}: {
models: ModelOption[];
loraModels: LoraModelOption[];
externalModels: ExternalModelOption[];
value?: string;
onSelect: (id: string, meta: ModelSelectorChangeMeta) => void;
onEject?: () => void;
@ -139,6 +226,20 @@ function ModelSelectorContent({
}) {
const hasSelection = Boolean(value);
const chatOnly = usePlatformStore((s) => s.isChatOnly());
const hasExternal = externalModels.length > 0;
const chatOnlyTabsDefault = useMemo(
() => (value && externalModels.some((model) => model.id === value) ? "external" : "hub"),
[externalModels, value],
);
const studioTabsDefault = useMemo((): "hub" | "lora" | "external" => {
if (value && externalModels.some((model) => model.id === value)) {
return "external";
}
if (value && loraModels.some((model) => model.id === value)) {
return "lora";
}
return "hub";
}, [externalModels, loraModels, value]);
return (
<PopoverContent
@ -150,12 +251,32 @@ function ModelSelectorContent({
)}
>
{chatOnly ? (
<HubModelPicker models={models} value={value} onSelect={onSelect} onFoldersChange={onFoldersChange} />
hasExternal ? (
<Tabs defaultValue={chatOnlyTabsDefault} className="w-full">
<TabsList className="mb-2 w-full">
<TabsTrigger value="hub">Hub models</TabsTrigger>
<TabsTrigger value="external">External</TabsTrigger>
</TabsList>
<TabsContent value="hub" className="m-0">
<HubModelPicker models={models} value={value} onSelect={onSelect} onFoldersChange={onFoldersChange} />
</TabsContent>
<TabsContent value="external" className="m-0">
<ExternalModelPicker
externalModels={externalModels}
value={value}
onSelect={onSelect}
/>
</TabsContent>
</Tabs>
) : (
<HubModelPicker models={models} value={value} onSelect={onSelect} onFoldersChange={onFoldersChange} />
)
) : (
<Tabs defaultValue="hub" className="w-full">
<Tabs defaultValue={studioTabsDefault} className="w-full">
<TabsList className="mb-2 w-full">
<TabsTrigger value="hub">Hub models</TabsTrigger>
<TabsTrigger value="lora">Fine-tuned</TabsTrigger>
{hasExternal ? <TabsTrigger value="external">External</TabsTrigger> : null}
</TabsList>
<TabsContent value="hub" className="m-0">
@ -171,6 +292,16 @@ function ModelSelectorContent({
deleteDisabled={deleteDisabled}
/>
</TabsContent>
{hasExternal ? (
<TabsContent value="external" className="m-0">
<ExternalModelPicker
externalModels={externalModels}
value={value}
onSelect={onSelect}
/>
</TabsContent>
) : null}
</Tabs>
)}
@ -207,6 +338,7 @@ function ModelSelectorContent({
export function ModelSelector({
models,
loraModels = [],
externalModels = [],
value,
defaultValue,
activeGgufVariant,
@ -224,6 +356,7 @@ export function ModelSelector({
onOpenChange,
triggerDataTour,
contentDataTour,
showCloudIndicator = false,
}: ModelSelectorProps) {
const [uncontrolledOpen, setUncontrolledOpen] = useState(false);
const open = controlledOpen ?? uncontrolledOpen;
@ -266,8 +399,21 @@ export function ModelSelector({
description: tag,
});
}
for (const externalModel of externalModels) {
all.set(externalModel.id, {
...externalModel,
description: externalModel.providerName,
icon: (
<ExternalProviderLogo
providerType={externalModel.providerType}
className="size-4"
title={externalModel.providerName}
/>
),
});
}
return all;
}, [loraModels, models]);
}, [externalModels, loraModels, models]);
const currentModel = useMemo(() => {
if (!selected) return undefined;
@ -303,6 +449,7 @@ export function ModelSelector({
<ModelSelectorTrigger
currentModel={currentModel}
isLoaded={isLoaded}
showCloudIndicator={showCloudIndicator}
variant={variant}
size={size}
className={className}
@ -311,6 +458,7 @@ export function ModelSelector({
<ModelSelectorContent
models={models}
loraModels={loraModels}
externalModels={externalModels}
value={selected}
onSelect={handleSelect}
onEject={onEject ? handleEject : undefined}
@ -327,3 +475,105 @@ export function ModelSelector({
ModelSelector.Trigger = ModelSelectorTrigger;
ModelSelector.Content = ModelSelectorContent;
function normalizeForSearch(value: string): string {
return value.toLowerCase().replace(/[\s_.-]/g, "");
}
function ExternalModelPicker({
externalModels,
value,
onSelect,
}: {
externalModels: ExternalModelOption[];
value?: string;
onSelect: (id: string, meta: ModelSelectorChangeMeta) => void;
}) {
const [query, setQuery] = useState("");
const grouped = useMemo(() => {
const needle = normalizeForSearch(query.trim());
const byProvider = new Map<
string,
{ providerName: string; models: ExternalModelOption[] }
>();
for (const model of externalModels) {
const searchText = normalizeForSearch(
`${model.name} ${model.providerName} ${model.id}`,
);
if (needle && !searchText.includes(needle)) continue;
const prev = byProvider.get(model.providerId);
if (prev) {
prev.models.push(model);
} else {
byProvider.set(model.providerId, {
providerName: model.providerName,
models: [model],
});
}
}
return [...byProvider.entries()]
.map(([providerId, group]) => ({
providerId,
providerName: group.providerName,
models: group.models.sort((a, b) => a.name.localeCompare(b.name)),
}))
.sort((a, b) => a.providerName.localeCompare(b.providerName));
}, [externalModels, query]);
return (
<div className="space-y-2">
<div className="relative">
<HugeiconsIcon
icon={Search01Icon}
className="pointer-events-none absolute left-2.5 top-2.5 size-4 text-muted-foreground"
/>
<Input
value={query}
onChange={(event) => setQuery(event.target.value)}
placeholder="Search external models"
className="h-9 pl-8"
/>
</div>
<div className="max-h-64 overflow-y-auto">
<div className="space-y-2 p-1">
{grouped.length === 0 ? (
<div className="px-2.5 py-2 text-xs text-muted-foreground">
No external models configured.
</div>
) : (
grouped.map((group) => (
<div key={group.providerId}>
<div className="flex items-center gap-2 px-2.5 py-1.5 text-[10px] font-semibold uppercase tracking-wider text-muted-foreground">
<ExternalProviderLogo
providerType={group.models[0]?.providerType}
className="size-3.5"
title={group.providerName}
/>
<span className="min-w-0 truncate">{group.providerName}</span>
</div>
{group.models.map((model) => (
<button
key={model.id}
type="button"
onClick={() =>
onSelect(model.id, {
source: "external",
isLora: false,
})
}
className={cn(
"flex w-full items-center rounded-md px-2.5 py-1.5 text-left text-sm transition-colors hover:bg-accent",
value === model.id && "bg-accent/60",
)}
>
<span className="min-w-0 truncate">{model.name}</span>
</button>
))}
</div>
))
)}
</div>
</div>
</div>
);
}

9
studio/frontend/src/components/assistant-ui/model-selector/types.ts
View file

@ -18,8 +18,15 @@ export interface LoraModelOption extends ModelOption {
exportType?: "lora" | "merged" | "gguf";
}
export interface ExternalModelOption extends ModelOption {
providerId: string;
providerName: string;
/** Registry key (e.g. openai, gemini) for provider branding. */
providerType: string;
}
export interface ModelSelectorChangeMeta {
source: "hub" | "lora" | "exported" | "local";
source: "hub" | "lora" | "exported" | "local" | "external";
isLora: boolean;
ggufVariant?: string;
isDownloaded?: boolean;

9
studio/frontend/src/components/assistant-ui/reasoning.tsx
View file

@ -19,10 +19,8 @@ import {
useAuiState,
} from "@assistant-ui/react";
import { copyToClipboard } from "@/lib/copy-to-clipboard";
import { Idea01Icon } from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
import { type VariantProps, cva } from "class-variance-authority";
import { ChevronDownIcon, CopyIcon, CheckIcon } from "lucide-react";
import { CheckIcon, ChevronDownIcon, CopyIcon, LightbulbIcon } from "lucide-react";
import {
type CSSProperties,
type ComponentProps,
@ -128,10 +126,7 @@ function ReasoningTrigger({
)}
{...props}
>
<HugeiconsIcon
icon={Idea01Icon}
className="aui-reasoning-trigger-icon size-4 shrink-0"
/>
<LightbulbIcon className="aui-reasoning-trigger-icon size-4 shrink-0" />
<span
data-slot="reasoning-trigger-label"
className="aui-reasoning-trigger-label-wrapper relative inline-block leading-none"

204
studio/frontend/src/components/assistant-ui/thread.tsx
View file

@ -13,6 +13,7 @@ import { Reasoning, ReasoningGroup } from "@/components/assistant-ui/reasoning";
import { Sources, SourcesGroup } from "@/components/assistant-ui/sources";
import { ToolFallback } from "@/components/assistant-ui/tool-fallback";
import { ToolGroup } from "@/components/assistant-ui/tool-group";
import { CodeExecutionToolUI } from "@/components/assistant-ui/tool-ui-code-execution";
import { PythonToolUI } from "@/components/assistant-ui/tool-ui-python";
import { TerminalToolUI } from "@/components/assistant-ui/tool-ui-terminal";
import { WebSearchToolUI } from "@/components/assistant-ui/tool-ui-web-search";
@ -31,6 +32,9 @@ import {
DropdownMenuTrigger,
} from "@/components/ui/dropdown-menu";
import { sentAudioNames } from "@/features/chat/api/chat-adapter";
import { parseExternalModelId } from "@/features/chat/external-providers";
import { getExternalReasoningCapabilities } from "@/features/chat/provider-capabilities";
import { useExternalProvidersStore } from "@/features/chat/stores/external-providers-store";
import { useChatRuntimeStore } from "@/features/chat/stores/chat-runtime-store";
import { applyQwenThinkingParams } from "@/features/chat/utils/qwen-params";
import { isTauri } from "@/lib/api-base";
@ -210,13 +214,28 @@ const ThreadScrollToBottom: FC = () => {
};
const ThreadWelcome: FC<{ hideComposer?: boolean }> = ({ hideComposer }) => {
const [currentEmoji, setCurrentEmoji] = useState("large sloth drink.png");
useEffect(() => {
const hour = new Date().getHours();
if (hour >= 6 && hour < 12) setCurrentEmoji("large sloth drink.png");
else if (hour >= 12 && hour < 17) setCurrentEmoji("sloth magnify final.png");
else if (hour >= 17 && hour < 21) setCurrentEmoji("sloth shy large.png");
else setCurrentEmoji("unsloth-gem.png");
}, []);
const currentEmojiSrc =
currentEmoji === "unsloth-gem.png"
? `/${currentEmoji}`
: `/Sloth emojis/${currentEmoji}`;
return (
<div className="aui-thread-welcome-root mx-auto my-auto flex w-full max-w-(--thread-max-width) grow flex-col">
<div className="aui-thread-welcome-center flex w-full grow flex-col items-center justify-center pb-[48px]">
<div className="aui-thread-welcome-message flex w-full flex-col justify-center gap-6 px-4">
<div className="flex flex-col items-center gap-2 text-center">
<img
src="/Sloth emojis/sloth pc square.png"
src={currentEmojiSrc}
alt="Sloth mascot"
className="size-20"
/>
@ -459,15 +478,76 @@ const ReasoningToggle: FC = () => {
const modelLoaded = useChatRuntimeStore(
(s) => !!s.params.checkpoint && !s.modelLoading,
);
const checkpoint = useChatRuntimeStore((s) => s.params.checkpoint);
const supportsReasoning = useChatRuntimeStore((s) => s.supportsReasoning);
const reasoningAlwaysOn = useChatRuntimeStore((s) => s.reasoningAlwaysOn);
const reasoningEnabled = useChatRuntimeStore((s) => s.reasoningEnabled);
const setReasoningEnabled = useChatRuntimeStore((s) => s.setReasoningEnabled);
const reasoningStyle = useChatRuntimeStore((s) => s.reasoningStyle);
const reasoningEffort = useChatRuntimeStore((s) => s.reasoningEffort);
const supportsReasoningOff = useChatRuntimeStore((s) => s.supportsReasoningOff);
const reasoningEffortLevels = useChatRuntimeStore((s) => s.reasoningEffortLevels);
const setReasoningEffort = useChatRuntimeStore((s) => s.setReasoningEffort);
const disabled = !(modelLoaded && supportsReasoning);
const lastOpenRouterChosenModel = useChatRuntimeStore(
(s) => s.lastOpenRouterChosenModel,
);
const externalProviders = useExternalProvidersStore((s) => s.providers);
const externalSelection = parseExternalModelId(checkpoint);
const selectedExternalProvider =
externalSelection != null
? externalProviders.find((p) => p.id === externalSelection.providerId)
: undefined;
const isKimiExternal = selectedExternalProvider?.providerType === "kimi";
const toolsEnabled = useChatRuntimeStore((s) => s.toolsEnabled);
const setToolsEnabled = useChatRuntimeStore((s) => s.setToolsEnabled);
const effectiveExternalModelId =
selectedExternalProvider?.providerType === "openrouter" &&
externalSelection?.modelId === "openrouter/free" &&
lastOpenRouterChosenModel
? lastOpenRouterChosenModel
: externalSelection?.modelId;
const externalReasoningCaps =
externalSelection != null
? getExternalReasoningCapabilities(
selectedExternalProvider?.providerType,
effectiveExternalModelId,
{
isReasoningProvider:
selectedExternalProvider?.isReasoningModel === true,
},
)
: null;
const effectiveReasoningStyle =
externalReasoningCaps?.reasoningStyle ?? reasoningStyle;
const effectiveReasoningAlwaysOn =
externalReasoningCaps?.reasoningAlwaysOn ?? reasoningAlwaysOn;
const effectiveSupportsReasoningOff =
externalReasoningCaps?.supportsReasoningOff ?? supportsReasoningOff;
const effectiveReasoningEffortLevels =
externalReasoningCaps?.reasoningEffortLevels ?? reasoningEffortLevels;
const effectiveSupportsReasoning =
externalReasoningCaps?.supportsReasoning ?? supportsReasoning;
const reasoningLockedOn =
effectiveSupportsReasoning &&
(effectiveReasoningAlwaysOn || !effectiveSupportsReasoningOff);
const effectiveReasoningEnabled = reasoningLockedOn ? true : reasoningEnabled;
const effectiveReasoningVisualEnabled =
effectiveReasoningEnabled && reasoningEffort !== "none";
const disabled = !(modelLoaded && effectiveSupportsReasoning);
const formatEffortLabel = (level: typeof reasoningEffort): string => {
if (level !== "xhigh") return level.charAt(0).toUpperCase() + level.slice(1);
const normalized = externalSelection?.modelId?.trim().toLowerCase() ?? "";
if (
normalized.startsWith("claude-opus-4-6") ||
normalized.startsWith("claude-sonnet-4-6")
) {
return "Max";
}
return "Extra High";
};
const effortLabel = formatEffortLabel(reasoningEffort);
if (reasoningStyle === "reasoning_effort") {
if (effectiveReasoningStyle === "reasoning_effort") {
return (
<DropdownMenu>
<DropdownMenuTrigger asChild={true}>
@ -478,26 +558,52 @@ const ReasoningToggle: FC = () => {
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
disabled
? "cursor-not-allowed opacity-40"
: "bg-primary/10 text-primary hover:bg-primary/20",
: effectiveReasoningVisualEnabled
? "bg-primary/10 text-primary hover:bg-primary/20"
: "text-muted-foreground hover:bg-muted-foreground/15",
)}
aria-label={`Reasoning effort: ${reasoningEffort}`}
>
<LightbulbIcon className="size-3.5" />
{effectiveReasoningVisualEnabled ? (
<LightbulbIcon className="size-3.5" />
) : (
<LightbulbOffIcon className="size-3.5" />
)}
<span>
Think:{" "}
{reasoningEffort.charAt(0).toUpperCase() +
reasoningEffort.slice(1)}
Think: {effectiveReasoningVisualEnabled ? effortLabel : "None"}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
{(["low", "medium", "high"] as const).map((level) => (
{effectiveSupportsReasoningOff && (
<DropdownMenuItem
onSelect={() => {
setReasoningEnabled(false);
applyQwenThinkingParams(false);
}}
>
None
{!effectiveReasoningVisualEnabled ? " \u2713" : ""}
</DropdownMenuItem>
)}
{effectiveReasoningEffortLevels
.filter((level) => level !== "none")
.map((level) => (
<DropdownMenuItem
key={level}
onSelect={() => setReasoningEffort(level)}
onSelect={() => {
setReasoningEffort(level);
setReasoningEnabled(true);
applyQwenThinkingParams(true);
// Kimi's $web_search builtin forbids thinking, so
// enabling thinking flips the Search pill off.
if (isKimiExternal && toolsEnabled) {
setToolsEnabled(false);
}
}}
>
{level.charAt(0).toUpperCase() + level.slice(1)}
{reasoningEffort === level ? " \u2713" : ""}
{formatEffortLabel(level)}
{effectiveReasoningVisualEnabled && reasoningEffort === level ? " \u2713" : ""}
</DropdownMenuItem>
))}
</DropdownMenuContent>
@ -508,17 +614,39 @@ const ReasoningToggle: FC = () => {
return (
<button
type="button"
disabled={disabled}
disabled={disabled || reasoningLockedOn}
aria-disabled={disabled || reasoningLockedOn}
title={
reasoningLockedOn
? "This model requires reasoning to stay on."
: undefined
}
onClick={() => {
if (reasoningLockedOn) return;
const next = !reasoningEnabled;
setReasoningEnabled(next);
applyQwenThinkingParams(next);
// Mutual exclusion with the Search pill on Kimi — see the
// dropdown branch above and shared-composer for the same rule.
if (isKimiExternal && next && toolsEnabled) {
setToolsEnabled(false);
}
}}
className="composer-pill-btn"
data-active={reasoningEnabled && !disabled ? "true" : "false"}
aria-label={reasoningEnabled ? "Disable thinking" : "Enable thinking"}
data-active={
reasoningLockedOn || (effectiveReasoningEnabled && !disabled)
? "true"
: "false"
}
aria-label={
reasoningLockedOn
? "Thinking is required for this model"
: effectiveReasoningEnabled
? "Disable thinking"
: "Enable thinking"
}
>
{reasoningEnabled && !disabled ? (
{reasoningLockedOn || (effectiveReasoningEnabled && !disabled) ? (
<LightbulbIcon className="size-3.5" />
) : (
<LightbulbOffIcon className="size-3.5" />
@ -570,16 +698,44 @@ const WebSearchToggle: FC = () => {
const modelLoaded = useChatRuntimeStore(
(s) => !!s.params.checkpoint && !s.modelLoading,
);
const checkpoint = useChatRuntimeStore((s) => s.params.checkpoint);
const supportsTools = useChatRuntimeStore((s) => s.supportsTools);
// External providers (OpenAI today) expose a server-side web_search tool
// even when the local tool runtime is unavailable — gate the Search pill
// on either source so it lights up on external models too. Mirror of
// shared-composer's searchDisabled.
const supportsBuiltinWebSearch = useChatRuntimeStore(
(s) => s.supportsBuiltinWebSearch,
);
const toolsEnabled = useChatRuntimeStore((s) => s.toolsEnabled);
const setToolsEnabled = useChatRuntimeStore((s) => s.setToolsEnabled);
const disabled = !(modelLoaded && supportsTools);
const setReasoningEnabled = useChatRuntimeStore((s) => s.setReasoningEnabled);
const externalProviders = useExternalProvidersStore((s) => s.providers);
const externalSelection = parseExternalModelId(checkpoint);
const selectedExternalProvider =
externalSelection != null
? externalProviders.find((p) => p.id === externalSelection.providerId)
: undefined;
const isKimiExternal = selectedExternalProvider?.providerType === "kimi";
const disabled =
!modelLoaded || !(supportsTools || supportsBuiltinWebSearch);
return (
<button
type="button"
disabled={disabled}
onClick={() => setToolsEnabled(!toolsEnabled)}
onClick={() => {
const next = !toolsEnabled;
setToolsEnabled(next);
// Kimi's $web_search builtin requires thinking=disabled (see
// https://platform.kimi.ai/docs/guide/use-web-search). Keep
// the two pills mutually exclusive so the visible state always
// matches what the backend ends up sending.
if (isKimiExternal) {
setReasoningEnabled(!next);
applyQwenThinkingParams(!next);
}
}}
className="composer-pill-btn"
data-active={toolsEnabled && !disabled ? "true" : "false"}
aria-label={toolsEnabled ? "Disable web search" : "Enable web search"}
@ -595,9 +751,18 @@ const CodeToolsToggle: FC = () => {
(s) => !!s.params.checkpoint && !s.modelLoading,
);
const supportsTools = useChatRuntimeStore((s) => s.supportsTools);
// External providers have no local tool runtime, but Anthropic's
// Claude 4.x dispatches code_execution_20250825 server-side. The
// chat-page resolver stashes that capability in the runtime store
// (next to supportsBuiltinWebSearch). Mirror of shared-composer's
// codeDisabled so this pill lights up in active threads too.
const supportsBuiltinCodeExecution = useChatRuntimeStore(
(s) => s.supportsBuiltinCodeExecution,
);
const codeToolsEnabled = useChatRuntimeStore((s) => s.codeToolsEnabled);
const setCodeToolsEnabled = useChatRuntimeStore((s) => s.setCodeToolsEnabled);
const disabled = !(modelLoaded && supportsTools);
const disabled =
!modelLoaded || !(supportsTools || supportsBuiltinCodeExecution);
return (
<button
@ -791,6 +956,7 @@ const AssistantMessage: FC = () => {
web_search: WebSearchToolUI,
python: PythonToolUI,
terminal: TerminalToolUI,
code_execution: CodeExecutionToolUI,
},
Fallback: ToolFallback,
},

126
studio/frontend/src/components/assistant-ui/tool-ui-code-execution.tsx Normal file
View file

@ -0,0 +1,126 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"use client";
import { type ToolCallMessagePartComponent, useAuiState } from "@assistant-ui/react";
import { FileTextIcon, LoaderIcon, TerminalIcon } from "lucide-react";
import { memo, useEffect, useState } from "react";
import {
ToolFallbackContent,
ToolFallbackRoot,
ToolFallbackTrigger,
} from "./tool-fallback";
/**
* Renders the synthetic `_toolEvent` chunks emitted by
* `_stream_anthropic` when Anthropic's `code_execution_20250825` tool
* fires. The backend collapses Anthropic's two sub-tools
* (`bash_code_execution`, `text_editor_code_execution`) into a single
* `tool_name: "code_execution"`, with `arguments.kind` ("bash" or
* "text_editor") and a per-kind argument shape:
*
* kind=bash: { command: "<shell command>" }
* kind=text_editor: { command: "view"|"create"|"str_replace", path, ... }
*
* The `result` payload is preformatted text:
* - bash: stdout, then "--- stderr ---" block + return_code if non-zero
* - text_editor view: file contents verbatim
* - text_editor create: "Created <path>" / "Updated <path>"
* - text_editor str_replace: unified-diff `lines` joined with "\n"
* - error: "Error: <error_code>"
*/
interface CodeExecutionArgs {
kind?: "bash" | "text_editor";
command?: string;
path?: string;
}
const CodeExecutionToolUIImpl: ToolCallMessagePartComponent = ({
args,
result,
status,
}) => {
const parsedArgs = (args as CodeExecutionArgs) ?? {};
const kind = parsedArgs.kind ?? "bash";
const command = parsedArgs.command ?? "";
const path = parsedArgs.path ?? "";
const isRunning = status?.type === "running";
let runningLabel: string;
let completedLabel: string;
let Icon = TerminalIcon;
if (kind === "text_editor") {
Icon = FileTextIcon;
if (command === "view") {
runningLabel = path ? `Viewing ${path}…` : "Viewing file…";
completedLabel = path ? `Viewed ${path}` : "Viewed file";
} else if (command === "create") {
runningLabel = path ? `Writing ${path}…` : "Writing file…";
completedLabel = path ? `Wrote ${path}` : "Wrote file";
} else if (command === "str_replace") {
runningLabel = path ? `Editing ${path}…` : "Editing file…";
completedLabel = path ? `Edited ${path}` : "Edited file";
} else {
runningLabel = "Running file operation…";
completedLabel = "File operation";
}
} else {
runningLabel = "Running command…";
completedLabel = command ? `Ran \`${command}\`` : "Ran command";
}
// Collapse the card once the model has resumed streaming prose after
// the tool call. Mirrors WebSearchToolUI's behavior so the tool-card
// doesn't crowd the final answer once the run is done.
const hasText = useAuiState(({ message }) =>
message.content.some(
(p) =>
p.type === "text" &&
"text" in p &&
(p as { text: string }).text.length > 0,
),
);
const [open, setOpen] = useState(isRunning);
useEffect(() => {
if (isRunning) {
setOpen(true);
} else if (hasText) {
setOpen(false);
}
}, [isRunning, hasText]);
const resultText =
typeof result === "string"
? result
: result != null
? JSON.stringify(result, null, 2)
: "";
return (
<ToolFallbackRoot open={open} onOpenChange={setOpen}>
<ToolFallbackTrigger
toolName={isRunning ? runningLabel : completedLabel}
status={status}
icon={Icon}
/>
<ToolFallbackContent>
{isRunning ? (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<LoaderIcon className="size-3.5 animate-spin" />
<span>{runningLabel}</span>
</div>
) : resultText ? (
<pre className="max-h-64 overflow-auto whitespace-pre-wrap break-words rounded bg-muted/50 p-2 text-xs">
{resultText}
</pre>
) : null}
</ToolFallbackContent>
</ToolFallbackRoot>
);
};
export const CodeExecutionToolUI = memo(
CodeExecutionToolUIImpl,
) as unknown as ToolCallMessagePartComponent;
CodeExecutionToolUI.displayName = "CodeExecutionToolUI";

72
studio/frontend/src/features/chat/api-provider-logo.tsx Normal file
View file

@ -0,0 +1,72 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { cn } from "@/lib/utils";
import { DashboardSquare01Icon } from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
import { isCustomProviderType } from "./external-providers";
/**
* Registry logos live at `public/provider-logos/{provider_type}.{ext}` where `provider_type`
* matches `PROVIDER_REGISTRY` keys exactly (lowercase). Extension varies by asset (svg preferred).
*/
const PROVIDER_LOGO_EXT: Record<string, "svg" | "png" | "jpg"> = {
openai: "svg",
mistral: "svg",
gemini: "svg",
anthropic: "svg",
deepseek: "svg",
huggingface: "svg",
kimi: "jpg",
qwen: "png",
openrouter: "svg",
vllm: "svg",
ollama: "svg",
llama_cpp: "svg",
};
export function apiProviderLogoSrc(
providerType: string | undefined | null,
): string | undefined {
if (!providerType) return undefined;
const ext = PROVIDER_LOGO_EXT[providerType];
if (!ext) return undefined;
return `${import.meta.env.BASE_URL}provider-logos/${providerType}.${ext}`;
}
interface ApiProviderLogoProps {
providerType: string | undefined | null;
className?: string;
title?: string;
}
/**
* Renders the logo for a registry provider type when `provider_type.{ext}` exists under
* `public/provider-logos/`.
* OpenAI's asset is black-on-transparent; it is inverted in dark mode for contrast.
*/
export function ApiProviderLogo({ providerType, className, title }: ApiProviderLogoProps) {
const src = apiProviderLogoSrc(providerType);
if (!src && isCustomProviderType(providerType)) {
return (
<span title={title} aria-hidden className="inline-flex shrink-0">
<HugeiconsIcon icon={DashboardSquare01Icon} className={cn("shrink-0", className)} />
</span>
);
}
if (!src) return null;
return (
<img
src={src}
alt=""
title={title}
aria-hidden
className={cn(
"shrink-0 object-contain",
providerType === "openai" && "dark:invert",
className,
)}
/>
);
}

797
studio/frontend/src/features/chat/api/chat-adapter.ts
View file

@ -15,7 +15,34 @@ import {
streamChatCompletions,
validateModel,
} from "./chat-api";
import { pickFriendlyContainerName } from "../lib/friendly-names";
import { createOpenAIContainer } from "./openai-containers";
import {
encryptProviderApiKey,
isProviderKeyRotationError,
} from "./providers-api";
import { db } from "../db";
import type {
OpenAIChatCompletionsRequest,
OpenAIMessageContent,
} from "../types/api";
import {
getExternalProviderApiKey,
isCustomProviderType,
loadExternalProviders,
parseExternalModelId,
supportsProviderPromptCaching,
toExternalBackendProviderType,
} from "../external-providers";
import {
EXTERNAL_MAX_OUTPUT_TOKENS,
clampReasoningEffortToLevels,
getExternalMinOutputTokens,
getExternalReasoningCapabilities,
getProviderCapabilities,
providerSupportsBuiltinCodeExecution,
providerSupportsBuiltinWebSearch,
} from "../provider-capabilities";
import { useChatRuntimeStore } from "../stores/chat-runtime-store";
import { isMultimodalResponse } from "../types/api";
import type { ChatModelSummary } from "../types/runtime";
@ -118,6 +145,70 @@ function estimateTokenCount(text: string): number | undefined {
return Math.max(1, Math.round(trimmed.length / 4));
}
/**
* Normalize a streamed `delta.content` to a plain text string.
*
* OpenAI Chat Completions originally typed `delta.content` as a string, but
* a number of providers now emit it as an array of structured content parts.
* Concatenating that with `cumulativeText += delta` would stringify each
* part as `[object Object]` — this function is the guard against that.
*
* Handled part shapes:
* { type: "text" | "output_text", text | content: "..." } → text body
* { type: "thinking" | "reasoning", thinking | text: "..." } → wrapped as
* inline `<think>...</think>` so the downstream parser
* (`parseAssistantContent`) lifts it into a reasoning part the same way
* it does for providers that emit thinking inline. Without this wrap,
* Mistral magistral and similar reasoning-part providers would lose
* their thinking panel.
*
* Unknown part types are skipped — better to drop a stray field than to
* stringify an object and pollute the rendered chat with `[object Object]`.
*/
function extractDeltaText(delta: unknown): string {
const extractReasoningText = (payload: unknown): string => {
if (typeof payload === "string") return payload;
if (Array.isArray(payload)) {
return payload.map((item) => extractReasoningText(item)).join("");
}
if (!payload || typeof payload !== "object") return "";
const obj = payload as Record<string, unknown>;
for (const key of ["thinking", "text", "content", "reasoning", "summary"]) {
if (key in obj) {
const text = extractReasoningText(obj[key]);
if (text) return text;
}
}
return "";
};
if (typeof delta === "string") return delta;
if (!Array.isArray(delta)) return "";
let out = "";
for (const part of delta) {
if (typeof part === "string") {
out += part;
continue;
}
if (!part || typeof part !== "object") continue;
const obj = part as {
type?: string;
text?: string;
content?: string;
thinking?: string;
};
if (obj.type === "text" || obj.type === "output_text") {
if (typeof obj.text === "string") out += obj.text;
else if (typeof obj.content === "string") out += obj.content;
} else if (obj.type === "thinking" || obj.type === "reasoning") {
const thinking = extractReasoningText(obj);
if (thinking) out += `<think>${thinking}</think>`;
}
}
return out;
}
function buildTiming(
streamStartTime: number,
totalChunks: number,
@ -162,9 +253,51 @@ function collectTextParts(message: RunMessage): string[] {
return textParts;
}
function collectImageParts(
message: RunMessage,
): Array<{ type: "image_url"; image_url: { url: string } }> {
const parts: Array<{ type: "image_url"; image_url: { url: string } }> = [];
for (const part of message.content ?? []) {
if (part.type === "image" && "image" in part) {
const src = (part as { image: string }).image;
if (src) {
parts.push({
type: "image_url",
image_url: {
url: src.startsWith("data:") ? src : `data:image/png;base64,${src}`,
},
});
}
}
}
if ("attachments" in message && (message.attachments?.length ?? 0) > 0) {
for (const attachment of message.attachments ?? []) {
for (const part of attachment.content ?? []) {
if (part.type === "image" && "image" in part) {
const src = (part as { image: string }).image;
if (src) {
parts.push({
type: "image_url",
image_url: {
url: src.startsWith("data:")
? src
: `data:image/png;base64,${src}`,
},
});
}
}
}
}
}
return parts;
}
function toOpenAIMessage(message: RunMessage): {
role: "system" | "user" | "assistant";
content: string;
content: OpenAIMessageContent;
} | null {
if (
message.role !== "system" &&
@ -174,17 +307,25 @@ function toOpenAIMessage(message: RunMessage): {
return null;
}
let content = collectTextParts(message).join("\n");
let textContent = collectTextParts(message).join("\n");
// Strip inline audio base64 from prior assistant messages to avoid
// inflating token counts (e.g. audio-player responses with embedded WAV).
if (message.role === "assistant") {
content = content.replace(
textContent = textContent.replace(
/data:audio\/[a-z0-9.+-]+;base64,[A-Za-z0-9+/=]+/g,
"[audio]",
);
}
return { role: message.role, content };
const imageParts = collectImageParts(message);
if (imageParts.length > 0) {
return {
role: message.role,
content: [{ type: "text", text: textContent }, ...imageParts],
};
}
return { role: message.role, content: textContent };
}
function extractImageBase64(input: string): string | undefined {
@ -594,6 +735,33 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
toolsEnabled,
codeToolsEnabled,
} = runtime;
const externalSelection = parseExternalModelId(params.checkpoint);
const isExternalRequest = externalSelection !== null;
const externalProvider = isExternalRequest
? loadExternalProviders().find(
(provider) => provider.id === externalSelection.providerId,
)
: null;
const externalApiKey = externalProvider
? getExternalProviderApiKey(externalProvider.id).trim()
: "";
if (isExternalRequest && !externalProvider) {
toast.error("External provider not found.", {
description: "Open Connections and re-add this provider.",
});
throw new Error("External provider not found.");
}
// Local providers (llama.cpp / vLLM / Ollama) allow an empty key — only block hosted providers.
const externalProviderIsCustom = externalProvider
? isCustomProviderType(externalProvider.providerType)
: false;
if (isExternalRequest && !externalApiKey && !externalProviderIsCustom) {
toast.error("Missing API key for selected external provider.", {
description: "Open Connections and set the API key again.",
});
throw new Error("Missing external provider API key.");
}
const outboundMessages = messages
.map(toOpenAIMessage)
@ -711,6 +879,17 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
let cumulativeText = "";
let reasoningStartAt: number | null = null;
let reasoningDuration = 0;
// Tracks whether we are currently inside a `<think>` block opened by
// a `delta.reasoning_content` chunk. Kimi (kimi-k2.6, kimi-k2-thinking)
// and DeepSeek's reasoner stream their thinking as a separate
// `reasoning_content` field on the chat-completion delta — not as
// `content`, not as a structured part. We wrap those chunks with
// inline `<think>...</think>` so the existing parseAssistantContent
// lifts them into the reasoning panel the same way it does for
// local Harmony models. State has to live outside the SSE loop
// because the close tag fires when the next chunk carries content
// (or when the stream ends).
let reasoningContentOpen = false;
// Tool call content parts — accumulated and yielded cumulatively.
// result is set directly on the tool-call part when tool_end arrives.
const toolCallParts: ToolCallMessagePart[] = [];
@ -760,8 +939,260 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
supportsPreserveThinking,
preserveThinking,
} = runtime;
const stream = streamChatCompletions(
{
const externalBackendProviderType = toExternalBackendProviderType(
externalProvider?.providerType,
);
const externalCapabilities = getProviderCapabilities(
externalProvider?.providerType,
);
const externalReasoningCaps: ReturnType<
typeof getExternalReasoningCapabilities
> =
externalSelection && externalProvider
? getExternalReasoningCapabilities(
externalProvider.providerType,
externalSelection.modelId,
{
isReasoningProvider:
externalProvider.isReasoningModel === true,
},
)
: {
supportsReasoning,
reasoningStyle,
reasoningAlwaysOn: false,
supportsReasoningOff: false,
reasoningEffortLevels: ["low", "medium", "high"] as const,
};
type RequestReasoningEffort = Extract<
NonNullable<OpenAIChatCompletionsRequest["reasoning_effort"]>,
"none" | "minimal" | "low" | "medium" | "high" | "max" | "xhigh"
>;
const fallbackExternalEffort =
(externalReasoningCaps.reasoningEffortLevels[0] ??
"low") as RequestReasoningEffort;
const selectedExternalEffort: RequestReasoningEffort =
clampReasoningEffortToLevels(
reasoningEffort,
externalReasoningCaps.reasoningEffortLevels,
) as RequestReasoningEffort;
const localReasoningEffort =
reasoningEffort === "low" || reasoningEffort === "medium" || reasoningEffort === "high"
? reasoningEffort
: "low";
const externalReasoningEnabled =
!externalReasoningCaps.supportsReasoningOff ? true : reasoningEnabled;
const buildRequestPayload = async (
forceRefreshPublicKey = false,
): Promise<OpenAIChatCompletionsRequest> => {
if (externalSelection && externalProvider) {
// OpenAI shell-tool container reuse: pull the per-thread
// container_id (if any) so subsequent turns in the same
// thread reference the existing container instead of
// auto-creating a fresh one. Empty string / undefined →
// backend falls back to container_auto. Anthropic doesn't
// use this (server-side per-turn container).
let openaiCodeExecContainerId: string | null = null;
const codeExecEnabledForThisTurn =
codeToolsEnabled &&
providerSupportsBuiltinCodeExecution(
externalProvider.providerType,
externalSelection.modelId,
externalProvider.baseUrl,
);
if (codeExecEnabledForThisTurn && resolvedThreadId) {
try {
const thread = await db.threads.get(resolvedThreadId);
openaiCodeExecContainerId =
thread?.openaiCodeExecContainerId ?? null;
} catch {
openaiCodeExecContainerId = null;
}
// Cross-thread inheritance: when the active thread has
// no container yet, default to the one most recently
// used on *any* other thread (provider-scoped).
// Matches what the Code Execution settings section
// shows in the picker, and keeps the user from getting
// a fresh container on every new thread. The picker
// can still be set to "Auto-create per thread"
// explicitly to opt into a fresh container — but
// that's done via the dropdown, not silently.
if (
!openaiCodeExecContainerId &&
externalProvider.providerType === "openai"
) {
try {
const others = await db.threads
.orderBy("createdAt")
.reverse()
.toArray();
for (const t of others) {
if (t.id === resolvedThreadId) continue;
if (t.openaiCodeExecContainerId) {
openaiCodeExecContainerId = t.openaiCodeExecContainerId;
void db.threads
.update(resolvedThreadId, {
openaiCodeExecContainerId,
})
.catch(() => {});
break;
}
}
} catch {
/* fall through to lazy-create below */
}
}
// Lazy pre-create when there's no inherited container.
// We always POST /v1/containers ourselves (rather than
// letting the backend send container_auto) so every
// container shows up in the picker with a friendly
// English-word name and the user's configured TTL.
// Falls back to container_auto only if the POST fails
// — keeps the chat moving in that case.
if (
!openaiCodeExecContainerId &&
externalProvider.providerType === "openai"
) {
const ttl = externalProvider.openaiContainerTtlMinutes;
const ttlToUse =
typeof ttl === "number" && ttl >= 1 ? ttl : 20;
try {
const created = await createOpenAIContainer(
{
apiKey: externalApiKey,
baseUrl: externalProvider.baseUrl || null,
},
{
// Friendly English-word name so the container
// is human-readable in the picker list (e.g.
// "kestrel-3f9c") instead of a thread-id slug
// or OpenAI's default blank name.
name: pickFriendlyContainerName(),
ttlMinutes: ttlToUse,
},
);
openaiCodeExecContainerId = created.id;
void db.threads
.update(resolvedThreadId, {
openaiCodeExecContainerId: created.id,
})
.catch(() => {});
} catch {
// Fall back to backend's container_auto path on
// failure — keeps the chat moving; the next turn
// can retry. The auto-created container will be
// unnamed, but the chat doesn't break.
openaiCodeExecContainerId = null;
}
}
}
return {
model: externalSelection.modelId,
messages: outboundMessages,
stream: true,
// Reasoning-class models (OpenAI gpt-5.x / o3) reject temperature
// and top_p; only forward when the active provider supports them.
...(externalCapabilities?.temperature !== false
? { temperature: params.temperature }
: {}),
...(externalCapabilities?.topP !== false
? { top_p: params.topP }
: {}),
// Clamp to the cross-provider output cap so a maxTokens value
// carried over from a local-model session does not blow past
// provider limits (e.g. Claude Opus 400s on >128k). Also
// floor to the provider's documented minimum — Kimi's
// thinking models need >=16k or the response truncates
// before the answer fits alongside reasoning_content.
max_tokens: Math.min(
Math.max(
params.maxTokens,
getExternalMinOutputTokens(externalProvider?.providerType),
),
EXTERNAL_MAX_OUTPUT_TOKENS,
),
// Only forward sampling knobs the provider actually accepts; the
// backend's external-provider proxy is param-permissive and would
// surface a 400 from providers that reject unknown fields (e.g.
// OpenAI rejects top_k, Anthropic/DeepSeek reject presence_penalty).
...(externalCapabilities?.topK ? { top_k: params.topK } : {}),
...(externalCapabilities?.presencePenalty
? { presence_penalty: params.presencePenalty }
: {}),
// Built-in tools: Search pill maps to provider-side
// web_search (currently OpenAI / Anthropic / OpenRouter /
// Kimi); Code pill maps to Anthropic's server-side
// code_execution_20250825 tool (Anthropic is the only
// external provider that ships one today). Backend
// translates enabled_tools into each provider's tool
// schema — for Anthropic that's the entries appended to
// body["tools"] inside _stream_anthropic.
...((toolsEnabled &&
providerSupportsBuiltinWebSearch(externalProvider.providerType)) ||
(codeToolsEnabled &&
providerSupportsBuiltinCodeExecution(
externalProvider.providerType,
externalSelection.modelId,
externalProvider.baseUrl,
))
? {
enable_tools: true,
enabled_tools: [
...(toolsEnabled &&
providerSupportsBuiltinWebSearch(
externalProvider.providerType,
)
? ["web_search"]
: []),
...(codeToolsEnabled &&
providerSupportsBuiltinCodeExecution(
externalProvider.providerType,
externalSelection.modelId,
externalProvider.baseUrl,
)
? ["code_execution"]
: []),
],
}
: {}),
provider_id: externalProvider.id,
provider_type: externalBackendProviderType,
external_model: externalSelection.modelId,
...(externalApiKey
? {
encrypted_api_key: await encryptProviderApiKey(
externalApiKey,
forceRefreshPublicKey,
),
}
: {}),
provider_base_url: externalProvider.baseUrl || null,
...(openaiCodeExecContainerId
? {
openai_code_exec_container_id: openaiCodeExecContainerId,
}
: {}),
...(supportsProviderPromptCaching(externalProvider.providerType)
? {
enable_prompt_caching:
externalProvider.enablePromptCaching ?? true,
}
: {}),
...(externalReasoningCaps.supportsReasoning
? externalReasoningCaps.reasoningStyle === "reasoning_effort"
? externalReasoningEnabled
? { reasoning_effort: selectedExternalEffort }
: externalReasoningCaps.supportsReasoningOff
? { reasoning_effort: "none" }
: {
reasoning_effort: fallbackExternalEffort,
}
: { enable_thinking: reasoningEnabled }
: {}),
};
}
return {
model: params.checkpoint,
messages: outboundMessages,
stream: true,
@ -779,7 +1210,9 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
...(useAdapter === undefined ? {} : { use_adapter: useAdapter }),
...(supportsReasoning
? reasoningStyle === "reasoning_effort"
? { reasoning_effort: reasoningEffort }
? reasoningEnabled
? { reasoning_effort: localReasoningEffort }
: {}
: { enable_thinking: reasoningEnabled }
: {}),
...(supportsPreserveThinking ? { preserve_thinking: preserveThinking } : {}),
@ -798,116 +1231,262 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
})(),
}
: {}),
},
abortSignal,
);
};
};
for await (const chunk of stream) {
// Handle tool status events
const toolStatusText = (chunk as unknown as { _toolStatus?: string })._toolStatus;
if (toolStatusText !== undefined) {
runtime.setToolStatus(toolStatusText || null);
continue;
}
let retriedWithRefreshedKey = false;
while (true) {
try {
const stream = streamChatCompletions(
await buildRequestPayload(retriedWithRefreshedKey),
abortSignal,
);
// Emit tool-call content parts for assistant-ui.
// On tool_start: add a new tool-call part (renders in "running" state).
// On tool_end: set result on the existing part (transitions to "complete").
const toolEvent = (chunk as unknown as { _toolEvent?: Record<string, unknown> })._toolEvent;
if (toolEvent !== undefined) {
if (toolEvent.type === "tool_start") {
const id = (toolEvent.tool_call_id as string) || `${toolEvent.tool_name}_${Date.now()}`;
const toolArgs = (toolEvent.arguments ?? {}) as ToolCallMessagePart["args"];
toolCallParts.push({
type: "tool-call" as const,
toolCallId: id,
toolName: toolEvent.tool_name as string,
argsText: JSON.stringify(toolArgs),
args: toolArgs,
});
} else if (toolEvent.type === "tool_end") {
const id = (toolEvent.tool_call_id as string) ||
toolCallParts[toolCallParts.length - 1]?.toolCallId || "";
const idx = toolCallParts.findIndex((p) => p.toolCallId === id);
if (idx !== -1) {
const rawResult = (toolEvent.result as string) ?? "";
const imgMarker = "\n__IMAGES__:";
const imgIdx = rawResult.lastIndexOf(imgMarker);
let parsedResult: string | { text: string; images: string[]; sessionId: string };
if (imgIdx !== -1) {
const text = rawResult.slice(0, imgIdx);
// Fall back to "_default" to match the backend sandbox directory
// used when no session_id is provided (see tools.py _get_workdir).
const sessionId = resolvedThreadId || "_default";
try {
const images = JSON.parse(rawResult.slice(imgIdx + imgMarker.length)) as string[];
parsedResult = { text, images, sessionId };
} catch {
parsedResult = rawResult;
for await (const chunk of stream) {
// Handle tool status events
const toolStatusText = (chunk as unknown as { _toolStatus?: string })._toolStatus;
if (toolStatusText !== undefined) {
runtime.setToolStatus(toolStatusText || null);
continue;
}
// Emit tool-call content parts for assistant-ui.
// On tool_start: add a new tool-call part (renders in "running" state).
// On tool_end: set result on the existing part (transitions to "complete").
const toolEvent = (chunk as unknown as { _toolEvent?: Record<string, unknown> })._toolEvent;
if (toolEvent !== undefined) {
// OpenAI shell-tool container persistence — see
// ThreadRecord.openaiCodeExecContainerId. The backend
// emits these synthetic events on the OpenAI Responses
// SSE stream after capturing the container_id from a
// response, or detecting an expired-container error.
if (toolEvent.type === "container_ready") {
const newContainerId = toolEvent.container_id as
| string
| undefined;
if (newContainerId && resolvedThreadId) {
void db.threads
.update(resolvedThreadId, {
openaiCodeExecContainerId: newContainerId,
})
.catch(() => {});
}
} else {
parsedResult = rawResult;
continue;
}
toolCallParts[idx] = { ...toolCallParts[idx], result: parsedResult };
if (toolEvent.type === "container_invalidated") {
if (resolvedThreadId) {
void db.threads
.update(resolvedThreadId, {
openaiCodeExecContainerId: null,
})
.catch(() => {});
}
continue;
}
if (toolEvent.type === "tool_start") {
const id = (toolEvent.tool_call_id as string) || `${toolEvent.tool_name}_${Date.now()}`;
const toolArgs = (toolEvent.arguments ?? {}) as ToolCallMessagePart["args"];
toolCallParts.push({
type: "tool-call" as const,
toolCallId: id,
toolName: toolEvent.tool_name as string,
argsText: JSON.stringify(toolArgs),
args: toolArgs,
});
} else if (toolEvent.type === "tool_end") {
const id = (toolEvent.tool_call_id as string) ||
toolCallParts[toolCallParts.length - 1]?.toolCallId || "";
const idx = toolCallParts.findIndex((p) => p.toolCallId === id);
if (idx !== -1) {
const rawResult = (toolEvent.result as string) ?? "";
const imgMarker = "\n__IMAGES__:";
const imgIdx = rawResult.lastIndexOf(imgMarker);
let parsedResult: string | { text: string; images: string[]; sessionId: string };
if (imgIdx !== -1) {
const text = rawResult.slice(0, imgIdx);
// Fall back to "_default" to match the backend sandbox directory
// used when no session_id is provided (see tools.py _get_workdir).
const sessionId = resolvedThreadId || "_default";
try {
const images = JSON.parse(rawResult.slice(imgIdx + imgMarker.length)) as string[];
parsedResult = { text, images, sessionId };
} catch {
parsedResult = rawResult;
}
} else {
parsedResult = rawResult;
}
toolCallParts[idx] = { ...toolCallParts[idx], result: parsedResult };
}
}
// Yield cumulative state so tool UI updates (tools first, text after)
const textParts = parseAssistantContent(cumulativeText);
yield {
content: [...toolCallParts, ...textParts],
metadata: {
timing: buildTiming(streamStartTime, totalChunks, firstTokenTime),
custom: { reasoningDuration },
},
};
continue;
}
// OpenAI-standard usage chunk: choices=[], usage populated
if (chunk.choices?.length === 0 && chunk.usage) {
serverMetadata = {
usage: chunk.usage,
timings: (chunk as Record<string, unknown>).timings as ServerTimings | undefined,
};
continue;
}
totalChunks += 1;
// OpenRouter's free router (openrouter/free) picks a different
// underlying free model per request and reports it in every
// chunk's top-level `model` field. Latch the first non-empty
// value that differs from the requested checkpoint so the
// header chip can render "openrouter/free:<chosen>".
if (
isExternalRequest &&
externalProvider?.providerType === "openrouter" &&
externalSelection?.modelId === "openrouter/free"
) {
const chunkModel = (chunk as { model?: unknown }).model;
if (
typeof chunkModel === "string" &&
chunkModel.length > 0 &&
chunkModel !== externalSelection.modelId
) {
const storeState = useChatRuntimeStore.getState();
if (storeState.lastOpenRouterChosenModel !== chunkModel) {
storeState.setLastOpenRouterChosenModel(chunkModel);
}
}
}
const rawDelta = chunk.choices?.[0]?.delta?.content;
// Providers like Mistral's magistral return delta.content as an
// array of structured parts; normalize to text (with thinking
// parts re-wrapped as inline <think> tags) so the rest of the
// accumulator stays string-based.
const delta = extractDeltaText(rawDelta);
// Kimi (kimi-k2.6, kimi-k2-thinking) and DeepSeek reasoner
// stream thinking via `delta.reasoning_content` as a plain
// string field — separate from `delta.content` which carries
// the answer. Wrap reasoning chunks inline as <think>...
// </think> so parseAssistantContent treats them like any
// other reasoning. The close tag fires when the next chunk
// brings content, or when the stream ends.
const rawReasoning = (
chunk.choices?.[0]?.delta as
| { reasoning_content?: unknown }
| undefined
)?.reasoning_content;
// OpenRouter uses a third reasoning shape: a structured
// `delta.reasoning_details` array of parts (each carrying
// `text`). The router emits this regardless of which
// underlying provider it picked, so we extract here and
// merge into the same <think>...</think> wrap path used
// for Kimi / DeepSeek reasoning_content. See
// https://openrouter.ai/docs/guides/best-practices/reasoning-tokens
const rawReasoningDetails = (
chunk.choices?.[0]?.delta as
| { reasoning_details?: unknown }
| undefined
)?.reasoning_details;
const reasoningFromDetails = Array.isArray(rawReasoningDetails)
? rawReasoningDetails
.map((part) => {
if (!part || typeof part !== "object") return "";
const text = (part as { text?: unknown }).text;
return typeof text === "string" ? text : "";
})
.join("")
: "";
const reasoning =
(typeof rawReasoning === "string" ? rawReasoning : "") +
reasoningFromDetails;
if (!delta && !reasoning) {
continue;
}
if (waitingFirstChunk) {
waitingFirstChunk = false;
firstTokenTime = Date.now() - streamStartTime;
settleFirstTokenOk();
runtime.setGeneratingStatus(null);
}
if (reasoning) {
if (!reasoningContentOpen) {
cumulativeText += `<think>${reasoning}`;
reasoningContentOpen = true;
} else {
cumulativeText += reasoning;
}
}
if (delta) {
if (reasoningContentOpen) {
cumulativeText += "</think>";
reasoningContentOpen = false;
}
cumulativeText += delta;
}
// Mistral's magistral occasionally emits a trailing
// template-literal artifact (e.g. "${response}") at the end of
// an otherwise complete answer. It is never part of a real
// reply, so strip a trailing `${...}` token from external
// provider streams. The regex anchors to end-of-string and is
// idempotent — fragments mid-stream (e.g. "${re") leave the
// string untouched and only collapse once the closing brace
// arrives. Local-model output is left alone.
if (isExternalRequest) {
cumulativeText = cumulativeText.replace(
/\s*\$\{[^}]*\}\s*$/,
"",
);
}
const parts = parseAssistantContent(cumulativeText);
if (parts.some((part) => part.type === "reasoning") && !reasoningStartAt) {
reasoningStartAt = Date.now();
}
if (hasClosedThinkTag(cumulativeText) && reasoningStartAt && !reasoningDuration) {
reasoningDuration = Math.round((Date.now() - reasoningStartAt) / 1000);
}
if (parts.length > 0 || toolCallParts.length > 0) {
yield {
content: [...toolCallParts, ...parts],
metadata: {
timing: buildTiming(
streamStartTime,
totalChunks,
firstTokenTime,
),
custom: { reasoningDuration },
},
};
}
}
// Yield cumulative state so tool UI updates (tools first, text after)
const textParts = parseAssistantContent(cumulativeText);
yield {
content: [...toolCallParts, ...textParts],
metadata: {
timing: buildTiming(streamStartTime, totalChunks, firstTokenTime),
custom: { reasoningDuration },
},
};
continue;
}
// OpenAI-standard usage chunk: choices=[], usage populated
if (chunk.choices?.length === 0 && chunk.usage) {
serverMetadata = {
usage: chunk.usage,
timings: (chunk as Record<string, unknown>).timings as ServerTimings | undefined,
};
continue;
}
totalChunks += 1;
const delta = chunk.choices?.[0]?.delta?.content;
if (!delta) {
continue;
}
if (waitingFirstChunk) {
waitingFirstChunk = false;
firstTokenTime = Date.now() - streamStartTime;
settleFirstTokenOk();
runtime.setGeneratingStatus(null);
}
cumulativeText += delta;
const parts = parseAssistantContent(cumulativeText);
if (parts.some((part) => part.type === "reasoning") && !reasoningStartAt) {
reasoningStartAt = Date.now();
}
if (hasClosedThinkTag(cumulativeText) && reasoningStartAt && !reasoningDuration) {
reasoningDuration = Math.round((Date.now() - reasoningStartAt) / 1000);
}
if (parts.length > 0 || toolCallParts.length > 0) {
yield {
content: [...toolCallParts, ...parts],
metadata: {
timing: buildTiming(
streamStartTime,
totalChunks,
firstTokenTime,
),
custom: { reasoningDuration },
},
};
break;
} catch (streamError) {
if (
isExternalRequest &&
!retriedWithRefreshedKey &&
isProviderKeyRotationError(streamError)
) {
retriedWithRefreshedKey = true;
continue;
}
throw streamError;
}
}
// If the stream ended while we were still inside a
// delta.reasoning_content block (Kimi / DeepSeek path), close
// the open <think> tag so the reasoning panel parses cleanly.
if (reasoningContentOpen) {
cumulativeText += "</think>";
reasoningContentOpen = false;
}
settleFirstTokenOk();
// Extract source parts from completed web_search tool calls

124
studio/frontend/src/features/chat/api/openai-containers.ts Normal file
View file

@ -0,0 +1,124 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
/**
* Wrappers for the three OpenAI shell-tool container management
* endpoints exposed by the backend (studio/backend/routes/inference.py).
* Each one proxies to OpenAI's /v1/containers REST surface using the
* user's encrypted API key. Backend rejects any base URL that isn't
* api.openai.com — the shell tool only exists on the managed cloud.
*/
import { authFetch } from "@/features/auth";
import { encryptProviderApiKey } from "./providers-api";
export interface OpenAIContainerSummary {
id: string;
name?: string | null;
createdAt?: number | null;
lastActiveAt?: number | null;
expiresAfterMinutes?: number | null;
status?: string | null;
}
interface RawSummary {
id: string;
name?: string | null;
created_at?: number | null;
last_active_at?: number | null;
expires_after_minutes?: number | null;
status?: string | null;
}
function fromRaw(raw: RawSummary): OpenAIContainerSummary {
return {
id: raw.id,
name: raw.name ?? null,
createdAt: raw.created_at ?? null,
lastActiveAt: raw.last_active_at ?? null,
expiresAfterMinutes: raw.expires_after_minutes ?? null,
status: raw.status ?? null,
};
}
async function parseError(response: Response): Promise<string> {
try {
const body = (await response.json()) as { detail?: string };
if (body && typeof body.detail === "string") return body.detail;
} catch {
/* fall through */
}
return `HTTP ${response.status}`;
}
interface AuthInputs {
apiKey: string;
baseUrl: string | null;
}
async function buildAuthBody(auth: AuthInputs) {
return {
encrypted_api_key: await encryptProviderApiKey(auth.apiKey),
provider_base_url: auth.baseUrl,
};
}
export async function listOpenAIContainers(
auth: AuthInputs,
): Promise<OpenAIContainerSummary[]> {
const response = await authFetch(
"/api/inference/external/openai/containers/list",
{
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(await buildAuthBody(auth)),
},
);
if (!response.ok) throw new Error(await parseError(response));
const body = (await response.json()) as { containers?: RawSummary[] };
return (body.containers ?? []).map(fromRaw);
}
export async function createOpenAIContainer(
auth: AuthInputs,
params: { name: string; ttlMinutes: number },
): Promise<OpenAIContainerSummary> {
const response = await authFetch(
"/api/inference/external/openai/containers/create",
{
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({
...(await buildAuthBody(auth)),
name: params.name,
ttl_minutes: params.ttlMinutes,
}),
},
);
if (!response.ok) throw new Error(await parseError(response));
const raw = (await response.json()) as RawSummary;
return fromRaw(raw);
}
export async function deleteOpenAIContainer(
auth: AuthInputs,
containerId: string,
): Promise<void> {
const response = await authFetch(
"/api/inference/external/openai/containers/delete",
{
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({
...(await buildAuthBody(auth)),
container_id: containerId,
}),
},
);
// 404 = container already gone (deleted elsewhere, or expired-then-purged).
// Treat as idempotent success so a stale list entry doesn't surface as a
// confusing error — the caller will refresh and the entry will disappear.
if (!response.ok && response.status !== 204 && response.status !== 404) {
throw new Error(await parseError(response));
}
}

234
studio/frontend/src/features/chat/api/providers-api.ts Normal file
View file

@ -0,0 +1,234 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import forge from "node-forge";
import { authFetch } from "@/features/auth";
export interface ProviderRegistryEntry {
provider_type: string;
display_name: string;
base_url: string;
default_models: string[];
supports_streaming: boolean;
supports_vision: boolean;
supports_tool_calling: boolean;
/** remote = fetch /models; curated = huge catalogs — UI uses defaults + manual IDs only */
model_list_mode?: "remote" | "curated";
}
export interface ProviderConfig {
id: string;
provider_type: string;
display_name: string;
base_url: string;
is_enabled: boolean;
created_at: string;
updated_at: string;
}
export interface ProviderModelInfo {
id: string;
display_name: string;
context_length?: number | null;
owned_by?: string | null;
}
export interface ProviderTestResult {
success: boolean;
message: string;
models_count?: number | null;
}
function parseErrorText(status: number, body: unknown): string {
if (
body &&
typeof body === "object" &&
"detail" in body &&
typeof body.detail === "string"
) {
return body.detail;
}
if (
body &&
typeof body === "object" &&
"message" in body &&
typeof body.message === "string"
) {
return body.message;
}
return `Request failed (${status})`;
}
async function parseJsonOrThrow<T>(response: Response): Promise<T> {
const body = await response.json().catch(() => null);
if (!response.ok) {
throw new Error(parseErrorText(response.status, body));
}
return body as T;
}
export function isProviderKeyRotationError(error: unknown): boolean {
if (!(error instanceof Error)) return false;
const normalized = error.message.toLowerCase();
return (
normalized.includes("public key may have changed") ||
normalized.includes("server key may have changed")
);
}
let cachedPublicKeyPem: string | null = null;
let cachedForgeKey: forge.pki.rsa.PublicKey | null = null;
export function clearProviderPublicKeyCache(): void {
cachedPublicKeyPem = null;
cachedForgeKey = null;
}
async function importProviderPublicKey(
forceRefresh = false,
): Promise<forge.pki.rsa.PublicKey> {
if (!forceRefresh && cachedForgeKey) {
return cachedForgeKey;
}
const response = await authFetch("/api/providers/public-key");
const body = await parseJsonOrThrow<{ public_key: string }>(response);
const publicKeyPem = body.public_key?.trim();
if (!publicKeyPem) {
throw new Error("Provider public key is missing.");
}
if (!forceRefresh && cachedPublicKeyPem === publicKeyPem && cachedForgeKey) {
return cachedForgeKey;
}
const forgeKey = forge.pki.publicKeyFromPem(publicKeyPem);
cachedPublicKeyPem = publicKeyPem;
cachedForgeKey = forgeKey;
return forgeKey;
}
export async function encryptProviderApiKey(
plaintextApiKey: string,
forceRefresh = false,
): Promise<string> {
const key = await importProviderPublicKey(forceRefresh);
const encrypted = key.encrypt(plaintextApiKey, "RSA-OAEP", {
md: forge.md.sha256.create(),
mgf1: { md: forge.md.sha256.create() },
});
return forge.util.encode64(encrypted);
}
export async function listProviderRegistry(): Promise<ProviderRegistryEntry[]> {
const response = await authFetch("/api/providers/registry");
return parseJsonOrThrow<ProviderRegistryEntry[]>(response);
}
export async function listProviderConfigs(): Promise<ProviderConfig[]> {
const response = await authFetch("/api/providers/");
return parseJsonOrThrow<ProviderConfig[]>(response);
}
export async function createProviderConfig(payload: {
providerType: string;
displayName: string;
baseUrl?: string | null;
}): Promise<ProviderConfig> {
const response = await authFetch("/api/providers/", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
provider_type: payload.providerType,
display_name: payload.displayName,
base_url: payload.baseUrl ?? null,
}),
});
return parseJsonOrThrow<ProviderConfig>(response);
}
export async function deleteProviderConfig(providerId: string): Promise<void> {
const response = await authFetch(`/api/providers/${providerId}`, {
method: "DELETE",
});
if (!response.ok) {
const body = await response.json().catch(() => null);
throw new Error(parseErrorText(response.status, body));
}
}
export async function updateProviderConfig(
providerId: string,
payload: {
displayName?: string;
baseUrl?: string | null;
isEnabled?: boolean;
},
): Promise<ProviderConfig> {
const response = await authFetch(`/api/providers/${providerId}`, {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
...(payload.displayName === undefined ? {} : { display_name: payload.displayName }),
...(payload.baseUrl === undefined ? {} : { base_url: payload.baseUrl }),
...(payload.isEnabled === undefined ? {} : { is_enabled: payload.isEnabled }),
}),
});
return parseJsonOrThrow<ProviderConfig>(response);
}
async function withApiKeyEncryptionRetry<T>(
plaintextApiKey: string,
call: (encryptedApiKey: string | null) => Promise<T>,
): Promise<T> {
// Empty key (local providers): skip RSA round-trip and let the backend omit auth.
if (!plaintextApiKey) {
return await call(null);
}
try {
const encrypted = await encryptProviderApiKey(plaintextApiKey, false);
return await call(encrypted);
} catch (error) {
if (!isProviderKeyRotationError(error)) {
throw error;
}
clearProviderPublicKeyCache();
const encrypted = await encryptProviderApiKey(plaintextApiKey, true);
return await call(encrypted);
}
}
export async function testProviderConnection(payload: {
providerType: string;
apiKey: string;
baseUrl?: string | null;
}): Promise<ProviderTestResult> {
return withApiKeyEncryptionRetry(payload.apiKey, async (encryptedApiKey) => {
const response = await authFetch("/api/providers/test", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
provider_type: payload.providerType,
encrypted_api_key: encryptedApiKey,
base_url: payload.baseUrl ?? null,
}),
});
return parseJsonOrThrow<ProviderTestResult>(response);
});
}
export async function listProviderModels(payload: {
providerType: string;
apiKey: string;
baseUrl?: string | null;
}): Promise<ProviderModelInfo[]> {
return withApiKeyEncryptionRetry(payload.apiKey, async (encryptedApiKey) => {
const response = await authFetch("/api/providers/models", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
provider_type: payload.providerType,
encrypted_api_key: encryptedApiKey,
base_url: payload.baseUrl ?? null,
}),
});
return parseJsonOrThrow<ProviderModelInfo[]>(response);
});
}

377
studio/frontend/src/features/chat/chat-page.tsx
View file

@ -3,6 +3,7 @@
import {
type DeletedModelRef,
type ExternalModelOption,
type LoraModelOption,
type ModelOption,
ModelSelector,
@ -40,6 +41,18 @@ import { ChatSettingsPanel } from "./chat-settings-sheet";
import { ContextUsageBar } from "./components/context-usage-bar";
import { ModelLoadInlineStatus } from "./components/model-load-status";
import { db } from "./db";
import {
buildExternalModelId,
isExternalModelId,
parseExternalModelId,
} from "./external-providers";
import {
clampReasoningEffortToLevels,
getExternalReasoningCapabilities,
getProviderCapabilities,
providerSupportsBuiltinCodeExecution,
providerSupportsBuiltinWebSearch,
} from "./provider-capabilities";
import { useChatModelRuntime } from "./hooks/use-chat-model-runtime";
import {
clearTrainingCompareHandoff,
@ -54,6 +67,7 @@ import {
SharedComposer,
} from "./shared-composer";
import { useChatRuntimeStore } from "./stores/chat-runtime-store";
import { useExternalProvidersStore } from "./stores/external-providers-store";
import { buildChatTourSteps } from "./tour";
import type { ChatView, MessageRecord } from "./types";
@ -536,6 +550,8 @@ export function ChatPage(): ReactElement {
const settingsOpen = useChatRuntimeStore((s) => s.settingsPanelOpen);
const setSettingsOpen = useChatRuntimeStore((s) => s.setSettingsPanelOpen);
const externalProviders = useExternalProvidersStore((s) => s.providers);
const setExternalProviders = useExternalProvidersStore((s) => s.setProviders);
useEffect(() => {
const threadId = search.thread;
@ -596,7 +612,9 @@ export function ChatPage(): ReactElement {
loadProgress,
loadToastDismissed,
} = useChatModelRuntime();
const pendingNativeModelIntent = useNativeIntentStore((state) => state.pendingModelIntent);
const pendingNativeModelIntent = useNativeIntentStore(
(state) => state.pendingModelIntent,
);
const nativePathLeasesSupported = useNativePathLeasesSupported();
const refreshRef = useRef(refresh);
const selectModelRef = useRef(selectModel);
@ -605,9 +623,154 @@ export function ChatPage(): ReactElement {
refreshRef.current = refresh;
selectModelRef.current = selectModel;
}, [refresh, selectModel]);
const isExternalModel = useMemo(
() => isExternalModelId(inferenceParams.checkpoint),
[inferenceParams.checkpoint],
);
const reasoningEnabled = useChatRuntimeStore((s) => s.reasoningEnabled);
const reasoningStyle = useChatRuntimeStore((s) => s.reasoningStyle);
const reasoningEffort = useChatRuntimeStore((s) => s.reasoningEffort);
const supportsReasoningOff = useChatRuntimeStore((s) => s.supportsReasoningOff);
const activeExternalProvider = useMemo(() => {
const selection = parseExternalModelId(inferenceParams.checkpoint);
if (!selection) return null;
return (
externalProviders.find(
(p) => p.id === selection.providerId,
) ?? null
);
}, [externalProviders, inferenceParams.checkpoint]);
const activeExternalProviderType = activeExternalProvider?.providerType ?? null;
const activeProviderCapabilities = useMemo(() => {
const selection = parseExternalModelId(inferenceParams.checkpoint);
if (!selection) return null;
const provider = externalProviders.find(
(p) => p.id === selection.providerId,
);
const baseCapabilities = getProviderCapabilities(provider?.providerType);
if (!baseCapabilities) return baseCapabilities;
const anthropicThinkingEnabled =
provider?.providerType === "anthropic" &&
reasoningStyle === "reasoning_effort" &&
(supportsReasoningOff ? reasoningEnabled : true) &&
reasoningEffort !== "none";
if (!anthropicThinkingEnabled) return baseCapabilities;
return {
...baseCapabilities,
temperature: false,
topK: false,
};
}, [
externalProviders,
inferenceParams.checkpoint,
reasoningEnabled,
reasoningStyle,
reasoningEffort,
supportsReasoningOff,
]);
useEffect(() => {
const selection = parseExternalModelId(inferenceParams.checkpoint);
if (!selection) return;
const provider = externalProviders.find((p) => p.id === selection.providerId);
const reasoningCaps = getExternalReasoningCapabilities(
provider?.providerType,
selection.modelId,
{ isReasoningProvider: provider?.isReasoningModel === true },
);
const state = useChatRuntimeStore.getState();
const preferredEffort = state.reasoningEffort;
const effortLevels = reasoningCaps.reasoningEffortLevels;
const clampedEffort = clampReasoningEffortToLevels(
preferredEffort,
effortLevels,
);
// Per-provider default effort. Anthropic gets the highest available
// level (xhigh on 4.6/4.7, high on 4.5) since Claude's adaptive
// thinking adjusts cost per turn — sitting at the top of the dial
// gives users the strongest answers and the model can still skip
// thinking when the turn is trivial. OpenAI gets "high" by default
// — the gpt-5.x reasoning models accept high across the board and
// it's the right cost/quality sweet spot for Responses-API tools
// (web search included). Everyone else gets "medium" as a balanced
// default. Users can pick another level via the Think dropdown.
const isAnthropic = provider?.providerType === "anthropic";
const isOpenAI = provider?.providerType === "openai";
const anthropicTopEffort = effortLevels.includes("xhigh")
? "xhigh"
: effortLevels.includes("high")
? "high"
: clampedEffort;
const openaiDefaultEffort = effortLevels.includes("high")
? "high"
: effortLevels.includes("medium")
? "medium"
: clampedEffort;
const nextReasoningEffort = reasoningCaps.supportsReasoning
? isAnthropic
? anthropicTopEffort
: isOpenAI
? openaiDefaultEffort
: effortLevels.includes("medium")
? "medium"
: clampedEffort
: state.reasoningEffort;
const supportsBuiltinWebSearch = providerSupportsBuiltinWebSearch(
provider?.providerType,
);
const supportsBuiltinCodeExecution = providerSupportsBuiltinCodeExecution(
provider?.providerType,
selection.modelId,
provider?.baseUrl,
);
// Kimi's k2.6/k2.5 default to thinking enabled on the server side
// (per https://platform.kimi.ai/docs/models). Mirror that default
// in the UI so the Think pill comes up clicked when the user picks
// a Kimi model. The Search pill stays off by default; the mutual-
// exclusion handlers in the composer flip the two when needed.
const isKimi = provider?.providerType === "kimi";
// Web search is on by default for the two providers we trust most
// for it: Anthropic (web_search_20250305 server tool, structured
// citations) and OpenAI (/v1/responses web_search, structured
// citations). Other providers stay off-by-default — OpenRouter's
// plugins shape and Kimi's $web_search builtin still work when the
// user opts in via the pill, but they're a notch less reliable so
// we don't pre-enable them.
const searchOnByDefault =
supportsBuiltinWebSearch &&
(provider?.providerType === "anthropic" ||
provider?.providerType === "openai");
useChatRuntimeStore.setState({
supportsReasoning: reasoningCaps.supportsReasoning,
reasoningAlwaysOn: reasoningCaps.reasoningAlwaysOn,
reasoningStyle: reasoningCaps.reasoningStyle,
supportsReasoningOff: reasoningCaps.supportsReasoningOff,
reasoningEffortLevels: effortLevels,
reasoningEffort: nextReasoningEffort,
reasoningEnabled: reasoningCaps.supportsReasoning
? reasoningCaps.supportsReasoningOff
? isKimi
? true
: state.reasoningEnabled
: true
: state.reasoningEnabled,
supportsPreserveThinking: false,
// External models never give us a local tool runtime (no
// python sandbox), so `supportsTools` must be false. The two
// `supportsBuiltin*` flags pick up the slack for providers that
// run the tool server-side: `supportsBuiltinWebSearch` lights
// up the Search pill (OpenAI / Anthropic / OpenRouter / Kimi),
// `supportsBuiltinCodeExecution` lights up the Code pill
// (Anthropic Claude 4.x only, today).
supportsTools: false,
supportsBuiltinWebSearch,
supportsBuiltinCodeExecution,
toolsEnabled: searchOnByDefault,
codeToolsEnabled: false,
});
}, [externalProviders, inferenceParams.checkpoint]);
const canCompare = useMemo(() => {
return Boolean(inferenceParams.checkpoint);
}, [inferenceParams.checkpoint]);
return Boolean(inferenceParams.checkpoint) && !isExternalModel;
}, [inferenceParams.checkpoint, isExternalModel]);
// Derive view from URL search params
const view = useMemo<ChatView>(() => {
@ -632,7 +795,8 @@ export function ChatPage(): ReactElement {
const hasActiveModel = Boolean(inferenceParams.checkpoint);
const loadNativeModelIntent = useCallback(
async (intent: NativeIntent, loadingDescription: string) => {
const label = intent.path.displayLabel || intent.displayLabel || "Local GGUF model";
const label =
intent.path.displayLabel || intent.displayLabel || "Local GGUF model";
await selectModel({
id: label,
nativePathToken: intent.path.token,
@ -687,6 +851,7 @@ export function ChatPage(): ReactElement {
(
value: string,
meta?: {
source?: string;
isLora: boolean;
ggufVariant?: string;
isDownloaded?: boolean;
@ -702,6 +867,115 @@ export function ChatPage(): ReactElement {
(meta?.ggufVariant ?? null) === (currentVariant ?? null))
)
return;
if (meta?.source === "external" || isExternalModelId(value)) {
const selectedExternal = parseExternalModelId(value);
const selectedProvider = selectedExternal
? externalProviders.find((p) => p.id === selectedExternal.providerId)
: null;
const reasoningCaps = getExternalReasoningCapabilities(
selectedProvider?.providerType,
selectedExternal?.modelId,
{
isReasoningProvider:
selectedProvider?.isReasoningModel === true,
},
);
const preferredEffort = store.reasoningEffort;
const effortLevels = reasoningCaps.reasoningEffortLevels;
const clampedEffort = clampReasoningEffortToLevels(
preferredEffort,
effortLevels,
);
// Same per-provider default policy as the useEffect path above:
// Anthropic picks the highest available level, OpenAI picks
// "high", everyone else picks "medium".
const isAnthropic = selectedProvider?.providerType === "anthropic";
const isOpenAI = selectedProvider?.providerType === "openai";
const anthropicTopEffort = effortLevels.includes("xhigh")
? "xhigh"
: effortLevels.includes("high")
? "high"
: clampedEffort;
const openaiDefaultEffort = effortLevels.includes("high")
? "high"
: effortLevels.includes("medium")
? "medium"
: clampedEffort;
const nextReasoningEffort = reasoningCaps.supportsReasoning
? isAnthropic
? anthropicTopEffort
: isOpenAI
? openaiDefaultEffort
: effortLevels.includes("medium")
? "medium"
: clampedEffort
: store.reasoningEffort;
// Clear any cached router-picked openrouter/free model unless the
// user is staying on openrouter/free — otherwise the chip would
// keep showing a stale ":<chosen>" suffix from a previous model.
const stillOnOpenRouterFree =
selectedProvider?.providerType === "openrouter" &&
selectedExternal?.modelId === "openrouter/free";
setInferenceParams({
...store.params,
checkpoint: value,
});
const supportsBuiltinWebSearch = providerSupportsBuiltinWebSearch(
selectedProvider?.providerType,
);
const supportsBuiltinCodeExecution = providerSupportsBuiltinCodeExecution(
selectedProvider?.providerType,
selectedExternal?.modelId,
selectedProvider?.baseUrl,
);
// See sibling useEffect above: Kimi's k2.x default to thinking
// enabled, so the Think pill comes up clicked. Search pill stays
// off by default; mutual exclusion flips them via the composer.
const isKimi = selectedProvider?.providerType === "kimi";
// Mirror of sibling useEffect: Anthropic and OpenAI get Search
// on-by-default since their server tools emit structured
// citations end-to-end. OpenRouter and Kimi stay off-by-default.
const searchOnByDefault =
supportsBuiltinWebSearch &&
(selectedProvider?.providerType === "anthropic" ||
selectedProvider?.providerType === "openai");
useChatRuntimeStore.setState({
activeGgufVariant: null,
ggufContextLength: null,
ggufMaxContextLength: null,
ggufNativeContextLength: null,
activeNativePathToken: null,
supportsReasoning: reasoningCaps.supportsReasoning,
reasoningAlwaysOn: reasoningCaps.reasoningAlwaysOn,
reasoningStyle: reasoningCaps.reasoningStyle,
supportsReasoningOff: reasoningCaps.supportsReasoningOff,
reasoningEffortLevels: effortLevels,
reasoningEffort: nextReasoningEffort,
reasoningEnabled: reasoningCaps.supportsReasoning
? reasoningCaps.supportsReasoningOff
? isKimi
? true
: store.reasoningEnabled
: true
: store.reasoningEnabled,
supportsPreserveThinking: false,
// External models have no local tool runtime → supportsTools
// stays false. The two supportsBuiltin* flags carry the
// server-side capability info for each pill:
// - Search → providerSupportsBuiltinWebSearch
// - Code → providerSupportsBuiltinCodeExecution
// (Anthropic Claude 4.x only, today)
supportsTools: false,
supportsBuiltinWebSearch,
supportsBuiltinCodeExecution,
toolsEnabled: searchOnByDefault,
codeToolsEnabled: false,
...(stillOnOpenRouterFree ? {} : { lastOpenRouterChosenModel: null }),
});
return;
}
// Local model picked → drop any cached openrouter/free chosen model.
useChatRuntimeStore.setState({ lastOpenRouterChosenModel: null });
void (async () => {
let showImageCompatibilityWarning = false;
if (view.mode === "single" && activeThreadId) {
@ -738,7 +1012,14 @@ export function ChatPage(): ReactElement {
});
})();
},
[activeThreadId, modelsFromStore, selectModel, view],
[
activeThreadId,
externalProviders,
modelsFromStore,
selectModel,
setInferenceParams,
view,
],
);
const handleEject = useCallback(() => {
void ejectModel();
@ -813,6 +1094,47 @@ export function ChatPage(): ReactElement {
})),
[modelsFromStore],
);
const lastOpenRouterChosenModel = useChatRuntimeStore(
(s) => s.lastOpenRouterChosenModel,
);
const externalModels = useMemo<ExternalModelOption[]>(
() =>
externalProviders.flatMap((provider) =>
provider.models.map((model) => {
// For OpenRouter's free router we know which underlying free
// model the gateway actually picked once a stream completes
// (chat-adapter latches `chunk.model` into the runtime store).
// Render the chip as `openrouter:<short-chosen>` — drop the
// redundant `/free` from the router id and the org prefix
// from the chosen id (e.g.
// openrouter/free + inclusionai/ring-2.6-1t-20260508:free
// -> openrouter:ring-2.6-1t-20260508:free
// ). The `:free` suffix on the chosen id already conveys
// 'free model', so the leading `/free` is noise.
let displayName = model;
if (
provider.providerType === "openrouter" &&
model === "openrouter/free" &&
lastOpenRouterChosenModel
) {
const lastSlash = lastOpenRouterChosenModel.lastIndexOf("/");
const shortChosen =
lastSlash >= 0
? lastOpenRouterChosenModel.slice(lastSlash + 1)
: lastOpenRouterChosenModel;
displayName = `openrouter:${shortChosen}`;
}
return {
id: buildExternalModelId(provider.id, model),
name: displayName,
providerId: provider.id,
providerName: provider.name,
providerType: provider.providerType,
};
}),
),
[externalProviders, lastOpenRouterChosenModel],
);
const [localModels, setLocalModels] = useState<LoraModelOption[]>([]);
@ -847,20 +1169,24 @@ export function ChatPage(): ReactElement {
.catch(() => {});
}, [navigate]);
const refreshModelLists = useCallback((deletedModel?: DeletedModelRef) => {
const { checkpoint } = useChatRuntimeStore.getState().params;
const activeGgufVariant = useChatRuntimeStore.getState().activeGgufVariant;
if (
modelMatchesDeleted(
{ id: checkpoint, ggufVariant: activeGgufVariant },
deletedModel,
)
) {
useChatRuntimeStore.getState().clearCheckpoint();
}
void refresh();
refreshLocalModels();
}, [refresh, refreshLocalModels]);
const refreshModelLists = useCallback(
(deletedModel?: DeletedModelRef) => {
const { checkpoint } = useChatRuntimeStore.getState().params;
const activeGgufVariant =
useChatRuntimeStore.getState().activeGgufVariant;
if (
modelMatchesDeleted(
{ id: checkpoint, ggufVariant: activeGgufVariant },
deletedModel,
)
) {
useChatRuntimeStore.getState().clearCheckpoint();
}
void refresh();
refreshLocalModels();
},
[refresh, refreshLocalModels],
);
const loraModels = useMemo<LoraModelOption[]>(() => {
const fromLoras = lorasFromStore.map((lora) => ({
@ -1001,6 +1327,7 @@ export function ChatPage(): ReactElement {
<ModelSelector
models={models}
loraModels={loraModels}
externalModels={externalModels}
value={inferenceParams.checkpoint}
activeGgufVariant={activeGgufVariant}
onValueChange={handleCheckpointChange}
@ -1014,6 +1341,7 @@ export function ChatPage(): ReactElement {
onOpenChange={handleModelSelectorOpenChange}
triggerDataTour="chat-model-selector"
contentDataTour="chat-model-selector-popover"
showCloudIndicator={isExternalModel}
className="max-w-[62vw] !pr-3 sm:max-w-none !h-[34px]"
/>
)}
@ -1120,6 +1448,17 @@ export function ChatPage(): ReactElement {
onOpenChange={setSettingsOpen}
params={inferenceParams}
onParamsChange={setInferenceParams}
isExternalModel={isExternalModel}
providerCapabilities={activeProviderCapabilities}
activeExternalProvider={activeExternalProvider}
onExternalProviderChange={(updatedProvider) => {
setExternalProviders(
externalProviders.map((provider) =>
provider.id === updatedProvider.id ? updatedProvider : provider,
),
);
}}
externalProviderType={activeExternalProviderType}
onReloadModel={() => {
const state = useChatRuntimeStore.getState();
if (state.params.checkpoint) {

1454
studio/frontend/src/features/chat/chat-providers-dialog.tsx Normal file
View file

File diff suppressed because it is too large Load diff

268
studio/frontend/src/features/chat/chat-settings-sheet.tsx
View file

@ -64,6 +64,12 @@ import { Fragment, type ReactNode } from "react";
import { useEffect, useMemo, useRef, useState } from "react";
import { toast } from "sonner";
import { useChatRuntimeStore } from "./stores/chat-runtime-store";
import {
type ExternalProviderConfig,
getExternalProviderApiKey,
parseExternalModelId,
supportsProviderPromptCaching,
} from "./external-providers";
import {
applyPresetParams,
BUILTIN_PRESET_NAMES,
@ -80,6 +86,13 @@ import {
toPresetParams,
type Preset,
} from "./presets/preset-policy";
import { OpenAICodeExecSection } from "./components/openai-code-exec-section";
import {
EXTERNAL_MAX_OUTPUT_TOKENS,
getExternalMinOutputTokens,
providerSupportsBuiltinCodeExecution,
type ProviderCapabilities,
} from "./provider-capabilities";
import type { InferenceParams } from "./types/runtime";
export { defaultInferenceParams, type Preset } from "./presets/preset-policy";
@ -505,6 +518,21 @@ interface ChatSettingsPanelProps {
onOpenChange?: (open: boolean) => void;
params: InferenceParams;
onParamsChange: (params: InferenceParams) => void;
isExternalModel?: boolean;
/**
* Sampling-param capability set for the active external provider, or `null`
* for local models (in which case every knob is rendered). Drives the
* per-param visibility in the sampling section.
*/
providerCapabilities?: ProviderCapabilities | null;
activeExternalProvider?: ExternalProviderConfig | null;
onExternalProviderChange?: (provider: ExternalProviderConfig) => void;
/**
* Backend provider type for the active external model (e.g. "kimi",
* "anthropic", "openai"), or `null` for local models. Drives the
* per-provider Max Tokens floor in the slider.
*/
externalProviderType?: string | null;
onReloadModel?: () => void;
}
@ -513,11 +541,30 @@ export function ChatSettingsPanel({
onOpenChange,
params,
onParamsChange,
isExternalModel = false,
providerCapabilities = null,
activeExternalProvider = null,
onExternalProviderChange,
externalProviderType = null,
onReloadModel,
}: ChatSettingsPanelProps) {
// For non-external (local) models we show every knob — providerCapabilities
// is only consulted when `isExternalModel` is true. An external model with an
// unknown provider falls back to the OpenAI-compat shape via
// getProviderCapabilities, so these flags never undercount support.
const showTemperature =
!isExternalModel || Boolean(providerCapabilities?.temperature);
const showTopP = !isExternalModel || Boolean(providerCapabilities?.topP);
const showTopK = !isExternalModel || Boolean(providerCapabilities?.topK);
const showMinP = !isExternalModel || Boolean(providerCapabilities?.minP);
const showRepetitionPenalty =
!isExternalModel || Boolean(providerCapabilities?.repetitionPenalty);
const showPresencePenalty =
!isExternalModel || Boolean(providerCapabilities?.presencePenalty);
const isMobile = useIsMobile();
const isGguf = useChatRuntimeStore((s) => s.activeGgufVariant) != null;
const hasModelContent = isGguf || Boolean(params.checkpoint);
const hasModelContent =
!isExternalModel && (isGguf || Boolean(params.checkpoint));
const speculativeType = useChatRuntimeStore((s) => s.speculativeType);
const setSpeculativeType = useChatRuntimeStore((s) => s.setSpeculativeType);
const loadedSpeculativeType = useChatRuntimeStore(
@ -627,6 +674,26 @@ export function ChatSettingsPanel({
Boolean(currentCheckpoint) &&
modelRequiresTrustRemoteCode &&
!(params.trustRemoteCode ?? false);
const showPromptCachingControl =
activeExternalProvider != null &&
supportsProviderPromptCaching(activeExternalProvider.providerType);
const promptCachingEnabled =
activeExternalProvider?.enablePromptCaching !== false;
const externalSelection = currentCheckpoint
? parseExternalModelId(currentCheckpoint)
: null;
const showOpenAICodeExecSection =
activeExternalProvider != null &&
providerSupportsBuiltinCodeExecution(
activeExternalProvider.providerType,
externalSelection?.modelId,
activeExternalProvider.baseUrl,
) &&
activeExternalProvider.providerType === "openai";
const activeThreadId = useChatRuntimeStore((s) => s.activeThreadId);
const openAiApiKeyForSection = activeExternalProvider
? getExternalProviderApiKey(activeExternalProvider.id) || null
: null;
function set<K extends keyof InferenceParams>(key: K) {
return (v: InferenceParams[K]) => {
@ -1110,6 +1177,43 @@ export function ChatSettingsPanel({
</div>
</CollapsibleSection>
{showPromptCachingControl && activeExternalProvider ? (
<CollapsibleSection label="Provider" defaultOpen={true}>
<div className="flex items-center justify-between gap-3 pt-1">
<div className="flex min-w-0 items-center gap-1.5">
<span className="min-w-0 text-[13px] font-medium leading-[1.25] tracking-nav text-nav-fg">
Prompt caching
</span>
<InfoHint>
Reuse compatible prompt prefixes for lower latency and cost.
</InfoHint>
</div>
<Switch
className="panel-switch shrink-0"
checked={promptCachingEnabled}
onCheckedChange={(checked) => {
onExternalProviderChange?.({
...activeExternalProvider,
enablePromptCaching: checked,
});
}}
aria-label="Enable prompt caching"
/>
</div>
</CollapsibleSection>
) : null}
{showOpenAICodeExecSection && activeExternalProvider ? (
<CollapsibleSection label="Code Execution" defaultOpen={false}>
<OpenAICodeExecSection
provider={activeExternalProvider}
apiKey={openAiApiKeyForSection}
activeThreadId={activeThreadId}
onProviderChange={(p) => onExternalProviderChange?.(p)}
/>
</CollapsibleSection>
) : null}
<CollapsibleSection label="System Prompt" defaultOpen={true}>
<button
type="button"
@ -1131,65 +1235,79 @@ export function ChatSettingsPanel({
<CollapsibleSection label="Sampling" defaultOpen={true}>
<div className="flex flex-col gap-5 pt-1">
<ParamSlider
label="Temperature"
value={params.temperature}
min={0}
max={2}
step={0.01}
onChange={set("temperature")}
info="Controls randomness. Lower values make output focused and deterministic; higher values increase variety and creativity."
/>
<ParamSlider
label="Top P"
value={params.topP}
min={0}
max={1}
step={0.05}
onChange={set("topP")}
displayValue={params.topP === 1 ? "Off" : undefined}
info="Nucleus sampling. Restricts choices to the smallest set of tokens whose cumulative probability reaches this threshold. 1.0 = off."
/>
<ParamSlider
label="Top K"
value={params.topK}
min={0}
max={100}
step={1}
onChange={set("topK")}
displayValue={params.topK === 0 ? "Off" : undefined}
info="Limits sampling to the K most likely tokens at each step. 0 = off."
/>
<ParamSlider
label="Min P"
value={params.minP}
min={0}
max={1}
step={0.01}
onChange={set("minP")}
info="Drops tokens whose probability is below this fraction of the top token's probability. Filters unlikely candidates."
/>
<ParamSlider
label="Repetition Penalty"
value={params.repetitionPenalty}
min={1}
max={2}
step={0.05}
onChange={set("repetitionPenalty")}
displayValue={params.repetitionPenalty === 1 ? "Off" : undefined}
info="Down-weights tokens that have already appeared, reducing repetition. 1.0 = off; higher values penalize more strongly."
/>
<ParamSlider
label="Presence Penalty"
value={params.presencePenalty}
min={0}
max={2}
step={0.1}
onChange={set("presencePenalty")}
displayValue={params.presencePenalty === 0 ? "Off" : undefined}
info="Penalizes any token that has already appeared at least once, encouraging the model to introduce new topics. 0 = off."
/>
{!isGguf && (
{showTemperature ? (
<ParamSlider
label="Temperature"
value={params.temperature}
min={0}
max={2}
step={0.01}
onChange={set("temperature")}
info="Controls randomness. Lower values make output focused and deterministic; higher values increase variety and creativity."
/>
) : null}
{showTopP ? (
<ParamSlider
label="Top P"
value={params.topP}
min={0}
max={1}
step={0.05}
onChange={set("topP")}
displayValue={params.topP === 1 ? "Off" : undefined}
info="Nucleus sampling. Restricts choices to the smallest set of tokens whose cumulative probability reaches this threshold. 1.0 = off."
/>
) : null}
{showTopK ? (
<ParamSlider
label="Top K"
value={params.topK}
min={0}
max={100}
step={1}
onChange={set("topK")}
displayValue={params.topK === 0 ? "Off" : undefined}
info="Limits sampling to the K most likely tokens at each step. 0 = off."
/>
) : null}
{showMinP ? (
<ParamSlider
label="Min P"
value={params.minP}
min={0}
max={1}
step={0.01}
onChange={set("minP")}
info="Drops tokens whose probability is below this fraction of the top token's probability. Filters unlikely candidates."
/>
) : null}
{showRepetitionPenalty ? (
<ParamSlider
label="Repetition Penalty"
value={params.repetitionPenalty}
min={1}
max={2}
step={0.05}
onChange={set("repetitionPenalty")}
displayValue={
params.repetitionPenalty === 1 ? "Off" : undefined
}
info="Down-weights tokens that have already appeared, reducing repetition. 1.0 = off; higher values penalize more strongly."
/>
) : null}
{showPresencePenalty ? (
<ParamSlider
label="Presence Penalty"
value={params.presencePenalty}
min={0}
max={2}
step={0.1}
onChange={set("presencePenalty")}
displayValue={params.presencePenalty === 0 ? "Off" : undefined}
info="Penalizes any token that has already appeared at least once, encouraging the model to introduce new topics. 0 = off."
/>
) : null}
{!isExternalModel && !isGguf && (
<ParamSlider
label="Max Seq Length"
value={params.maxSeqLength}
@ -1203,8 +1321,18 @@ export function ChatSettingsPanel({
<ParamSlider
label="Max Tokens"
value={params.maxTokens}
min={64}
max={isGguf && ggufContextLength ? ggufContextLength : 32768}
min={
isExternalModel
? getExternalMinOutputTokens(externalProviderType)
: 64
}
max={
isExternalModel
? EXTERNAL_MAX_OUTPUT_TOKENS
: isGguf && ggufContextLength
? ggufContextLength
: 32768
}
step={64}
onChange={set("maxTokens")}
displayValue={
@ -1219,13 +1347,15 @@ export function ChatSettingsPanel({
</div>
</CollapsibleSection>
<CollapsibleSection label="Tools">
<div className="flex flex-col gap-5 pt-1">
<AutoHealToolCallsToggle />
<MaxToolCallsSlider />
<ToolCallTimeoutSlider />
</div>
</CollapsibleSection>
{!isExternalModel ? (
<CollapsibleSection label="Tools">
<div className="flex flex-col gap-5 pt-1">
<AutoHealToolCallsToggle />
<MaxToolCallsSlider />
<ToolCallTimeoutSlider />
</div>
</CollapsibleSection>
) : null}
</div>
</div>
<Dialog

659
studio/frontend/src/features/chat/components/openai-code-exec-section.tsx Normal file
View file

@ -0,0 +1,659 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
/**
* Settings-sheet section for OpenAI shell-tool container management.
* Renders only when:
* - active provider is OpenAI cloud (api.openai.com base URL), AND
* - the active model is gpt-5.5 or gpt-5.5-pro (the only families
* where the shell tool is wired through today).
*
* Surfaces three controls:
* 1. Default container idle-timeout (minutes). Persists on the
* provider record; pre-fills the create dialog and is used by
* the chat-adapter's lazy-create path on the first turn of a
* thread.
* 2. Container picker for the *active thread* — pick any of the
* user's existing OpenAI containers, or "Auto-create per thread"
* (default; lets the auto-create path manage it).
* 3. Create-new-container inline form. Refresh + delete actions
* per row.
*
* State persistence:
* - TTL → ExternalProviderConfig.openaiContainerTtlMinutes
* - Active container for this thread → ThreadRecord.openaiCodeExecContainerId
*
* No new global stores — list is fetched on open / refresh and held
* in component state.
*/
"use client";
import { useCallback, useEffect, useMemo, useState } from "react";
import { toast } from "sonner";
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
} from "@/components/ui/alert-dialog";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { Skeleton } from "@/components/ui/skeleton";
import { TrashIcon, RefreshCwIcon, PlusIcon } from "lucide-react";
import {
createOpenAIContainer,
deleteOpenAIContainer,
listOpenAIContainers,
type OpenAIContainerSummary,
} from "../api/openai-containers";
import { db } from "../db";
import type { ExternalProviderConfig } from "../external-providers";
import { useLiveQuery } from "../db";
import { ensureThreadRecord } from "../runtime-provider";
const AUTO_OPTION_VALUE = "__auto__";
const DEFAULT_TTL_MINUTES = 20;
const TTL_MIN = 1;
const TTL_MAX = 20; // OpenAI hard cap on expires_after.minutes
// Cadence for re-fetching the container list while the section is
// mounted. OpenAI's container TTL flips at minute granularity, so 30s
// is fast enough that an expired container loses its ACTIVE pill within
// half a minute without hammering /v1/containers.
const REFRESH_POLL_MS = 30_000;
function ageLabel(epochSeconds: number | null | undefined): string {
if (!epochSeconds) return "";
const ageSec = Math.max(0, Math.floor(Date.now() / 1000) - epochSeconds);
if (ageSec < 60) return `${ageSec}s ago`;
const ageMin = Math.floor(ageSec / 60);
if (ageMin < 60) return `${ageMin}m ago`;
const ageHr = Math.floor(ageMin / 60);
if (ageHr < 48) return `${ageHr}h ago`;
const ageDay = Math.floor(ageHr / 24);
return `${ageDay}d ago`;
}
function shortContainerId(id: string): string {
// Mid-truncate keeps the "cntr_" prefix readable and still surfaces the
// tail digits users sometimes copy off OpenAI's dashboard.
if (id.length <= 18) return id;
return `${id.slice(0, 12)}…${id.slice(-4)}`;
}
function isContainerRunning(c: OpenAIContainerSummary): boolean {
// OpenAI's containers API reports `status: "running"` while idle TTL is
// valid and `status: "expired"` once the idle window has passed. Treat
// a missing status as running so we don't false-positive on any older
// payloads that didn't include the field.
return c.status == null || c.status === "running";
}
interface OpenAICodeExecSectionProps {
provider: ExternalProviderConfig;
apiKey: string | null;
activeThreadId: string | null;
onProviderChange: (provider: ExternalProviderConfig) => void;
}
export function OpenAICodeExecSection({
provider,
apiKey,
activeThreadId,
onProviderChange,
}: OpenAICodeExecSectionProps) {
const [containers, setContainers] = useState<OpenAIContainerSummary[]>([]);
const [isLoading, setIsLoading] = useState(false);
const [creating, setCreating] = useState(false);
const [createOpen, setCreateOpen] = useState(false);
const [createName, setCreateName] = useState("");
const [createTtl, setCreateTtl] = useState<number>(
provider.openaiContainerTtlMinutes ?? DEFAULT_TTL_MINUTES,
);
// Ids that have been deleted in this session. Once tombstoned, an id
// stays hidden from the picker for the lifetime of the page — OpenAI's
// /containers list can keep returning a freshly-deleted id for an
// undocumented and variable amount of time, and an automatic re-show
// creates more confusion than it solves. Refreshing the page resets
// the tombstone naturally.
const [tombstones, setTombstones] = useState<Set<string>>(() => new Set());
// Target row for the destructive confirmation dialog. Held in state
// (rather than blocking with window.confirm) so the dialog sits inside
// the settings sheet instead of a native browser alert.
const [pendingDelete, setPendingDelete] =
useState<OpenAIContainerSummary | null>(null);
const [deleting, setDeleting] = useState(false);
const thread = useLiveQuery(
async () => (activeThreadId ? db.threads.get(activeThreadId) : undefined),
[activeThreadId],
);
const activeContainerId = thread?.openaiCodeExecContainerId ?? null;
// Hide just-deleted containers even if OpenAI's list still returns them.
// This is the single chokepoint — every downstream view (sorted picker,
// auto-bind candidate, all-containers list) derives from visibleContainers.
const visibleContainers = useMemo(() => {
if (tombstones.size === 0) return containers;
return containers.filter((c) => !tombstones.has(c.id));
}, [containers, tombstones]);
// Containers sorted newest-first by lastActiveAt so the dropdown's
// default (auto-bind target) shows up first.
const sortedContainers = useMemo(
() =>
[...visibleContainers].sort(
(a, b) => (b.lastActiveAt ?? 0) - (a.lastActiveAt ?? 0),
),
[visibleContainers],
);
// First running container by lastActiveAt — the auto-bind target and
// also what we surface visually before Dexie catches up.
const firstRunningContainer = useMemo(
() => sortedContainers.find(isContainerRunning) ?? null,
[sortedContainers],
);
// What the picker should treat as "active" right now. We decouple
// this from `activeContainerId` (Dexie state) so the user immediately
// sees the most-recent running container while the auto-bind effect's
// async write propagates. If the Dexie-bound container has since
// expired, fall back to the first running candidate — the stale-bind
// sweeper below will clear Dexie shortly after.
const boundContainer = useMemo(
() => sortedContainers.find((c) => c.id === activeContainerId) ?? null,
[sortedContainers, activeContainerId],
);
const displayedContainerId =
(boundContainer && isContainerRunning(boundContainer)
? boundContainer.id
: firstRunningContainer?.id) ?? null;
const refresh = useCallback(async () => {
if (!apiKey) return;
setIsLoading(true);
try {
const list = await listOpenAIContainers({
apiKey,
baseUrl: provider.baseUrl || null,
});
setContainers(list);
} catch (err) {
toast.error(
`Failed to list containers: ${err instanceof Error ? err.message : "Unknown"}`,
);
} finally {
setIsLoading(false);
}
}, [apiKey, provider.baseUrl]);
// Fetch once when the section mounts (or provider changes), then
// poll on a low cadence so an expired container's ACTIVE pill clears
// without the user clicking the refresh button. Also re-fetch when
// the tab regains visibility — covers the common case of leaving the
// sheet open across a long idle period.
useEffect(() => {
void refresh();
const interval = window.setInterval(() => {
if (document.visibilityState === "visible") {
void refresh();
}
}, REFRESH_POLL_MS);
const onVisibility = () => {
if (document.visibilityState === "visible") {
void refresh();
}
};
document.addEventListener("visibilitychange", onVisibility);
return () => {
window.clearInterval(interval);
document.removeEventListener("visibilitychange", onVisibility);
};
}, [refresh]);
// Auto-bind the active thread to the most-recently-active container
// whenever the thread has none set and at least one container exists
// on the user's OpenAI account. Sorting by `lastActiveAt` matches
// what feels "most recent" from the user's perspective.
//
// We eagerly materialize the thread row via `ensureThreadRecord` so
// the bind actually lands in Dexie before the user has sent a first
// message. This does NOT create anything at OpenAI — only a local
// ThreadRecord — so it does not bypass the user's expectation that
// a fresh OpenAI container is not created until first send.
//
// If `containers` is empty (no OpenAI containers exist yet), this
// effect short-circuits: the picker renders an empty-state hint and
// the chat-adapter's lazy-create path will mint the first container
// on first send.
useEffect(() => {
if (
!activeThreadId ||
activeContainerId ||
visibleContainers.length === 0
) {
return;
}
const sorted = [...visibleContainers].sort(
(a, b) => (b.lastActiveAt ?? 0) - (a.lastActiveAt ?? 0),
);
const candidate = sorted[0];
if (!candidate) return;
void (async () => {
try {
await ensureThreadRecord({
threadId: activeThreadId,
modelType: "base",
});
await db.threads.update(activeThreadId, {
openaiCodeExecContainerId: candidate.id,
});
} catch {
// Best-effort; the chat-adapter will inherit/create on send.
}
})();
}, [activeThreadId, activeContainerId, visibleContainers]);
const ttlValue = provider.openaiContainerTtlMinutes ?? DEFAULT_TTL_MINUTES;
const onTtlChange = (raw: string) => {
const n = parseInt(raw, 10);
if (Number.isNaN(n)) return;
const clamped = Math.min(Math.max(n, TTL_MIN), TTL_MAX);
onProviderChange({ ...provider, openaiContainerTtlMinutes: clamped });
};
const onPick = async (value: string) => {
if (!activeThreadId || !value) return;
// value is always a container id now — the "Auto-create per thread"
// option has been removed in favour of always defaulting to the
// most-recently-active container. The chat-adapter still handles
// the no-containers-exist case (lazy-create on first send).
//
// ensureThreadRecord materializes the thread row eagerly (modelType
// "base" — settings sheet is single-thread-mode only) so the update
// actually lands when the user hasn't sent a message yet.
try {
await ensureThreadRecord({ threadId: activeThreadId, modelType: "base" });
const affected = await db.threads.update(activeThreadId, {
openaiCodeExecContainerId: value,
});
if (affected === 0) {
toast.error("Could not update thread.");
}
} catch (err) {
toast.error(
`Could not update thread: ${err instanceof Error ? err.message : "Unknown"}`,
);
}
};
const onCreate = async () => {
if (!apiKey) return;
const name = createName.trim();
if (!name) {
toast.error("Container name is required");
return;
}
setCreating(true);
try {
const created = await createOpenAIContainer(
{ apiKey, baseUrl: provider.baseUrl || null },
{ name, ttlMinutes: createTtl },
);
toast.success(`Created container ${name}`);
setCreateName("");
setCreateOpen(false);
// Auto-bind the just-created container to the active thread.
// ensureThreadRecord first so the bind lands even when the user
// creates a container before sending the first message — without
// it, db.threads.update silently affects 0 rows and the chat
// adapter falls back to cross-thread inheritance / lazy-create,
// which can pick a stale container that fails with "container
// does not exist" on the first turn.
if (activeThreadId) {
try {
await ensureThreadRecord({
threadId: activeThreadId,
modelType: "base",
});
await db.threads.update(activeThreadId, {
openaiCodeExecContainerId: created.id,
});
} catch {
/* best-effort; toast above already confirmed creation */
}
}
} catch (err) {
toast.error(
`Create failed: ${err instanceof Error ? err.message : "Unknown"}`,
);
} finally {
setCreating(false);
// Refresh even on failure: the request may have partially succeeded
// server-side (created container, lost response), and a re-fetch
// keeps the picker in sync with OpenAI's actual state.
await refresh();
}
};
const confirmDelete = async () => {
if (!apiKey || !pendingDelete) return;
const { id, name } = pendingDelete;
setDeleting(true);
try {
await deleteOpenAIContainer(
{ apiKey, baseUrl: provider.baseUrl || null },
id,
);
// Tombstone the id so the picker hides it immediately even if
// OpenAI's list keeps returning it for a while.
setTombstones((prev) => {
if (prev.has(id)) return prev;
const next = new Set(prev);
next.add(id);
return next;
});
// Clear any thread bindings pointing at the now-deleted id.
const affected = await db.threads
.filter((t) => t.openaiCodeExecContainerId === id)
.toArray();
await Promise.all(
affected.map((t) =>
db.threads.update(t.id, { openaiCodeExecContainerId: null }),
),
);
toast.success(`Deleted container ${name || id}`);
} catch (err) {
toast.error(
`Delete failed: ${err instanceof Error ? err.message : "Unknown"}`,
);
} finally {
setDeleting(false);
setPendingDelete(null);
// Always refresh so a stale list entry (e.g. container deleted
// elsewhere, or already expired) is purged from the UI even when
// the delete call itself errored.
await refresh();
}
};
const displayActiveId = displayedContainerId;
return (
<div className="flex flex-col gap-3 pt-1">
{/* TTL */}
<div className="flex items-center justify-between gap-3">
<label
htmlFor="openai-container-ttl"
className="min-w-0 text-[13px] font-medium leading-[1.25] tracking-nav text-nav-fg"
>
New-container idle timeout (min, max 20)
</label>
<Input
id="openai-container-ttl"
type="number"
min={TTL_MIN}
max={TTL_MAX}
value={ttlValue}
onChange={(e) => onTtlChange(e.target.value)}
className="h-8 w-14 px-2 text-center text-sm tabular-nums"
/>
</div>
{/* Single container list. The previously-separate "Active for
this thread" picker collapses into this list: clicking a row
binds it to the active thread, and the ACTIVE pill marks
which one. Avoids duplicating state across two controls. */}
<div className="flex flex-col gap-1.5">
<div className="flex items-center justify-between gap-2">
<span className="text-[11px] uppercase tracking-wider text-muted-foreground">
Containers
</span>
<Button
size="sm"
variant="ghost"
className="-mr-1 h-6 w-6 p-0 text-muted-foreground"
onClick={() => void refresh()}
disabled={isLoading || !apiKey}
aria-label="Refresh container list"
>
<RefreshCwIcon
className={`size-3.5 ${isLoading ? "animate-spin" : ""}`}
/>
</Button>
</div>
{/* When no containers exist yet, render a disabled placeholder
instead of the picker. The first one is created by the
chat-adapter on first send (lazy-create) and will appear
here after the next refresh. */}
{sortedContainers.length === 0 ? (
<div className="h-9 w-full rounded-md border border-primary/40 bg-background px-2 flex items-center text-sm text-muted-foreground">
(none yet — will be created on first send)
</div>
) : (
<select
value={displayedContainerId ?? sortedContainers[0].id}
onChange={(e) => onPick(e.target.value)}
disabled={!activeThreadId}
className="h-9 w-full rounded-md border border-primary/40 bg-background px-2 text-sm font-medium shadow-sm focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-primary/40"
>
{sortedContainers.map((c) => (
<option key={c.id} value={c.id}>
{c.name ?? "(unnamed)"} · {c.id.slice(0, 14)}…
{c.lastActiveAt ? ` · active ${ageLabel(c.lastActiveAt)}` : ""}
</option>
))}
</select>
)}
</div>
{/* Container list with delete actions — labeled and visually
quieter so it's clearly the "all containers, manage them"
area rather than the active selector above. */}
<div className="flex flex-col gap-1.5">
<span className="text-[11px] uppercase tracking-wider text-muted-foreground">
All containers
</span>
{isLoading && visibleContainers.length === 0 ? (
<Skeleton className="h-16 w-full" />
) : sortedContainers.length > 0 ? (
<ul className="flex max-h-52 flex-col gap-1 overflow-auto">
{sortedContainers.map((c) => {
const running = isContainerRunning(c);
const isActive = running && c.id === displayActiveId;
const ttlMinutes = c.expiresAfterMinutes ?? DEFAULT_TTL_MINUTES;
const canActivate =
activeThreadId != null && !isActive && running;
const statusLabel = !running ? (c.status ?? "expired") : null;
return (
<li
key={c.id}
className={`flex items-center gap-2 rounded-md border px-2 py-1.5 text-xs transition-colors ${
isActive
? "border-primary/30 bg-primary/5"
: "border-border/60 hover:bg-muted/40"
} ${canActivate ? "cursor-pointer" : ""} ${
running ? "" : "opacity-60"
}`}
onClick={() => {
if (canActivate) void onPick(c.id);
}}
onKeyDown={(e) => {
if (!canActivate) return;
if (e.key === "Enter" || e.key === " ") {
e.preventDefault();
void onPick(c.id);
}
}}
tabIndex={canActivate ? 0 : undefined}
role={canActivate ? "button" : undefined}
aria-pressed={isActive}
title={
canActivate
? "Use this container for the active thread"
: !running
? `Container is ${statusLabel}`
: undefined
}
>
{/* min-w-0 + truncate keeps long OpenAI container ids
from spilling under the trash button on narrow
settings sheets. */}
<div className="flex min-w-0 flex-1 flex-col gap-0.5">
<div className="flex min-w-0 items-center gap-1.5">
<span className="min-w-0 truncate font-medium">
{c.name ?? "(unnamed)"}
</span>
{isActive ? (
<span className="shrink-0 rounded-sm bg-primary/15 px-1 py-px text-[9px] font-medium uppercase tracking-wider text-primary">
Active
</span>
) : statusLabel ? (
<span className="shrink-0 rounded-sm bg-muted px-1 py-px text-[9px] font-medium uppercase tracking-wider text-muted-foreground">
{statusLabel}
</span>
) : null}
</div>
<div
className="flex min-w-0 items-center gap-1.5 text-muted-foreground"
title={c.id}
>
<span className="min-w-0 truncate font-mono text-[11px]">
{shortContainerId(c.id)}
</span>
<span className="shrink-0 text-[10px] uppercase tracking-wider">
· {ttlMinutes}m
</span>
</div>
</div>
<Button
size="sm"
variant="ghost"
className="h-6 w-6 shrink-0 p-0 text-muted-foreground hover:text-destructive"
onClick={(e) => {
e.stopPropagation();
setPendingDelete(c);
}}
aria-label={`Delete container ${c.name ?? c.id}`}
>
<TrashIcon className="size-3.5" />
</Button>
</li>
);
})}
</ul>
) : (
<p className="text-xs text-muted-foreground">
None yet — one will be created on first send.
</p>
)}
</div>
{/* Create new */}
{createOpen ? (
<div className="flex flex-col gap-2 rounded-md border border-border/60 p-2">
<Input
placeholder="Container name (e.g. data-analysis)"
value={createName}
onChange={(e) => setCreateName(e.target.value)}
className="h-8 text-sm"
/>
<div className="flex items-center gap-2">
<Input
type="number"
min={TTL_MIN}
max={TTL_MAX}
value={createTtl}
onChange={(e) => {
const n = parseInt(e.target.value, 10);
if (!Number.isNaN(n))
setCreateTtl(Math.min(Math.max(n, TTL_MIN), TTL_MAX));
}}
className="h-8 w-24 text-sm"
aria-label="Idle timeout in minutes"
/>
<span className="text-xs text-muted-foreground">min idle</span>
<div className="flex-1" />
<Button
size="sm"
variant="ghost"
className="h-7"
onClick={() => {
setCreateOpen(false);
setCreateName("");
}}
disabled={creating}
>
Cancel
</Button>
<Button
size="sm"
className="h-7"
onClick={() => void onCreate()}
disabled={creating || !createName.trim() || !apiKey}
>
Create
</Button>
</div>
</div>
) : (
<Button
size="sm"
variant="outline"
className="h-8"
onClick={() => {
setCreateTtl(ttlValue);
setCreateOpen(true);
}}
disabled={!apiKey}
>
<PlusIcon className="size-3.5 mr-1" />
New container
</Button>
)}
<AlertDialog
open={pendingDelete !== null}
onOpenChange={(nextOpen) => {
if (!nextOpen && deleting) return;
if (!nextOpen) setPendingDelete(null);
}}
>
<AlertDialogContent size="sm">
<AlertDialogHeader>
<AlertDialogTitle>
Delete{" "}
<span className="font-mono">
{pendingDelete?.name ?? "container"}
</span>
?
</AlertDialogTitle>
<AlertDialogDescription>
Threads using this container will fall back to auto-create on
their next turn. This cannot be undone.
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel disabled={deleting}>Cancel</AlertDialogCancel>
<AlertDialogAction
variant="destructive"
disabled={deleting}
onClick={(e) => {
e.preventDefault();
void confirmDelete();
}}
>
{deleting ? "Deleting…" : "Delete"}
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</div>
);
}

387
studio/frontend/src/features/chat/external-providers.ts Normal file
View file

@ -0,0 +1,387 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
export interface ExternalProviderConfig {
id: string;
/** Backend provider type (e.g. openai, mistral, gemini). */
providerType: string;
/** Display name in UI. */
name: string;
/** Provider base URL (default from registry or backend-saved override). */
baseUrl: string;
/** Model ids user enabled from `/api/providers/models`. */
models: string[];
/** Cached available model ids from the provider's /models response. */
availableModels?: string[];
/** Whether to ask supported hosted providers to use prompt caching. */
enablePromptCaching?: boolean;
/** User-pinned: the loaded vLLM model supports `enable_thinking`. */
isReasoningModel?: boolean;
/**
* Default idle-timeout (in minutes) for newly created OpenAI shell
* containers. Pre-fills the "Create container" dialog and is the
* TTL the auto-create-per-thread path POSTs to /v1/containers with.
* OpenAI's hard default is 20. Only meaningful for OpenAI cloud.
*/
openaiContainerTtlMinutes?: number;
createdAt: number;
updatedAt: number;
}
const PROMPT_CACHING_PROVIDER_TYPES = new Set(["openai", "anthropic"]);
export function supportsProviderPromptCaching(
providerType: string | null | undefined,
): boolean {
return providerType != null && PROMPT_CACHING_PROVIDER_TYPES.has(providerType);
}
// Provider types that expose the connection-level "reasoning model"
// toggle. vLLM's OpenAI-compat endpoint doesn't advertise this per model.
const REASONING_TOGGLE_PROVIDER_TYPES = new Set(["vllm"]);
export function supportsProviderReasoningToggle(
providerType: string | null | undefined,
): boolean {
return (
providerType != null && REASONING_TOGGLE_PROVIDER_TYPES.has(providerType)
);
}
export const CUSTOM_BACKEND_PROVIDER_TYPE = "openai";
export const LEGACY_CUSTOM_PROVIDER_TYPE = "custom";
export const CUSTOM_PROVIDER_PRESETS = [
{
providerType: "llama_cpp",
displayName: "llama.cpp",
baseUrlPlaceholder: "http://localhost:8080/v1",
modelIdsPlaceholder: "gpt-oss-20b\nqwen3-14b",
},
{
providerType: "vllm",
displayName: "vLLM",
baseUrlPlaceholder: "https://my-vllm-server.com/v1",
modelIdsPlaceholder: "openai/gpt-oss-20b\nQwen/Qwen3-14B",
},
{
providerType: "ollama",
displayName: "Ollama",
baseUrlPlaceholder: "http://localhost:11434/v1",
modelIdsPlaceholder: "gpt-oss:20b\nqwen3:14b",
},
] as const;
const CUSTOM_PROVIDER_LABELS: Record<string, string> = {
[LEGACY_CUSTOM_PROVIDER_TYPE]: "Custom",
...Object.fromEntries(
CUSTOM_PROVIDER_PRESETS.map((preset) => [
preset.providerType,
preset.displayName,
]),
),
};
const CUSTOM_PROVIDER_BASE_URL_PLACEHOLDERS: Record<string, string> = {
[LEGACY_CUSTOM_PROVIDER_TYPE]: "https://my-vllm-server.com/v1",
...Object.fromEntries(
CUSTOM_PROVIDER_PRESETS.map((preset) => [
preset.providerType,
preset.baseUrlPlaceholder,
]),
),
};
const CUSTOM_PROVIDER_MODEL_IDS_PLACEHOLDERS: Record<string, string> = {
[LEGACY_CUSTOM_PROVIDER_TYPE]: "openai/gpt-oss-20b\nQwen/Qwen3-14B",
...Object.fromEntries(
CUSTOM_PROVIDER_PRESETS.map((preset) => [
preset.providerType,
preset.modelIdsPlaceholder,
]),
),
};
export function isCustomProviderType(
providerType: string | null | undefined,
): boolean {
if (!providerType) return false;
return providerType in CUSTOM_PROVIDER_LABELS;
}
export function customProviderDisplayName(
providerType: string | null | undefined,
): string {
if (!providerType) return "Custom";
return CUSTOM_PROVIDER_LABELS[providerType] ?? providerType;
}
export function customProviderBaseUrlPlaceholder(
providerType: string | null | undefined,
): string {
if (!providerType) {
return CUSTOM_PROVIDER_BASE_URL_PLACEHOLDERS[LEGACY_CUSTOM_PROVIDER_TYPE];
}
return (
CUSTOM_PROVIDER_BASE_URL_PLACEHOLDERS[providerType] ??
CUSTOM_PROVIDER_BASE_URL_PLACEHOLDERS[LEGACY_CUSTOM_PROVIDER_TYPE]
);
}
export function customProviderModelIdsPlaceholder(
providerType: string | null | undefined,
): string {
if (!providerType) {
return CUSTOM_PROVIDER_MODEL_IDS_PLACEHOLDERS[LEGACY_CUSTOM_PROVIDER_TYPE];
}
return (
CUSTOM_PROVIDER_MODEL_IDS_PLACEHOLDERS[providerType] ??
CUSTOM_PROVIDER_MODEL_IDS_PLACEHOLDERS[LEGACY_CUSTOM_PROVIDER_TYPE]
);
}
export function toExternalBackendProviderType(providerType: string): string;
export function toExternalBackendProviderType(
providerType: null | undefined,
): undefined;
export function toExternalBackendProviderType(
providerType: string | null | undefined,
): string | undefined;
export function toExternalBackendProviderType(
providerType: string | null | undefined,
): string | undefined {
if (!providerType) return undefined;
// vLLM's /v1/responses applies the loaded model's chat template, which
// 400s on strict-alternation templates (e.g. Gemma 3). Pass the actual
// type through so the backend routes vLLM to /v1/chat/completions instead
// of the OpenAI Responses path used for gpt-5.x.
if (providerType === "vllm") return "vllm";
return isCustomProviderType(providerType)
? CUSTOM_BACKEND_PROVIDER_TYPE
: providerType;
}
const EXTERNAL_PROVIDERS_KEY = "unsloth_chat_external_providers";
const EXTERNAL_PROVIDER_KEYS_KEY = "unsloth_chat_external_provider_keys";
const EXTERNAL_MODEL_PREFIX = "external::";
function canUseStorage(): boolean {
return typeof window !== "undefined";
}
export function isExternalModelId(
value: string | null | undefined,
): value is string {
return typeof value === "string" && value.startsWith(EXTERNAL_MODEL_PREFIX);
}
export function buildExternalModelId(providerId: string, modelId: string): string {
return `${EXTERNAL_MODEL_PREFIX}${providerId}::${encodeURIComponent(modelId)}`;
}
export function parseExternalModelId(
value: string | null | undefined,
): { providerId: string; modelId: string } | null {
if (!isExternalModelId(value)) return null;
const payload = value.slice(EXTERNAL_MODEL_PREFIX.length);
const separator = payload.indexOf("::");
if (separator < 0) return null;
const providerId = payload.slice(0, separator);
const encodedModelId = payload.slice(separator + 2);
if (!providerId || !encodedModelId) return null;
try {
return { providerId, modelId: decodeURIComponent(encodedModelId) };
} catch {
return null;
}
}
function isExternalProviderConfig(value: unknown): value is ExternalProviderConfig {
if (!value || typeof value !== "object") return false;
const maybe = value as Partial<ExternalProviderConfig>;
return (
typeof maybe.id === "string" &&
typeof maybe.providerType === "string" &&
typeof maybe.name === "string" &&
typeof maybe.baseUrl === "string" &&
Array.isArray(maybe.models)
);
}
function mapLegacyPresetToProviderType(presetId: string): string {
if (presetId === "google") return "gemini";
return presetId;
}
function normalizeProvider(raw: ExternalProviderConfig): ExternalProviderConfig {
const providerType = raw.providerType.trim();
return {
...raw,
providerType,
name: raw.name.trim(),
baseUrl: raw.baseUrl.trim(),
models: raw.models
.map((model) => model.trim())
.filter((model) => model.length > 0),
availableModels: (raw.availableModels ?? [])
.map((model) => model.trim())
.filter((model) => model.length > 0),
enablePromptCaching: supportsProviderPromptCaching(providerType)
? raw.enablePromptCaching !== false
: undefined,
isReasoningModel: supportsProviderReasoningToggle(providerType)
? raw.isReasoningModel === true
: undefined,
openaiContainerTtlMinutes:
providerType === "openai" &&
typeof raw.openaiContainerTtlMinutes === "number" &&
raw.openaiContainerTtlMinutes >= 1
? Math.min(raw.openaiContainerTtlMinutes, 20)
: undefined,
};
}
function isCompleteProvider(provider: ExternalProviderConfig): boolean {
if (!provider.id || !provider.name || !provider.providerType) return false;
return true;
}
type LegacyProviderConfig = {
id?: unknown;
presetId?: unknown;
name?: unknown;
baseUrl?: unknown;
models?: unknown;
createdAt?: unknown;
updatedAt?: unknown;
};
function fromUnknownProvider(value: unknown): ExternalProviderConfig | null {
if (!value || typeof value !== "object") return null;
if (isExternalProviderConfig(value)) {
return value;
}
const legacy = value as LegacyProviderConfig;
const id = typeof legacy.id === "string" ? legacy.id : "";
const presetId = typeof legacy.presetId === "string" ? legacy.presetId : "";
if (!id || !presetId || presetId === "custom") return null;
const providerType = mapLegacyPresetToProviderType(presetId);
if (!providerType) return null;
return {
id,
providerType,
name: typeof legacy.name === "string" ? legacy.name : providerType,
baseUrl: typeof legacy.baseUrl === "string" ? legacy.baseUrl : "",
models: Array.isArray(legacy.models)
? legacy.models.filter((item): item is string => typeof item === "string")
: [],
createdAt: typeof legacy.createdAt === "number" ? legacy.createdAt : Date.now(),
updatedAt: typeof legacy.updatedAt === "number" ? legacy.updatedAt : Date.now(),
};
}
export function loadExternalProviders(): ExternalProviderConfig[] {
if (!canUseStorage()) return [];
try {
const raw = localStorage.getItem(EXTERNAL_PROVIDERS_KEY);
if (!raw) return [];
const parsed = JSON.parse(raw) as unknown;
if (!Array.isArray(parsed)) return [];
return parsed
.map(fromUnknownProvider)
.filter((provider): provider is ExternalProviderConfig => provider !== null)
.map(normalizeProvider)
.filter(isCompleteProvider);
} catch {
return [];
}
}
/**
* Load the raw (encrypted or legacy plaintext) key map from localStorage.
* Values are opaque strings — either AES-GCM ciphertext or legacy plaintext.
*/
function loadRawKeyMap(): Record<string, string> {
if (!canUseStorage()) return {};
try {
const raw = localStorage.getItem(EXTERNAL_PROVIDER_KEYS_KEY);
if (!raw) return {};
const parsed = JSON.parse(raw) as unknown;
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) return {};
const out: Record<string, string> = {};
for (const [providerId, value] of Object.entries(parsed)) {
if (typeof providerId === "string" && typeof value === "string") {
out[providerId] = value;
}
}
return out;
} catch {
return {};
}
}
function saveRawKeyMap(map: Record<string, string>): void {
if (!canUseStorage()) return;
try {
localStorage.setItem(EXTERNAL_PROVIDER_KEYS_KEY, JSON.stringify(map));
} catch {
// ignore
}
}
export function saveExternalProviders(
providers: ExternalProviderConfig[],
): void {
if (!canUseStorage()) return;
try {
localStorage.setItem(EXTERNAL_PROVIDERS_KEY, JSON.stringify(providers));
// Prune keys for removed providers — works on raw ciphertext, no decryption needed
const allowedIds = new Set(providers.map((provider) => provider.id));
const keys = loadRawKeyMap();
const pruned: Record<string, string> = {};
for (const [providerId, value] of Object.entries(keys)) {
if (allowedIds.has(providerId)) {
pruned[providerId] = value;
}
}
saveRawKeyMap(pruned);
} catch {
// ignore
}
}
/**
* Retrieve a provider API key from localStorage.
* Returns "" if no key is stored.
*/
export function getExternalProviderApiKey(
providerId: string,
): string {
const keys = loadRawKeyMap();
return keys[providerId] ?? "";
}
/**
* Store a provider API key in localStorage.
*/
export function setExternalProviderApiKey(
providerId: string,
apiKey: string,
): void {
if (!canUseStorage()) return;
const keys = loadRawKeyMap();
keys[providerId] = apiKey;
saveRawKeyMap(keys);
}
export function removeExternalProviderApiKey(providerId: string): void {
if (!canUseStorage()) return;
try {
const keys = loadRawKeyMap();
delete keys[providerId];
saveRawKeyMap(keys);
} catch {
// ignore
}
}

61
studio/frontend/src/features/chat/hooks/use-chat-model-runtime.ts
View file

@ -23,7 +23,10 @@ import {
validateModel,
} from "../api/chat-api";
import { formatEta, formatRate } from "../utils/format-transfer";
import { useChatRuntimeStore } from "../stores/chat-runtime-store";
import {
type ReasoningEffort,
useChatRuntimeStore,
} from "../stores/chat-runtime-store";
import {
mergeBackendRecommendedInference,
resolveLoadMaxSeqLength,
@ -31,6 +34,7 @@ import {
import {
isMultimodalResponse,
} from "../types/api";
import { isExternalModelId } from "../external-providers";
import type {
ChatLoraSummary,
ChatModelSummary,
@ -143,6 +147,15 @@ function normalizeSpeculativeType(v: string | null | undefined): string | null {
return "default";
}
type LocalReasoningEffort = Extract<ReasoningEffort, "low" | "medium" | "high">;
function clampLocalReasoningEffort(value: ReasoningEffort): LocalReasoningEffort {
if (value === "low" || value === "medium" || value === "high") {
return value;
}
return "low";
}
export function useChatModelRuntime() {
const params = useChatRuntimeStore((state) => state.params);
const models = useChatRuntimeStore((state) => state.models);
@ -221,7 +234,9 @@ export function useChatModelRuntime() {
setModels(listRes.models.map(toChatModelSummary));
setLoras(lorasRes.loras.map(toLoraSummary));
if (statusRes.active_model) {
const selectedCheckpoint = useChatRuntimeStore.getState().params.checkpoint;
const isExternalSelectionActive = isExternalModelId(selectedCheckpoint);
if (statusRes.active_model && !isExternalSelectionActive) {
setCheckpoint(statusRes.active_model, statusRes.gguf_variant);
// Apply inference defaults on reconnect (page refresh with model already loaded)
@ -241,6 +256,10 @@ export function useChatModelRuntime() {
const supportsReasoning = statusRes.supports_reasoning ?? false;
const reasoningAlwaysOn = statusRes.reasoning_always_on ?? false;
const reasoningStyle = statusRes.reasoning_style ?? "enable_thinking";
const reasoningEffortLevels =
reasoningStyle === "reasoning_effort"
? (["low", "medium", "high"] as const)
: (["low", "medium", "high"] as const);
const supportsPreserveThinking = statusRes.supports_preserve_thinking ?? false;
const supportsTools = statusRes.supports_tools ?? false;
const currentGgufContextLength = statusRes.is_gguf
@ -262,6 +281,9 @@ export function useChatModelRuntime() {
// Otherwise we'd clobber the values the load path just applied and
// the UI would appear to revert the user's changes.
const prevState = useChatRuntimeStore.getState();
const clampedReasoningEffort = clampLocalReasoningEffort(
prevState.reasoningEffort,
);
const nextDefaultChatTemplate =
statusRes.chat_template === undefined
? prevState.defaultChatTemplate
@ -270,12 +292,25 @@ export function useChatModelRuntime() {
supportsReasoning,
reasoningAlwaysOn,
reasoningStyle,
supportsReasoningOff: reasoningStyle !== "reasoning_effort",
reasoningEffortLevels,
reasoningEffort: clampedReasoningEffort,
supportsPreserveThinking,
supportsTools,
// Reset per-turn reasoning flag so models that do not support
// reasoning do not inherit a stale off state from a prior model.
// Reset per-turn reasoning flag so:
// 1. models that do not support reasoning do not inherit a stale
// off state from a prior model, and
// 2. local reasoning-effort models (where the composer hides
// the Off option via supportsReasoningOff=false) cannot end
// up with reasoningEnabled=false carried over from an
// external model where Off was selected — the composer would
// keep showing "Think: <level>" via effectiveReasoningEnabled,
// but the chat-adapter would omit the kwarg and the Harmony
// template would fall back to its own default effort.
reasoningEnabled: supportsReasoning
? useChatRuntimeStore.getState().reasoningEnabled
? reasoningStyle === "reasoning_effort"
? true
: useChatRuntimeStore.getState().reasoningEnabled
: true,
ggufContextLength: currentGgufContextLength,
ggufMaxContextLength,
@ -313,7 +348,7 @@ export function useChatModelRuntime() {
}
useChatRuntimeStore.getState().setReasoningEnabled(reasoningDefault);
}
} else {
} else if (!statusRes.active_model && !isExternalSelectionActive) {
useChatRuntimeStore.setState({
modelRequiresTrustRemoteCode: false,
loadedIsMultimodal: false,
@ -569,6 +604,15 @@ export function useChatModelRuntime() {
// context state and display the backend-reported effective context.
const keepCustomCtx = null;
const reasoningAlwaysOn = loadResponse.reasoning_always_on ?? false;
const reasoningStyle = loadResponse.reasoning_style ?? "enable_thinking";
const reasoningEffortLevels =
reasoningStyle === "reasoning_effort"
? (["low", "medium", "high"] as const)
: (["low", "medium", "high"] as const);
const existingReasoningEffort = useChatRuntimeStore.getState().reasoningEffort;
const clampedReasoningEffort = clampLocalReasoningEffort(
existingReasoningEffort,
);
const ggufMaxContextLength = reportedMaxCtx;
useChatRuntimeStore.setState({
ggufContextLength: nativeCtx,
@ -579,7 +623,10 @@ export function useChatModelRuntime() {
supportsReasoning: loadResponse.supports_reasoning ?? false,
reasoningAlwaysOn,
reasoningEnabled: reasoningAlwaysOn ? true : reasoningDefault,
reasoningStyle: loadResponse.reasoning_style ?? "enable_thinking",
reasoningStyle,
supportsReasoningOff: reasoningStyle !== "reasoning_effort",
reasoningEffortLevels,
reasoningEffort: clampedReasoningEffort,
supportsPreserveThinking: loadResponse.supports_preserve_thinking ?? false,
supportsTools: loadResponse.supports_tools ?? false,
toolsEnabled: loadResponse.supports_tools ?? false,

244
studio/frontend/src/features/chat/lib/friendly-names.ts Normal file
View file

@ -0,0 +1,244 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
/**
* Friendly default names for auto-created OpenAI shell containers.
* Used by the chat-adapter when the lazy-create path fires (Code pill
* on, no thread container yet, user has set a non-default TTL). The
* goal is a human-memorable label like "otter" or "harbor" instead of
* "chat-abc12345" — the user can still rename via the Studio-side
* alias map.
*
* The list is curated to:
* - Be unambiguous, non-offensive nouns from natural categories
* (animals, plants, geography, materials, weather).
* - Avoid technical / political / brand words that might read as
* odd in a chat UI.
* - Stay reasonably small so the bundle cost is negligible (~200
* entries × ~7 bytes ≈ 1.5 KB).
*
* Collisions are tolerated — the container's real unique key is its
* ``cntr_*`` id, not its name. A short random hex suffix is appended
* to make accidental same-name collisions visually distinct in the
* picker list.
*/
const WORDS = [
// animals
"otter",
"falcon",
"heron",
"lynx",
"marten",
"stoat",
"raven",
"magpie",
"salmon",
"trout",
"perch",
"tortoise",
"gecko",
"iguana",
"axolotl",
"narwhal",
"manatee",
"dolphin",
"porpoise",
"octopus",
"cuttlefish",
"nautilus",
"starfish",
"urchin",
"anemone",
"coral",
"puffin",
"kestrel",
"osprey",
"buzzard",
"kingfisher",
"robin",
"wren",
"finch",
"sparrow",
"thrush",
"siskin",
"warbler",
"tanager",
"oriole",
"hare",
"badger",
"weasel",
"ferret",
"polecat",
"civet",
"tapir",
"okapi",
"ibex",
"chamois",
// plants & trees
"alder",
"aspen",
"birch",
"cedar",
"cypress",
"elder",
"elm",
"fir",
"ginkgo",
"hawthorn",
"hazel",
"hemlock",
"holly",
"juniper",
"larch",
"linden",
"maple",
"oak",
"olive",
"pine",
"rowan",
"spruce",
"sycamore",
"willow",
"yew",
"thistle",
"fern",
"moss",
"ivy",
"clover",
"heather",
"lavender",
"rosemary",
"sage",
"thyme",
"myrtle",
"laurel",
"magnolia",
// geography / landscape
"harbor",
"atoll",
"lagoon",
"estuary",
"fjord",
"delta",
"isthmus",
"mesa",
"plateau",
"valley",
"ridge",
"summit",
"glade",
"meadow",
"moor",
"heath",
"tundra",
"savanna",
"prairie",
"steppe",
"bayou",
"marsh",
"fen",
"grotto",
"cavern",
"canyon",
"ravine",
"gorge",
"knoll",
"dell",
"vale",
"coast",
// materials / minerals / colors
"amber",
"agate",
"onyx",
"opal",
"jade",
"quartz",
"obsidian",
"basalt",
"granite",
"marble",
"slate",
"flint",
"lapis",
"topaz",
"garnet",
"pearl",
"coral",
"ivory",
"ebony",
"copper",
"cobalt",
"indigo",
"saffron",
"vermilion",
"ochre",
"umber",
"sienna",
"russet",
// weather / sky / time
"aurora",
"comet",
"ember",
"frost",
"gale",
"harvest",
"monsoon",
"nebula",
"solstice",
"twilight",
"zephyr",
"drizzle",
"tempest",
"halcyon",
"equinox",
"rainbow",
"horizon",
"meridian",
"zenith",
"comet",
// misc tactile / cozy nouns
"lantern",
"kettle",
"compass",
"anchor",
"beacon",
"harbor",
"voyage",
"trellis",
"cottage",
"thicket",
"orchard",
"bramble",
"haystack",
"snowfall",
"campfire",
];
/** RFC 4122-ish 4-character lowercase hex suffix using crypto.randomUUID. */
function randomHexSuffix(): string {
if (
typeof crypto !== "undefined" &&
typeof crypto.randomUUID === "function"
) {
return crypto.randomUUID().replace(/-/g, "").slice(0, 4);
}
// Older browser fallback. Math.random is fine here — this is a
// display suffix, not a security token.
return Math.floor(Math.random() * 0xffff)
.toString(16)
.padStart(4, "0");
}
/**
* Returns a single English-word name with a short random hex suffix.
*
* Example output: "kestrel-3f9c", "harbor-a012".
*
* The suffix keeps containers visually distinguishable in the picker
* when the same word recurs across creations.
*/
export function pickFriendlyContainerName(): string {
const word = WORDS[Math.floor(Math.random() * WORDS.length)] ?? "container";
return `${word}-${randomHexSuffix()}`;
}

598
studio/frontend/src/features/chat/provider-capabilities.ts Normal file
View file

@ -0,0 +1,598 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
/**
* Per-provider sampling parameter capability matrix.
*
* Values are derived from each provider's published chat-completion docs as of
* 2026-05. They describe which of our UI knobs map cleanly onto the provider's
* request body; the panel hides params a provider does not accept so users
* cannot dial a value that gets silently dropped or rejected.
*
* "Local" models (anything that is not an external provider) are represented by
* a null capability — every knob renders for them.
*/
export interface ProviderCapabilities {
/**
* Temperature sampling. Reasoning-class models (OpenAI's gpt-5.x / o3 via
* /v1/responses) reject this with `Unsupported parameter`.
*/
temperature: boolean;
/** Nucleus (top_p) sampling. Same restriction as `temperature` on OpenAI. */
topP: boolean;
/** top-k token sampling (only Anthropic on the providers we ship). */
topK: boolean;
/** min-p token cutoff (no SaaS provider currently exposes this). */
minP: boolean;
/** Repetition penalty (no SaaS provider currently exposes this). */
repetitionPenalty: boolean;
/** OpenAI-style presence penalty. */
presencePenalty: boolean;
}
export type ExternalReasoningCapabilities = {
supportsReasoning: boolean;
reasoningStyle: "enable_thinking" | "reasoning_effort";
reasoningAlwaysOn: boolean;
supportsReasoningOff: boolean;
reasoningEffortLevels: readonly (
| "none"
| "minimal"
| "low"
| "medium"
| "high"
| "max"
| "xhigh"
)[];
};
/**
* Prefer a stored reasoning effort level that exists in ``effortLevels``,
* mapping legacy "xhigh" to "max" when the model only exposes the latter
* (Claude 4.6 adaptive thinking).
*/
export function clampReasoningEffortToLevels(
preferred: ExternalReasoningCapabilities["reasoningEffortLevels"][number],
effortLevels: ExternalReasoningCapabilities["reasoningEffortLevels"],
): ExternalReasoningCapabilities["reasoningEffortLevels"][number] {
let candidate = preferred;
if (
candidate === "xhigh" &&
!effortLevels.includes("xhigh") &&
effortLevels.includes("max")
) {
candidate = "max";
}
if (effortLevels.includes(candidate)) {
return candidate;
}
return effortLevels[0] ?? "low";
}
/**
* Output-token cap for any external provider request. Picked to stay below the
* tightest declared limit across the providers we ship (Anthropic Claude Opus
* tops out at 128k, GPT-5.x ~128k, Gemini 2.5 ~65k, DeepSeek 8k) while staying
* well above what a typical chat reply needs. The local-model path is not
* subject to this — local backends honour whatever the loaded context allows.
*
* If a user's stored maxTokens (e.g. carried over from a prior local-model
* session with a 128k+ context) exceeds this, chat-adapter clamps the
* outbound request so the provider does not 400 on it.
*/
export const EXTERNAL_MAX_OUTPUT_TOKENS = 32768;
/**
* Whether the external provider offers a built-in web-search tool that the
* model invokes server-side. When `true`, the chat composer's Search button
* is available for that provider and the chat-adapter forwards
* `enable_tools: true, enabled_tools: ["web_search"]` on the request — the
* backend routes the call through the provider's tool schema:
* - OpenAI: `tools: [{type: "web_search"}]` on /v1/responses
* - Anthropic: `tools: [{type: "web_search_20250305", name: "web_search",
* max_uses: 5}]` on /v1/messages
* - OpenRouter: `plugins: [{id: "web"}]` on /v1/chat/completions (the
* router's universal web-search shape; works for every
* underlying model including the `openrouter/free` router).
* - Kimi: `tools: [{type: "builtin_function", function: {name:
* "$web_search"}}]` with `thinking: {type:
* "disabled"}`. Requires a client round-trip:
* the first call returns the search args; the backend
* echoes them back as a role=tool message; the second
* call streams the answer. Handled in
* _stream_kimi_web_search on the backend.
*
* Mistral is intentionally excluded: their `web_search` connector lives on
* the Agents API (`/v1/agents` + `/v1/conversations`), not chat completions,
* and returns `"WebSearchTool connector is not supported"` if injected into
* /v1/chat/completions. Wiring it would require a dedicated Agents streaming
* path. Gemini's grounded-search can be added with the same pattern when
* matching backend translation lands.
*/
export function providerSupportsBuiltinWebSearch(
providerType: string | null | undefined,
): boolean {
return (
providerType === "openai" ||
providerType === "anthropic" ||
providerType === "openrouter" ||
providerType === "kimi"
);
}
/**
* Whether the selected external provider/model exposes a server-side
* code-execution tool. Two providers ship one today:
*
* - **Anthropic** (`code_execution_20250825`): Python + bash +
* str_replace-based file edits inside a 5 GB sandboxed container
* per request. Documented at
* https://platform.claude.com/docs/en/agents-and-tools/tool-use/code-execution-tool
*
* - **OpenAI cloud** (`shell` on /v1/responses): bash inside a
* reusable container; we auto-create one on the first turn of a
* chat thread and reference it on subsequent turns via the
* thread's stored `openaiCodeExecContainerId`. Documented at
* https://developers.openai.com/api/docs/guides/tools-shell
*
* Returns false for every other provider. The backend additionally
* gates the OpenAI shell tool on `is_openai_cloud` so custom
* OpenAI-compat servers (ollama / llama.cpp / vLLM) that also report
* `provider_type="openai"` never receive the tool — but in practice
* none of those catalogs surface the `gpt-5.5` ids anyway, so the
* frontend prefix match is enough.
*
* v1 wires the tools themselves; file uploads (Anthropic
* `container_upload` / OpenAI `input_file`) are a deliberate follow-up.
*/
const ANTHROPIC_CODE_EXECUTION_MODEL_PREFIXES = [
"claude-opus-4-7",
"claude-opus-4-6",
"claude-sonnet-4-6",
"claude-opus-4-5",
"claude-sonnet-4-5",
"claude-haiku-4-5",
// Deprecated upstream but the registry still exposes the ids, so the
// pill should remain functional for users on those snapshots.
"claude-opus-4-1",
"claude-opus-4",
"claude-sonnet-4",
] as const;
// OpenAI cloud shell-tool gating. Docs only explicitly demonstrate
// gpt-5.5; gpt-5.5-pro is included because the family share the same
// /v1/responses contract. `gpt-5.5-pro` is checked first so the prefix
// match doesn't collide with a hypothetical `gpt-5.5-turbo` etc.
const OPENAI_CODE_EXECUTION_MODEL_PREFIXES = [
"gpt-5.5-pro",
"gpt-5.5",
] as const;
/**
* Strict check that a provider configuration points at OpenAI's
* managed cloud (api.openai.com), as opposed to a custom OpenAI-compat
* backend (ollama / llama.cpp / vLLM / generic "custom" preset). The
* shell tool ONLY exists on OpenAI cloud; sending it to anything else
* 400s the request. Mirror of the backend's
* `is_openai_cloud = "api.openai.com" in self.base_url` guard.
*/
function isOpenAICloudBaseUrl(baseUrl: string | null | undefined): boolean {
if (!baseUrl) return true; // No override → uses the default openai.com base.
return baseUrl.trim().toLowerCase().includes("api.openai.com");
}
export function providerSupportsBuiltinCodeExecution(
providerType: string | null | undefined,
modelId: string | null | undefined,
baseUrl?: string | null,
): boolean {
const normalized = modelId?.trim().toLowerCase() ?? "";
if (!normalized) return false;
if (providerType === "anthropic") {
return ANTHROPIC_CODE_EXECUTION_MODEL_PREFIXES.some((prefix) =>
normalized.startsWith(prefix),
);
}
if (providerType === "openai") {
if (!isOpenAICloudBaseUrl(baseUrl)) return false;
return OPENAI_CODE_EXECUTION_MODEL_PREFIXES.some((prefix) =>
normalized.startsWith(prefix),
);
}
return false;
}
/**
* Per-provider minimum on the outbound max_tokens. Kimi's docs require
* `max_tokens >= 16000` whenever a thinking model is in use so the
* reasoning_content and final answer both fit in the budget — anything
* lower truncates the response mid-stream. Other providers don't have a
* documented floor, so they fall through to the generic min of 64 in
* the slider.
*
* The chat-adapter resolves the effective floor on send and bumps the
* outbound max_tokens up to this value if the user's stored maxTokens
* sits below it. The settings panel reflects the same floor as the
* slider min so the displayed value never drifts from what's sent.
*/
const EXTERNAL_MIN_OUTPUT_TOKENS_BY_PROVIDER: Record<string, number> = {
kimi: 16000,
};
export function getExternalMinOutputTokens(
providerType: string | null | undefined,
): number {
if (!providerType) return 64;
return EXTERNAL_MIN_OUTPUT_TOKENS_BY_PROVIDER[providerType] ?? 64;
}
const OPENAI_COMPAT_BASE: ProviderCapabilities = {
temperature: true,
topP: true,
topK: false,
minP: false,
repetitionPenalty: false,
presencePenalty: true,
};
const ALL_SUPPORTED: ProviderCapabilities = {
temperature: true,
topP: true,
topK: true,
minP: true,
repetitionPenalty: true,
presencePenalty: true,
};
const PROVIDER_CAPABILITIES: Record<string, ProviderCapabilities> = {
// OpenAI's flagship models (gpt-5.x / o3 / gpt-4.5) are reasoning-class
// models served via /v1/responses, which rejects temperature, top_p, and
// presence/frequency penalty. See backend
// external_provider._stream_openai_responses for the proxy.
openai: {
temperature: false,
topP: false,
topK: false,
minP: false,
repetitionPenalty: false,
presencePenalty: false,
},
// Anthropic's Messages API accepts top_k on 3.x and 4.5/4.6, but Claude
// 4.7 (Opus/Sonnet/Haiku) deprecated it and returns 400 if it is set.
// We surface top_k in the panel for all Anthropic providers and let the
// backend strip it per-model — see _stream_anthropic in
// studio/backend/core/inference/external_provider.py.
// Presence/frequency penalty is not part of the Messages API on any
// Claude generation.
anthropic: {
temperature: true,
topP: true,
topK: true,
minP: false,
repetitionPenalty: false,
presencePenalty: false,
},
mistral: OPENAI_COMPAT_BASE,
gemini: OPENAI_COMPAT_BASE,
// Kimi k2.5/k2.6 are reasoning-class — the API locks temperature and
// top_p to fixed defaults and 400s on any other value:
// "invalid temperature: only 1 is allowed for this model".
// Hide both sliders so the user is not offered knobs the model
// silently overrides. Backend additionally strips these fields via
// PROVIDER_REGISTRY['kimi']['body_omit'].
kimi: {
temperature: false,
topP: false,
topK: false,
minP: false,
repetitionPenalty: false,
presencePenalty: true,
},
// DeepSeek deprecated presence/frequency penalty in their current docs.
deepseek: {
temperature: true,
topP: true,
topK: false,
minP: false,
repetitionPenalty: false,
presencePenalty: false,
},
qwen: OPENAI_COMPAT_BASE,
huggingface: OPENAI_COMPAT_BASE,
// OpenRouter silently drops params the target model does not support, so we
// surface every knob and let the gateway handle the per-model fan-out.
openrouter: ALL_SUPPORTED,
// Local OpenAI-compatible connections are proxied through the OpenAI backend
// path, but vLLM/Ollama/llama.cpp users often want top_k / min_p /
// repetition controls, so be permissive.
custom: ALL_SUPPORTED,
vllm: ALL_SUPPORTED,
ollama: ALL_SUPPORTED,
llama_cpp: ALL_SUPPORTED,
};
const DEFAULT_EXTERNAL_CAPABILITIES = OPENAI_COMPAT_BASE;
/**
* Resolve the capability set for an external provider. Returns `null` for
* a local model (i.e. when `providerType` is null/undefined), which callers
* should treat as "every knob applies".
*/
export function getProviderCapabilities(
providerType: string | null | undefined,
): ProviderCapabilities | null {
if (!providerType) return null;
return PROVIDER_CAPABILITIES[providerType] ?? DEFAULT_EXTERNAL_CAPABILITIES;
}
const DEFAULT_EFFORT_LEVELS = ["low", "medium", "high"] as const;
const OPENROUTER_MANDATORY_REASONING_MODELS = new Set([
"google/gemini-pro-latest",
"baidu/cobuddy:free",
"inclusionai/ring-2.6-1t:free",
"deepseek/deepseek-r1",
]);
function isOpenRouterMandatoryReasoningModel(modelId: string): boolean {
const normalized = modelId.trim().toLowerCase();
const canonical = normalized.startsWith("~") ? normalized.slice(1) : normalized;
return OPENROUTER_MANDATORY_REASONING_MODELS.has(canonical);
}
type ReasoningCaps = {
supportsReasoning: boolean;
supportsReasoningOff: boolean;
reasoningEffortLevels: ExternalReasoningCapabilities["reasoningEffortLevels"];
};
const DEFAULT_EXTERNAL_REASONING_CAPABILITIES: ExternalReasoningCapabilities = {
supportsReasoning: false,
reasoningStyle: "enable_thinking",
reasoningAlwaysOn: false,
supportsReasoningOff: false,
reasoningEffortLevels: DEFAULT_EFFORT_LEVELS,
};
const NO_REASONING_CAPS: ReasoningCaps = {
supportsReasoning: false,
supportsReasoningOff: false,
reasoningEffortLevels: DEFAULT_EFFORT_LEVELS,
};
const ANTHROPIC_REASONING_MODELS = [
{
prefixes: ["claude-opus-4-7"],
levels: ["none", "low", "medium", "high", "xhigh", "max"],
},
{
prefixes: ["claude-opus-4-6", "claude-sonnet-4-6"],
levels: ["none", "low", "medium", "high", "max"],
},
{
prefixes: ["claude-opus-4-5", "claude-sonnet-4-5", "claude-haiku-4-5"],
// Backend maps semantic levels to manual budget_tokens.
levels: ["none", "low", "medium", "high"],
},
] as const;
function matchesModelPrefix(
modelId: string,
prefixes: readonly string[],
): boolean {
return prefixes.some((prefix) => modelId.startsWith(prefix));
}
function resolveAnthropicReasoningEffortCapabilities(modelId: string): ReasoningCaps {
const normalized = modelId.trim().toLowerCase();
const matched = ANTHROPIC_REASONING_MODELS.find((entry) =>
matchesModelPrefix(normalized, entry.prefixes),
);
if (matched) {
return {
supportsReasoning: true,
supportsReasoningOff: true,
reasoningEffortLevels: matched.levels,
};
}
return NO_REASONING_CAPS;
}
const OPENAI_REASONING_MODELS = [
{
prefixes: ["gpt-5.5-pro", "gpt-5.4-pro"],
supportsOff: false,
levels: ["medium", "high", "xhigh"],
},
{
prefixes: ["gpt-5.5", "gpt-5.4"],
supportsOff: true,
levels: ["none", "low", "medium", "high", "xhigh"],
},
{
prefixes: ["gpt-5.3-chat-latest"],
supportsOff: false,
levels: ["medium"],
},
{
prefixes: ["gpt-5.3-codex"],
supportsOff: true,
levels: ["none", "low", "medium", "high", "xhigh"],
},
{
prefixes: ["gpt-5", "gpt-5.1", "gpt-5.2"],
supportsOff: false,
levels: ["minimal", "low", "medium", "high"],
},
{
prefixes: ["o3"],
supportsOff: false,
levels: DEFAULT_EFFORT_LEVELS,
},
] as const;
function resolveOpenAIReasoningEffortCapabilities(modelId: string): ReasoningCaps {
const normalized = modelId.trim().toLowerCase();
const matched = OPENAI_REASONING_MODELS.find((entry) =>
matchesModelPrefix(normalized, entry.prefixes),
);
if (matched) {
return {
supportsReasoning: true,
supportsReasoningOff: matched.supportsOff,
reasoningEffortLevels: matched.levels,
};
}
return NO_REASONING_CAPS;
}
function withEnableThinkingStyle(
overrides?: Partial<ExternalReasoningCapabilities>,
): ExternalReasoningCapabilities {
return {
...DEFAULT_EXTERNAL_REASONING_CAPABILITIES,
...overrides,
reasoningStyle: "enable_thinking",
};
}
function withReasoningEffortStyle(caps: ReasoningCaps): ExternalReasoningCapabilities {
return {
...DEFAULT_EXTERNAL_REASONING_CAPABILITIES,
supportsReasoning: true,
reasoningStyle: "reasoning_effort",
supportsReasoningOff: caps.supportsReasoningOff,
reasoningEffortLevels: caps.reasoningEffortLevels,
};
}
function resolveKimiReasoningCapabilities(modelId: string): ExternalReasoningCapabilities {
// Kimi exposes a boolean thinking toggle rather than an effort scale.
// - kimi-k2.6: thinking enabled by default, toggleable
// via extra_body: {thinking: {type: enabled|disabled}}
// - kimi-k2-thinking: thinking always on, no off switch
// - kimi-k2.5 (and anything else): no thinking
if (modelId === "kimi-k2-thinking") {
return withEnableThinkingStyle({
supportsReasoning: true,
reasoningAlwaysOn: true,
});
}
if (modelId === "kimi-k2.6") {
return withEnableThinkingStyle({
supportsReasoning: true,
supportsReasoningOff: true,
});
}
return withEnableThinkingStyle();
}
function resolveMistralReasoningCapabilities(modelId: string): ExternalReasoningCapabilities {
if (modelId === "magistral-medium-latest") {
return withReasoningEffortStyle({
supportsReasoning: true,
supportsReasoningOff: false,
// Native reasoning model: present baseline as Medium in the UI.
reasoningEffortLevels: ["medium", "high"] as const,
});
}
if (modelId === "mistral-small-latest" || modelId === "mistral-vibe-cli-latest") {
return withReasoningEffortStyle({
supportsReasoning: true,
supportsReasoningOff: true,
reasoningEffortLevels: ["none", "high"] as const,
});
}
return withEnableThinkingStyle();
}
export interface ExternalReasoningResolveOptions {
/** vLLM connection flagged as a reasoning model in provider config. */
isReasoningProvider?: boolean;
}
// vLLM has no per-model reasoning signal on OpenAI-compat — pin via user toggle.
function resolveConnectionLevelReasoning(
normalizedProvider: string,
options: ExternalReasoningResolveOptions | undefined,
): ExternalReasoningCapabilities | null {
if (normalizedProvider === "vllm" && options?.isReasoningProvider) {
return withEnableThinkingStyle({
supportsReasoning: true,
supportsReasoningOff: true,
});
}
return null;
}
/**
* resolve external-model thinking capabilities.
* provider-specific matching lives in the OpenAI/Anthropic resolvers.
* other providers default to no reasoning controls.
*/
export function getExternalReasoningCapabilities(
providerType: string | null | undefined,
modelId: string | null | undefined,
options?: ExternalReasoningResolveOptions,
): ExternalReasoningCapabilities {
const normalizedModel = modelId?.trim().toLowerCase() ?? "";
const normalizedProvider = providerType?.trim().toLowerCase() ?? "";
const connectionLevel = resolveConnectionLevelReasoning(
normalizedProvider,
options,
);
if (connectionLevel) {
return connectionLevel;
}
if (!normalizedModel) {
return withEnableThinkingStyle();
}
// Some OpenRouter-routed ids are mandatory-reasoning and must stay on even
// if they arrive through aliased/custom provider routes.
if (isOpenRouterMandatoryReasoningModel(normalizedModel)) {
return withEnableThinkingStyle({
supportsReasoning: true,
reasoningAlwaysOn: true,
supportsReasoningOff: false,
});
}
// OpenRouter ids are namespaced (e.g. "openai/gpt-5.5").
const modelForMatching =
normalizedProvider === "openrouter" && normalizedModel.includes("/")
? normalizedModel.split("/").at(-1) ?? normalizedModel
: normalizedModel;
const isOpenAIProvider = normalizedProvider === "openai";
const isAnthropicProvider = normalizedProvider === "anthropic";
const isKimiProvider = normalizedProvider === "kimi";
const isMistralProvider = normalizedProvider === "mistral";
const isOpenRouterProvider = normalizedProvider === "openrouter";
if (isOpenRouterProvider) {
// OpenRouter's unified `reasoning` parameter is accepted on every
// chat-completion request; the gateway silently no-ops for models
// that don't reason. Mandatory-reasoning ids are handled by the
// early guard above; everything else exposes a toggleable control.
return {
supportsReasoning: true,
reasoningStyle: "enable_thinking",
reasoningAlwaysOn: false,
supportsReasoningOff: true,
reasoningEffortLevels: DEFAULT_EFFORT_LEVELS,
};
}
if (isKimiProvider) return resolveKimiReasoningCapabilities(modelForMatching);
if (isMistralProvider) return resolveMistralReasoningCapabilities(modelForMatching);
if (!isOpenAIProvider && !isAnthropicProvider) {
return withEnableThinkingStyle();
}
const providerCaps = isOpenAIProvider
? resolveOpenAIReasoningEffortCapabilities(modelForMatching)
: resolveAnthropicReasoningEffortCapabilities(modelForMatching);
if (providerCaps.supportsReasoning) {
return withReasoningEffortStyle(providerCaps);
}
return withEnableThinkingStyle();
}

2
studio/frontend/src/features/chat/runtime-provider.tsx
View file

@ -396,7 +396,7 @@ function toThreadMessage(m: MessageRecord): ThreadMessage {
};
}
async function ensureThreadRecord({
export async function ensureThreadRecord({
threadId,
modelType,
pairId,

244
studio/frontend/src/features/chat/shared-composer.tsx
View file

@ -18,7 +18,16 @@ import { useAui } from "@assistant-ui/react";
import { ArrowUpIcon, GlobeIcon, HeadphonesIcon, LightbulbIcon, LightbulbOffIcon, MicIcon, PlusIcon, SquareIcon, XIcon } from "lucide-react";
import { toast } from "sonner";
import { loadModel, validateModel } from "./api/chat-api";
import { useChatRuntimeStore } from "./stores/chat-runtime-store";
import { parseExternalModelId } from "./external-providers";
import { useExternalProvidersStore } from "./stores/external-providers-store";
import {
type ReasoningEffort,
useChatRuntimeStore,
} from "./stores/chat-runtime-store";
import {
getExternalReasoningCapabilities,
providerSupportsBuiltinCodeExecution,
} from "./provider-capabilities";
import {
type CompositionEvent,
type KeyboardEvent,
@ -66,6 +75,33 @@ function fileToBase64DataURL(file: File): Promise<string> {
});
}
function formatReasoningEffortLabel(level: ReasoningEffort, modelId?: string): string {
if (level === "max") return "Max";
if (level === "xhigh") {
const normalized = modelId?.trim().toLowerCase() ?? "";
if (
normalized.startsWith("claude-opus-4-6") ||
normalized.startsWith("claude-sonnet-4-6")
) {
return "Max";
}
return "Extra High";
}
return level.charAt(0).toUpperCase() + level.slice(1);
}
function formatReasoningDisabledLabel(
supportsReasoningOff: boolean,
isExternalOpenAIReasoning: boolean,
modelId?: string,
): string {
const normalized = modelId?.trim().toLowerCase() ?? "";
// Magistral keeps the "none" wire value, but UX should present this floor
// as "Medium" rather than a disabled state label.
if (normalized.includes("magistral-medium-latest")) return "Medium";
return supportsReasoningOff && isExternalOpenAIReasoning ? "None" : "Off";
}
function useDictation(
setText: (value: string | ((prev: string) => string)) => void,
) {
@ -253,6 +289,8 @@ export function SharedComposer({
const checkpoint = s.params.checkpoint;
return s.models.find((m) => m.id === checkpoint);
});
const checkpoint = useChatRuntimeStore((s) => s.params.checkpoint);
const externalProviders = useExternalProvidersStore((s) => s.providers);
const modelLoaded = useChatRuntimeStore(
(s) => !!s.params.checkpoint && !s.modelLoading,
);
@ -262,17 +300,94 @@ export function SharedComposer({
const setReasoningEnabled = useChatRuntimeStore((s) => s.setReasoningEnabled);
const reasoningStyle = useChatRuntimeStore((s) => s.reasoningStyle);
const reasoningEffort = useChatRuntimeStore((s) => s.reasoningEffort);
const supportsReasoningOff = useChatRuntimeStore((s) => s.supportsReasoningOff);
const reasoningEffortLevels = useChatRuntimeStore((s) => s.reasoningEffortLevels);
const setReasoningEffort = useChatRuntimeStore((s) => s.setReasoningEffort);
const supportsPreserveThinking = useChatRuntimeStore((s) => s.supportsPreserveThinking);
const preserveThinking = useChatRuntimeStore((s) => s.preserveThinking);
const setPreserveThinking = useChatRuntimeStore((s) => s.setPreserveThinking);
const supportsTools = useChatRuntimeStore((s) => s.supportsTools);
const supportsBuiltinWebSearch = useChatRuntimeStore(
(s) => s.supportsBuiltinWebSearch,
);
const toolsEnabled = useChatRuntimeStore((s) => s.toolsEnabled);
const setToolsEnabled = useChatRuntimeStore((s) => s.setToolsEnabled);
const codeToolsEnabled = useChatRuntimeStore((s) => s.codeToolsEnabled);
const setCodeToolsEnabled = useChatRuntimeStore((s) => s.setCodeToolsEnabled);
const reasoningDisabled = !modelLoaded || !supportsReasoning;
const toolsDisabled = !modelLoaded || !supportsTools;
const lastOpenRouterChosenModel = useChatRuntimeStore(
(s) => s.lastOpenRouterChosenModel,
);
const externalSelection = parseExternalModelId(checkpoint);
const selectedExternalProvider =
externalSelection != null
? externalProviders.find((p) => p.id === externalSelection.providerId)
: undefined;
const effectiveExternalModelId =
selectedExternalProvider?.providerType === "openrouter" &&
externalSelection?.modelId === "openrouter/free" &&
lastOpenRouterChosenModel
? lastOpenRouterChosenModel
: externalSelection?.modelId;
const externalReasoningCaps =
externalSelection != null
? getExternalReasoningCapabilities(
selectedExternalProvider?.providerType,
effectiveExternalModelId,
{
isReasoningProvider:
selectedExternalProvider?.isReasoningModel === true,
},
)
: null;
const isExternalOpenAIReasoning =
externalReasoningCaps?.supportsReasoning === true &&
externalReasoningCaps.reasoningStyle === "reasoning_effort";
const effectiveReasoningStyle =
externalReasoningCaps?.reasoningStyle ?? reasoningStyle;
const effectiveReasoningAlwaysOn =
externalReasoningCaps?.reasoningAlwaysOn ?? reasoningAlwaysOn;
const effectiveSupportsReasoningOff =
externalReasoningCaps?.supportsReasoningOff ?? supportsReasoningOff;
const effectiveReasoningEffortLevels =
externalReasoningCaps?.reasoningEffortLevels ?? reasoningEffortLevels;
const effectiveSupportsReasoning =
externalReasoningCaps?.supportsReasoning ?? supportsReasoning;
const reasoningLockedOn =
effectiveSupportsReasoning &&
(effectiveReasoningAlwaysOn || !effectiveSupportsReasoningOff);
// Kimi's $web_search builtin mandates thinking=disabled per the docs at
// https://platform.kimi.ai/docs/guide/use-web-search. Both pills stay
// clickable for Kimi, but turning one on flips the other off — the
// click handlers below enforce this mutual exclusion so the visible
// state always matches what the backend actually sends.
const isKimiExternal = selectedExternalProvider?.providerType === "kimi";
const effectiveReasoningEnabled = reasoningLockedOn ? true : reasoningEnabled;
const effectiveReasoningVisualEnabled =
effectiveReasoningEnabled && reasoningEffort !== "none";
const reasoningDisabled = !modelLoaded || !effectiveSupportsReasoning;
const showReasoningControl =
effectiveSupportsReasoning || effectiveReasoningAlwaysOn;
// Two-pill gating: Search pill lights up when the runtime has either
// a local tool runtime (supportsTools, gives us our Code/python + local
// web_search) OR a server-side web_search the provider runs for us
// (supportsBuiltinWebSearch, currently OpenAI / Anthropic / OpenRouter
// / Kimi). Code pill lights up on the local runtime OR when Anthropic
// is selected with a model that accepts the server-side
// code_execution_20250825 tool — see
// providerSupportsBuiltinCodeExecution. Anthropic is the only external
// provider that ships a code-execution tool today.
const supportsBuiltinCodeExecution = providerSupportsBuiltinCodeExecution(
selectedExternalProvider?.providerType,
effectiveExternalModelId,
selectedExternalProvider?.baseUrl,
);
const searchDisabled =
!modelLoaded || !(supportsTools || supportsBuiltinWebSearch);
const codeDisabled =
!modelLoaded || !(supportsTools || supportsBuiltinCodeExecution);
// Backwards-compatible alias for any other call site that may still
// reference `toolsDisabled` (rare; both pills used it before).
const toolsDisabled = codeDisabled;
const setPendingAudioStore = useChatRuntimeStore((s) => s.setPendingAudio);
const clearPendingAudioStore = useChatRuntimeStore((s) => s.clearPendingAudio);
@ -625,7 +740,8 @@ export function SharedComposer({
</TooltipIconButton>
</>
)}
{reasoningStyle === "reasoning_effort" ? (
{showReasoningControl ? (
effectiveReasoningStyle === "reasoning_effort" ? (
<DropdownMenu>
<DropdownMenuTrigger asChild={true}>
<button
@ -635,26 +751,66 @@ export function SharedComposer({
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
reasoningDisabled
? "cursor-not-allowed opacity-40"
: "bg-primary/10 text-primary hover:bg-primary/20",
: effectiveReasoningVisualEnabled
? "bg-primary/10 text-primary hover:bg-primary/20"
: "text-muted-foreground hover:bg-muted-foreground/15",
)}
aria-label={`Reasoning effort: ${reasoningEffort}`}
>
<LightbulbIcon className="size-3.5" />
{effectiveReasoningVisualEnabled ? (
<LightbulbIcon className="size-3.5" />
) : (
<LightbulbOffIcon className="size-3.5" />
)}
<span>
Think:{" "}
{reasoningEffort.charAt(0).toUpperCase() +
reasoningEffort.slice(1)}
{effectiveReasoningVisualEnabled
? formatReasoningEffortLabel(
reasoningEffort,
externalSelection?.modelId,
)
: formatReasoningDisabledLabel(
effectiveSupportsReasoningOff,
isExternalOpenAIReasoning,
checkpoint,
)}
</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
{(["low", "medium", "high"] as const).map((level) => (
{effectiveSupportsReasoningOff && (
<DropdownMenuItem
onSelect={() => {
setReasoningEnabled(false);
applyQwenThinkingParams(false);
}}
>
{formatReasoningDisabledLabel(
effectiveSupportsReasoningOff,
isExternalOpenAIReasoning,
checkpoint,
)}
{!effectiveReasoningVisualEnabled ? " \u2713" : ""}
</DropdownMenuItem>
)}
{effectiveReasoningEffortLevels
.filter((level) => level !== "none")
.map((level) => (
<DropdownMenuItem
key={level}
onSelect={() => setReasoningEffort(level)}
onSelect={() => {
setReasoningEffort(level);
setReasoningEnabled(true);
applyQwenThinkingParams(true);
// Mutual exclusion: turning thinking on for a
// Kimi model forces the web_search builtin off.
if (isKimiExternal && toolsEnabled) {
setToolsEnabled(false);
}
}}
>
{level.charAt(0).toUpperCase() + level.slice(1)}
{reasoningEffort === level ? " \u2713" : ""}
{formatReasoningEffortLabel(level, externalSelection?.modelId)}
{effectiveReasoningVisualEnabled && reasoningEffort === level ? " \u2713" : ""}
</DropdownMenuItem>
))}
</DropdownMenuContent>
@ -662,31 +818,53 @@ export function SharedComposer({
) : (
<button
type="button"
disabled={reasoningDisabled}
disabled={reasoningDisabled || reasoningLockedOn}
aria-disabled={reasoningDisabled || reasoningLockedOn}
title={
reasoningLockedOn
? "This model requires reasoning to stay on."
: undefined
}
onClick={() => {
if (reasoningAlwaysOn) return;
if (reasoningLockedOn) return;
const next = !reasoningEnabled;
setReasoningEnabled(next);
applyQwenThinkingParams(next);
// Mutual exclusion: Kimi's $web_search builtin
// requires thinking off, so turning thinking on flips
// the Search pill off (and vice versa).
if (isKimiExternal && next && toolsEnabled) {
setToolsEnabled(false);
}
}}
className={cn(
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
reasoningDisabled
? "cursor-not-allowed opacity-40"
: (reasoningEnabled || reasoningAlwaysOn)
? "bg-primary/10 text-primary hover:bg-primary/20"
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
reasoningLockedOn
? "cursor-not-allowed bg-primary/10 text-primary"
: reasoningDisabled
? "cursor-not-allowed opacity-40"
: effectiveReasoningEnabled
? "bg-primary/10 text-primary hover:bg-primary/20"
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
)}
aria-label={reasoningEnabled ? "Disable thinking" : "Enable thinking"}
aria-label={
reasoningLockedOn
? "Thinking is required for this model"
: effectiveReasoningEnabled
? "Disable thinking"
: "Enable thinking"
}
>
{(reasoningEnabled || reasoningAlwaysOn) && !reasoningDisabled ? (
{reasoningLockedOn ||
(effectiveReasoningEnabled && !reasoningDisabled) ? (
<LightbulbIcon className="size-3.5" />
) : (
<LightbulbOffIcon className="size-3.5" />
)}
<span>Think</span>
</button>
)}
)
) : null}
{supportsPreserveThinking && (
<button
type="button"
@ -714,10 +892,22 @@ export function SharedComposer({
)}
<button
type="button"
disabled={toolsDisabled}
onClick={() => setToolsEnabled(!toolsEnabled)}
disabled={searchDisabled}
onClick={() => {
const next = !toolsEnabled;
setToolsEnabled(next);
// Kimi's $web_search builtin requires thinking=disabled
// (https://platform.kimi.ai/docs/guide/use-web-search).
// Toggle the Think pill off when Search comes on, and
// back on when Search goes off — mutual exclusion that
// mirrors what the backend enforces.
if (isKimiExternal) {
setReasoningEnabled(!next);
applyQwenThinkingParams(!next);
}
}}
className="composer-pill-btn"
data-active={toolsEnabled && !toolsDisabled ? "true" : "false"}
data-active={toolsEnabled && !searchDisabled ? "true" : "false"}
aria-label={toolsEnabled ? "Disable web search" : "Enable web search"}
>
<GlobeIcon className="size-3.5" />
@ -725,10 +915,10 @@ export function SharedComposer({
</button>
<button
type="button"
disabled={toolsDisabled}
disabled={codeDisabled}
onClick={() => setCodeToolsEnabled(!codeToolsEnabled)}
className="composer-pill-btn"
data-active={codeToolsEnabled && !toolsDisabled ? "true" : "false"}
data-active={codeToolsEnabled && !codeDisabled ? "true" : "false"}
aria-label={codeToolsEnabled ? "Disable code execution" : "Enable code execution"}
>
<CodeToggleIcon className="size-3.5" />

Some files were not shown because too many files have changed in this diff Show more