* Chat search: match across all messages, not just title and preview
The Cmd/Ctrl+K chat search only matched the thread title and a 120
character preview of the newest message, so keywords anywhere else in a
conversation were unfindable. Build a per-thread haystack from the title
plus every message's text and match against that.
The haystack is lowercased once at index build time, so the keystroke
filter only normalizes the short query instead of re-lowercasing the
whole conversation per item.
* Chat search: search user messages first, expand to all only if no hit
User messages are short while assistant replies can be very long, so
matching the whole conversation on every keystroke scales with the
assistant text. Store a separate userSearchText (title plus user
messages) and match it first, expanding to the full per thread
searchText only when nothing matches user text anywhere. Row filtering
moves into selectVisibleChats with cmdk shouldFilter disabled so the
tier choice is deterministic.
* Chat search: include tool calls and sources in the full-text tier
extractText now also pulls reasoning/thinking, tool call name/args/result
and cited source title/url, so the expanded full-conversation tier finds
keywords that only appear in tool activity. Drop the now-unused preview
field since search no longer matches on it.
* Chat search: drop base64 image/audio payloads from the index
extractText stringified tool-call results wholesale, so an
image_generation result (image_b64) or audio payload would pour megabytes
of base64 into searchText and get lowercased on every rebuild. Add
searchableText, which keeps readable tool args/results (tool name, prompt,
text) but skips binary keys and strips data URLs, long base64 runs and the
__IMAGES__ suffix.
* Studio: log transformers version-switching decisions and stop swallowing MLX activation failures
Two logging gaps in dynamic transformers version switching (issue #6103):
1. get_transformers_tier returned a tier with no trace of why. Add an
info log at each decision point naming the model and the trigger
(which substring matched, or which config check fired), so a model
landing on the wrong tier is diagnosable.
2. The MLX fast-path in run_training_process activated the transformers
version inside a bare 'except Exception: pass', silently swallowing
failures while the non-MLX path reports them. A missing or broken
version venv (e.g. Gemma-4 needing 5.5.0) left no trace and only a
confusing downstream crash. Extract a small _activate_transformers_version_or_warn
helper that logs a warning on failure while keeping the non-fatal
fall-through, and call it from the MLX path.
Adds tier-selection logging tests and helper warn/silent tests.
* Studio: clarify path-prepend log, warn on venv version mismatch, log per-package install progress
Completes the remaining logging items of #6103 in studio/backend/utils/transformers_version.py:
- activate_transformers_for_subprocess: the early "Activated transformers X.X.X" line was misleading because at that point only the venv directory has been prepended to sys.path, not imported. It now says it prepended the venv to sys.path and notes the loaded version is confirmed later by "Subprocess loaded transformers ...".
- _venv_dir_is_valid: a detected version mismatch is logged at warning instead of info, since it immediately triggers a full venv wipe and reinstall that should be visible in the logs.
- _ensure_venv_dir: log each package as it starts installing with an N/M progress counter, so a slow runtime install is not mistaken for a hang (pip/uv output is piped and only surfaced on error).
Adds tests covering all three behaviours; pre-existing unused imports are left untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make tier log-capture tests independent of import order
The new issue #6103 caplog assertions in test_transformers_version.py
relied on the module-level sys.modules.setdefault("loggers", stub)
winning the import race. In a full backend pytest run another module
(for example test_log_filter_no_truncation, collected earlier) imports
the real loggers first, so the setdefault is a no-op and
transformers_version.logger becomes a structlog/stdout logger that
caplog cannot capture -- the tier, activation, venv-mismatch and
install-progress log assertions then fail even though the line was
emitted.
Bind a real stdlib logger to transformers_version.logger for the
duration of each test via an autouse fixture, so the module logs through
logging and caplog captures them regardless of collection order.
* Studio: log local checkpoint tier decisions and warn on MLX inference activation
- get_transformers_tier: the local config.json fast path returned a tier
without logging it, so local checkpoints stayed opaque while HF ids were
traceable. Log each decision there too, with a caplog regression test.
- inference worker: the MLX path swallowed _activate_transformers_version
failures with a bare except, the same gap issue #6103 fixed for training.
Warn instead, keeping the non-fatal fall-through.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: warn when a GPU model silently loaded on CPU
llama-server can serve HTTP 200 while running a model entirely on CPU when its GPU backend fails to init, so Studio could run a GGUF on CPU without saying so (#5807 / #5106 / #5830). The silent-CPU warning already exists but stopped firing on current llama.cpp because _classify_gpu_offload keyed only on the dropped 'model buffer size' lines. Add a shared classify_gpu_offload_lines (offloaded N/M counts, GPU model-buffer markers excluding _Host, device_info disconfirm-only) and delegate to it so the warning fires again. Log-only: no install or load behavior changes.
Pure classification of already-captured startup log lines, run once after load; no new subprocess, no slowdown.
* Studio: key the CPU-offload warning on the main model, not a draft
With MTP/speculative decoding llama-server logs 'offloaded N/M layers to GPU' twice: once for the main model and once for the small draft model. The old scan returned True on any non-zero count, so a drafter that fits on GPU while the main GGUF runs on CPU suppressed the warning (the Qwen3.6-27B-MTP case). Decide on the line with the most layers (the main model) instead, so a drafter cannot mask a main model on CPU.
* Studio: make lifespan shutdown resilient to a dead default executor
On an abrupt shutdown (closing the Windows console window, or interpreter
teardown racing uvicorn's graceful stop) the event loop's default thread-pool
executor can already be shut down by the time the FastAPI lifespan shutdown
runs. The first post-yield statement was an unguarded
`await asyncio.to_thread(terminate_hub_downloads)`, so executor.submit raised
`RuntimeError: cannot schedule new futures after shutdown`. That raise
propagated up through every nested merged_lifespan __aexit__, aborted the rest
of the cleanup (DEVICE reset, compiled-cache clear), and surfaced as
"Application shutdown failed. Exiting."
Extract the post-yield cleanup into utils/lifespan_shutdown.run_lifespan_shutdown
and guard each step independently. On the to_thread RuntimeError, fall back to
running the (already best-effort, quick) terminate inline on the loop thread so
shutdown still completes cleanly. The helper is dependency-injected and free of
the heavy backend import graph, so it is unit-tested in isolation.
Add tests/test_lifespan_shutdown.py (4 cases: dead-executor survival, normal
path, terminate error, clear error). Validated on windows-latest and
ubuntu-latest runners on Python 3.13.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: only retry terminate inline when scheduling fails, not when the body raises
Address review feedback on the shutdown helper: the previous
`except RuntimeError` after `asyncio.to_thread(terminate_downloads)` could
not tell a dead-executor scheduling failure from a RuntimeError raised by
terminate_downloads itself, so a body-side RuntimeError on a healthy executor
ran the cleanup a second time inline.
Schedule via loop.run_in_executor and await separately: a dead default executor
raises synchronously at submit time (inline fallback), while a body exception
only surfaces when the future is awaited (logged, never retried). Add a
regression test that a body RuntimeError runs terminate exactly once.
* Studio: run terminate cleanup with a copied context (parity with asyncio.to_thread)
Simulation across the executor-state x exception x DEVICE matrix surfaced the one
behavioural difference from the original implementation: asyncio.to_thread copies
the caller's contextvars into the worker thread, while a bare
run_in_executor(None, fn) does not. Restore exact parity by scheduling
ctx.run(terminate_downloads) from a contextvars.copy_context(), so the refactor
is a behavioural no-op apart from the intended dead-executor recovery. Add a
regression test asserting the copied context is visible to terminate_downloads.
* Studio: tighten comments in lifespan_shutdown helper and tests
Condense the verbose docstrings/comments to the non-obvious rationale and drop
the self-evident ones. Verified comment-only with comment_tools.py check
--strip-docstrings (code signature unchanged); tests and sims still green.
* Studio: address review nits on lifespan shutdown helper
Annotate hw_module as types.ModuleType, reword the schedule/await comment
(inline fallback runs, it does not retry), and add a test for the public
loop.shutdown_default_executor() path (the 'Executor shutdown has been called'
RuntimeError that real uvicorn shutdown takes, distinct from the submit-time
'cannot schedule new futures after shutdown').
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* chore: add Studio version info to General settings
* chore: cache Studio version fetch
* fix: clear cached Studio version request on failure
* fix: retry incomplete Studio version fetch
* fix: remove cached Studio version fetch
* chore: move connections below API
* fix: localize studio version rows
* feat: implement thread forking functionality with associated database updates and UI components
* fix(studio/chat): register fork-count listener even when thread unsaved
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Polish thread fork action menu
* fix-studio-fork-project-test-order
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Fix the libaray path for probe_server_capabilities()
even when running something as simple as `./llama-server --help`,
the binary still requires correct LD_LIBRARY_PATH to work - or it
returns merely an "error while loading shared libraries":
"libllama-server-impl.so: cannot open shared object file: No such file or directory"
For a local installation with no LD_LIBRARY_PATH specifically set,
the probe_server_capabilities() run of `./llana-server --help` should
share the same libaray resolution logic as start_llama_server().
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Adjust and readd comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* feat(studio): expose provider_type selector in model provider dialog
Use the configured provider_type in the model provider payload instead of hardcoding openai. Add a provider type selector to the model provider dialog for the provider types supported by Data Designer.
* feat(studio): add MiniMax-M2.7 inference defaults
Add MiniMax-M2.7 model family entry to inference_defaults.json
with recommended parameters (temperature=1.0, top_p=0.95, top_k=40).
Pattern placed before minimax-m2.5 for correct longest-match-first
ordering.
* Fix/adjust provider type support for PR #4277
* Clarify provider type validation for PR #4277
* Fix provider type default validation for PR #4277
---------
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: PR Bot <pr-bot@minimaxi.com>
* Studio: guard the training import against a namespace-package shadow of unsloth
A directory named unsloth without an __init__.py on sys.path (a stray checkout,
a partial clone, or a polluted PYTHONPATH) makes the path finder return a
namespace package, so the worker's 'from unsloth import FastLanguageModel' fails
with 'cannot import name ... (unknown location)'. A normal site-packages install
always wins this race, so only source/editable installs are exposed. Before the
import, drop the offending sys.path entries, bind the real packages, then
restore sys.path so other modules on those entries keep importing. It is a
no-op when unsloth already resolves to a real package.
* Studio: import unsloth before unsloth_zoo in the namespace-shadow guard
_ensure_real_packages imported the requested names in argument order, so a
unsloth_zoo namespace shadow made it import unsloth_zoo directly before
unsloth. That skips unsloth.__init__ -> _gpu_init, which runs its ROCm and
Windows bitsandbytes fixes before its own import unsloth_zoo, so the recovery
path could import zoo with those guards skipped and fail on the ROCm/Windows
cases those fixes handle.
Import parent-first via reversed(names) so unsloth is imported first and pulls
in the real unsloth_zoo after _gpu_init has run; the later cached import is a
no-op. This also covers the case where only unsloth_zoo is shadowed: the bad
sys.path entry is still dropped and unsloth owns the zoo import. Detection,
sys.path pruning, shadow-cache clearing, and restoration are unchanged.
Add tests/test_namespace_shadow_guard_pr6269.py: CPU-only subprocess scenarios
(only zoo shadowed, both shadowed, only unsloth shadowed, healthy no-op, real
package absent, multiple shadow entries) that assert unsloth imports before
unsloth_zoo and that sys.path is restored.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: restore sys.path even if invalidate_caches fails in the shadow guard
Move importlib.invalidate_caches() inside the try/finally so a failure there
still restores sys.path, and tighten the import-order comment. Add a test that
forces invalidate_caches to raise and asserts sys.path is restored.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: show llama.cpp version and GPU specs in the About panel
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: run the hardware endpoint off the event loop
* Studio: show the full llama.cpp release tag (incl -mix-<sha>) in the About panel
* Studio: order About-panel GPUs by visible ordinal and skip the llama.cpp probe during updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix hardware info refresh and endpoint scope
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* Studio: use the shared spinner for loading toasts
The loading toast used a different spinner from the model download toast,
so 'Unloading model' and 'Downloading model' looked inconsistent. Switch
the sonner loading icon to the app-wide Spinner so they match.
* Studio: consolidate loading spinners onto the shared Spinner
Replace the duplicate ToolCallSpinner and the remaining inline loading
spinners (hugeicons Loading03, lucide Loader2/Loader/LoaderCircle, and
the hand-rolled border rings) with the shared Spinner component, then
delete tool-call-spinner.tsx. Refresh and regenerate action icons and
the shimmer button are left alone since they are not loading spinners.
* Studio: align and even out sidebar hover pills
The Recents chat rows started further left than the nav items above them
and the hover pill widths varied. Give every nav and chat group the same
pl-1.5 pr-2 so the pills line up and share one width, and on hover the
chat rows only reserve room for the kebab so the title keeps one more
character.
* Studio: address review feedback on spinner and sidebar polish
- recipe-graph-node: restore rounded-full bg-background on the spinner
wrapper so it keeps masking the grid lines behind the floating spinner
(the original hand-rolled ring had this mask).
- app-sidebar: revert the pinned-row open-state padding to pr-8 so the
unpin button stays clear of the title when the kebab menu is open.
- app-sidebar: correct the now-stale inset comment (pl-1.5 = 6px, 18px).
* Fix undefined variable 'e' in Version parsing exception
* Fix SyntaxError for tuple unpacking in array index for Python < 3.11
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
When an empty assistant turn is dropped (a 0-token model reply, or a Stop-button
sentinel handled by _drop_empty_assistant_sentinels) the GGUF chat history can be
left with two user turns back to back. Strict Jinja chat templates (Gemma 3, some
Mistral variants) call raise_exception("Conversation roles must alternate ...") on
the first role-parity break, so llama-server returns a 400 and the thread becomes
unsendable.
Add _coalesce_consecutive_user_turns (merging only adjacent user turns, never
assistant/tool turns, preserving multimodal parts) and apply it in
_openai_messages_for_gguf_chat. The tool path inherits the fix for free because it
rebuilds from this same normalized history via _set_or_prepend_system_message. The
passthrough path is left untouched (it forwards messages verbatim). No-op for
already-alternating histories. Adds unit and end-to-end coverage.
* fix: use partial hipinfo output on crash to avoid CPU fallback (#6043)
`hipinfo.exe` on some RDNA 4 hosts (e.g. RX 9060 XT / gfx1200) exits
with STATUS_ACCESS_VIOLATION (0xC0000005) after printing the
gcnArchName line. The previous guard `$LASTEXITCODE -eq 0` in
studio/setup.ps1 and `if result.returncode == 0` in
install_python_stack.py discarded this partial-but-valid output,
causing the installer to fall through to WMI name inference which sets
HasROCm=false and installs CPU PyTorch instead of the ROCm wheel.
Fix: check for gcnArchName in stdout first; accept the arch regardless
of exit code. Only fall through to the amd-smi / WMI path when no
gcnArchName is present at all (crash before any output, or a genuine
"no device" error). A cyan INFO substep is emitted when the arch is
recovered from a crashed hipinfo run so users can see what happened.
Adds a regression test covering the crash-with-valid-output path.
Fixes#6043
* Fix/adjust hipinfo crash fallback for PR #6292
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: fix Mac IME input-method switch leaving composer Send disabled
On macOS, switching input method (Ctrl+Space / menu-bar language icon)
fires compositionstart but never compositionend — leaving composingRef
pinned at true and the Send button permanently disabled even after
switching back to English.
Two immediate recovery paths added to useImeComposerInputHandlers
(thread.tsx) and SharedComposer (shared-composer.tsx):
* onKeyDown else-if: clears composingRef on the first non-IME keystroke
after a stuck composition, unblocking Send on that very keydown rather
than waiting for the 2500ms watchdog.
* onBlur handler: clears composingRef unconditionally on textarea focus
loss — safe because the OS always commits or cancels any active
composition before surrendering focus to another element.
Two new Playwright regression steps (6e, 6f) added to
playwright_chat_ime_i18n.py assert recovery within 1500ms (well below
the 2500ms watchdog), covering both recovery paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix IME regression test idle handoff for PR #5762
* Fix IME cleanup console guard for PR #5762
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix IME Enter guard for PR #5762
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
On a restart the constructor reaps the previous run's orphaned llama-server, but the driver does not reclaim that VRAM synchronously. _kill_orphaned_servers now returns the number of processes it killed, and __init__ arms _last_kill_monotonic when that count is positive, so the first load_model waits for VRAM to settle before ranking GPUs by free memory instead of pinning the model onto the smaller card. The compute-graph / auto-fit reserve is handled separately in #6312.
* Studio: add --secure Cloudflare-only mode and revamp API usage examples
--secure / --not-secure on `unsloth studio` and `unsloth studio run`:
- --secure binds 127.0.0.1, requires the Cloudflare tunnel, and advertises only
the Cloudflare link. cloudflared reaches the server over localhost, so the raw
port is never exposed on a public interface.
- If the tunnel cannot start, fail closed with a clear message instead of
silently leaving a raw 0.0.0.0 link.
- Default stays not-secure (no behavior change); coexists with the existing
--cloudflare/--no-cloudflare flag. Host defaults are unchanged.
- /api/health (authed) now reports the live tunnel URL.
API usage examples (Profile > API):
- Example tabs for curl, Python, curl + tools, Python + tools, plus an OS row
(Linux/macOS/WSL vs Windows) auto-detected from the platform.
- Windows curl passes the JSON body via a file so PowerShell does not strip the
quotes when calling curl.exe.
- Python + tools forwards enable_tools/enabled_tools through extra_body and
guards chunk.choices, since tool-lifecycle events carry no choices.
- Shows the loaded model name and the real API key while it is still revealed.
- A Cloudflare Tunnel toggle (default on) shows the public tunnel URL and uses
it as the base_url in the examples when a tunnel is running.
Tests cover the tunnel start gate and the --secure flag on both commands.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: gate --secure tools on public exposure and harden API examples
In secure mode the server binds loopback but is reachable via the public
Cloudflare tunnel, so resolve the tool policy against the public exposure
(0.0.0.0) rather than the loopback bind. This keeps server-side tools off by
default and prompts before enabling them, instead of inheriting the loopback
default of on. The startup tool notice now names the public surface.
Also reject --secure with --no-cloudflare directly in run_server and the
run.py argparse (not only the CLI), JSON-encode interpolated model names so
Windows paths and quotes cannot produce invalid JSON or broken snippets, and
force-refresh /api/health on the API panel so a tunnel that starts after the
first health read still surfaces its URL.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: API examples show direct host when tunnel toggle is off; move Copy onto code
The Cloudflare Tunnel toggle had no visible effect when Studio was opened
through the tunnel: the off state fell back to window.location.origin, which
equals the tunnel URL in that case. /api/health now reports the direct
host:port (server_url), and the API panel uses it for the off state so it shows
the real non-tunnel base. Also move the Copy button out of the tab row and onto
the code block.
* Studio: highlight API examples, add advanced tabs, fix tunnel toggle row
Syntax-highlight the curl/PowerShell/Python snippets with the app's shared
shiki plugin (bash/powershell/python). Add 'curl + advanced' and
'Python + advanced' tabs that set temperature/top_p/top_k/min_p/
repetition_penalty/max_tokens, enable thinking, and turn on all tools.
The Cloudflare Tunnel row no longer shifts the code block: the tunnel URL is
always rendered (dimmed when off) so toggling keeps the row height constant.
Key the highlighted block on its content so it remounts when only the base URL
changes (the renderer's block memo otherwise kept a stale URL).
* Studio: rename API tunnel toggle to Secure HTTPS, hint --secure when exposed
Rename the API examples toggle from Cloudflare Tunnel to Secure HTTPS. When the
server was not launched with --secure, show an info tooltip noting the raw
0.0.0.0 port is still globally reachable and pointing at --secure. /api/health
now reports whether --secure was used so the hint is hidden in secure mode.
* Studio: force tools off for plain network/secure launches
The plain 'unsloth studio --secure' (and '-H 0.0.0.0') launcher re-execs run.py
and never installed a tool policy, so the process default (honor per-request
enable_tools) let any API-key holder run Python/terminal tools over the public
endpoint. Force the policy off at the run.py entrypoint when network-reachable
(0.0.0.0 or --secure); 'unsloth studio run' still installs its own resolved
policy and does not go through this path.
* Studio: apply default tool policy in run_server, not the run.py entrypoint
The plain launcher runs from the studio venv and calls run_server directly, so
it never hit the run.py __main__ guard. Move the network/secure default-off tool
policy into run_server so every launch path (plain, --secure, direct run.py)
gets it; the run subcommand still overrides it with its resolved policy.
* Studio: clarify --secure help text on the network exposure tradeoff
Spell out in --help (both unsloth studio and unsloth studio run, plus the
run.py argparse) that --not-secure also serves the raw 0.0.0.0 port reachable
from anywhere on the network, matching the API panel's Secure HTTPS hint.
* Studio: cache API-key PBKDF2 derivation to cut per-request /v1 auth overhead
validate_api_key re-ran the 100k-round PBKDF2 on every authenticated
request, adding ~15ms to each /v1 call made with an sk-unsloth- key.
Benchmarked against the bare llama-server it proxies to, API-key requests
carried ~22ms of fixed overhead vs ~7ms for the JWT path; the gap was
entirely this redundant key derivation (Pydantic validation measured
0.005ms, so it is not a factor).
The raw-key to hash mapping is a pure deterministic function of the fixed
server salt, so memoize it per process, keyed by a salted HMAC of the key
(never the key or a recoverable digest). The cached value equals what is
already stored at rest. Revocation and expiry remain enforced by the
SQLite read on every call, so a cache hit only skips the KDF, never the
active or expiry checks. Only keys that exist in the DB are cached, so
unknown-key spam cannot grow it.
After the change the API-key /v1 overhead drops to ~8ms, at parity with
JWT, while the at-rest PBKDF2 hashing is unchanged.
Adds test_api_key_expiry.py covering API-key and JWT expiry enforcement
and the new cache: it skips the KDF on repeat and still rejects revoked
or expired keys.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten comments across the secure-tunnel and API-key changes
Condense multi-line comments and docstrings to one or two lines, drop the
ones that restate obvious code, and remove an orphaned test section header.
Comment-only: verified with comment_tools.py check (9/9 code unchanged), the
auth/secure-tunnel/CLI test suites, and a clean frontend typecheck and build.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix race in tool-call confirmation gate
* Studio: gate built-in tool calls and harden the confirmation handshake
The Allow / Always allow / Deny controls only lived in the fallback tool
card, but the built-in tools (web search, python, terminal, code
execution, image generation) render with their own components and so
never showed the buttons. Those calls paused after tool_start with no way
to approve them, hanging until the 1 hour timeout. Only MCP tools, which
use the fallback renderer, actually worked.
Render the controls for every tool card by wrapping each registered tool
component (and the fallback) in thread.tsx with a shared
ToolConfirmationControls, so the gate applies uniformly.
Also make the handshake robust:
- The gate keys on a per-call approval_id minted by the backend and
echoed in tool_start, instead of session_id alone, so a stale or
concurrent confirmation can no longer resolve the wrong call.
- The approval slot is registered before tool_start is yielded, closing
the race where a fast click or an auto "Always allow" could reach the
backend before the waiter existed.
- The frontend resolves with the same session id the request was sent
with (plus the approval_id), fixing the new-thread mismatch where the
confirmation targeted a different session than the blocked stream.
- The confirm endpoint returns {resolved}; the UI keeps the buttons and
shows a retry hint until the backend confirms a match, instead of
hiding them on a failed or mistargeted post.
- The gate runs after the disabled-tool and duplicate-call checks, so a
call that will not execute is not put up for approval. A denied call is
still excluded from duplicate detection, so re-issuing and approving it
works.
- "Always allow" is scoped per session to match the backend gate.
Add backend tests for the approval registry, the SSE no-deadlock
handshake, and the loop integration (allow, deny, disabled, duplicate,
re-issue after deny).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move "Confirm tool calls" to the Tools section
* Studio: add Bypass Permissions (skip confirmation, disable tool sandbox)
Adds an opt-in Bypass Permissions toggle next to Confirm tool calls. When on,
no tool call shows a confirmation prompt and the python/terminal sandbox is
disabled: safety checks, command blocklist, and resource limits are skipped.
Secret env vars are still stripped and HOME stays repointed at the session
workdir. Default off keeps current behavior, and it takes precedence over
Confirm tool calls. Enabling it requires accepting a warning each time.
* Studio: harden Bypass Permissions secret handling and fix Anthropic tool path
Follow-up to the Bypass Permissions feature. Addresses the review findings:
- Anthropic /v1/messages 500: declare bypass_permissions on
AnthropicMessagesRequest so tool requests that omit the field default to
False instead of raising AttributeError (extra='allow' does not set absent
attributes).
- /proc parent-env leak: stripping the child env did not stop a same-uid
bypassed child from reading /proc/<parent>/environ to recover the
tool-executing process's unfiltered secrets. Clear PR_SET_DUMPABLE on that
process before the first bypass exec so its /proc entries become root-owned.
Hardening is fail-closed: if prctl is denied, bypass execution is refused
rather than run with the parent environ still readable. Mitigation, not a
full boundary; documented in the code.
- Broker/capability vars: strip SSH_AUTH_SOCK, SSH_AGENT_PID, GPG_AGENT_INFO,
GNUPGHOME, KUBECONFIG, DOCKER_HOST so a bypassed tool cannot use the
operator's live agents.
- Credential-bearing URL values: drop any env var whose value embeds URL
userinfo (scheme://user:pass@ and token-only scheme://token@) regardless of
the variable name. Benign proxy/index URLs without credentials are kept, so
proxy-only and internal-index setups still work in bypass mode.
- Windows temp isolation: repoint TEMP and TMP (not just TMPDIR) at the
per-session sandbox dir.
- Frontend: stop persisting bypassPermissions; a reload now starts with the
sandbox/confirmation bypass off and requires re-accepting the warning dialog.
Adds regression tests for each finding in test_bypass_permissions.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: strip cred-location env vars (HF_HOME etc.) in Bypass Permissions
Repointing HOME did not stop SDKs auto-reading cached creds via vars that
point at the real home/cache/config: HF_HOME (startup always sets it; token
lives under $HF_HOME/token), HF/XDG cache roots, NETRC/BOTO_CONFIG/
PIP_CONFIG_FILE, and Windows HOMEDRIVE/HOMEPATH. Drop those, and repoint
USERPROFILE/APPDATA/LOCALAPPDATA at the per-session workdir. Adds regression
tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: lock in bypass HF token resolution with an end-to-end test
The drop-based fix relies on the whole HF_HOME/XDG fallback chain being
removed so huggingface_hub resolves under the repointed HOME. Add a test
that sets HF_HOME and XDG_CACHE_HOME at a real cache and asserts the
resolved token path lands under the workdir, not the operator's cache.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: strip npm _auth, MYSQL_PWD, and BASH_ENV from bypass env
Three more credential vectors dodged the bypass scrubber: NPM_CONFIG__AUTH
(npm _auth, base64 so no URL userinfo and no AUTH marker), MYSQL_PWD (markers
use PASSWD, not PWD, since PWD is the cwd var), and BASH_ENV (bash -c sources
it for non-interactive shells, so a startup file can re-export stripped
secrets). Add an AUTH marker, the exact MYSQL_PWD name, and drop BASH_ENV plus
PGPASSFILE. Adds regression tests incl. an end-to-end check that a bypass
terminal call does not source BASH_ENV.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: extend bypass env scrubber and enforce confirm precedence in loops
From a parallel review pass over the bypass changes:
- Drop more credential-location vars in _build_bypass_env: npm/yarn/git/cargo/
rclone config pointers (NPM_CONFIG_USERCONFIG, NPM_CONFIG_GLOBALCONFIG,
YARN_RC_FILENAME, GIT_CONFIG_GLOBAL, GIT_CONFIG_SYSTEM, CARGO_HOME,
RCLONE_CONFIG) and the GIT_ASKPASS/SSH_ASKPASS auth helpers.
- Enforce confirm_tool_calls AND NOT bypass_permissions inside the safetensors
and GGUF tool loops, not just at the route, so a direct internal caller
passing both flags never prompts.
- Soften the toggle hint: environment secrets are stripped, but bypassed code
can still read files and credentials on the machine (no overclaim that keys
stay hidden).
Adds regression tests for the new names and the loop-level precedence.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add GGUF loop test for bypass-over-confirm precedence
The safetensors loop precedence is covered behaviorally; the GGUF loop needs a
live llama-server so add an AST guard asserting its _needs_confirm gate
references both confirm_tool_calls and bypass_permissions, matching the other
llama_cpp source-inspection tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add red Bypass Permissions badge in the composer
When Bypass Permissions is on, show a persistent red pill in the composer
tool-pill row (like the Search/Code pills), matching Claude Code's always-
visible bypass indicator. Clicking it turns bypass off, mirroring the other
composer toggles. Enabling still goes through the settings toggle + warning
dialog. Adds a data-variant=danger style for the destructive-colored pill.
* Studio: show Bypass Permissions badge in the Thread composer too
The empty-state and active Thread render their own composer (thread.tsx),
not shared-composer, so the badge only appeared in the split layout. Mirror
the red dismissible pill in ComposerAction so it shows in every composer.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep the Bypass Permissions badge visible when the composer is collapsed
The Thread composer only renders the pill row when expanded, so the active-mode
badge vanished on the default (collapsed) empty state. Render it before the
expand gate (it returns null when bypass is off) so the red indicator always
shows while bypass is on.
* Studio: make the Bypass Permissions confirm button a solid red button
The destructive button variant is a subtle 10% tint that read as bare red text
next to the outlined Cancel. Force the solid destructive fill (the variant's
class loses to the tint through AlertDialogAction's Slot merge, so use the !
override the codebase already uses for this case) and shorten the label to
'I understand' so it fits the small dialog's two-column footer.
* Studio: add Bypass Permissions to the composer + More menu
Adds a 'Bypass Permissions' entry to the composer plus-menu (under More by
default) in both composers, so it can be toggled without opening Run settings.
Enabling routes through the same danger warning dialog; disabling is immediate.
A shared BypassPermissionsMenuItem keeps the two composers in sync.
* Studio: harden bypass env scrubber for IMDS opt-out and connection strings
Two gaps in the Bypass Permissions secret scrubber:
- The broad AWS_ prefix also dropped AWS_EC2_METADATA_DISABLED, a non-secret
opt-out. Removing it re-opens the IMDS instance-role credential path that the
operator explicitly disabled, so a bypassed boto/AWS-CLI call could recover
cloud creds. Keep that flag (and AWS_EC2_METADATA_V1_DISABLED) via a keep-list
while still stripping the real AWS credential vars.
- Azure App Service connection strings (SQLCONNSTR_/CUSTOMCONNSTR_/...,
WEBSITE_CONTENTAZUREFILECONNECTIONSTRING) and values like Password=/AccountKey=
/SharedAccessKey= slipped past the name and URL-only value classifiers. Add
CONNSTR/CONNECTIONSTRING name markers and a connection-string value matcher.
* Studio: let Bypass Permissions suppress the confirm-tool-calls guards
The confirm-vs-bypass precedence (confirm and not bypass) was applied at the
loop call sites but not at the earlier request guards, so a client sending
confirm_tool_calls + bypass_permissions together was rejected (stream=true
required / unsupported for external or Anthropic tools) before the precedence
took effect. Gate all four confirm guards on not bypass_permissions so both
flags together proceed with the gate suppressed, matching the documented rule.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: add Anthropic-compatible thinking parameter
Add `thinking` parameter using Anthropic's format ({type: 'disabled'} /
{type: 'enabled'}) alongside the existing `enable_thinking` boolean for
backward compatibility.
The new parameter is mapped internally to `enable_thinking` at the route
layer so all downstream templates and backends continue to work unchanged.
Changes:
- Add ThinkingConfig model and `thinking` field to ChatCompletionRequest
- Add mapping logic in routes: thinking.type -> enable_thinking
- Add `thinking` field to frontend TypeScript types
- Update frontend request building to send thinking parameter
- Add tests for new thinking parameter
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: move thinking→enable_thinking mapping to model_validator
The Gemini review correctly identified that the route-level mapping
bypasses normalization for external provider requests. Moving the
mapping into a @model_validator on ChatCompletionRequest ensures it
runs during Pydantic validation regardless of routing path.
* Document ThinkingConfig scope and thinking validation behavior
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* fix(studio): Windows GGUF cancel hang + CPU spinlock overhead (#5692)
Two fixes for Windows-native GGUF inference via llama-server:
**Issue 1 — GPU/CUDA Hang on Stream Cancellation:**
- Add `Connection: close` header to all httpx requests proxying to
llama-server, preventing Keep-Alive from masking downstream socket
closure.
- Introduce `_await_disconnect_then_close` background watcher that
polls `request.is_disconnected()` every 100ms and calls
`resp.aclose()` immediately when the client disconnects. This runs
alongside the existing cancel-POST watcher and covers client aborts
that never reach the /cancel endpoint (tab close, proxy aborts,
Colab, mobile navigation, etc.).
- Change all StreamingResponse `Connection: keep-alive` headers to
`Connection: close`.
**Issue 2 — High CPU Spinlock & KV Cache Backup Overhead:**
- Set OMP_WAIT_POLICY=PASSIVE and OMP_NUM_THREADS=2 in the
llama-server subprocess environment on Windows to prevent OpenMP
from spin-waiting on all logical cores while the GPU decodes.
- Limit `--threads` to 2 on Windows when the model is fully
GPU-offloaded (`-ngl -1`). Auto-detect otherwise.
- Pass `--cache-ram 0 --ctx-checkpoints 0 --no-cache-prompt
--checkpoint-every-n-tokens -1` on Windows to disable prompt-cache
snapshots that copy KV cache to system RAM over the WDDM/PCI-E bus.
Closes#5692.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: use local import to avoid ruff F823 (sys used before assignment)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* review: address gemini review feedback
- Simplify _fully_gpu_offloaded init: default to False, only set True
in the gpu_indices branch, drop redundant else.
- Log exceptions in _await_disconnect_then_close at debug level instead
of silent pass, per review suggestion.
* Adjust review feedback for PR #5749
- _await_disconnect_then_close: set cancel_event before resp.aclose() so
the streamer's RemoteProtocolError handler treats the watcher-driven
close as cancellation, not an upstream error. Both call sites pass
cancel_event through.
- Windows --cache-ram / --no-cache-prompt / --ctx-checkpoints block: gate
on _fully_gpu_offloaded so CPU and partial-offload Windows runs keep
prompt-cache reuse across turns.
- Windows OMP_WAIT_POLICY / OMP_NUM_THREADS env: same gate so CPU and
partial-offload Windows runs keep default OpenMP parallelism.
* Shorten code comments touched by PR #5749
* Clean up local imports and rename underscore locals in PR #5749
- Drop the function-local `import sys as _sys` introduced as an F823
workaround; remove the redundant in-function `import os`/`import sys`
block so module-level imports resolve sys/os instead. F823 no longer
triggers because no shadowing import remains inside load_model.
- Rename `_fully_gpu_offloaded` and `_t` to `fully_gpu_offloaded` and
`threads_arg`. Underscore-prefixed names usually mean private/module-
level; plain locals match Python style for in-function temporaries.
No behavior change. ruff clean, py_compile clean, 35 studio cancel-
infra tests + 13 launch-gating AST locks + 6 disconnect-watcher locks
+ 4 spoof live-import tests all pass.
* Fix Windows GGUF follow-ups for PR #5749
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix cache flag gating for PR #5749
* Fix Python 3.9 annotations for PR #5749
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Anmol Mishra <anmolx.work@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: enable stdio MCP servers on a loopback bind
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address codex review on stdio MCP loopback gate
* Studio: fix banner URL and preserve stdio MCP env opt-in on network binds
* Studio: scope loopback to exact aliases and honor force-disable on run_server reuse
* Studio: cover force-disable across a public re-bind and fix a stale test comment
* Studio: keep stdio MCP off on Colab loopback launches
* Studio: set tool policy before server startup
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* Expose MLX grad value clipping in Studio
* update test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* dataset ordering + wd
* fix mlx smoke step expectations
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* cast norm activation output back to original input dtype
* address mlx studio review feedback
* Fix present-but-None seed override for PR #5656
studio/backend/core/training/worker.py
`config.get("model_random_state", random_seed)` only fills the
default when the key is absent. When a caller passes
`config["model_random_state"] = None` explicitly (which happens
any time a JSON payload sends an explicit `null`), the old code
forwarded `None` to FastMLXModel and disabled deterministic init
silently. Same for `lora_random_state`. Treat absent and explicit
None the same way: fall back to random_seed.
studio/backend/tests/test_training_raw_support.py
Update the source-string assertions to match the new lines.
* Guard optional MLXTrainingConfig fields and normalize random_seed for PR #5656
The MLX worker now passes `cast_norm_output_to_input_dtype` and
`dataset_order` only when the linked unsloth-zoo dataclass actually
declares them. Released zoo trees that predate the paired PR can still
construct `MLXTrainingConfig` without raising
`TypeError: unexpected keyword argument`. Once the dependency floor is
bumped to a release that contains both fields, the feature-detect
guards become no-ops.
`random_seed = config.get("random_seed", 3407)` was unguarded against
explicit `None` from raw / backend callers. The same value seeded the
trainer and was the fallback target for `model_random_state` /
`lora_random_state`. Normalize once at the top of the function and use
the normalized value everywhere so an explicit `None` cannot reach
FastMLXModel / get_peft_model / MLXTrainingConfig.
Existing seed source-pattern test updated to match the new normalize
helper. New test asserts the feature-detection guards exist and that
the unconditional kwargs do not include the gated fields.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Normalize seed / cast / max_grad_value at TrainingBackend for PR #5656
Round-3 review consensus: the per-field guards that landed in the MLX
worker only protect the MLX path. The same `TrainingBackend.start_training`
config still reaches the CUDA/text trainer at `worker.py:2267`, the
embedding LoRA init at `worker.py:2450`, and embedding TrainingArguments
at `worker.py:2624` with raw `None` values, so an explicit
`random_seed=None` from a raw / backend caller still breaks non-MLX
training even after the previous fix.
Move the normalization into `TrainingBackend.start_training` itself,
where it runs once for every training mode:
- `_coerce_seed(value)`: explicit `None`, non-int, or absent all become
3407. Every downstream worker now sees an int.
- `_coerce_optional_bool(value, default)`: explicit `None` falls back
to `default` instead of `bool(None) == False`. Also normalizes the
common raw-config / YAML string aliases ("true" / "false" / "0" /
"1"). Used for `cast_norm_output_to_input_dtype`.
- `_coerce_optional_nonneg_float(name, value)`: rejects negative
numerics from raw / backend callers, matching the Pydantic
`ge=0` constraint the HTTP route already enforces. Used for
`max_grad_value`.
worker.py MLX path: the existing `bool(config.get(key, True))` for
`cast_norm_output_to_input_dtype` was changed to also fall back on
explicit `None`, so direct worker callers (bypassing
`TrainingBackend.start_training`) are equally safe. `max_grad_value`
also raises on negative values inside the worker for the same reason.
TrainingStartRequest.random_seed default bumped from 42 to 3407 so
direct REST callers that omit the field receive the same default as
the Studio frontend and the MLX worker.
New regression test exercises the three new helpers across explicit
None, valid values, string aliases, and negative-value rejection.
* Tighten feature-detect test paren tracking for PR #5656
The block-extraction used , which stops at the
first inner closing paren (e.g. )
and would silently miss a future unconditional
/ added later in the same dict literal. Switched to
proper paren-depth tracking so the unconditional block is checked end-to-end.
* Shorten verbose comments in MLX Studio backend
* Handle MLX Studio EOS appending by mode
* Wire MLX leaf norm clipping through Studio
* Respect VLM layer filters for explicit LoRA targets
Rationale / guardrails for the local Studio/vision push:
When callers provide explicit VLM LoRA target_modules together with layer filters, FastVisionModel still needs to route the explicit targets through get_peft_regex. Otherwise the layer filters are ignored and adapters can be attached outside the requested language/vision scope.
Do not revert this to plain list(target_modules) for explicit module lists. The CUDA/Studio-facing contract is that explicit targets and layer filters compose: target_modules selects module names, while finetune_language_layers / finetune_vision_layers / finetune_attention_modules / finetune_mlp_modules constrain where those targets are allowed.
The regression test covers the language-only explicit q_proj case and source-checks that explicit targets are wrapped through get_peft_regex when filters are active.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refresh MLX smoke clip-config note for leaf_norm default
Trim the 11-line comment block to 5 lines and correct the stale claim
that MLXTrainingConfig defaults to max_grad_value=1.0. The new default
is max_grad_leaf_norm=1.0 (same memory profile as elementwise but
direction-preserving). The smoke still pins max_grad_value=1.0
explicitly to keep the 13-seed pass-rate fixture stable.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Forward max_grad_leaf_norm through the training route and warn when layer filters constrain explicit target_modules for PR #5656
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han-Chen <info@unsloth.ai>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Rename the banner heading from "New llama.cpp version" to "New llama.cpp
update", and add a small hint under the version line ("No restart needed
after update"). Tighten the gap above the action row so the hint and the
buttons read as one group.
* Studio: decide diffusion routing before the SWA resolver
Loading a DiffusionGemma GGUF could fail with "llama-server failed to start. Check that the GGUF file is valid" while llama-diffusion-cli ran the same model fine.
_read_gguf_metadata set self._is_diffusion after calling _resolve_swa_pattern, both inside one try/except. The resolver reaches into transformers/HF, which can raise for an architecture transformers does not know (diffusion-gemma); the shared except then swallowed it and left _is_diffusion False, so the model was routed to plain llama-server instead of the diffusion runner. It only reproduced where the SWA pattern was not already cached/inline.
Set _is_diffusion right after the KV parse loop (before the resolver) so a resolver error can no longer drop the routing, and skip the resolver for diffusion models, which do not use Studio's SWA pattern.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
UNSLOTH_STUDIO_LLAMA_ONLY=1, the llama.cpp-only fast path used by 'unsloth studio update', skipped the base-install block where STUDIO_HOME, _STUDIO_HOME_IS_CUSTOM, _STUDIO_OWNED_MARKER and _assert_studio_owned_or_absent are defined. setup.sh runs under 'set -euo pipefail', so the llama.cpp section aborted with '_STUDIO_HOME_IS_CUSTOM: unbound variable' before installing anything.
Hoist the studio-home and ownership resolution above the llama-only guard so both paths share it. Full installs are unchanged: the values are computed earlier but identically.
The update banner is driven by update_available from
/api/llama/update-status. Prebuilt (marker) installs decide this via
freshness.is_behind, which is mix-aware: a release at the same upstream
base build but with a new -mix-<sha> suffix counts as behind.
Source-build installs went through _source_build_status, which compared
only the numeric base build (installed_build < latest_build). When a new
prebuilt shared the installed upstream base (our usual mix re-tag at the
same base), it returned update_available=False and the banner never
showed. This is the common macOS case, where a failed prebuilt fetch
falls back to a source build that lacks the mix patches.
Make the source-build path mix-aware to match the marker path: same base
plus a -mix-<sha> tag now offers the update (and displays the mix tag),
a bare same-base rebuild does not, and the downgrade guard and unknown
-version fallback are unchanged.
* Studio: derive training output dir from model basename for local-drive models
A LoRA/QLoRA run started from a model loaded by absolute path (common when
models live on a non-system drive, e.g. G:\modelsAI\...\gemma-4-12B-it) seeded
the default output dir with that full path. resolve_output_dir then raised
"path escapes root ... is not under the studio outputs folder", so training
could not start from a model stored off the system drive.
Add default_run_dir_name(): Hugging Face repo ids keep their namespace
(org/model becomes org_model), while local paths collapse to their final
component so an absolute source path can no longer leak into the output dir.
Use it at the three worker derivation sites and add a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cap run dir name length and drop redundant resolve
Apply PR review feedback: length-cap the auto-generated output dir component so an unusually long model name stays under the filesystem name limit, and drop the redundant double resolve_output_dir at the embedding site so all three derivation sites match.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The generated install command builds unsloth from git with
--no-build-isolation, so pip uses the environment's existing setuptools
rather than the pinned build-system requirement. On setuptools < 77 the
PEP 639 license string in pyproject.toml fails to validate and the
install aborts. Upgrade setuptools and wheel up front so the source
build always has PEP 639 support.
* Studio: UI polish for sidebar, menus, hub and toasts
- Pinned chats: add a Pinned section above Recents with a Pin/Unpin menu
action and a hover unpin button on each pinned row
- Sidebar: align the Recents and Pinned section labels with Train, and
switch the chat Archive icon to archive-03
- Dropdown menus: pull separators in to match item width, and even out the
projects menu padding and radius to match the composer menu
- Account menu: match the divider width to the row hover width
- Hub toolbar: equalize the format, capability and sort filter widths so
the search field has more room
- Model run bar: make the quant selector hover a full pill
- Tooltips: slightly larger corner radius
- Auth: reduce horizontal padding on the submit button
- Toasts: center the model load content so top and bottom padding match
* Studio: shrink pinned unpin icon, match projects menu to composer menu
* Studio: inset dropdown separators slightly from item width
* Studio: move archived chats from the sidebar into chat settings
Archive now shows a toast pointing to Settings. Archived chats are listed
and managed (open, unarchive, delete) under Settings, Chat, Data.
* Studio: make the archive toast open the archived chats list
* Studio: keep chats archived when opened from the archived list
* Chat settings: optional delete confirmation and menu cleanup
Add a "Confirm before deleting" toggle in Settings > Chat > Data. When
turned off, deleting a chat from the sidebar skips the confirm dialog
and removes it instantly. The preference is stored client-side.
Add a separator above Archive in the chat row menu so Archive and
Delete read as a group separate from Export.
Drop the duplicate divider above "Clear all chats". The section already
draws a divide-y separator between rows, so the destructive row no
longer adds its own top border.
* Hub, dialog and page polish
Hub catalog:
- Drop "inference" from the Hub subtitle.
- Make the HF token shield a circle in the header.
- Equal, narrower filter triggers that stay a fixed width; long labels
truncate with an ellipsis instead of clipping.
- Tighter gap between the label and the dropdown chevron so one more
character fits.
- Keep the toolbar on one row at desktop widths.
- Align the model list and row hover with the Discover tab, add right
side breathing room.
- Make the dark-mode scrollbar lighter so it is easier to see.
- Filled size and quant chips no longer draw a border.
- Column divider uses the sidebar border color.
- Hub card extends into the bottom space.
Filters and channels:
- Add an MLX format option; detect MLX repos by library or tag.
- "Fine-tune ready" now includes 16bit safetensors, not only bnb-4bit.
- Shorten the curated channel descriptions.
Shared UI:
- New standard chevron icons (down and submenu right) shared across the
Hub, dropdown menu, menubar, project switcher and model selector.
- Dialogs get more top and bottom padding and a larger heading.
- Guided Tour hides itself on pages with no tour.
Other pages:
- Export and Data Recipes use the same page padding as the Hub.
* Soften popup overlays and refine hub filters
Dialogs, alert dialogs and sheets now use a light blur with a slight
darken instead of a heavy black scrim, so popups stay legible without
dimming the whole screen. Edit System Prompt and Edit Chat Template
drop their custom overlays and share the same look.
Also limit the Fine-tune ready channel to checkpoints Unsloth can
actually fine-tune (drops fp8, nvfp4, w4a16 and similar), revert chip
borders so filled chips stay visible, nudge the Run icon to center it,
and shrink the submenu chevron a touch.
* Inline chat rename and review-feedback fixes
Rename a chat inline as a rounded pill on the row instead of opening a
dialog. Enter saves, Escape cancels, clicking away saves. Projects and
training runs keep the existing dialog.
Address review feedback:
- Route the no-confirm chat delete through the shared cleanup helper so
it gets the same error toast and pin removal as the confirmed path.
- Archived chats now confirm before deleting (respecting the setting)
and pass the active thread, so deleting the open chat resets to a new
chat instead of leaving a stale thread URL.
- Only offer the MLX format filter on Discover, since downloaded
inventory rows are never tagged mlx and would filter to empty.
- Tighten the non-finetunable name match so a dot-delimited token such
as .tflite is also excluded from the Fine-tune ready channel.
* Sidebar: borderless inline rename, no name flash, align titles
Rename now edits the title as plain highlighted text with no pill or
box. While the debounced sidebar refresh catches up after a rename the
row shows the new name optimistically, so the old name no longer flashes
back in. Recent chat titles now line up flush under the Recents label.
* Profile: preferred name and avatar shape options
Add a "What should Unsloth call you?" preferred name that drives the
new-chat greeting, falling back to the first name then the login id.
Add a toggle to show the avatar as a full circle or a rounded
rectangle, applied everywhere the avatar appears.
* Settings: show version block at the top of General
Surface the Unsloth and package versions at the top of General, not
just in About. Both tabs share one StudioVersionSection so the version
fetch and markup live in a single place.
* Profile: pick a sloth sticker as your avatar
Add a curated set of sloth stickers users can choose as a profile
picture, shown below the picture shape toggle in Settings > Profile.
- Only squarish, low-whitespace stickers are listed so each one fills
the avatar frame cleanly.
- Selecting one reuses the existing avatar persistence (localStorage),
so it sits alongside the photo upload with no backend changes.
- Image avatars now get a neutral background so transparent stickers
read cleanly in both shapes.
* Profile: keep transparent avatars transparent
A transparent upload is now kept as WebP and shrunk to fit instead of
falling back to JPEG, which has no alpha and was painting a background
behind the image. The image avatar is also explicitly transparent.
* Profile: fall back to PNG for transparent avatars on Safari
Safari cannot encode WebP via canvas, so a transparent upload was
throwing instead of saving. Fall back to PNG, which keeps alpha and
works in every browser, before giving up. Still never uses JPEG.
* Archived chats: reset nav when deleting the open compare too
Compare panes do not write the active thread to the store, so the pair
id only lives in the route search. Derive the open chat id from the
route (thread or compare) like the sidebar does, so deleting the open
archived compare resets to a new chat instead of a dead URL.
* Profile: trim three sloth stickers from the picker
Drop the duplicate computer sloth without text on the screen (keeping
the Local one), the nature-screen sloth, and the gift sloth.
* Chat: tighten model selector right padding
Use less right padding than left so the trailing chevron does not leave
a heavy gap on the right of the trigger.
* Address review feedback on profile, reset and rename
- Give each sloth avatar button an accessible name so screen readers
can tell the options apart.
- Reset all local preferences now clears the delete-confirmation pref,
so it returns to the safe confirm default.
- A no-op Enter in inline rename now closes the editor instead of
leaving the row stuck as an input with its blur suppressed.
* Chat: rebalance model selector padding and chevron
More left padding than right and pull the chevron close to the label so
the trigger reads balanced around the text instead of heavy on the right.
* Hub: shrink the external link dialog icon
Set the icon to size-6 so it sits proportionally in the media circle
instead of filling it.
* Hub: shrink the external link dialog media circle
Override the media circle to size-12 with a size-5 icon on this dialog
only, so it is more compact without changing the shared component.
* Installer: drop redundant -WindowStyle Hidden from the Windows launcher VBS
The desktop / Start Menu shortcut launches Studio through a generated
launch-studio.vbs that runs:
shell.Run "powershell ... -WindowStyle Hidden -File launch-studio.ps1", 0, False
The second argument to shell.Run is intWindowStyle 0 (hidden), so WScript
already launches the child windowless. The child -WindowStyle Hidden is
therefore redundant: dropping it keeps the launcher hidden and behaviour
identical, while removing the WScript-spawns-hidden-ExecutionPolicy-Bypass
PowerShell token combination that antivirus heuristics weight. That shape was
reported as a Kaspersky HEUR:Trojan.VBS.Agent false positive during install.
Adds tests/studio/install/test_launch_studio_launcher.py to stop the flag from
being reintroduced and to assert the launcher stays windowless via
shell.Run(cmd, 0, False).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix llama.cpp prebuilt: skip the already-installed same-release fallback
install_prebuilt computes diffusion_visual_server_backfill_needed from the
newest candidate (plan.attempts[0]); when that is True it passed
existing_install_dir=None to validate_prebuilt_attempts, which disabled the
"existing install already matches this candidate" skip for the WHOLE plan. So
when the newest bundle failed validation the installer re-downloaded and
re-extracted an older fallback bundle that was already correctly installed.
Pass the real install dir always and gate the skip per-attempt: a matching
candidate is skipped unless that specific candidate still needs the
DiffusionGemma backfill re-extract.
Also make test_llama_cpp_search_roots_handles_studio_root_oserror read the full
_find_llama_server_binary / _kill_orphaned_servers method bodies instead of a
fixed 4000-char window. The except handler it asserts already exists, but the
function grew past the window so the guard silently failed; slicing to the next
sibling def keeps the check correct as the file grows.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: keep llama-server discovery from crashing on an access-denied candidate
_find_llama_server_binary probed candidates with Path.is_file(), which raises
PermissionError (WinError 5) when a path exists but is momentarily inaccessible
(antivirus lock, an install replace in flight, an elevated-install ACL),
aborting model validation. Treat a denied-but-present path as the real binary
so discovery returns it; absent paths still skip.
* Retry a transiently locked binary instead of returning a denied path
Returning a still-denied path only moved the PermissionError to the next
is_file() (probe_server_capabilities). Retry briefly so a transient lock
clears and discovery returns an accessible path; on a persistent lock return
nothing rather than a path downstream cannot stat.
* Studio: do not fall back to another llama-server when a pinned one is locked
A denied LLAMA_SERVER_PATH made discovery skip the explicit pin and run a
lower-priority managed or PATH binary, so a load could silently use a stale or
incompatible server. Split the probe into a file/absent/denied status: when the
pinned path exists but stays access-denied, warn and stop rather than falling
back to a different executable.
* Studio: never downgrade past a denied pinned or managed llama-server
Extend the no-fallback rule beyond LLAMA_SERVER_PATH: a present-but-denied
UNSLOTH_LLAMA_CPP_PATH or managed ($STUDIO_HOME/llama.cpp, ~/.unsloth/llama.cpp)
binary now reports temporarily-unavailable instead of silently launching a
lower-priority legacy or PATH server. Shared _scan_pinned/_unavailable helpers;
legacy in-tree and PATH stay genuine fallbacks (a denied candidate there just
continues).
* Studio: let diffusion asset lookup use a locked llama-server path for its dir
DiffusionGemma does not run llama-server; _find_diffusion_assets only needs the
install dir to find the adjacent llama-diffusion-gemma-visual-server. The
no-fallback rule returning None on a transiently locked llama-server therefore
hid an available visual-server and raised 'runner not found'. Add an
include_denied option so diffusion lookup gets the locked path (its dir is all
it needs), while inference keeps the no-denied-path, no-downgrade behavior.
* Studio: report a locked llama-server as temporarily unavailable, not missing
When the pinned/managed binary stays access-denied through the retries, discovery
returns None and load_model raised 'binary not found', a terminal error that
points users at reinstalling rather than retrying a transient AV/install lock.
Reuse include_denied to detect the locked path and raise a distinct
temporarily-unavailable, retry message instead.
* Studio: GGUF preflight treats a locked llama-server as present
The pre-download preflight (and so /api/inference/validate) used the default
discovery, which returns None for a transiently access-denied binary, so it
raised 'binary not found' for a binary that merely needs the lock to clear. Use
include_denied so the existence check counts a locked binary as present; the
load itself still reports a still-locked binary as temporarily unavailable.
* Studio: backfill the DiffusionGemma visual-server on a tag-matching update
An install made before the visual-server entered the copy allowlist (#6254)
matches on tag yet lacks the binary, so the update skip never backfills it and
DiffusionGemma fails with "runner not found". Re-extract the bundle when a
matching install is missing it, gated to the fork bundles that ship it so
upstream installs never thrash and so it self-limits once the binary lands.
* Studio: make the visual-server backfill bypass the existing-install short-circuit
The tag-matching skip is checked twice: once in install_prebuilt and again
inside validate_prebuilt_attempts via existing_install_matches_choice, which
only compares metadata and primary executables, not the DiffusionGemma
visual-server. So a stale install missing that binary still raised
ExistingInstallSatisfied and skipped the reinstall. Pass existing_install_dir
as None when a backfill is needed so the bundle actually re-extracts.
