* Fix SyntheticDataKit.chunk_data dropping single-chunk documents
chunk_data turns the n boundary points from np.linspace into n-1 ranges
via the boundaries[:-1] / [1:] pairing. When a document fits in a single
chunk (n_chunks == 1) that produces zero ranges, so the loop writes no
files and the whole document is silently dropped. Emit the full
[0, length] range when n_chunks <= 1; the multi-chunk path is unchanged.
Added a regression test covering the single-chunk and multi-chunk cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* chunk_data: emit nothing for an empty document (no empty chunk file)
Addresses review feedback: when the input document is empty (length == 0),
return no chunks instead of writing a single empty chunk file. Added a
regression test for the empty-document case.
* chunk_data: reject overlap >= chunk size (non-positive stride)
Per review feedback: when overlap >= max_tokens the chunk stride is
non-positive, which would divide by zero or silently emit one oversized
chunk. Raise a clear RuntimeError for that unusable configuration. Added
a regression test.
* Broaden single-chunk guard to length <= max_tokens (also fixes sub-overlap docs); expand tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
* Studio: drop OpenEnv and unused ExecuTorch/open_spiel install deps
* Studio: drop 8 more unused install deps from extras
* Studio: restore tomli<3.11 for kernels; tidy dep-cleanup comments and tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: refresh scan-packages baseline for scipy _external + unsloth-zoo tests
scipy moved its vendored array_api_compat from scipy/_lib to
scipy/_external, so the four allowlisted array_api_compat __init__.py
entries stopped matching and resurfaced as unsuppressed CRITICAL
"Downloads and executes remote code" findings on all three pip
scan-packages shards (extras, hf-stack, studio). Add the _external
paths next to the existing _lib ones so both scipy layouts stay covered.
Allowlist two unsloth-zoo test-file false positives now present in the
hf-stack shard: tests/test_mlx_save_export_regressions.py (writes to
/tmp dropper) and tests/test_mlx_trainer_internals.py (obfuscation plus
exec/eval).
Drop nine stale entries for packages removed from the Studio
requirements and no longer in any shard closure (evaluate, pytest,
hypothesis, kgb, langid), confirmed absent via with-deps resolution of
all three shards.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Harden MLX self-heal install against supply-chain execution
The Apple Silicon MLX self-heal runs uv pip install on a daemon thread
during Studio startup, default-on with only an env opt-out, before the
post-install stack check. Two things widened the supply-chain surface:
- it accepted source distributions, whose PEP 517 build backends run
arbitrary code at install time; and
- it forwarded the full process environment, exposing Studio secrets to
that code and letting a poisoned env (UV_FIND_LINKS / UV_DEFAULT_INDEX)
repoint the install at a hostile source.
Require pre-built wheels (--only-binary=:all:) and forward only an
allowlist of variables uv needs (PATH/HOME, proxy + CA settings, cache
dir), setting UV_OVERRIDE ourselves. mlx/mlx-metal ship wheels only and
mlx-lm/mlx-vlm publish py3-none-any wheels, so a healthy self-heal is
unaffected; an unavailable wheel just leaves Studio chat-only as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop cache-dir env vars from the self-heal allowlist
Address review: a poisoned process env could set UV_CACHE_DIR / XDG_CACHE_HOME
to redirect uv at an attacker-staged cache (cache poisoning, symlink writes),
which partly undercut the index-redirect protection. Drop them from the
allowlist; uv falls back to its safe user-owned default cache, still reused
across runs, so there is no normal-path cost. Test now asserts both are
excluded from the install env.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix Qwen3 NaN: self-heal vision pad_token in load_correct_tokenizer
Text-only Qwen3 (and Qwen2.5) models share Qwen3-VL's vocab, so their
Hub tokenizer configs ship <|vision_pad|> as pad_token. Padding text-only
training with a vision token corrupts attention/loss and produces NaN
losses and gradients on affected stacks.
patch_tokenizer already heals this, but only when a model with config is
passed. The standalone load_correct_tokenizer path (and custom training
loops) still returned <|vision_pad|>. This adds a model-independent guard
in load_correct_tokenizer that replaces a vision pad_token on text-only
tokenizers with the first safe text token (<|endoftext|>, <pad>, [PAD],
<unk>), falling back to eos_token only if it differs from pad_token.
The result now matches upstream Qwen configs (pad_token <|endoftext|>,
id 151643) with no new token added. Vision processors (image_processor
present) and non-vision pad tokens (Llama, Qwen2) are left untouched.
Fixes#3155
* Format pad_token helper for ruff kwarg-spacing hook (pre-commit)
* Harden vision pad_token fix: drop unk candidate, guard get_vocab, skip vision eos
* Tighten code comments (no logic change)
* Delegate pad_token fix to shared unsloth_zoo.pad_token
Generalize the narrow _fix_vision_pad_token by delegating to unsloth_zoo's
shared fix_pad_token (single source of truth, AGPL-3.0), which scans the
reserved-token families instead of only the vision-pad case. A guarded import
keeps this working against an older unsloth_zoo that has not shipped the module
yet: on ImportError it falls back to _fix_vision_pad_token.
allow_add=False is passed so the early load_correct_tokenizer call stays
side-effect free (no model here to resize embeddings); the later model-aware
patch_tokenizer call finishes the job and is idempotent.
Adds tests/python/test_pad_token_fix.py covering both dispatch paths offline.
* Fix CPOTrainer crash with multimodal processors
CPOTrainer shares build_tokenized_answer/tokenize_row and __init__ with
ORPOTrainer, but the ORPO replacement functions that route tokenization
through the underlying text tokenizer and resolve pad_token_id were only
registered for orpo_trainer. With a multimodal processing class (e.g.
Gemma4Processor) the positional self.processing_class(prompt, ...) call binds
prompt to images=, leaving text=None and raising
TypeError: 'NoneType' object is not subscriptable.
Register the existing orpo_trainer_text_tokenizer and
orpo_trainer_processor_pad_token under cpo_trainer as well so CPO/SimPO
fine-tuning of multimodal models works. No change for plain tokenizers.
* Add CPO processor tokenizer regression test
Static, CPU-only checks that cpo_trainer registers the same
orpo_trainer_text_tokenizer and orpo_trainer_processor_pad_token rewriters as
orpo_trainer, and that the rewriter drops the broken positional
self.processing_class(prompt, ...) call. Guards against issue #4952 regressing.
* Format CPO test assert for ruff line length (pre-commit)
* Bind CPO __init__ pad/eos token reads to underlying tokenizer
TRL 0.28+ CPOTrainer.__init__ reads bare processing_class.pad_token and
processing_class.eos_token before pad_token_id, which raises AttributeError
for multimodal processors (e.g. Gemma) where those live on .tokenizer.
Extend orpo_trainer_processor_pad_token to route that block through the
underlying tokenizer, and add a regression test.
* Tighten code comments (no logic change)
* Make CPO/ORPO rewriters reach the trainer on TRL 1.x
TRL 1.x moved CPOTrainer and ORPOTrainer out of trl.trainer into
trl.experimental.<algo> and dropped the trl.trainer.<algo>_trainer shim that
older TRL (0.26 - 0.28) kept. patch_trl_rl_trainers() discovers trainers via
dir(trl.trainer), so on TRL 1.x cpo_trainer and orpo_trainer are never found and
the multimodal-processor tokenization fix (#4952) silently stops applying, even
though the rewriters themselves still match the source.
Re-expose experimental-only trainers that Unsloth has rewriters for (RL_FUNCTIONS
keys) under trl.trainer before discovery, so the existing patch machinery and its
thin-wrapper resolution work unchanged. The alias is a no-op on older TRL where
trl.trainer.<algo>_trainer already exists.
Also rebind the patched Trainer/Config into every already-imported trl.* module
that holds the original class so the fix is visible at the experimental import
site (from trl.experimental.cpo import CPOTrainer), not only via trl.trainer.
Verified on transformers 4.57.6 + trl 0.22.2, transformers 4.57.6 + trl 0.27.1,
and transformers 5.12.1 + trl 1.6.0: CPOTrainer with a multimodal processor
tokenizes through the underlying text tokenizer with no crash on all three, and
the SFT/GRPO/DPO patch paths are unchanged.
* Format for ruff (pre-commit)
* Simplify CPO fix to mirror ORPO registrations (#4952)
Register the existing ORPO row-tokenizer/pad-token rewriters for cpo_trainer.
Under the trl<=0.24.0 pin CPOTrainer lives in trl.trainer.cpo_trainer (found by
dir(trl.trainer)), shares ORPO's build_tokenized_answer and uses
processing_class.pad_token_id, so the two registrations are sufficient.
Drop the trl 1.x experimental aliasing/rebind machinery in rl.py and the
bare-pad_token rewriter: trl 1.x (CPO in trl.experimental) and the bare
pad_token pattern (trl>=0.28) are not installable under the pin.
* CPO: route bare pad_token/eos_token default through inner tokenizer
TRL 1.x CPO/ORPO __init__ (the trl.experimental source unsloth resolves on TRL
0.26+) defaults processing_class.pad_token from processing_class.eos_token
before tokenizing. Multimodal processors (Gemma3/Gemma4 Processor) expose those
attributes on .tokenizer, not on the processor, so that bare access raises
AttributeError during __init__ even with the pad_token_id fallback registered.
Extend orpo_trainer_processor_pad_token to rewrite that defaulting block to run
on the inner tokenizer. The pinned TRL range (<=0.24.0) has no such block, so
the regex is a no-op there and only the existing pad_token_id fallback applies.
Verified the rewrite against the real trl 1.6.0 experimental CPOTrainer.__init__
(bare access removed, result compiles, a processor without pad_token no longer
raises) and added offline regression tests for both the rewrite and its no-op.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* vision/llama: thread finetune_audio_layers through get_peft_model
Pass finetune_audio_layers to get_peft_regex so FastModel / FastVisionModel can
adapt the Gemma 4 / Gemma 3N audio encoder and the audio/vision embedder
projectors. Enabled automatically under target_modules="all-linear", included in
the explicit-target layer-scope filter, and added to the UNSLOTH_USE_NEW_MODEL
kwargs defaults. Requires the matching unsloth-zoo get_peft_regex change.
* vision: address review feedback on finetune_audio_layers threading
- Drop finetune_audio_layers from the explicit-target_modules filter condition: it
defaults False, so including it forced every explicit list through get_peft_regex
and printed the scope warning even for text / vision models that never opted in.
- Guard the kwarg by get_peft_regex's signature so an older unsloth_zoo without the
parameter does not raise TypeError; warn once if requested but unsupported.
* vision: honor finetune_audio_layers with explicit targets; fail fast on old zoo
- Route an explicit target_modules list through get_peft_regex when audio is opted
in (positive `or finetune_audio_layers`), so finetune_audio_layers=True is no
longer a no-op for explicit lists. The scope warning is printed only when a layer
family is actually disabled, so default text/vision calls are not spammed.
- Raise (instead of warn) when finetune_audio_layers=True but the installed
unsloth_zoo lacks the parameter, rather than silently training a language-only
adapter.
* vision: keyword-safe audio flag, graceful all-linear on old zoo
- Move finetune_audio_layers to the end of get_peft_model's signature so
positional callers of finetune_last_n_layers (and the params after it) keep
their slots.
- Track whether the caller EXPLICITLY requested audio before all-linear flips
the flag on, and only raise the 'needs newer unsloth_zoo' error for explicit
requests. A plain all-linear (text/vision) run now degrades gracefully on an
old unsloth_zoo instead of failing.
* studio: add sidebar update button (static design only)
Adds a clock-icon update card above the account button in the sidebar footer. Visual/layout only; update detection and click behavior are wired in follow-ups.
* studio: show installed version in sidebar update card + collapse to icon
Replaces the placeholder version with the real installed app version via @tauri-apps/api/app getVersion() (Tauri-only; hidden in browser). Collapsed sidebar now shows just the clock icon instead of hiding the card.
* studio: open Settings About (update section) when the sidebar update card is clicked
* studio: i18n the sidebar update button label and add aria-label
Address Gemini Code Assist review on #6545:
- wrap the hardcoded "Update available" label in t() (shell.updateAvailable,
en + zh-CN), matching the rest of the sidebar
- add aria-label so the collapsed icon-only button has an accessible name
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* studio: hide sidebar update card unless an update is available
Gates the card on useWebUpdateCheck so it stays hidden by default on both web and desktop, appearing only when the installed PyPI version is behind the latest release. Includes a TEMP localStorage dev override (devForceUpdateCard) to preview the card where there is no real update; remove before merge. Keeps the i18n label/aria-label; desktop (Tauri updater) detection not wired yet.
* Polish Studio sidebar update affordance
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* Studio: lazy-import matplotlib so the server starts when the wheel is blocked
matplotlib.pyplot was imported at the top of core/training/training.py, on the
server boot path. When matplotlib's native extension fails to load (e.g. an
unsigned wheel blocked by Windows Smart App Control), that import crashed the
whole Studio server at startup instead of just disabling loss plots.
Move it into a lazy _load_pyplot() helper called from _create_loss_plot, using
the headless Agg backend, and return None when matplotlib is unavailable so
plotting degrades gracefully. The plot return was already Optional, so callers
need no changes. Keep the type-only import under TYPE_CHECKING and quote the
annotations.
Fixes#6588
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: pin matplotlib==3.11.0
Pin matplotlib to the current latest so a new unsigned release does not
reintroduce the Smart App Control block on Windows. Belt-and-suspenders on
top of the lazy import. Pinned in both studio.txt and extras.txt.
* Pin matplotlib to 3.10.9 so Studio still installs on Python 3.10
matplotlib 3.11.0 requires Python >=3.11, so the pin had no installable wheel on
Python 3.10 (still supported) and pip install failed there. 3.10.9 is the latest
3.10.x (requires-python >=3.10) and covers Python 3.10 through 3.13. Also tighten
the lazy-import docstrings.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Fix Gemma 4 GGUF OpenAI API streams
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid duplicate Responses stream disconnect watcher
* Keep reasoning-only Responses output hidden
* Address Gemma stream review comments
* Avoid Responses stream task-group cleanup
* Harden OpenAI chat completion streams
* Address OpenAI stream review issues
* Clean up Studio OpenAI stream helpers
* Fix Studio passthrough cold stream timeout
* Fix tool parser compatibility exports lint
* Preserve audio stream disconnect cancellation
* Avoid synthetic finish after passthrough errors
* Address stream cleanup and Gemma parser reviews
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gemma 4: parse bare-string tool args and keep safetensors tools for native <|tool_call>
- Quote bare unquoted string values in Gemma native tool-call args (e.g.
{location:Tokyo,unit:celsius}) so they parse; JSON scalars stay typed.
- Stop _detect_safetensors_features from suppressing supports_tools for
templates that emit Gemma native <|tool_call>, which the shared parser
now reads.
- Add tests for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden Gemma tool-call parsing and stream-error detection
Address three issues in the Gemma-native tool-call path:
- _quote_gemma_object_keys stopped a bare (unquoted) string value at the
first comma, so an argument like `location:New York, NY` was split
mid-value and the synthesized JSON failed to parse, dropping the whole
tool call. A bare value now ends only at `}` or a comma that begins the
next `key:` pair.
- parse_tool_calls_from_text scanned the entire response for Gemma markers
even inside a tool call already parsed from a `<tool_call>{...}` JSON
block, so a marker-like string inside an argument (data) was promoted to
a second, unintended tool call. Matches inside an already-consumed call
span are now skipped.
- _openai_passthrough_stream relied on _monitor_openai_sse_line to flag a
stream error, which returns early when monitor_id is None
(skip_api_monitor), so an upstream error chunk left saw_stream_error
unset and the synthetic-finish guard emitted a successful finish_reason
after a failed stream. Error chunks are now detected independently of API
monitoring.
Adds tests/test_gemma_tool_parse_edge_cases.py covering the comma and
marker-injection cases.
* Emit the terminal finish_reason chunk in GGUF streams
The OpenAI chat-completions GGUF tool stream and plain stream both built a
final ChatCompletionChunk carrying finish_reason but never yielded it, so
clients received the optional usage chunk and [DONE] with no chunk carrying
finish_reason. OpenAI-compatible consumers rely on that terminal choice to
distinguish stop/length/tool_calls. Yield it before the usage chunk and
[DONE], matching the other streaming paths.
* Parse tool calls in document order and skip nested markers both ways
Unify the JSON- and Gemma-format tool-call passes into a single
position-ordered scan:
- Calls are now emitted in byte order across both formats, so a mixed
output like `<|tool_call>call:create{...}<tool_call|> ... <tool_call>
{"name":"read",...}</tool_call>` executes create before read, matching
the order they appear in (tools run in returned order).
- A candidate that starts inside an already-accepted call's span is
skipped, in both directions: a JSON marker inside a Gemma argument and a
Gemma marker inside a JSON argument are treated as data, not promoted to
a second executable tool call.
Extends tests/test_gemma_tool_parse_edge_cases.py with the ordering and
JSON-in-Gemma nesting cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Quote bare Gemma array elements; order finish before trailing usage
- _quote_gemma_object_keys skipped array values, so a Gemma call with a
bare-string array argument like labels:[bug,ui] produced invalid JSON and
the whole tool call was dropped. Array values are now scanned and bare
string elements quoted, while numbers, quoted strings, and JSON literals
are preserved.
- In the OpenAI passthrough stream, a trailing usage-only chunk
(stream_options.include_usage) that arrived before any finish chunk was
relayed before the synthetic finish, producing usage -> finish -> [DONE].
Emit the synthetic finish before that usage chunk so the order matches the
other streams (finish -> usage -> [DONE]).
Extends tests/test_gemma_tool_parse_edge_cases.py with the bare-array cases.
* Harden Gemma array parsing, XML-parameter guard, and stream teardown
Address five review findings on the Gemma tool-call and OpenAI passthrough
streaming paths:
- parse_tool_calls_from_text collected JSON and Gemma markers without the
_inside_open_parameter guard, so a marker embedded in an existing
<function=...><parameter=...> value was promoted to a separate tool call.
Candidates that start inside an open XML parameter are now skipped, matching
the guard the XML-style parser already applies.
- _quote_gemma_array_elements preserved array elements starting with { or [
verbatim, so an array of objects (items:[{path:a}]) or a nested array failed
json.loads and the whole call was dropped. Object and nested-array elements
are now normalised recursively.
- _openai_passthrough_stream synthesized a finish chunk before a trailing
usage-only chunk and set saw_finish_reason, which made the EOF guard skip the
[DONE] sentinel. The EOF path now emits [DONE] whenever the upstream omitted
it, even after a finish chunk was already synthesized.
- /generate/stream drove generation through asyncio.to_thread with no
disconnect watcher, so a client disconnect during a long generation went
unnoticed until the next send. It now runs _await_disconnect_then_cancel
against the request, matching the other local streaming endpoints.
- _SameTaskStreamingResponse closed the body iterator with aclose() on a
send-side disconnect, raising GeneratorExit so the generators' cancellation
handlers (which finish the api_monitor entry) never ran. It now throws
CancelledError, falling back to aclose() when athrow is unavailable.
Extends tests/test_gemma_tool_parse_edge_cases.py with array-of-objects,
nested-array, and marker-inside-XML-parameter cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Watch disconnects on Anthropic streams; keep timestamps in Gemma values
Two follow-ups on the streaming and tool-parse paths:
- _anthropic_tool_stream and _anthropic_plain_stream drove generation through
asyncio.to_thread(next, gen, ...) and only polled is_disconnected() between
events, so a client disconnect during prefill or a long generation/tool step
held the decode slot until the next event or a failed send. Both now run the
_await_disconnect_then_cancel watcher used by the other local streams, stop it
in finally, and break promptly when cancel_event is set.
- _GEMMA_NEXT_KEY_RE treated any comma followed by word-chars-then-colon as the
next key, so a bare value such as "meet at 10:00, 11:00 tomorrow" was split
into bogus keys. The next-key token must now be identifier-shaped (start with
a letter or underscore), so a comma before a timestamp, ratio, or other
numeric-then-colon text stays part of the value.
Adds a timestamp-in-bare-value regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard nested markers, reset on disconnect, clean unstarted streams
Three follow-ups on the tool-parse and streaming paths:
- parse_tool_calls_from_text only skipped markers that fell inside a span it
had already parsed successfully, so when an unquoted Gemma argument contained
a literal marker (code:<|tool_call>call:terminal{...}<tool_call|>) the outer
object failed to normalize, its span was never recorded, and the inner marker
was promoted to a standalone terminal call. Candidates nested inside any other
candidate's brace span are now skipped regardless of whether the enclosing
candidate parsed, so a marker in malformed outer data is never executed.
- /generate/stream skipped backend.reset_generation_state() when the disconnect
watcher set cancel_event between chunks: the loop broke and the finally's reset
is guarded on cancel_event being unset. A subprocess backend kept decoding
after the client left. The cancel-break path now resets the backend.
- _SameTaskStreamingResponse threw CancelledError / called aclose() on the body
iterator on a send-side disconnect, but neither runs the try/finally of a
generator that never started (early disconnect on http.response.start), so the
passthrough's eagerly-opened upstream httpx stream and cancel-registry entry
leaked. It now tracks whether the body started and, when it did not, runs an
optional unstarted_cleanup hook; the OpenAI passthrough wires it to close the
upstream resp/client and exit the cancel tracker.
Adds a nested-unquoted-marker regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* studio: report the true reasoning duration and fix the Stop button for thinking models
For a local GGUF the "Thought for N" label was timed entirely on the client by a
brittle edge-detector, so an always-think model (Qwen3 MTP) that buffers its whole
reasoning and flushes it in one chunk showed "1 second" instead of the real
minute-plus. The client cannot time reasoning it receives atomically, so make the
timing backend-authoritative.
Backend: generate_chat_completion_with_tools measures wall-clock reasoning and
emits a Studio reasoning_summary event (duration_ms) at the moment reasoning ends
-- the first answer token, or end-of-stream for a reasoning-only reply -- for both
the tool-detection pass and the final-answer pass. Timing resets per tool
iteration so the final answer's thinking time wins on the client (which takes the
latest reasoning_summary). routes/inference.py forwards the event in the GGUF tool
stream.
Frontend: parse the reasoning_summary SSE into a _reasoningDurationMs chunk and
use it as the authoritative reasoning duration (last write wins), clamped to >= 0
and guarded to a finite number so a malformed or proxied chunk cannot produce a
NaN label; the persisted value wins for the final "Thought for N" label, with the
previous live timer kept only as a fallback when no metadata arrives.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
#6579 reworded these comments to attribute the failure to a half-resolved
install and claimed a clean 4.14 is fine on 3.13. That is wrong: #6483 is a
genuine anyio 4.14 + Python 3.13 regression. 4.14 added a per-task cancel
scope in its asyncio backend (TaskHandle/_run_coro) that gets exited in the
wrong task under starlette's collapsing task group, raising the cancel-scope
RuntimeError on streaming; 4.13 has no such code and is unaffected (the
reporter confirmed 4.13.0 fixes it). The TaskHandle ImportError is only the
secondary macOS-arm symptom from the mlx-vs-cap version fight. Comments only.
* Studio: start the Cloudflare tunnel for --secure even in --api-only, and add --api-only to `unsloth studio run`
--secure exposes ONLY the Cloudflare link (it forces a loopback bind), but
_cloudflare_tunnel_should_start gated the tunnel on `not api_only`, so
`run.py --secure --api-only` started no tunnel and then fail-closed with
"A secure Cloudflare link is not allowed". That blocked the natural headless
use: serve just the API (no web UI) over the authenticated tunnel.
Make --secure start the tunnel regardless of api_only (the non-secure path is
unchanged: tunnel only a 0.0.0.0 bind, never api-only Tauri or Colab). Then
expose --api-only on `unsloth studio run` and forward it through both the
re-exec args and the in-venv run_server call, so
`unsloth studio run --secure --api-only --model ...` is a one-liner secure API
server.
Verified end to end: `run.py --secure --api-only` now brings up the tunnel and
serves /api/health over it (200), with / returning 404 (no UI).
Tests: update the tunnel-gate truth table (secure+api-only now tunnels;
secure+colab still does not) and add --api-only registration + re-exec/in-venv
forwarding coverage to the run CLI tests.
* Trim comments to be succinct (no behavior change)
* studio: address review on parent --api-only and secure api-only CORS
- Reject --api-only on the parent `unsloth studio` group when a subcommand
is invoked, with the same redirect guidance used for --parallel/--secure;
otherwise the flag was silently dropped and the UI served anyway.
- Keep CORS any-origin for secure api-only serving: that mode publishes the
API over Cloudflare for remote browser clients, so the Tauri-only lockdown
(still applied to plain local api-only) would break preflight. Factored the
decision into cors_origins_for_mode() and gate it on api_only and not secure;
run_server exports UNSLOTH_SECURE before importing main.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: suppress TAURI_PORT and de-dup test for headless run --api-only
- run_server gains emit_tauri_port (default True, unchanged for the Tauri/
desktop path). The new headless `run --api-only` path passes False so the
Tauri-only TAURI_PORT= line no longer prepends the documented URL/API key
banner (it ran even under --silent and could break one-liner parsers).
- Remove a duplicate test_reexec_forwards_api_only that shadowed the
parametrized one; fold the --secure --api-only case into it so the secure
headless path is actually collected.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
torchao has no working Windows ROCm build. transformers.quantizers imports it,
and it loads torch's c10d distributed backend at module level, which the AMD
Windows wheels omit (no RCCL). The import aborts, transformers can no longer
expose PreTrainedModel, and the sentence-transformers embedder silently falls
back to the llama-server GGUF embedder. Linux ROCm and NVIDIA are unaffected
(the c10d ops are present / torchao is real there).
The training and export workers already install the shared torchao stub before
importing transformers, but the RAG embedder runs in the main backend process,
which never did. Two fixes, both no-ops off Windows ROCm:
- embeddings.py: install_torchao_windows_rocm_stub() before the first
sentence-transformers import, so an already-installed torchao is neutralized
(fixes existing venvs).
- install_python_stack.py: stop installing torchao on Windows ROCm; it can only
crash on import there, so new venvs never ship it.
Add tests covering the embedder stub call and the install skip.
PR #6364 added a branch that overrode the unsloth owner avatar with the
bundled circle-logo-small.png sticker. Revert it so unsloth uploads use
the live Hugging Face org avatar again, falling back to the colored
initial tile. Upstream re-uploads are unaffected since they still
resolve through provider logos.
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
* Chat: match reasoning thinking icon to the composer bulb
The reasoning "Thinking..." indicator used lucide's LightbulbIcon while
the composer thinking toggle used a custom bulb glyph, so the two did not
match. Move that glyph into lib/bulb-icon.tsx and use it in both places
so they render the same icon.
* Let BulbIcon take and override svg props
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
* Studio: fall back to anonymous HF browsing on a malformed token
The Discover/Recommended feeds call the Hugging Face JS client
(`listModels`/`listDatasets`) directly from the browser. That client
throws `Your access token must start with 'hf_'` when handed a non-empty
token that isn't a well-formed HF token, instead of falling back to
anonymous access. A single bad value left in the HF token field (e.g. a
placeholder someone typed) therefore takes down the entire discovery feed
even though it works fine with no token at all.
Add `hfApiToken()` to the HF token store, which returns the token only
when it looks like a real `hf_...` credential and `undefined` otherwise,
and route hub-page's four HF call sites through it. Malformed tokens now
degrade to anonymous public browsing rather than erroring. The raw token
is still stored and shown in the settings field unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Studio: trim hfApiToken comments
Collapse the 11-line JSDoc to a 2-line note and drop the redundant
call-site comment in hub-page. AST signature check confirms code is
unchanged (comments only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Studio: treat data-center Blackwell (sm_100/sm_103) as Blackwell in llama.cpp prebuilt selection
_host_is_blackwell gated on _BLACKWELL_MIN_SM = 120, but data-center Blackwell
parts report a lower compute capability than consumer Blackwell: B100/B200 are
sm_100 and B300/GB300 are sm_103, while RTX 50 is sm_120 and DGX Spark is
sm_121. Because 100 and 103 are both < 120, every data-center Blackwell host was
classified as non-Blackwell, so two GPU-targeting paths never fired for a
B200/B300:
- the Linux blackwell_runtime_override that prefers the highest CUDA-major
runtime line shipping a bundle covering the host SMs (so a cu12x torch could
pin a cuda12 bundle over a native cuda13 one), and
- _drop_blackwell_incapable_windows_cuda, which removes cuda-12.4 builds that
load and validate but run Blackwell on a slow PTX-JIT path.
The result is a B200/B300 being handed a prebuilt that does not natively offload
its SM, i.e. the llama.cpp prebuilt is not really for the GPU. The Blackwell
floor is sm_100, so set _BLACKWELL_MIN_SM = 100. The toolkit floor (12.8) is
unchanged and already correct for sm_100/sm_103.
Surfaced loading unsloth/GLM-5.2-GGUF UD-IQ1_S on 8x B200.
Adds tests covering the sm_100/sm_103 classification, the Linux cuda13
preference for a data-center host, and the Windows cuda-12.4 drop.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be succinct (no behavior change)
* studio: require CUDA 12.9 for sm_103/sm_121 Blackwell prebuilts
sm_103 (B300/GB300) and sm_121 (DGX Spark) have no native compiler
target before CUDA 12.9; the family floor of 12.8 only covers
sm_100/101/120. Make the Windows-CUDA Blackwell filter SM-aware so a
legacy win-cuda-12.8 bundle is dropped on an sm_103/sm_121 host while
sm_100/sm_120 hosts keep the 12.8 floor.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio): sort export checkpoints by step
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Withhold HF_TOKEN from pull_request runs of CI workflows
The pull_request-triggered CI workflows check out and execute PR-controlled
code (install.sh, .github/scripts/**, tests/**) with secrets.HF_TOKEN in the
step environment. For a same-repo PR, GitHub provides repository secrets to the
run, so a malicious or compromised branch could modify a checked-out script to
read and exfiltrate HF_TOKEN, including by writing it into the uploaded logs/
artifact. HF_TOKEN is an external Hugging Face credential of unknown scope, so
this is the high-value exposure.
Gate every HF_TOKEN reference in these workflows with
`github.event_name != 'pull_request' && secrets.HF_TOKEN || ''`, so the real
token flows only on the trusted schedule/push/workflow_dispatch runs and PR runs
see an empty string. All model repos used by these jobs are public
(unsloth/*-GGUF), so anonymous download still works on PRs; install_llama_prebuilt.py
only sends HF auth to Hugging Face hosts and tolerates an absent token.
GITHUB_TOKEN (passed as GH_TOKEN) is intentionally left in place: it is the
auto-provisioned, job-scoped, contents:read token that expires with the job and
gives a same-repo PR author nothing they do not already have, and
install_llama_prebuilt.py needs it to authenticate the GitHub releases API or
the prebuilt llama.cpp download hits the anonymous rate-limit bucket and 403s.
* Trim the HF_TOKEN gating comments to one line per site
Comment/whitespace-only: collapse the per-step rationale to a single line and
shorten the local-agent-guides header note. No workflow logic changes (verified
each file's parsed YAML is identical before/after).
* Fix FlashAttention fp32 crash with DoRA (use_dora=True)
DoRA upcasts lora_magnitude_vector to fp32 for the optimizer, which promotes
the q/k/v_proj output to fp32. FlashAttention only accepts fp16/bf16, so the
fp32 q/k/v raised 'FlashAttention only support fp16 and bf16 data type'.
Downcast q/k/v to the compute dtype before the flash kernels.
Fixes#1013
* Apply kwarg-spacing format hook to DoRA dtype test (pre-commit)
* DoRA+FA2: downcast any fp32 among Q/K/V and clamp to a flash-supported dtype
* Tighten code comments (no logic change)
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Fix misleading 'only for image models' error for Qwen3-VL when torchvision is missing
transformers >= 5.4 hard-requires torchvision for VLM image/video processors and
no longer falls back to a slow processor. Without torchvision the processor load
raises ImportError, unsloth degrades to a text-only tokenizer, and the vision data
collator later fails with 'UnslothVisionDataCollator is only for image models!'.
Detect this case at load time and raise a clear, actionable error pointing at the
missing torchvision dependency instead.
Fixesunslothai/unsloth#4202
* Apply kwarg-spacing format hook to vision torchvision guard (pre-commit)
* Make torchvision-missing detection precise: check availability first, match specific error text
* Tighten code comments (no logic change)
* Make missing-torchvision VLM error version-agnostic
The raise also fires on transformers 4.57.x for VLMs with a video processor
(Qwen2.5-VL, Qwen3-VL), where AutoVideoProcessor requires torchvision. The old
message claimed 'transformers >= 5.4 requires torchvision', which is inaccurate
on 4.57.x. Reword to state torchvision is required for this model's vision
processors without a version-specific claim.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Fix GRPOTrainer evaluate() crash when called without prior training
GRPOTrainer.compute_loss read self.current_gradient_accumulation_steps
directly. That attribute is only set by the transformers training loop, so
calling trainer.evaluate() standalone (no prior trainer.train()) raised
AttributeError. Read it via getattr with a fallback to
args.gradient_accumulation_steps so standalone evaluation works.
* GRPO eval: fall back accumulation steps to 1 so standalone eval_loss is not underreported
* Tighten code comments (no logic change)
* fix: use EMPTY_LOGITS on the fused-CE not-return_dict path (#2068)
CausalLM_fast_forward's fused cross-entropy path (small batch, labels set,
UNSLOTH_RETURN_LOGITS off) computes the loss straight from hidden_states
via unsloth_fused_ce_loss and never materializes `logits`. The
return_dict=True branch returns EMPTY_LOGITS, but the `not return_dict`
branch returned `(logits,) + outputs[1:]`, raising
"UnboundLocalError: cannot access local variable 'logits'" whenever it ran
(e.g. training with return_dict=False). Same bug in the llama and mistral
fast-forward paths.
Return EMPTY_LOGITS on that branch too, matching the adjacent return_dict
output. Verified on GPU: a forward(return_dict=False, labels=...) that
raised UnboundLocalError now returns (loss, EMPTY_LOGITS, ...) and
backward() succeeds.
Adds tests/test_fused_ce_not_return_dict_logits.py, a CPU source-drift guard
(the fused path itself is GPU/triton only) asserting both fast-forward paths
keep using EMPTY_LOGITS there.
* Address review: parse the fused-CE drift line with whitespace-tolerant regexes
The drift detector sliced the source with exact string matching
(source.index("output = (") + the next newline), so a formatter respacing or
rewrapping the assignment would break the parse. Switch to anchored regexes that
tolerate whitespace and line wrapping, keeping the match anchored after the
fused guard so it targets the fused-CE branch and not the normal
output = (logits,) path. Behavior and the two drift assertions are unchanged.
* Tighten code comments (no logic change)
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* studio/setup.sh: guard empty CUDA arch detection in the source build
PR #5826 hardened setup.sh for fresh CUDA toolkits, but the source build
still set -DCMAKE_CUDA_ARCHITECTURES only when nvidia-smi reported a
compute capability. When that query returns nothing the build proceeded
with no explicit arch list, so llama.cpp built PTX only. On a driver older
than the toolkit that binary fails at runtime with "the provided PTX was
compiled with an unsupported toolchain" - the build succeeds, so neither
the build-time check nor the CPU fallback caught it (issue #5854).
Resolve the arch list before committing to a CUDA build. A new pure helper
_resolve_cuda_archs parses and de-duplicates the nvidia-smi compute_cap
output and honors an explicit UNSLOTH_LLAMA_CUDA_ARCHS override. When the
result is empty, build CPU llama.cpp instead of a PTX-only binary, with a
clear message pointing at the override - so the user still ends up with a
working llama-server. The override also lets advanced users force a native
build on hosts where nvidia-smi cannot report compute_cap.
No behavior change when an arch is detected: -DGGML_CUDA=ON plus the arch,
CUDA flags and NVCC_PREPEND_FLAGS are assembled exactly as before.
Adds tests/sh/test_resolve_cuda_archs.sh (single/multi/dedup/empty/garbage/
whitespace/override cases), wired into tests/run_all.sh and the
studio-backend-ci.yml shell-test loop.
* studio/setup.sh: resolve nvidia-smi via /usr/bin fallback for arch detection
Addresses review feedback on the empty-CUDA-arch guard: _setup_has_usable_nvidia_gpu
classifies a host as NVIDIA-usable using nvidia-smi on PATH OR /usr/bin/nvidia-smi,
but the new arch detection probed only `command -v nvidia-smi`. On a GPU host where
nvidia-smi is off PATH (reachable only at /usr/bin), arch detection returned empty
and the new empty-arch branch dropped the build to CPU, losing CUDA. Mirror the same
PATH-then-/usr/bin resolution so those hosts still get a native CUDA build.
Also scope _resolve_cuda_archs locals with `local` (no behavior change; it already
runs under command substitution).
* tests: update compute_cap-probe assertion for $_smi_bin resolution
The nvidia-smi /usr/bin fallback parameterized the binary in the compute_cap
probe (_setup_run_smi "$_smi_bin" ...), so the literal-string assertion in
test_compute_cap_probe_timeout_wrapped no longer matched. Assert the probe is
preceded by _setup_run_smi (timeout-wrapped) instead, scanning all occurrences
so the comment mention is ignored. Same intent, binary-agnostic.
* tests: ruff-format the compute_cap probe assertion (pre-commit)
Collapse the backslash-continued assert onto one line and normalize slice
spacing so the ruff-format pre-commit hook (0.6.9) is satisfied. Formatting
only; no behavior change.
* Tighten code comments (no logic change)
* studio(windows): build CPU when CUDA arch is undetectable (#5854)
The Windows source build added -DGGML_CUDA=ON unconditionally but only set
-DCMAKE_CUDA_ARCHITECTURES when $CudaArch was detected. With no detectable
compute capability that produced a PTX-only binary, the same hole the Linux
fix closed. Build CPU llama.cpp in that case, and honor UNSLOTH_LLAMA_CUDA_ARCHS
to force a CUDA build, matching setup.sh. Detected-arch builds are unchanged.
* test: anchor NVCC_PREPEND_FLAGS scope check on the final CPU branch
The undetectable-arch CPU fallback adds an earlier -DGGML_CUDA=OFF, so the
ordering check now anchors on -DGGML_CUDA=ON and the last -DGGML_CUDA=OFF
instead of the first.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* fix(studio): handle multimodal list content in inference text paths
Studio receives chat message content in two shapes: the legacy string
form, and the OpenAI multimodal list form
([{"type": "text", "text": ...}, {"type": "image_url", ...}]).
Several string-only paths called .strip()/re.sub()/f-string interpolation
on content directly, raising "'list' object has no attribute 'replace'"
for vision models (issue #4383), or rendering the list repr into the
prompt for the manual chat-template formatters.
Add core/inference/message_content.py with content_to_text(), a pure
helper (no heavy imports) that returns strings unchanged and joins the
text parts of a list while dropping image/audio parts. Apply it at every
string-only content site: _generate_vision_response, the audio user-text
extraction, format_chat_prompt, and the llama3/mistral/chatml/alpaca/
generic template formatters. The plain-string path is a no-op, so
existing behavior is unchanged.
Adds tests/test_message_content.py covering str/None/list/tuple,
multimodal drop, multi-part join and empty-part skipping.
* Tighten code comments (no logic change)
* studio: join multimodal text parts with newline for llama.cpp parity
llama.cpp joins multiple text content parts with a newline (common/chat.cpp),
so match that in content_to_text instead of a single space.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Three independent upstream CI fixes that currently fail on every open PR:
verify_import_hoist.py: TARGET-CHANGED only flags a genuine swap (a BEFORE
target no longer reachable in AFTER). A pure superset growth such as adding
import urllib.error next to import urllib.request binds the same top-level
package and loses nothing, so it is no longer a blocker (transformers_version.py).
test_vision_cache.py: run each test from a fresh empty cwd. is_vision_model
calls is_local_path first, and a relative model id that happens to exist on
disk short-circuits before the mocked detection runs; the CI cwd and HF cache
can contain dirs colliding with the synthetic ids, causing 'called 0 times'.
Production code is correct; only the test needed cwd isolation.
consolidated-tests-ci.yml: the llama.cpp smoke probes the first of
llama-cli / llama-mtmd-cli / llama-server that exists instead of hard-requiring
llama-cli, which upstream no longer always builds. llama-cli stays first so it
is preferred when present. Adds Windows .exe + build/bin/Release handling.
* Studio: refresh chat tour for the redesigned model picker
- Pick a model step describes the Recommended and On Device tabs instead of the old Hub and Fine-tuned split
- Find a model step (was Two tabs) covers Unsloth search vs Search Hub, the format and sort filters, and the OOM tag
- Settings step now anchors to the run settings panel on the right. The old anchor sat on the open settings button, which unmounts when settings opens, so the tooltip lost its target and drifted left
* Add guided tour step for the composer + menu
Polish for the in-chat model picker popover and its guided-tour step.
- Search box placeholder reads Search Unsloth models, matching the Unsloth-only listing.
- Search Hub button shows a Search all models tooltip on hover.
- Floating Eject pill moves 1px lower so it sits closer to the bottom edge.
- Results list max height trimmed by 1px (21rem to 335px) from the bottom only.
- Chat guided tour Two tabs step updated to describe Unsloth-scoped search plus Search Hub for all of Hugging Face.
- Discover defaults to the whole Hub instead of the unsloth org; an explicit
Unsloth choice is still remembered
- Discover models placeholder reads Search all models to match
- Give the Unsloth/All scope pill a min width so it stays readable
* Use UTF-8 for Python code-execution subprocess I/O
Studio's code-execution tool already tells the child to emit UTF-8
(PYTHONIOENCODING=utf-8 in _build_safe_env), but _python_exec writes the
temp script and decodes the subprocess pipe with the OS default codec.
On Windows (cp1252), non-ASCII in model-written code or its output --
arrows, CJK, emoji -- raises UnicodeEncodeError / UnicodeDecodeError and
breaks execution.
Complete the UTF-8 wiring in core/inference/tools.py:
- write the temp script with encoding="utf-8"
- decode _python_exec stdout as utf-8, errors="replace"
- set PYTHONIOENCODING=utf-8 in _build_bypass_env too (matches
_build_safe_env, so the bypass path's child also emits utf-8)
The child is python with PYTHONIOENCODING=utf-8, so it emits UTF-8
regardless of the console code page and the decode is always correct.
Shell execution via cmd.exe has a separate console-code-page story and
is left to a follow-up.
Refs unslothai/unsloth#6489
* Scope Python exec UTF-8 env to Python tool
* Make bash bypass test robust to a host-set PYTHONIOENCODING for PR #6548
Bypass mode preserves benign host env vars, so a host-set PYTHONIOENCODING was
inherited into the bash bypass env and tripped the new assertion even though
_bash_exec never adds it. Clear it in the test so the assertion checks _bash_exec,
not the runner environment.
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: correct the anyio<4.14 pin rationale (mixed-install ImportError)
The pin comments said "anyio 4.14+ breaks cancel scope on Python 3.13", but
a clean anyio 4.14.0 works on 3.13 (cancel scopes, Event, and the asyncio
backend import all pass). The actual failure is a half-resolved install:
anyio 4.14 added TaskHandle, imported by __init__.py and _backends/_asyncio
from _core/_tasks. When a stale 4.13 _core/_tasks (no TaskHandle) sits under
4.14's importers, the import raises ImportError and 500s the server. Correct
the rationale; the <4.14 pin still stands as the way to keep one consistent
anyio version.
* Clarify the anyio override comment (mixed-install ImportError, not a 4.14 cancel-scope bug)
* Studio: keep model downloads running across navigation and loads
Downloads started from the chat model selector were tied to the staged
pick lifecycle, so they were cancelled in cases where Hub downloads keep
going. This makes the chat download flow behave like the Hub.
- Leaving the chat route or switching thread/project/new chat now detaches
the staging UI but keeps the in-flight transfer running in the global
download manager (new keepDownload option on abandonStagedModel).
- Staging a second pick no longer cancels the previous pick's download, so
multiple models/variants can download at once.
- Picking a model to download while another model is loading now starts the
download in the background instead of refusing, since a download is
independent of a load.
* Studio: also background-download remote GGUF quants while a model loads
isDownloadableHubRepo (wantManagerDownload) excludes GGUF sources, so an
uncached remote GGUF quant picked from the chat selector while another model
was loading fell through to the 'Another model is already loading' toast
instead of downloading in the background. Treat an uncached remote hub GGUF as
a background download too, matching the staged-pick download path.
Addresses review feedback from gemini-code-assist and codex on PR #6573.
* Studio: only toast a background download once it actually starts
The chat background-download path (used when a model is already loading)
fired the "Downloading in the background" toast unconditionally, but
requestStart can return without starting a job: a cross-transport partial
records a conflict that is only resolvable from the Hub download card, and
a busy sibling variant returns after its own toast. So the user could be
told a download started when none did, with no way to resolve the conflict
from chat.
requestStart now reports an outcome (started/conflict/busy/error). The
chat path only shows the success toast on an actual start and points the
user to the Hub when a transport conflict needs resolving. The Hub card
surface keeps its existing behavior (it renders the conflict resolver, so
it ignores the outcome).
* Studio: report background-download outcome from real job state
The chat background-download toast trusted requestStart's optimistic
"started", but a start can no-op without throwing: startJob finalizes the
job as "error" when the backend refuses or fails apiStart, its peer guard
skips a fresh start, and hasActiveOrPendingStart trips on a snapshot, peer
variant, or pending preflight that is not this request. So the user could
be told a download started when none did.
Derive the outcome from the actual job state of the exact key
(running/cancelling = started, otherwise error/busy), so the toast only
fires for a transfer that is really live.
Also guard against re-downloading the model that is already loading: the
/load flow downloads before it sets the checkpoint, and that fetch is not
a download-manager job, so picking the same id+variant again would start a
second transfer against the same cache. Detect that pick and surface a
"this model is already loading" toast instead.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Restore sys.modules in test_pre_import_gate_is_transformers_free
The test pops transformers and utils.models.model_config from sys.modules to
assert the pre-import security gate does not re-import them, but never put them
back. A later importer then rebound a fresh utils.models.model_config, so tests
that had captured the original instance missed their patches and hit the real
path: test_vision_cache patches _is_vision_model_uncached on the original
module, but is_vision_model (still bound to that original) ran the real network
lookup instead. This produced 17 spurious failures whenever test_ssm_runtime
ran before test_vision_cache in the same process.
Snapshot the removed modules and restore the original objects in a finally, so
the assertions still run against a clean slate while later tests see the same
module instances they captured at import time.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The macOS-arm studio venv still installs anyio 4.14.0 despite the
constraints.txt cap from #6546. mlx-vlm / mlx-lm pull anyio>=4.14, which
conflicts with the anyio<4.14.0 constraint; a uv -c constraint loses that
conflict so 4.14.0 gets installed, reintroducing the cancel-scope
RuntimeError on Python 3.13 (#6483). UV_OVERRIDE is already applied on
macOS-arm via overrides-darwin-arm64.txt and a uv override wins the
conflict, so cap anyio there too. macOS-arm now resolves anyio 4.13.0.
* Studio: hide RAG embedder from the On Device list
The bge-small-en-v1.5 RAG embedder (and other infra models) were already
hidden from Discover but still showed up in the On Device browse list,
cluttering the user's downloaded models. They are now filtered out of On
Device the same way, while a search that matches still reveals the row so
the user can confirm it is already downloaded.
* Studio: also check path/title when hiding infra models from On Device
isHiddenModelId only saw row.id and row.repoId, but local inventory rows can
have a null repoId and an id that is a hash rather than the file path/name, so
the llama.cpp validation probe (stories260K.gguf) could slip into the On Device
list. Pass the local row's path and title too, mirroring the backend's
_is_hidden_model(m.id, m.path).
Addresses review feedback from gemini-code-assist on PR #6572.
* Studio: exclude infra models from On Device count and dataset list
The On Device hidden-model filter was applied to datasets too, so a
dataset whose id/title/path contained an infra needle (bge-small-en-v1.5,
stories260k.gguf) was wrongly hidden. Bypass the filter for datasets, the
same way Discover and the format filter already do.
The On Device header count and the Cache/Local stat pills still used the
unfiltered row counts, so a fresh install with only the bge embedder
cached read 1 over an empty list. Count visible (non-infra) rows instead,
keeping full counts for datasets.
* Studio: count search-revealed infra rows in the On Device tally
The visible-row counts excluded every hidden row unconditionally, but the
On Device list reveals a hidden row when the search query matches it. So
with only the bge embedder cached and a "bge" search, the list showed one
row while the header and Cache stat stayed 0. Reuse isVisibleInventoryRow
for the counts so a query-revealed row is counted, keeping them in step
with the list.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio Playwright: snooze update banner before sending
The llama.cpp update banner is a fixed bottom-right toast (z-9998). When an
update is available it overlaps the composer's Send button and its subtree
intercepts the click, so send_and_wait times out (flaky; surfaces on the
Windows studio UI smoke, passes otherwise). Snooze the banner if it is
showing before each send, then wait for it to detach.
* Also snooze the web update banner before sending
The web update banner (web-update-banner, z-9999) is a fixed bottom-right
toast like the llama.cpp one and can overlap the Send button too. Loop over
both banners and snooze whichever is showing.
* Studio: honor custom HF_HOME for model download and load
_setup_cache_env always derived HF_HUB_CACHE and HF_XET_CACHE from
XDG_CACHE_HOME / ~/.cache, ignoring a user-set HF_HOME. Because it sets
HF_HUB_CACHE explicitly and that variable takes precedence over HF_HOME
in huggingface_hub, the hub cache was pinned to the standard location: a
model already present under a custom HF_HOME was detected but then
re-downloaded from scratch on load.
Seed HF_HUB_CACHE and HF_XET_CACHE from HF_HOME when the user set it
(HF's own default is $HF_HOME/hub and $HF_HOME/xet), and honor the legacy
HUGGINGFACE_HUB_CACHE alias. The hub download workers call
snapshot_download without a cache_dir for both the Xet and HTTP-fallback
paths, so they follow HF_HUB_CACHE; fixing it here unifies detection and
both transports on one root. Explicit HF_HUB_CACHE / HF_XET_CACHE stay
untouched. Adds tests for the custom-HF_HOME, default, explicit-override,
and legacy-alias cases. Fixes#5182.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: do not crash startup when a custom HF_HOME is not writable
Seeding HF_HUB_CACHE/HF_XET_CACHE from HF_HOME means _setup_cache_env now
mkdir's under a user-controlled path. A non-writable or not-yet-mounted
HF_HOME (typo, offline drive) would raise and crash startup, where the old
code silently fell back. Make the mkdir best-effort; the env var is still
set, so HF reports a clear error at download time. Adds a regression test.
* Studio: strip blank HF_HOME and isolate cache-env tests
Address review: a whitespace-only HF_HOME no longer derives " /hub";
strip it and fall back to the default (matches studio_root). Tests set
UNSLOTH_STUDIO_HOME to a tmp dir so _setup_cache_env's UV/VLLM mkdirs do
not touch the real ~/.unsloth/studio. Adds a whitespace regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Resolve the transformers tier by probing AutoConfig instead of guessing
When the only signal is a 5.x tokenizer class, get_transformers_tier guessed the
lowest 5.x sidecar (530). That misroutes models whose built-in config parser needs
a higher tier: dense NemotronH ships a 5.x tokenizer but its '-' (MLP) layer only
transformers 5.10 can parse, so 5.3/5.5 raise KeyError '-'. The config.json
transformers_version field records the saving version, not the minimum to load, so
it cannot drive routing either.
Replace the weak tokenizer->530 guesses (local and remote) with a probe: parse
config.json with the built-in parser (trust_remote_code=False) in each sidecar,
escalating 530->550->510, and pick the first that succeeds. This generalizes to any
architecture without hardcoded lists. Strong signals stay fast paths (no subprocess);
the probe runs only when the tier is otherwise ambiguous and is cached by (model,
commit sha). It never executes repo code, never downloads weights, never raises, and
falls back to the legacy 530 guess on a transient/auth/offline failure or when no
sidecar is available. UNSLOTH_DISABLE_TIER_PROBE restores the old behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: tier probe fallbacks and cross-platform robustness
Codex:
- Never escalate to 510 on uncertainty. When every sidecar was probed and none
parsed with the built-in parser, the model is a remote-code / custom model_type
that loads via its own code; keep the legacy 530 route instead of jumping to
510 (which would change the behavior of models that worked on the 5.3 stack).
- Only cache the 530 fallback when the result is conclusive (every tier actually
probed). If a sidecar was missing/uninstallable the environment is incomplete,
so return 530 uncached and retry on the next call.
- Do not pin the tier cache under an unknown revision: _resolve_commit_sha no
longer memoizes a None sha (a transient Hub failure is retried), and _probe_tier
only caches a tier when the commit sha is known.
Gemini:
- Wrap Path.exists() in the sha resolver in try/except OSError (a remote repo id
can raise WinError 123 on Windows).
- Probe script writes the error to sys.stderr.buffer as UTF-8 bytes so a non-ASCII
message cannot itself raise UnicodeEncodeError under cp1252.
- subprocess.run decodes stderr with errors="replace" to avoid UnicodeDecodeError
on non-UTF-8 consoles.
Tests: 72 passed (added partial-sidecar uncached, sha-unresolved not cached,
all-failed stays 530 + cached, sha resolver retries None / handles OSError).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review round 2: authenticate tier checks, stop memoizing local sigs
Codex:
- Thread hf_token through _check_config_needs_510/550 and
_check_tokenizer_config_needs_v5 (and the underlying raw fetches). Previously a
gated/private model whose only 5.x signal is tokenizer_config.json never reached
the authenticated probe: the unauthenticated raw fetch failed and cached False,
so the model fell through to the default 4.x tier. The per-check caches are now
keyed by (model, token) so an unauthenticated miss cannot poison a later authed
read, mirroring _load_config_json.
- _resolve_commit_sha no longer memoizes a local directory signature. A local
signature is mutable (size/mtime of config/tokenizer), so a reused/overwritten
checkpoint path would otherwise keep selecting the previous tier; it is now
recomputed every call. Only the immutable remote commit sha is memoized.
Tests: 75 passed (added token-cache isolation + auth header, local signature not
memoized, token threaded into all checks/probe).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review round 3: reach activation with the token, drop SHA tier cache
Codex round 3:
- Thread hf_token into the activation path that actually selects a sidecar. The
token-aware tier checks added last round were unreachable:
activate_transformers_for_subprocess called get_transformers_tier without a
token, and the inference/training/export workers passed only the model name even
though they hold a request-scoped hf_token. activate_transformers_for_subprocess
now takes hf_token and the three workers forward config["hf_token"], so a
gated/private model whose only 5.x signal is an authenticated config/tokenizer is
routed to the right sidecar instead of falling to default 4.x.
- Stop importing huggingface_hub during tier detection. _probe_tier no longer
resolves a commit sha, so it never pulls huggingface_hub into the worker before
the sidecar venv is prepended to sys.path (activation only prepends, never
purges), which would otherwise pin the default-env hub over the sidecar's
pinned huggingface_hub==1.8.0.
- The tier cache is now keyed by model_name for the process lifetime (a model's
required tier is a property of its architecture; cleared on restart). This drops
the mutable-SHA memo that masked remote revision changes and the mutable
local-signature memo, removing _resolve_commit_sha / _local_dir_signature /
_probe_sha_cache entirely.
- Do not cache a probe success that depended on a skipped lower tier: if a lower
sidecar was unavailable, the lowest valid tier may change once it installs, so
the result is returned uncached and re-probed next call.
Tests: 73 passed (probe imports no hub; success uncached when a lower tier is
skipped; activation forwards the token).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Re-probe overwritten local checkpoints and authenticate the probe child
The AutoConfig tier probe cached its result under the bare model_name, so a
local checkpoint overwritten in place (same path, new config.json) kept serving
the stale sidecar. Fold a cheap config.json signature (size + mtime) into the
cache key for local paths; remote ids stay name-keyed so no huggingface_hub
import lands before the sidecar is activated.
The probe relies on the implicit HF_TOKEN env, so an inherited
HF_HUB_DISABLE_IMPLICIT_TOKEN=1 left it unauthenticated and a gated repo 401ed
into the 530 fail-safe. Clear that flag in the child env when a token is set.
* Keep tier probes off the log-only path and probe new 5.x archs default-first
- get_transformers_tier gains probe=True/False. needs_transformers_5 (a coarse
4-vs-5 boolean used only for a spawn log and a vision-check branch) now passes
probe=False, so a parent/log-only caller never spawns sidecar probes. The real
activation path keeps probe=True and resolves the exact tier in the worker.
- A config.json saved by transformers 5.x but matched by no fast path is now probed
default-first: _probe_tier gains include_default + floor, prepending the ambient
4.57.x tier to the escalation. A model that still parses on the default is left on
it (no mis-route onto a sidecar); only a config the default parser cannot read
escalates to the lowest 5.x tier that parses. The transformers_version field is a
cheap 'worth probing' hint only, read from the already-fetched config (no extra
network); ordinary 4.x configs never probe.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Separate probe cache by mode and keep version-field 5.x visible to needs_transformers_5
- _probe_tier cache was keyed only by config.json signature, so a default-first probe
that returned 'default' could be handed back to a later tokenizer/known-5.x caller
(floor=530), leaving a model with a 5.x-only tokenizer on transformers 4.x. Key the
cache by probe mode (floor + include_default); the legacy 530 mode keeps the bare key.
- The version-field 5.x detection is a cheap config read, not a probe, so run it even
when probe=False: a standard-tokenizer model whose only signal is transformers_version
>= 5 now classifies as 5.x via needs_transformers_5 (returns '530' without spawning a
probe), so the vision-routing fallback uses the 5.x subprocess instead of failing the
default parser and marking it non-vision. The real activation path still probes
default-first and may resolve 'default'.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Don't treat local checkpoints as Hub ids, and fix stale activation test double
- _load_config_json / _check_tokenizer_config_needs_v5: a local checkpoint dir whose
config.json / tokenizer_config.json is not yet present was being fetched from the Hub
as if the path were a repo id, and the 404 miss was cached. A later call after the
file is written (in-progress checkpoint) then served the stale miss, so a
TokenizersBackend checkpoint fell through to the default tier. Skip the Hub fetch for
local dirs and do not cache the miss, so the file is read once it appears.
- test_activate_transformers_version_or_warn_*: the worker now threads hf_token into
_activate_transformers_version (model_name, hf_token); update the one-arg test doubles
to the real two-arg signature so the silent-success path stays silent.
* Tighten comments in the AutoConfig probe and tier-selection paths
* Address review: canonical probe cache key and reuse _token_cache_key
- _probe_cache_key resolves config.json to its absolute realpath before
keying, so a relative path or a changed cwd can't collide with or miss a
prior probe result. Remote ids still fall back to the name (stat raises,
caught).
- _cached_config_json reuses _token_cache_key instead of re-hashing the
token inline, keeping the (model, token) key derivation in one place.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio: let users change their password from Settings
The only day-to-day way to change credentials was the destructive console command
'unsloth studio reset-password' (it deletes auth.db); the in-app change-password
page is the forced first-login flow and bounces non-forced users to /login.
Add a Change password control to Settings > General > Account: a small dialog
that takes the current and new password and calls the existing
POST /api/auth/change-password, then stores the rotated tokens it returns.
Username changes remain out of scope.
The dialog uses authFetch, so an expired access token is refreshed and the
request retried instead of failing with a spurious expired-token error for a user
who left Studio open past the token lifetime. The row is hidden in the Tauri
desktop app, which authenticates via desktop auto-auth with a generated secret:
there is no user-entered password to change there, and changing it would clear
the desktop secret.
* studio: harden settings password change
* studio: harden settings password dialog UX
---------
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
