Address review findings on the tool-strip and streaming paths:
- strip_tool_call_markup stripped Gemma-native spans with a plain regex that
stops at the first <tool_call|>, so a literal close marker inside a
<|"|>-quoted argument truncated the span and leaked its suffix into visible
text. A brace/quote-aware _strip_gemma_native_spans now removes complete
spans (keeping an incomplete one unless final), matching the parser's own
balance logic.
- The Gemma close pattern this PR added (<\|tool_call>.*?<tool_call\|>) had no
\Z fallback, so a run of unclosed markers backtracked from every open
position (quadratic, and the streaming stripper re-scans per token). It is
now anchored to (?:<tool_call|>|\Z) like routes/inference.py's _TOOL_XML_RE,
linear with identical output on well-formed input.
- _SameTaskStreamingResponse added unstarted_cleanup for the OpenAI passthrough,
but the local GGUF/safetensors streams that enter _TrackedCancel before
returning only unregister in the generator finally, which never runs if the
client disconnects before the body iterator starts, leaking cancel-registry
entries. Each such stream now passes unstarted_cleanup to exit its tracker.
- __call__ reads _unstarted_cleanup via getattr so a response built through
__new__ (the cancel-timing test) without __init__ does not raise
AttributeError; the test also sets the attribute explicitly.
- Document that the verbatim /v1/chat/completions passthrough delegates
<think>/<|tool_call> splitting to llama-server (--jinja, --reasoning-format
auto) and is intentionally not re-parsed locally, noting the llama.cpp
dependency.
Adds a regression test for the close-marker-inside-quoted-argument strip.
#6579 reworded these comments to attribute the failure to a half-resolved
install and claimed a clean 4.14 is fine on 3.13. That is wrong: #6483 is a
genuine anyio 4.14 + Python 3.13 regression. 4.14 added a per-task cancel
scope in its asyncio backend (TaskHandle/_run_coro) that gets exited in the
wrong task under starlette's collapsing task group, raising the cancel-scope
RuntimeError on streaming; 4.13 has no such code and is unaffected (the
reporter confirmed 4.13.0 fixes it). The TaskHandle ImportError is only the
secondary macOS-arm symptom from the mlx-vs-cap version fight. Comments only.
* Studio: start the Cloudflare tunnel for --secure even in --api-only, and add --api-only to `unsloth studio run`
--secure exposes ONLY the Cloudflare link (it forces a loopback bind), but
_cloudflare_tunnel_should_start gated the tunnel on `not api_only`, so
`run.py --secure --api-only` started no tunnel and then fail-closed with
"A secure Cloudflare link is not allowed". That blocked the natural headless
use: serve just the API (no web UI) over the authenticated tunnel.
Make --secure start the tunnel regardless of api_only (the non-secure path is
unchanged: tunnel only a 0.0.0.0 bind, never api-only Tauri or Colab). Then
expose --api-only on `unsloth studio run` and forward it through both the
re-exec args and the in-venv run_server call, so
`unsloth studio run --secure --api-only --model ...` is a one-liner secure API
server.
Verified end to end: `run.py --secure --api-only` now brings up the tunnel and
serves /api/health over it (200), with / returning 404 (no UI).
Tests: update the tunnel-gate truth table (secure+api-only now tunnels;
secure+colab still does not) and add --api-only registration + re-exec/in-venv
forwarding coverage to the run CLI tests.
* Trim comments to be succinct (no behavior change)
* studio: address review on parent --api-only and secure api-only CORS
- Reject --api-only on the parent `unsloth studio` group when a subcommand
is invoked, with the same redirect guidance used for --parallel/--secure;
otherwise the flag was silently dropped and the UI served anyway.
- Keep CORS any-origin for secure api-only serving: that mode publishes the
API over Cloudflare for remote browser clients, so the Tauri-only lockdown
(still applied to plain local api-only) would break preflight. Factored the
decision into cors_origins_for_mode() and gate it on api_only and not secure;
run_server exports UNSLOTH_SECURE before importing main.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: suppress TAURI_PORT and de-dup test for headless run --api-only
- run_server gains emit_tauri_port (default True, unchanged for the Tauri/
desktop path). The new headless `run --api-only` path passes False so the
Tauri-only TAURI_PORT= line no longer prepends the documented URL/API key
banner (it ran even under --silent and could break one-liner parsers).
- Remove a duplicate test_reexec_forwards_api_only that shadowed the
parametrized one; fold the --secure --api-only case into it so the secure
headless path is actually collected.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
torchao has no working Windows ROCm build. transformers.quantizers imports it,
and it loads torch's c10d distributed backend at module level, which the AMD
Windows wheels omit (no RCCL). The import aborts, transformers can no longer
expose PreTrainedModel, and the sentence-transformers embedder silently falls
back to the llama-server GGUF embedder. Linux ROCm and NVIDIA are unaffected
(the c10d ops are present / torchao is real there).
The training and export workers already install the shared torchao stub before
importing transformers, but the RAG embedder runs in the main backend process,
which never did. Two fixes, both no-ops off Windows ROCm:
- embeddings.py: install_torchao_windows_rocm_stub() before the first
sentence-transformers import, so an already-installed torchao is neutralized
(fixes existing venvs).
- install_python_stack.py: stop installing torchao on Windows ROCm; it can only
crash on import there, so new venvs never ship it.
Add tests covering the embedder stub call and the install skip.
PR #6364 added a branch that overrode the unsloth owner avatar with the
bundled circle-logo-small.png sticker. Revert it so unsloth uploads use
the live Hugging Face org avatar again, falling back to the colored
initial tile. Upstream re-uploads are unaffected since they still
resolve through provider logos.
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
* Chat: match reasoning thinking icon to the composer bulb
The reasoning "Thinking..." indicator used lucide's LightbulbIcon while
the composer thinking toggle used a custom bulb glyph, so the two did not
match. Move that glyph into lib/bulb-icon.tsx and use it in both places
so they render the same icon.
* Let BulbIcon take and override svg props
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
* Studio: fall back to anonymous HF browsing on a malformed token
The Discover/Recommended feeds call the Hugging Face JS client
(`listModels`/`listDatasets`) directly from the browser. That client
throws `Your access token must start with 'hf_'` when handed a non-empty
token that isn't a well-formed HF token, instead of falling back to
anonymous access. A single bad value left in the HF token field (e.g. a
placeholder someone typed) therefore takes down the entire discovery feed
even though it works fine with no token at all.
Add `hfApiToken()` to the HF token store, which returns the token only
when it looks like a real `hf_...` credential and `undefined` otherwise,
and route hub-page's four HF call sites through it. Malformed tokens now
degrade to anonymous public browsing rather than erroring. The raw token
is still stored and shown in the settings field unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Studio: trim hfApiToken comments
Collapse the 11-line JSDoc to a 2-line note and drop the redundant
call-site comment in hub-page. AST signature check confirms code is
unchanged (comments only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Studio: treat data-center Blackwell (sm_100/sm_103) as Blackwell in llama.cpp prebuilt selection
_host_is_blackwell gated on _BLACKWELL_MIN_SM = 120, but data-center Blackwell
parts report a lower compute capability than consumer Blackwell: B100/B200 are
sm_100 and B300/GB300 are sm_103, while RTX 50 is sm_120 and DGX Spark is
sm_121. Because 100 and 103 are both < 120, every data-center Blackwell host was
classified as non-Blackwell, so two GPU-targeting paths never fired for a
B200/B300:
- the Linux blackwell_runtime_override that prefers the highest CUDA-major
runtime line shipping a bundle covering the host SMs (so a cu12x torch could
pin a cuda12 bundle over a native cuda13 one), and
- _drop_blackwell_incapable_windows_cuda, which removes cuda-12.4 builds that
load and validate but run Blackwell on a slow PTX-JIT path.
The result is a B200/B300 being handed a prebuilt that does not natively offload
its SM, i.e. the llama.cpp prebuilt is not really for the GPU. The Blackwell
floor is sm_100, so set _BLACKWELL_MIN_SM = 100. The toolkit floor (12.8) is
unchanged and already correct for sm_100/sm_103.
Surfaced loading unsloth/GLM-5.2-GGUF UD-IQ1_S on 8x B200.
Adds tests covering the sm_100/sm_103 classification, the Linux cuda13
preference for a data-center host, and the Windows cuda-12.4 drop.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be succinct (no behavior change)
* studio: require CUDA 12.9 for sm_103/sm_121 Blackwell prebuilts
sm_103 (B300/GB300) and sm_121 (DGX Spark) have no native compiler
target before CUDA 12.9; the family floor of 12.8 only covers
sm_100/101/120. Make the Windows-CUDA Blackwell filter SM-aware so a
legacy win-cuda-12.8 bundle is dropped on an sm_103/sm_121 host while
sm_100/sm_120 hosts keep the 12.8 floor.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio): sort export checkpoints by step
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Withhold HF_TOKEN from pull_request runs of CI workflows
The pull_request-triggered CI workflows check out and execute PR-controlled
code (install.sh, .github/scripts/**, tests/**) with secrets.HF_TOKEN in the
step environment. For a same-repo PR, GitHub provides repository secrets to the
run, so a malicious or compromised branch could modify a checked-out script to
read and exfiltrate HF_TOKEN, including by writing it into the uploaded logs/
artifact. HF_TOKEN is an external Hugging Face credential of unknown scope, so
this is the high-value exposure.
Gate every HF_TOKEN reference in these workflows with
`github.event_name != 'pull_request' && secrets.HF_TOKEN || ''`, so the real
token flows only on the trusted schedule/push/workflow_dispatch runs and PR runs
see an empty string. All model repos used by these jobs are public
(unsloth/*-GGUF), so anonymous download still works on PRs; install_llama_prebuilt.py
only sends HF auth to Hugging Face hosts and tolerates an absent token.
GITHUB_TOKEN (passed as GH_TOKEN) is intentionally left in place: it is the
auto-provisioned, job-scoped, contents:read token that expires with the job and
gives a same-repo PR author nothing they do not already have, and
install_llama_prebuilt.py needs it to authenticate the GitHub releases API or
the prebuilt llama.cpp download hits the anonymous rate-limit bucket and 403s.
* Trim the HF_TOKEN gating comments to one line per site
Comment/whitespace-only: collapse the per-step rationale to a single line and
shorten the local-agent-guides header note. No workflow logic changes (verified
each file's parsed YAML is identical before/after).
* Fix FlashAttention fp32 crash with DoRA (use_dora=True)
DoRA upcasts lora_magnitude_vector to fp32 for the optimizer, which promotes
the q/k/v_proj output to fp32. FlashAttention only accepts fp16/bf16, so the
fp32 q/k/v raised 'FlashAttention only support fp16 and bf16 data type'.
Downcast q/k/v to the compute dtype before the flash kernels.
Fixes#1013
* Apply kwarg-spacing format hook to DoRA dtype test (pre-commit)
* DoRA+FA2: downcast any fp32 among Q/K/V and clamp to a flash-supported dtype
* Tighten code comments (no logic change)
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Fix misleading 'only for image models' error for Qwen3-VL when torchvision is missing
transformers >= 5.4 hard-requires torchvision for VLM image/video processors and
no longer falls back to a slow processor. Without torchvision the processor load
raises ImportError, unsloth degrades to a text-only tokenizer, and the vision data
collator later fails with 'UnslothVisionDataCollator is only for image models!'.
Detect this case at load time and raise a clear, actionable error pointing at the
missing torchvision dependency instead.
Fixesunslothai/unsloth#4202
* Apply kwarg-spacing format hook to vision torchvision guard (pre-commit)
* Make torchvision-missing detection precise: check availability first, match specific error text
* Tighten code comments (no logic change)
* Make missing-torchvision VLM error version-agnostic
The raise also fires on transformers 4.57.x for VLMs with a video processor
(Qwen2.5-VL, Qwen3-VL), where AutoVideoProcessor requires torchvision. The old
message claimed 'transformers >= 5.4 requires torchvision', which is inaccurate
on 4.57.x. Reword to state torchvision is required for this model's vision
processors without a version-specific claim.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Fix GRPOTrainer evaluate() crash when called without prior training
GRPOTrainer.compute_loss read self.current_gradient_accumulation_steps
directly. That attribute is only set by the transformers training loop, so
calling trainer.evaluate() standalone (no prior trainer.train()) raised
AttributeError. Read it via getattr with a fallback to
args.gradient_accumulation_steps so standalone evaluation works.
* GRPO eval: fall back accumulation steps to 1 so standalone eval_loss is not underreported
* Tighten code comments (no logic change)
* fix: use EMPTY_LOGITS on the fused-CE not-return_dict path (#2068)
CausalLM_fast_forward's fused cross-entropy path (small batch, labels set,
UNSLOTH_RETURN_LOGITS off) computes the loss straight from hidden_states
via unsloth_fused_ce_loss and never materializes `logits`. The
return_dict=True branch returns EMPTY_LOGITS, but the `not return_dict`
branch returned `(logits,) + outputs[1:]`, raising
"UnboundLocalError: cannot access local variable 'logits'" whenever it ran
(e.g. training with return_dict=False). Same bug in the llama and mistral
fast-forward paths.
Return EMPTY_LOGITS on that branch too, matching the adjacent return_dict
output. Verified on GPU: a forward(return_dict=False, labels=...) that
raised UnboundLocalError now returns (loss, EMPTY_LOGITS, ...) and
backward() succeeds.
Adds tests/test_fused_ce_not_return_dict_logits.py, a CPU source-drift guard
(the fused path itself is GPU/triton only) asserting both fast-forward paths
keep using EMPTY_LOGITS there.
* Address review: parse the fused-CE drift line with whitespace-tolerant regexes
The drift detector sliced the source with exact string matching
(source.index("output = (") + the next newline), so a formatter respacing or
rewrapping the assignment would break the parse. Switch to anchored regexes that
tolerate whitespace and line wrapping, keeping the match anchored after the
fused guard so it targets the fused-CE branch and not the normal
output = (logits,) path. Behavior and the two drift assertions are unchanged.
* Tighten code comments (no logic change)
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* studio/setup.sh: guard empty CUDA arch detection in the source build
PR #5826 hardened setup.sh for fresh CUDA toolkits, but the source build
still set -DCMAKE_CUDA_ARCHITECTURES only when nvidia-smi reported a
compute capability. When that query returns nothing the build proceeded
with no explicit arch list, so llama.cpp built PTX only. On a driver older
than the toolkit that binary fails at runtime with "the provided PTX was
compiled with an unsupported toolchain" - the build succeeds, so neither
the build-time check nor the CPU fallback caught it (issue #5854).
Resolve the arch list before committing to a CUDA build. A new pure helper
_resolve_cuda_archs parses and de-duplicates the nvidia-smi compute_cap
output and honors an explicit UNSLOTH_LLAMA_CUDA_ARCHS override. When the
result is empty, build CPU llama.cpp instead of a PTX-only binary, with a
clear message pointing at the override - so the user still ends up with a
working llama-server. The override also lets advanced users force a native
build on hosts where nvidia-smi cannot report compute_cap.
No behavior change when an arch is detected: -DGGML_CUDA=ON plus the arch,
CUDA flags and NVCC_PREPEND_FLAGS are assembled exactly as before.
Adds tests/sh/test_resolve_cuda_archs.sh (single/multi/dedup/empty/garbage/
whitespace/override cases), wired into tests/run_all.sh and the
studio-backend-ci.yml shell-test loop.
* studio/setup.sh: resolve nvidia-smi via /usr/bin fallback for arch detection
Addresses review feedback on the empty-CUDA-arch guard: _setup_has_usable_nvidia_gpu
classifies a host as NVIDIA-usable using nvidia-smi on PATH OR /usr/bin/nvidia-smi,
but the new arch detection probed only `command -v nvidia-smi`. On a GPU host where
nvidia-smi is off PATH (reachable only at /usr/bin), arch detection returned empty
and the new empty-arch branch dropped the build to CPU, losing CUDA. Mirror the same
PATH-then-/usr/bin resolution so those hosts still get a native CUDA build.
Also scope _resolve_cuda_archs locals with `local` (no behavior change; it already
runs under command substitution).
* tests: update compute_cap-probe assertion for $_smi_bin resolution
The nvidia-smi /usr/bin fallback parameterized the binary in the compute_cap
probe (_setup_run_smi "$_smi_bin" ...), so the literal-string assertion in
test_compute_cap_probe_timeout_wrapped no longer matched. Assert the probe is
preceded by _setup_run_smi (timeout-wrapped) instead, scanning all occurrences
so the comment mention is ignored. Same intent, binary-agnostic.
* tests: ruff-format the compute_cap probe assertion (pre-commit)
Collapse the backslash-continued assert onto one line and normalize slice
spacing so the ruff-format pre-commit hook (0.6.9) is satisfied. Formatting
only; no behavior change.
* Tighten code comments (no logic change)
* studio(windows): build CPU when CUDA arch is undetectable (#5854)
The Windows source build added -DGGML_CUDA=ON unconditionally but only set
-DCMAKE_CUDA_ARCHITECTURES when $CudaArch was detected. With no detectable
compute capability that produced a PTX-only binary, the same hole the Linux
fix closed. Build CPU llama.cpp in that case, and honor UNSLOTH_LLAMA_CUDA_ARCHS
to force a CUDA build, matching setup.sh. Detected-arch builds are unchanged.
* test: anchor NVCC_PREPEND_FLAGS scope check on the final CPU branch
The undetectable-arch CPU fallback adds an earlier -DGGML_CUDA=OFF, so the
ordering check now anchors on -DGGML_CUDA=ON and the last -DGGML_CUDA=OFF
instead of the first.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* fix(studio): handle multimodal list content in inference text paths
Studio receives chat message content in two shapes: the legacy string
form, and the OpenAI multimodal list form
([{"type": "text", "text": ...}, {"type": "image_url", ...}]).
Several string-only paths called .strip()/re.sub()/f-string interpolation
on content directly, raising "'list' object has no attribute 'replace'"
for vision models (issue #4383), or rendering the list repr into the
prompt for the manual chat-template formatters.
Add core/inference/message_content.py with content_to_text(), a pure
helper (no heavy imports) that returns strings unchanged and joins the
text parts of a list while dropping image/audio parts. Apply it at every
string-only content site: _generate_vision_response, the audio user-text
extraction, format_chat_prompt, and the llama3/mistral/chatml/alpaca/
generic template formatters. The plain-string path is a no-op, so
existing behavior is unchanged.
Adds tests/test_message_content.py covering str/None/list/tuple,
multimodal drop, multi-part join and empty-part skipping.
* Tighten code comments (no logic change)
* studio: join multimodal text parts with newline for llama.cpp parity
llama.cpp joins multiple text content parts with a newline (common/chat.cpp),
so match that in content_to_text instead of a single space.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Three independent upstream CI fixes that currently fail on every open PR:
verify_import_hoist.py: TARGET-CHANGED only flags a genuine swap (a BEFORE
target no longer reachable in AFTER). A pure superset growth such as adding
import urllib.error next to import urllib.request binds the same top-level
package and loses nothing, so it is no longer a blocker (transformers_version.py).
test_vision_cache.py: run each test from a fresh empty cwd. is_vision_model
calls is_local_path first, and a relative model id that happens to exist on
disk short-circuits before the mocked detection runs; the CI cwd and HF cache
can contain dirs colliding with the synthetic ids, causing 'called 0 times'.
Production code is correct; only the test needed cwd isolation.
consolidated-tests-ci.yml: the llama.cpp smoke probes the first of
llama-cli / llama-mtmd-cli / llama-server that exists instead of hard-requiring
llama-cli, which upstream no longer always builds. llama-cli stays first so it
is preferred when present. Adds Windows .exe + build/bin/Release handling.
* Studio: refresh chat tour for the redesigned model picker
- Pick a model step describes the Recommended and On Device tabs instead of the old Hub and Fine-tuned split
- Find a model step (was Two tabs) covers Unsloth search vs Search Hub, the format and sort filters, and the OOM tag
- Settings step now anchors to the run settings panel on the right. The old anchor sat on the open settings button, which unmounts when settings opens, so the tooltip lost its target and drifted left
* Add guided tour step for the composer + menu
Polish for the in-chat model picker popover and its guided-tour step.
- Search box placeholder reads Search Unsloth models, matching the Unsloth-only listing.
- Search Hub button shows a Search all models tooltip on hover.
- Floating Eject pill moves 1px lower so it sits closer to the bottom edge.
- Results list max height trimmed by 1px (21rem to 335px) from the bottom only.
- Chat guided tour Two tabs step updated to describe Unsloth-scoped search plus Search Hub for all of Hugging Face.
- Discover defaults to the whole Hub instead of the unsloth org; an explicit
Unsloth choice is still remembered
- Discover models placeholder reads Search all models to match
- Give the Unsloth/All scope pill a min width so it stays readable
Three follow-ups on the tool-parse and streaming paths:
- parse_tool_calls_from_text only skipped markers that fell inside a span it
had already parsed successfully, so when an unquoted Gemma argument contained
a literal marker (code:<|tool_call>call:terminal{...}<tool_call|>) the outer
object failed to normalize, its span was never recorded, and the inner marker
was promoted to a standalone terminal call. Candidates nested inside any other
candidate's brace span are now skipped regardless of whether the enclosing
candidate parsed, so a marker in malformed outer data is never executed.
- /generate/stream skipped backend.reset_generation_state() when the disconnect
watcher set cancel_event between chunks: the loop broke and the finally's reset
is guarded on cancel_event being unset. A subprocess backend kept decoding
after the client left. The cancel-break path now resets the backend.
- _SameTaskStreamingResponse threw CancelledError / called aclose() on the body
iterator on a send-side disconnect, but neither runs the try/finally of a
generator that never started (early disconnect on http.response.start), so the
passthrough's eagerly-opened upstream httpx stream and cancel-registry entry
leaked. It now tracks whether the body started and, when it did not, runs an
optional unstarted_cleanup hook; the OpenAI passthrough wires it to close the
upstream resp/client and exit the cancel tracker.
Adds a nested-unquoted-marker regression test.
* Use UTF-8 for Python code-execution subprocess I/O
Studio's code-execution tool already tells the child to emit UTF-8
(PYTHONIOENCODING=utf-8 in _build_safe_env), but _python_exec writes the
temp script and decodes the subprocess pipe with the OS default codec.
On Windows (cp1252), non-ASCII in model-written code or its output --
arrows, CJK, emoji -- raises UnicodeEncodeError / UnicodeDecodeError and
breaks execution.
Complete the UTF-8 wiring in core/inference/tools.py:
- write the temp script with encoding="utf-8"
- decode _python_exec stdout as utf-8, errors="replace"
- set PYTHONIOENCODING=utf-8 in _build_bypass_env too (matches
_build_safe_env, so the bypass path's child also emits utf-8)
The child is python with PYTHONIOENCODING=utf-8, so it emits UTF-8
regardless of the console code page and the decode is always correct.
Shell execution via cmd.exe has a separate console-code-page story and
is left to a follow-up.
Refs unslothai/unsloth#6489
* Scope Python exec UTF-8 env to Python tool
* Make bash bypass test robust to a host-set PYTHONIOENCODING for PR #6548
Bypass mode preserves benign host env vars, so a host-set PYTHONIOENCODING was
inherited into the bash bypass env and tripped the new assertion even though
_bash_exec never adds it. Clear it in the test so the assertion checks _bash_exec,
not the runner environment.
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: correct the anyio<4.14 pin rationale (mixed-install ImportError)
The pin comments said "anyio 4.14+ breaks cancel scope on Python 3.13", but
a clean anyio 4.14.0 works on 3.13 (cancel scopes, Event, and the asyncio
backend import all pass). The actual failure is a half-resolved install:
anyio 4.14 added TaskHandle, imported by __init__.py and _backends/_asyncio
from _core/_tasks. When a stale 4.13 _core/_tasks (no TaskHandle) sits under
4.14's importers, the import raises ImportError and 500s the server. Correct
the rationale; the <4.14 pin still stands as the way to keep one consistent
anyio version.
* Clarify the anyio override comment (mixed-install ImportError, not a 4.14 cancel-scope bug)
* Studio: keep model downloads running across navigation and loads
Downloads started from the chat model selector were tied to the staged
pick lifecycle, so they were cancelled in cases where Hub downloads keep
going. This makes the chat download flow behave like the Hub.
- Leaving the chat route or switching thread/project/new chat now detaches
the staging UI but keeps the in-flight transfer running in the global
download manager (new keepDownload option on abandonStagedModel).
- Staging a second pick no longer cancels the previous pick's download, so
multiple models/variants can download at once.
- Picking a model to download while another model is loading now starts the
download in the background instead of refusing, since a download is
independent of a load.
* Studio: also background-download remote GGUF quants while a model loads
isDownloadableHubRepo (wantManagerDownload) excludes GGUF sources, so an
uncached remote GGUF quant picked from the chat selector while another model
was loading fell through to the 'Another model is already loading' toast
instead of downloading in the background. Treat an uncached remote hub GGUF as
a background download too, matching the staged-pick download path.
Addresses review feedback from gemini-code-assist and codex on PR #6573.
* Studio: only toast a background download once it actually starts
The chat background-download path (used when a model is already loading)
fired the "Downloading in the background" toast unconditionally, but
requestStart can return without starting a job: a cross-transport partial
records a conflict that is only resolvable from the Hub download card, and
a busy sibling variant returns after its own toast. So the user could be
told a download started when none did, with no way to resolve the conflict
from chat.
requestStart now reports an outcome (started/conflict/busy/error). The
chat path only shows the success toast on an actual start and points the
user to the Hub when a transport conflict needs resolving. The Hub card
surface keeps its existing behavior (it renders the conflict resolver, so
it ignores the outcome).
* Studio: report background-download outcome from real job state
The chat background-download toast trusted requestStart's optimistic
"started", but a start can no-op without throwing: startJob finalizes the
job as "error" when the backend refuses or fails apiStart, its peer guard
skips a fresh start, and hasActiveOrPendingStart trips on a snapshot, peer
variant, or pending preflight that is not this request. So the user could
be told a download started when none did.
Derive the outcome from the actual job state of the exact key
(running/cancelling = started, otherwise error/busy), so the toast only
fires for a transfer that is really live.
Also guard against re-downloading the model that is already loading: the
/load flow downloads before it sets the checkpoint, and that fetch is not
a download-manager job, so picking the same id+variant again would start a
second transfer against the same cache. Detect that pick and surface a
"this model is already loading" toast instead.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Restore sys.modules in test_pre_import_gate_is_transformers_free
The test pops transformers and utils.models.model_config from sys.modules to
assert the pre-import security gate does not re-import them, but never put them
back. A later importer then rebound a fresh utils.models.model_config, so tests
that had captured the original instance missed their patches and hit the real
path: test_vision_cache patches _is_vision_model_uncached on the original
module, but is_vision_model (still bound to that original) ran the real network
lookup instead. This produced 17 spurious failures whenever test_ssm_runtime
ran before test_vision_cache in the same process.
Snapshot the removed modules and restore the original objects in a finally, so
the assertions still run against a clean slate while later tests see the same
module instances they captured at import time.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The macOS-arm studio venv still installs anyio 4.14.0 despite the
constraints.txt cap from #6546. mlx-vlm / mlx-lm pull anyio>=4.14, which
conflicts with the anyio<4.14.0 constraint; a uv -c constraint loses that
conflict so 4.14.0 gets installed, reintroducing the cancel-scope
RuntimeError on Python 3.13 (#6483). UV_OVERRIDE is already applied on
macOS-arm via overrides-darwin-arm64.txt and a uv override wins the
conflict, so cap anyio there too. macOS-arm now resolves anyio 4.13.0.
Two follow-ups on the streaming and tool-parse paths:
- _anthropic_tool_stream and _anthropic_plain_stream drove generation through
asyncio.to_thread(next, gen, ...) and only polled is_disconnected() between
events, so a client disconnect during prefill or a long generation/tool step
held the decode slot until the next event or a failed send. Both now run the
_await_disconnect_then_cancel watcher used by the other local streams, stop it
in finally, and break promptly when cancel_event is set.
- _GEMMA_NEXT_KEY_RE treated any comma followed by word-chars-then-colon as the
next key, so a bare value such as "meet at 10:00, 11:00 tomorrow" was split
into bogus keys. The next-key token must now be identifier-shaped (start with
a letter or underscore), so a comma before a timestamp, ratio, or other
numeric-then-colon text stays part of the value.
Adds a timestamp-in-bare-value regression test.
* Studio: hide RAG embedder from the On Device list
The bge-small-en-v1.5 RAG embedder (and other infra models) were already
hidden from Discover but still showed up in the On Device browse list,
cluttering the user's downloaded models. They are now filtered out of On
Device the same way, while a search that matches still reveals the row so
the user can confirm it is already downloaded.
* Studio: also check path/title when hiding infra models from On Device
isHiddenModelId only saw row.id and row.repoId, but local inventory rows can
have a null repoId and an id that is a hash rather than the file path/name, so
the llama.cpp validation probe (stories260K.gguf) could slip into the On Device
list. Pass the local row's path and title too, mirroring the backend's
_is_hidden_model(m.id, m.path).
Addresses review feedback from gemini-code-assist on PR #6572.
* Studio: exclude infra models from On Device count and dataset list
The On Device hidden-model filter was applied to datasets too, so a
dataset whose id/title/path contained an infra needle (bge-small-en-v1.5,
stories260k.gguf) was wrongly hidden. Bypass the filter for datasets, the
same way Discover and the format filter already do.
The On Device header count and the Cache/Local stat pills still used the
unfiltered row counts, so a fresh install with only the bge embedder
cached read 1 over an empty list. Count visible (non-infra) rows instead,
keeping full counts for datasets.
* Studio: count search-revealed infra rows in the On Device tally
The visible-row counts excluded every hidden row unconditionally, but the
On Device list reveals a hidden row when the search query matches it. So
with only the bge embedder cached and a "bge" search, the list showed one
row while the header and Cache stat stayed 0. Reuse isVisibleInventoryRow
for the counts so a query-revealed row is counted, keeping them in step
with the list.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio Playwright: snooze update banner before sending
The llama.cpp update banner is a fixed bottom-right toast (z-9998). When an
update is available it overlaps the composer's Send button and its subtree
intercepts the click, so send_and_wait times out (flaky; surfaces on the
Windows studio UI smoke, passes otherwise). Snooze the banner if it is
showing before each send, then wait for it to detach.
* Also snooze the web update banner before sending
The web update banner (web-update-banner, z-9999) is a fixed bottom-right
toast like the llama.cpp one and can overlap the Send button too. Loop over
both banners and snooze whichever is showing.
* Studio: honor custom HF_HOME for model download and load
_setup_cache_env always derived HF_HUB_CACHE and HF_XET_CACHE from
XDG_CACHE_HOME / ~/.cache, ignoring a user-set HF_HOME. Because it sets
HF_HUB_CACHE explicitly and that variable takes precedence over HF_HOME
in huggingface_hub, the hub cache was pinned to the standard location: a
model already present under a custom HF_HOME was detected but then
re-downloaded from scratch on load.
Seed HF_HUB_CACHE and HF_XET_CACHE from HF_HOME when the user set it
(HF's own default is $HF_HOME/hub and $HF_HOME/xet), and honor the legacy
HUGGINGFACE_HUB_CACHE alias. The hub download workers call
snapshot_download without a cache_dir for both the Xet and HTTP-fallback
paths, so they follow HF_HUB_CACHE; fixing it here unifies detection and
both transports on one root. Explicit HF_HUB_CACHE / HF_XET_CACHE stay
untouched. Adds tests for the custom-HF_HOME, default, explicit-override,
and legacy-alias cases. Fixes#5182.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: do not crash startup when a custom HF_HOME is not writable
Seeding HF_HUB_CACHE/HF_XET_CACHE from HF_HOME means _setup_cache_env now
mkdir's under a user-controlled path. A non-writable or not-yet-mounted
HF_HOME (typo, offline drive) would raise and crash startup, where the old
code silently fell back. Make the mkdir best-effort; the env var is still
set, so HF reports a clear error at download time. Adds a regression test.
* Studio: strip blank HF_HOME and isolate cache-env tests
Address review: a whitespace-only HF_HOME no longer derives " /hub";
strip it and fall back to the default (matches studio_root). Tests set
UNSLOTH_STUDIO_HOME to a tmp dir so _setup_cache_env's UV/VLLM mkdirs do
not touch the real ~/.unsloth/studio. Adds a whitespace regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Resolve the transformers tier by probing AutoConfig instead of guessing
When the only signal is a 5.x tokenizer class, get_transformers_tier guessed the
lowest 5.x sidecar (530). That misroutes models whose built-in config parser needs
a higher tier: dense NemotronH ships a 5.x tokenizer but its '-' (MLP) layer only
transformers 5.10 can parse, so 5.3/5.5 raise KeyError '-'. The config.json
transformers_version field records the saving version, not the minimum to load, so
it cannot drive routing either.
Replace the weak tokenizer->530 guesses (local and remote) with a probe: parse
config.json with the built-in parser (trust_remote_code=False) in each sidecar,
escalating 530->550->510, and pick the first that succeeds. This generalizes to any
architecture without hardcoded lists. Strong signals stay fast paths (no subprocess);
the probe runs only when the tier is otherwise ambiguous and is cached by (model,
commit sha). It never executes repo code, never downloads weights, never raises, and
falls back to the legacy 530 guess on a transient/auth/offline failure or when no
sidecar is available. UNSLOTH_DISABLE_TIER_PROBE restores the old behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: tier probe fallbacks and cross-platform robustness
Codex:
- Never escalate to 510 on uncertainty. When every sidecar was probed and none
parsed with the built-in parser, the model is a remote-code / custom model_type
that loads via its own code; keep the legacy 530 route instead of jumping to
510 (which would change the behavior of models that worked on the 5.3 stack).
- Only cache the 530 fallback when the result is conclusive (every tier actually
probed). If a sidecar was missing/uninstallable the environment is incomplete,
so return 530 uncached and retry on the next call.
- Do not pin the tier cache under an unknown revision: _resolve_commit_sha no
longer memoizes a None sha (a transient Hub failure is retried), and _probe_tier
only caches a tier when the commit sha is known.
Gemini:
- Wrap Path.exists() in the sha resolver in try/except OSError (a remote repo id
can raise WinError 123 on Windows).
- Probe script writes the error to sys.stderr.buffer as UTF-8 bytes so a non-ASCII
message cannot itself raise UnicodeEncodeError under cp1252.
- subprocess.run decodes stderr with errors="replace" to avoid UnicodeDecodeError
on non-UTF-8 consoles.
Tests: 72 passed (added partial-sidecar uncached, sha-unresolved not cached,
all-failed stays 530 + cached, sha resolver retries None / handles OSError).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review round 2: authenticate tier checks, stop memoizing local sigs
Codex:
- Thread hf_token through _check_config_needs_510/550 and
_check_tokenizer_config_needs_v5 (and the underlying raw fetches). Previously a
gated/private model whose only 5.x signal is tokenizer_config.json never reached
the authenticated probe: the unauthenticated raw fetch failed and cached False,
so the model fell through to the default 4.x tier. The per-check caches are now
keyed by (model, token) so an unauthenticated miss cannot poison a later authed
read, mirroring _load_config_json.
- _resolve_commit_sha no longer memoizes a local directory signature. A local
signature is mutable (size/mtime of config/tokenizer), so a reused/overwritten
checkpoint path would otherwise keep selecting the previous tier; it is now
recomputed every call. Only the immutable remote commit sha is memoized.
Tests: 75 passed (added token-cache isolation + auth header, local signature not
memoized, token threaded into all checks/probe).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review round 3: reach activation with the token, drop SHA tier cache
Codex round 3:
- Thread hf_token into the activation path that actually selects a sidecar. The
token-aware tier checks added last round were unreachable:
activate_transformers_for_subprocess called get_transformers_tier without a
token, and the inference/training/export workers passed only the model name even
though they hold a request-scoped hf_token. activate_transformers_for_subprocess
now takes hf_token and the three workers forward config["hf_token"], so a
gated/private model whose only 5.x signal is an authenticated config/tokenizer is
routed to the right sidecar instead of falling to default 4.x.
- Stop importing huggingface_hub during tier detection. _probe_tier no longer
resolves a commit sha, so it never pulls huggingface_hub into the worker before
the sidecar venv is prepended to sys.path (activation only prepends, never
purges), which would otherwise pin the default-env hub over the sidecar's
pinned huggingface_hub==1.8.0.
- The tier cache is now keyed by model_name for the process lifetime (a model's
required tier is a property of its architecture; cleared on restart). This drops
the mutable-SHA memo that masked remote revision changes and the mutable
local-signature memo, removing _resolve_commit_sha / _local_dir_signature /
_probe_sha_cache entirely.
- Do not cache a probe success that depended on a skipped lower tier: if a lower
sidecar was unavailable, the lowest valid tier may change once it installs, so
the result is returned uncached and re-probed next call.
Tests: 73 passed (probe imports no hub; success uncached when a lower tier is
skipped; activation forwards the token).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Re-probe overwritten local checkpoints and authenticate the probe child
The AutoConfig tier probe cached its result under the bare model_name, so a
local checkpoint overwritten in place (same path, new config.json) kept serving
the stale sidecar. Fold a cheap config.json signature (size + mtime) into the
cache key for local paths; remote ids stay name-keyed so no huggingface_hub
import lands before the sidecar is activated.
The probe relies on the implicit HF_TOKEN env, so an inherited
HF_HUB_DISABLE_IMPLICIT_TOKEN=1 left it unauthenticated and a gated repo 401ed
into the 530 fail-safe. Clear that flag in the child env when a token is set.
* Keep tier probes off the log-only path and probe new 5.x archs default-first
- get_transformers_tier gains probe=True/False. needs_transformers_5 (a coarse
4-vs-5 boolean used only for a spawn log and a vision-check branch) now passes
probe=False, so a parent/log-only caller never spawns sidecar probes. The real
activation path keeps probe=True and resolves the exact tier in the worker.
- A config.json saved by transformers 5.x but matched by no fast path is now probed
default-first: _probe_tier gains include_default + floor, prepending the ambient
4.57.x tier to the escalation. A model that still parses on the default is left on
it (no mis-route onto a sidecar); only a config the default parser cannot read
escalates to the lowest 5.x tier that parses. The transformers_version field is a
cheap 'worth probing' hint only, read from the already-fetched config (no extra
network); ordinary 4.x configs never probe.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Separate probe cache by mode and keep version-field 5.x visible to needs_transformers_5
- _probe_tier cache was keyed only by config.json signature, so a default-first probe
that returned 'default' could be handed back to a later tokenizer/known-5.x caller
(floor=530), leaving a model with a 5.x-only tokenizer on transformers 4.x. Key the
cache by probe mode (floor + include_default); the legacy 530 mode keeps the bare key.
- The version-field 5.x detection is a cheap config read, not a probe, so run it even
when probe=False: a standard-tokenizer model whose only signal is transformers_version
>= 5 now classifies as 5.x via needs_transformers_5 (returns '530' without spawning a
probe), so the vision-routing fallback uses the 5.x subprocess instead of failing the
default parser and marking it non-vision. The real activation path still probes
default-first and may resolve 'default'.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Don't treat local checkpoints as Hub ids, and fix stale activation test double
- _load_config_json / _check_tokenizer_config_needs_v5: a local checkpoint dir whose
config.json / tokenizer_config.json is not yet present was being fetched from the Hub
as if the path were a repo id, and the 404 miss was cached. A later call after the
file is written (in-progress checkpoint) then served the stale miss, so a
TokenizersBackend checkpoint fell through to the default tier. Skip the Hub fetch for
local dirs and do not cache the miss, so the file is read once it appears.
- test_activate_transformers_version_or_warn_*: the worker now threads hf_token into
_activate_transformers_version (model_name, hf_token); update the one-arg test doubles
to the real two-arg signature so the silent-success path stays silent.
* Tighten comments in the AutoConfig probe and tier-selection paths
* Address review: canonical probe cache key and reuse _token_cache_key
- _probe_cache_key resolves config.json to its absolute realpath before
keying, so a relative path or a changed cwd can't collide with or miss a
prior probe result. Remote ids still fall back to the name (stat raises,
caught).
- _cached_config_json reuses _token_cache_key instead of re-hashing the
token inline, keeping the (model, token) key derivation in one place.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Address five review findings on the Gemma tool-call and OpenAI passthrough
streaming paths:
- parse_tool_calls_from_text collected JSON and Gemma markers without the
_inside_open_parameter guard, so a marker embedded in an existing
<function=...><parameter=...> value was promoted to a separate tool call.
Candidates that start inside an open XML parameter are now skipped, matching
the guard the XML-style parser already applies.
- _quote_gemma_array_elements preserved array elements starting with { or [
verbatim, so an array of objects (items:[{path:a}]) or a nested array failed
json.loads and the whole call was dropped. Object and nested-array elements
are now normalised recursively.
- _openai_passthrough_stream synthesized a finish chunk before a trailing
usage-only chunk and set saw_finish_reason, which made the EOF guard skip the
[DONE] sentinel. The EOF path now emits [DONE] whenever the upstream omitted
it, even after a finish chunk was already synthesized.
- /generate/stream drove generation through asyncio.to_thread with no
disconnect watcher, so a client disconnect during a long generation went
unnoticed until the next send. It now runs _await_disconnect_then_cancel
against the request, matching the other local streaming endpoints.
- _SameTaskStreamingResponse closed the body iterator with aclose() on a
send-side disconnect, raising GeneratorExit so the generators' cancellation
handlers (which finish the api_monitor entry) never ran. It now throws
CancelledError, falling back to aclose() when athrow is unavailable.
Extends tests/test_gemma_tool_parse_edge_cases.py with array-of-objects,
nested-array, and marker-inside-XML-parameter cases.
* studio: let users change their password from Settings
The only day-to-day way to change credentials was the destructive console command
'unsloth studio reset-password' (it deletes auth.db); the in-app change-password
page is the forced first-login flow and bounces non-forced users to /login.
Add a Change password control to Settings > General > Account: a small dialog
that takes the current and new password and calls the existing
POST /api/auth/change-password, then stores the rotated tokens it returns.
Username changes remain out of scope.
The dialog uses authFetch, so an expired access token is refreshed and the
request retried instead of failing with a spurious expired-token error for a user
who left Studio open past the token lifetime. The row is hidden in the Tauri
desktop app, which authenticates via desktop auto-auth with a generated secret:
there is no user-entered password to change there, and changing it would clear
the desktop secret.
* studio: harden settings password change
* studio: harden settings password dialog UX
---------
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Add HF dataset streaming mode to Studio
* Added default value for datasetStreaming in training-config-store.ts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle None max_steps for streaming validation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fast-fail streaming validation and guard incompatible modes
Reject dataset_streaming at the API boundary when hf_dataset is empty,
the dataset is vision/audio, or max_steps is not set. Probe eval split
with get_dataset_split_names before the streaming load so typos fail
immediately instead of mid-training. Guard column_names=None after map
on iterables. Hide the UI toggle for non-text configurations and clear
the stale flag when config becomes incompatible.
* studio: add streaming dataset tests, iterable helper, and streaming template/format support (WIP)
Work-in-progress on top of feat/studio-dataset-streaming-mode (PR #4946):
- new test_training_streaming.py and iterable.py dataset helper
- streaming support in chat_templates.py and format_conversion.py
- additional streaming guards in trainer.py / models / routes
- frontend streaming wiring in params-section and training-config-store
Committed to preserve uncommitted work before merging latest main.
* studio: fix review-team findings for streaming + main merge
BLOCKER: streaming + raw-text/CPT crashed on len(IterableDataset). Guard it in the
start route (reject format_type=="raw" or training_type=="Continued Pretraining")
and in isStreamingSupported (datasetFormat !== "raw").
Also:
- models/training.py: validate hf_dataset/subset/split (charset+length, block ..//);
cap dataset slice indices (le=1e9); note validator ordering
- chat_templates.py: guard _apply_custom_mapping .map() for streaming
- trainer.py: warn when packing+streaming
- training-config-store.ts: persist-migration bump to v11 (standalone datasetStreaming
backfill); add isVisionModel to NON_PERSISTED; toast on silent streamingCompatiblePatch
mutations in the 4 indirect setters
- tests: route rejections (max_steps, raw/cpt), slice cap, unsafe hf_dataset
* studio: enable raw-text/CPT dataset streaming + streaming UX polish
- raw_text: keep the lazy filter but skip len()-based row counting for
IterableDatasets so raw-text / CPT can stream; guard the eval-size log
- routes/trainer: drop the raw/CPT streaming block; add a defensive
not-streaming guard on the eval auto-split (train_test_split)
- dataset-section: streaming toggle is visible-but-disabled and lists the
exact unmet requirement(s) in its tooltip; block embedding models
- training-start-overlay: show "streaming (no full download)" instead of a
stuck download bar for streaming runs
- trim the streaming test suite to the high-value cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: address streaming review (MLX/embedding guards, sliced eval split, rehydrate timing)
- routes: reject dataset_streaming for embedding training and on Apple Silicon
(MLX); both loaders materialize the full dataset instead of streaming
- trainer: validate the base eval split name so streaming eval accepts HF slice
syntax such as "validation[:1000]"
- training-config-store: defer the onRehydrateStorage setState to a microtask so
it doesn't hit the store's TDZ during synchronous hydration
- test: streaming start rejects embedding models
* studio: harden HF dataset streaming (column_names, split slicing, empty/eval bounds, gating)
Address a deeper streaming review:
- raw_text: resolve_column_names() guards IterableDataset.column_names=None
(from_generator / unresolved features) so raw-text and CPT streaming no longer
raise TypeError before training
- models/routes: reject HF slice syntax in train_split/eval_split when streaming
(load_dataset(streaming=True) raises "Bad split"); reject mixed sources
(local/S3) and embedding/MLX streaming at the API, not just in the UI
- trainer: an empty post-slice/filter stream fails preflight with a clear message;
streaming eval is capped (STREAMING_EVAL_MAX_SAMPLES) so each eval terminates;
the manual-slice shortcut falls back to a regular load when train_split is sliced
- format_conversion: streaming conversions preflight the first mapped row so
format errors surface before training, not mid-iteration
- frontend: block streaming on Apple Silicon; clear datasetStreaming when a
dataset is detected as image/audio at start
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix CI for streaming PR (lint blocker + no-torch sandbox + preflight test)
- trainer.py: drop unused `IterableDataset` import (hoist safety-net blocker).
- test_training_streaming.py: only select real classes (isinstance type) when
locating the trainer class, so a MagicMock-stubbed global is never passed to
object.__new__ (fixes TypeError on the Python 3.10-3.13 jobs).
- no-torch import sandboxes (test_e2e_no_torch_sandbox.py,
test_studio_import_no_torch.py): teach the chat_templates/format_conversion
exec stubs and the full-import-chain copy list about the new `.iterable`
module so the AFTER/runtime cases import without torch again.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* Installer: respect a declined Studio auto-start and keep Ctrl+C shutdown logs ordered
The `curl | sh` Studio auto-start prompt had two issues on Linux/macOS/WSL
(install.sh). install.ps1 already gates on input redirection, so Windows is
unaffected.
1. Typing n, or any closed/EOF /dev/tty, still launched Studio. The read
fallbacks defaulted to "y" (read failure, and the no-tty branch), so any
answer other than a cleanly delivered y/n line auto-started a blocking
foreground server. Default those to "n"; a real Enter still counts as yes
via ${_reply:-y}.
2. On Ctrl+C the shell prompt printed in the middle of Studio's shutdown logs.
The non-interactive installer shell took the default SIGINT action and died
before the child finished its graceful shutdown, so the prompt raced ahead
of "All subprocesses cleaned up". trap '' INT in the installer shell so it
waits for Studio's own graceful shutdown.
* Studio: wait for the uvicorn thread before the terminal returns on Ctrl+C
Builds on #6565 by @Imagineer99. The studio server runs uvicorn in a daemon
thread, so on Ctrl+C the process could return to the shell while that thread
was still writing its shutdown logs, interleaving them with the prompt.
Retain the uvicorn thread and join it (flushing stdout/stderr) before terminal
entrypoints return, from run.py's main shutdown path and the CLI shutdown paths.
Refinements over #6565:
- Bound the join at 5s (_SERVER_SHUTDOWN_JOIN_TIMEOUT, matching the existing
_graceful_shutdown subprocess timeouts) so a stalled uvicorn shutdown cannot
hang the terminal; the timeout warning branch is now reachable.
- Restore SIG_DFL for SIGINT/SIGTERM at the start of the signal handler so a
second Ctrl+C force-quits, and drop the redundant in-handler wait (the
post-loop wait already covers the signal path).
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: keep child Ctrl+C working and restore SIGBREAK
- install.sh: run studio in a subshell that resets INT to default
(trap - INT; exec ...) so the foreground child does not inherit the
installer shell's ignored SIGINT, which would otherwise swallow the
studio process's own Ctrl+C and graceful shutdown.
- run.py: also restore SIGBREAK to SIG_DFL in the signal handler so a
second Ctrl+Break force-quits on Windows, matching SIGINT/SIGTERM.
* install.sh: capture studio exit with || under set -e so the migration hint still prints
* Trim shutdown-fix comments to be terser (comments only, no code change)
* Dedup CLI shutdown-wait into finally blocks (review follow-up)
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add regression tests for the stray-forward compile-cache reset
Follow-up to #6511, which fixed the bug but whose squash merge did not
include the tests. These cover the two issues that fix addressed, under
the GPU-free tests/conftest.py harness:
- _unsloth_reset_stray_compile_cache is an exported module-level symbol in
unsloth.models._utils (it previously lived only inside the RL trainer
template string, so every non-RL import silently no-op'd)
- _unsloth_install_pretrain_detector keeps a recorded "seen" forward on an
idempotent reinstall with a live hook, and only resets it after teardown
- only a grad-enabled pre-train forward marks the cache poisoned
- the reset warns and clears seen when a stray forward was seen, tears the
hook down even on the clean path, and walks the .model/.base_model/.module
wrapper chain to reach a nested marker
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Pin UNSLOTH_COMPILE_DISABLE in the warn-path reset tests
The reset only warns and resets Dynamo when UNSLOTH_COMPILE_DISABLE != "1".
A GPU-free CI env that sets it to "1" would make the warn assertion in
test_reset_clears_seen_and_warns_when_a_stray_forward_was_seen flaky.
monkeypatch it to "0" in both warn-path tests so the warn / no-warn
assertions are deterministic and test the seen flag, not the env.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: accept --not-secure as a back-compat alias for --no-secure
PR #6560 renamed the negative secure flag from --not-secure to --no-secure
to match argparse.BooleanOptionalAction. Re-add --not-secure as a hidden,
deprecated alias at both CLI layers so existing scripts and muscle memory
keep working, while --no-secure stays the documented spelling.
- studio/backend/run.py: extract the CLI parser into _build_arg_parser() so
the flag wiring is unit-testable, and register --not-secure as a hidden
store_false alias for --no-secure. Last flag wins, matching
BooleanOptionalAction semantics.
- unsloth_cli/commands/studio.py: add a hidden --not-secure option to
`unsloth studio` and `unsloth studio run`; it forces secure off and
forwards the canonical --no-secure to the backend.
- Tests at both layers for the alias and its polarity.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address review on --not-secure alias
- run.py: use argparse.SUPPRESS for the --not-secure default so the alias
never contributes a namespace default (the canonical --secure owns it).
- studio.py: resolve --not-secure last-wins from argv via _resolve_secure()
so `--not-secure --secure` keeps secure on, matching the backend's
BooleanOptionalAction and how --secure/--no-secure already behave.
- Add a CLI last-wins test covering both flag orders.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- _quote_gemma_object_keys skipped array values, so a Gemma call with a
bare-string array argument like labels:[bug,ui] produced invalid JSON and
the whole tool call was dropped. Array values are now scanned and bare
string elements quoted, while numbers, quoted strings, and JSON literals
are preserved.
- In the OpenAI passthrough stream, a trailing usage-only chunk
(stream_options.include_usage) that arrived before any finish chunk was
relayed before the synthetic finish, producing usage -> finish -> [DONE].
Emit the synthetic finish before that usage chunk so the order matches the
other streams (finish -> usage -> [DONE]).
Extends tests/test_gemma_tool_parse_edge_cases.py with the bare-array cases.
* Studio: show tool-call progress for large GGUF tool arguments
The GGUF agentic tool loop only surfaced an early provisional tool card
for render_html, so any other tool (python, terminal, ...) was invisible
in the UI while its arguments streamed. For a large argument such as a
full HTML or code file this left the chat sitting on "Generating..." with
zero progress for tens of seconds while the model was clearly working.
Generalize the provisional tool_start to any enabled tool once its
streamed arguments grow past a threshold (render_html still surfaces
immediately, small-argument tools are unchanged). The provisional and the
real tool_start share the tool_call_id so the frontend reconciles them
into one card. Close the provisional on no-op, denial, parallel-drop,
post-loop, and on stream errors so a card can never spin forever, surface
each parallel call, and skip the early card while a human confirmation
gate is active. Apply the same confirmation-gate guard to the safetensors
agentic loop.
Additional hardening:
- Only emit a provisional card once a real, non-empty tool_call_id is
known. llama.cpp can stream a tool call with an empty id, and a card
keyed by "" cannot reconcile with the real tool_start (the frontend
mints its own id per event), so it would dangle.
- On a connection drop or other mid-iteration failure, close the dangling
provisional card with an error result instead of an empty success so the
UI renders it as failed rather than completed.
- Mirror the provisional cleanup in the safetensors loop: close a
provisional render_html card if the model generator raises mid-stream or
the controller turns the call into an internal no-op.
Adds regression tests for the empty-id guard, the error-result on a
dropped connection, and the safetensors mid-stream exception cleanup.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
