Resolves the app-sidebar conflict: main added Hub and Projects rows
inline while this branch renders the nav from navRows in the order and
pin state set under Settings -> Appearance. Kept the data-driven
rendering, having checked both of main's additions are already
represented there - the projects row carries the same icon, label,
active check, handlers and inline New project button.
Main also replaced the sidebar's inline name field with
NewProjectDialog, which owns its own state, so the button no longer
resets a name draft: it sets the move target and opens the dialog, as
main's other call sites do.
Follow-up to #7435, which fixed _smart_apt_install. Three sites were left.
studio/setup.sh: the WSL GGUF build-deps block is the pre-#7435 install.sh
pattern verbatim. It probes with 'test -r /dev/tty', assumes REPLY=y when that
fails, and then runs the elevated apt-get with stdin open. Its own guard
comment says a password is needed on WSL, so this is exactly the scenario from
issue #7307, and install.sh runs setup.sh in the same install. Give it the same
treatment: a real open probe, -n -k with stdin closed on the headless path, and
the manual command plus the existing _SKIP_GGUF_BUILD degradation on failure.
The helper is defined locally because setup.sh runs as its own process.
install.sh autostart prompt: still used 'test -r /dev/tty' and printed the
question before checking, leaving a dangling prompt in container logs. Reuse
_can_read_tty and move the printf inside the branch.
install.sh interactive escalation: a sudoers denial, a wrong password or an apt
error aborted on the bare message while the headless branch printed what to run
by hand. Make both symmetric.
Co-authored-by: danielhanchen <unslothai@gmail.com>
Chat cannot load one, so it was either hidden or failed on load. The unfiltered
picker now lists on-device diffusion models and navigates to the page that runs
them, passing the repo and quant so that page loads it.
LM Studio, ./models and custom-folder rows ignored it, so the Images picker listed
chat GGUFs that 400 on a diffusion load. The backend already tags every local model
with a task for this purpose.
A GGUF entry went out as a full snapshot, whose ignore list drops *.gguf: the job
finished at once having fetched only docs, and the repo landed on device unloadable.
Every entry is scoped now. The Hub also no longer tags image/video models as
unsupported (they run on their own pages), and those pickers name what they select.
They downloaded inline inside the load, so they had none of the manager's disk
preflight, manifest verification, resume or panel progress. Picks now stage as
scoped jobs carrying the loader's own file list, then load from a warm cache.
The quant rows never forwarded the .gguf filename, so every hub GGUF pick on
Images/Video fell through to a silent return. On Train the top bar now picks the
training base instead of a generation model, which is GGUF-only and untrainable.
* Fix the CPU-only ROCm routing errors and two font-scale UI flakes
Two unrelated causes of red CI on every PR, both reproduced before fixing.
ROCm routing: 12 errors on Repo tests (CPU). The spoof reports an AMD GPU, and
unsloth_zoo pulls in bitsandbytes, which picks a compute backend at import. Once
torch looks like a GPU is present, bnb loads its ROCm/CUDA ops, which a CPU-only
torch cannot satisfy (no libhipblas.so.2, no torch._C._cuda_getCurrentRawStream),
so the child died before printing RESULT. Nothing here tests bitsandbytes, so
import it first, under the honest hardware. Reproduced in a CPU-only torch venv:
11 passed with 12 errors before, 23 passed after. Still 23 passed on a CUDA build.
Font-scale UI: the select-viewport step pressed ArrowDown six times behind fixed
sleeps, but Radix moves focus into the listbox after the content opens, so on a
loaded runner the keys landed on the trigger and nothing scrolled. Wait on the
overflow and press until it moves, bounded at 40. The same fixed-sleep pattern
made open_appearance miss the dialog when the shortcut fired before the app wired
its handler; alternate both chords on a bounded retry and wait for the control the
caller is about to drive.
Both were reproduced locally by running the suite against a real Studio under full
CPU load. Original: 2 of 10 passed, with the exact CI signature 'keyboard did not
scroll the select viewport: 0' five times. Fixed: 10 of 10.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the ROCm routing assertion live on Apple Silicon for PR #7469
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* install.sh: do not assume sudo consent when there is no terminal (#7307 P7)
_smart_apt_install printed an "Accept? [Y/n]" prompt, and when /dev/tty was
unreadable it set REPLY=y and escalated anyway. Every sudo call in that branch
redirects stdin from /dev/null, so on any host where sudo needs a password the
install died on sudo's own error rather than the actionable message the no-sudo
path already prints. Containers, CI and locked-down corporate machines hit this.
Probe with `sudo -n true` first. If there is no terminal to prompt on and sudo
would need a password, exit with the missing packages and the exact command to
run, matching the no-sudo path. Passwordless sudo still escalates unattended,
which is the one case where that is legitimate, and says so in the log.
With a readable /dev/tty the behaviour is unchanged, and the prompt now only
prints when something can actually answer it.
Extend tests/sh/test_apt_distro_prompt.sh to drive the real function across all
four TTY/sudo combinations, rewriting /dev/tty to a fixture path the same way
the existing cases rewrite /etc/os-release. Against the old install.sh five of
these assertions fail. Register the file in studio-backend-ci.yml's shell suite,
which did not run it before.
* install.sh: probe the real tty and the real sudo commands (#7307)
Codex review follow-ups on the no-TTY sudo escalation guard.
`test -r /dev/tty` only reads the device node's permission bits. Inside
containers and systemd units those bits look fine while open() fails with
ENXIO, so the guard still fell through to a prompt nobody could answer.
_can_read_tty() does a real open. The subshell is load-bearing: in dash a
failed redirection on the special builtin `:` exits the script.
`sudo -n true` proves only that `true` is allowed. Under a command-specific
rule like `NOPASSWD: /usr/bin/apt-get` it is the wrong question in both
directions. _sudo_runs_unattended() asks the sudoers policy about the exact
argument vectors we are about to elevate, via `sudo -n -l --`, which checks
without running and fails instead of prompting.
Tests cover both: a NOPASSWD-on-trivia-but-not-apt-get sudoers stub, and a
readable-but-unopenable /dev/tty faked with a unix socket (skipped where the
platform cannot produce that shape).
* install.sh: test sudo by running it with -n, not by asking sudo -l
Codex follow-up. `sudo -n -l -- apt-get ...` answers authorization, not
authentication: on a host where apt-get is permitted but still carries the
PASSWD tag, list mode exits 0 while the actual run needs a password, so the
guard reported unattended and the escalation died exactly as #7307 described.
Inferring the answer from list output means parsing for `!authenticate`, which
is human-readable text that varies by sudo version. Drop the inference. In the
no-terminal branch, run the real commands with `sudo -n`: -n never prompts, so
it cannot block on a closed stdin, and its exit status is the question we were
trying to answer. If it is refused, print the actionable manual command as
before. The terminal branch is unchanged: prompt, then plain sudo, which may
ask for a password because someone is there to type it.
The test stub now models sudo properly (-n refuses and runs nothing when a
password is needed) instead of special-casing the probe's argv.
* install.sh: require a real NOPASSWD rule, and stop blaming the password for apt failures
Two review findings on the headless escalation branch.
A cached authentication timestamp from an earlier, unrelated elevation made
`-n` succeed for a PASSWD-tagged apt-get, so packages installed with nobody
having answered the prompt. Add `-k` so the probe ignores the timestamp and
only a real NOPASSWD rule counts as passwordless. Per sudo(8), `-k` alongside
a command ignores the cached credentials for that invocation and "will not
update the user's cached credentials", so an interactive session elsewhere
does not have to re-authenticate afterwards.
A nonzero status from the elevated apt-get was reported as "likely needs a
password" even when sudo had authenticated fine and apt itself failed on a bad
repository, a dpkg lock or a network outage. sudo returns the command's own
exit status when the command runs, so the two cases are not distinguishable
from the status alone. Report both possibilities and point at the real error.
tests/sh/test_apt_distro_prompt.sh: teach the sudo stub about -k, add a cached
mode, and assert both behaviours. The three new assertions fail against the
previous commit.
* install.sh: an unreadable answer at the consent prompt declines
_can_read_tty proves the device opens, not that anyone is there to answer. A
read that hits EOF still fell back to REPLY=y and escalated, so the branch that
does have a terminal kept the behaviour this change removes from the branch
that does not. A drained or half-closed terminal reached it.
Default to n instead, which is what the post-install autostart prompt at the
bottom of this file already does on the same condition. Enter still means yes:
that is a successful read of an empty line, not a failed read.
tests/sh/test_apt_distro_prompt.sh: add an eof tty fixture, which opens
normally and returns EOF immediately. Both new assertions fail against the
previous commit.
* install.sh: tighten the escalation comments, and correct the exit-status claim
Comment-only. The earlier note said a nonzero status from the elevated apt-get
was not distinguishable from the status alone; sudo(8) is more specific than
that. sudo exits 1 on an authentication or configuration failure and passes the
command's own status through when the command runs, while apt-get(8) returns
100 on error, so the two usually are distinguishable. sudo also exits 1 when
the command cannot be executed, which is why the message still states both
causes rather than naming one.
* install.sh, tests: tighten the comments added by this branch
Comment-only pass over the branch's own comments in both files. Same intent,
fewer lines: drop restatement, keep the parts a reader cannot derive from the
code (why test -r is the wrong probe, why the subshell around the redirection
is load-bearing under dash, what -k buys over -n, and why a nonzero status
does not by itself name the cause).
Verified to touch nothing but comments and blank lines.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* studio: shard export checkpoint loads across all visible GPUs
Export checkpoint loading always used unsloth's from_pretrained default of
device_map="sequential", which stacks the whole model on GPU0. On a multi-GPU
host this OOMs GPU0 while the other GPUs sit empty, so a GGUF export that would
comfortably fit across the machine fails with CUDA out of memory (#7053).
Add _multi_gpu_device_map_kwargs(): when the CUDA/ROCm host exposes more than
one visible GPU and get_device_map resolves to "balanced" (the same policy the
inference loader already uses), pass device_map="balanced" to every
from_pretrained in load_checkpoint. In every other case -- single GPU, CPU,
MLX, or any probe failure -- it returns {} so the loader default is untouched.
Fixes#7053
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/save: reach the UUID/MIG fallback, release sharded models before quantize
Two review fixes on the multi-GPU export sharding:
1. UUID/MIG CUDA_VISIBLE_DEVICES masks resolve to no numeric ids, so the
len(visible) > 1 gate skipped get_device_map entirely and large exports on
those hosts still stacked onto GPU0. An empty id list now routes to
get_device_map(None), whose visible-count fallback exists for exactly this
case; a genuinely GPU-less host still resolves "sequential" and keeps the
loader default.
2. The compressed (FP8/NVFP4) export freed GPU memory before its llm-compressor
subprocess only for single-device models -- a plain .to("cpu") is invalid on
an accelerate-dispatched model, so a multi-GPU-sharded checkpoint stayed
resident on every GPU while the subprocess loaded a second copy. The release
is factored into _offload_model_for_quantize_subprocess /
_restore_model_after_quantize_subprocess: dispatched all-GPU shards get their
accelerate hooks removed, move to CPU, and are re-dispatched over the
recorded hf_device_map afterwards. Maps with cpu/disk targets (already
offloading) and quantized models are left alone, as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/save: budget merged tensors per device, restore hooks if CPU offload fails
Two review fixes on the multi-GPU export path:
1. The LoRA-merge save path budgeted every merged tensor against GPU0
(get_device_properties(0) + unqualified memory_allocated()). A merged tensor
lives on the GPU of its source layer, so for a model sharded across GPUs
(the device_map="balanced" this PR enables) GPU1+ could OOM as their weights
accumulated while only GPU0's headroom was checked. Budget against W's own
device via a per-device cache; single-GPU behavior is unchanged (W on GPU0).
2. _offload_model_for_quantize_subprocess removed the accelerate hooks and then
moved a dispatched model to CPU; if that move raised (host RAM too small for
the sharded checkpoint) the model was left hookless and half-moved, breaking
later exports in the same worker. It now re-dispatches (or, for the
single-device path, moves back) on a failed move before aborting the offload.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/save: release sharded models before the torchao reload too
The portable torchao FP8/INT8 export freed the in-memory model only when every
parameter sat on one device, then reloaded a second copy with
device_map="auto". A checkpoint loaded through the new multi-GPU export map is
accelerate-dispatched across several GPUs, so that single-device gate never
fired and the original stayed resident on every GPU during the reload -- an OOM
for exactly the models large enough to have needed the sharded load.
It now uses the same _offload_model_for_quantize_subprocess /
_restore_model_after_quantize_subprocess pair as the compressed export, which
removes the accelerate hooks, moves to CPU, and re-dispatches over the recorded
hf_device_map afterwards. Those helpers are extended to XPU as well, since
torchao also runs on Intel GPUs and the path they replace covered both.
* studio/save: release quantized and cpu-spilled shards before quantize reloads
Two cases the release helper skipped outright, both of which leave GPU memory
held while the compressed subprocess or the torchao device_map="auto" reload
allocates a second copy:
- Quantized models. ExportBackend.load_checkpoint loads 4-bit by DEFAULT, so the
common Studio export hit the is_loaded_in_4bit guard and kept a quantized shard
on every visible GPU. They are now attempted like any other model: transformers
refuses .to() for some bitsandbytes builds, but that refusal raises before
anything moves, so the existing recovery path restores the model and returns
None -- best-effort where the stack allows it, old behaviour where it does not.
- Maps that spill to CPU. Any non-GPU target disqualified the whole model even
though the GPU-mapped modules were still resident and are exactly what needs
reclaiming. A cpu spill is safe to move (those weights are already in host RAM)
and is now released; only disk/meta targets are still skipped, because
accelerate keeps those parameters off the model and moving would try to
materialize the whole checkpoint. An all-CPU map is skipped as a no-op.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix multi-GPU offload for PEFT exports and fall back when sharding OOMs (#7215)
The dispatch branch of _offload_model_for_quantize_subprocess never ran for a
PEFT model: the wrapper proxies _hf_hook, so remove_hook_from_submodules raised
AttributeError and the bare except returned None. Studio always loads adapters,
so the new balanced map turned the offload off (0 percent freed against 91.8 on
the sequential path it replaces).
- resolve the real dispatch root before removing or replaying hooks
- snapshot and replay hooks, tensor placements and instance forwards; a plain
re-dispatch rebuilds hooks against the post-PEFT tree (395 to 1379) and drops
the fused kernels accelerate captured into _old_forward before unsloth patched
- drop the accelerator side of tied_params_map so the offload actually frees
- pass skip_keys on the fallback dispatch_model
- log the swallowed exception instead of returning None silently
- guard _unsloth_save_torchao_with_given_config like its two siblings
- retry the export load once on the loader default when the balanced map OOMs,
which happens when a training or chat job already owns the other GPUs
Measured on 4x B200 with Qwen3-0.6B: 89.9 percent freed bf16 and 79.7 percent
4bit under balanced, logits bit-identical, hooks and placements restored
exactly, 184 Params4bit round-tripped unchanged including nested state2.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the original offloaded until the torchao copy is released, and retie shared weights (#7215)
Two follow-ups from review of 8b6b4ca0b.
_unsloth_save_torchao_with_given_config restored the original inside a finally
that ran as soon as from_pretrained returned, so the original and the quantized
copy were both resident while the copy was still being saved. The restore now
sits in an outer finally that covers saving and releasing quantized_model, which
is what the two sibling paths already do.
The dispatch replay did not preserve tied embeddings. A CPU round trip repoints
every tensor and accelerate's tied_params_map is keyed on the old pointer, so
replaying the hooks produced two independent parameters. Reproduced on a tied
Llama: lm_head picked up its own storage, the embedding was duplicated in VRAM,
and an update to one no longer reached the other. The snapshot now records tied
groups (named_parameters(remove_duplicate=False), since the default hides one
half of every pair) and re-ties them after placements are restored.
Verified: tie preserved, no extra storages, live CUDA storage census identical
before and after, updates propagate again, logits bit-identical, and the 4 GPU
invariants unchanged at 89.9 percent freed bf16 and 79.7 percent 4bit.
* Keep meta tensors out of tie groups, restore accelerate move guards, retry CPU spills (#7215)
Four follow-ups from review of a58f1086b.
Meta tensors all report storage pointer 0, and accelerate parks every
CPU-offloaded parameter on meta, so grouping by pointer collapsed them into one
fake tied group. Reproduced with a balanced map that spills two blocks to CPU:
18 meta parameters in a single group with shapes 64x64, 32x64 and 128x64, which
the retie step would have overwritten with the first one. Meta and null-pointer
tensors are now skipped, and the retie also checks shape.
remove_hook_from_submodules deletes the to/cuda/xpu wrappers dispatch_model
installs to stop a caller moving an offloaded model. The snapshot now records
and replays those alongside forward and _old_forward.
The single-device retry only matched OOM, but a balanced map that spills to CPU
is refused by bitsandbytes with a plain ValueError saying modules were dispatched
to the CPU or the disk (transformers quantizers/quantizer_bnb_4bit.py:128), with
no memory wording. That is now retryable too, which matters because Studio loads
4-bit by default and busy secondary GPUs are exactly when balanced spills.
The torchao path dropped the quantized copy at the end of the try, so a failure
in save_pretrained left it resident while the original was restored. The del
moved into the finally, ahead of the restore.
Four regression tests added; suites now 25 and 9.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Retry exports whose multi-GPU load silently offloads to CPU, and clear the failed torchao traceback (#7215)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments for PR #7215
* Keep gradients across the export offload and release the failed torchao copy (#7215)
* Tighten comments for PR #7215
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
* Add fast fast_inference GRPO smoke test for the vLLM LoRA rollout path
Covers the vLLM >= 0.25.0 LoRA collision path (unsloth#7283, fixed in
unsloth-zoo#919) with all seven attention and MLP projections as LoRA targets so
both fused families (qkv_proj, gate_up_proj) are exercised. Kept tiny: the
ungated unsloth/Qwen2.5-0.5B-Instruct, max_steps=1 (the collision triggers on the
first rollout), short prompts/completions, and enforce_eager=True to skip CUDA
graph capture. Runs in ~89s cold and ~37s on a warm torch.compile cache.
Wrapped as a pytest test that skips without CUDA and still runs as a script; a
length-based reward gives non-zero GRPO advantages; asserts the vLLM engine is
attached at load and still bound on the trainer. Heavy imports are deferred into
the test so CPU-only collection stays import-free.
Co-authored-by: JoshuaL3000 <joshua.jian.ern.liew@intel.com>
* Assert GRPO metrics and pin seed in fast_inference test
Switch to unsloth/Qwen3-0.6B, disable vLLM torch.compile
(compilation_config=0) and run 3 steps so the updated LoRA adapter is
re-synced into vLLM on every step, not just loaded once.
Pin GRPOConfig(seed=...), which TRL forwards to vLLM SamplingParams, so
the run is reproducible, and assert per-step metrics (loss, grad_norm,
completion length, reward, reward spread, kl) instead of only checking
that train() returned. Verified across seeds 42/123/2024/7.
* Correct the seed comment and drop the pytest return
GRPOConfig(seed=...) does not reach vLLM SamplingParams: TRL's
generation_kwargs carries no seed key. Reproducibility comes from the
Trainer's set_seed pinning the global RNG the colocated sampler draws
from, so describe that instead.
Returning a value from a test triggers PytestReturnNotNoneWarning, which
pytest intends to make an error; the value was unused.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* guard llama.cpp prebuilt against out-of-disk instead of doomed source build
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review comments on out-of-disk guard
* keep reusable installs and Windows parity in the out-of-disk guard
* preserve the ENOSPC cause when re-raising fallback errors
* catch out-of-disk before the attempt loop and accept all llama-server layouts
* Fix out-of-disk detection gaps and false positives for PR #7420
Follow-ups found while testing the guard against a real ENOSPC (LD_PRELOAD
shim returning errno 28 under a path prefix, real network, real release):
- hydrate_source_tree retried the next mirror after an ENOSPC and only raised
on the last URL. Both source fallbacks 404 for the published mix commit, so
the reported cause was HTTP 404 and the run fell through to the source build
exactly like before the guard. Stop at the first environment-fatal error.
- The 5 GB preflight rejected hosts that install fine. A full CUDA install
peaks at 0.87 GB, the largest published bundle is 0.77 GB and macOS is
0.01 GB, so at 3 GB free the install succeeded before and exited 4 after,
with the source-build fallback suppressed too. It is now advisory, and a
real ENOSPC still exits 4. This also drops the case where an install
matching an older release plan was rejected before its reuse check.
- ENOSPC raised inside shutil.copytree arrives as shutil.Error with errno
None and no __cause__ or __context__, so it was never classified. That path
covers the hydrated source tree, the runtime overlay and the activation
fallback copy.
- _causal_chain followed __context__ even when __suppress_context__ was set,
so `raise ... from None` over an unrelated ENOSPC reported disk full and
wrongly suppressed the source build.
- TemporaryDirectory now ignores cleanup errors: an rmtree failure on the way
out replaced the in-flight SystemExit and lost EXIT_NO_SPACE.
- setup.sh skips the arm64 CPU last resort after exit 4; it re-ran the same
disk-rejected installer and buried the hint under a second error dump.
- The in-app updater turns exit 4 into a readable message instead of
"installer exited 4" plus a log tail.
Adds tests/studio/install/test_llama_prebuilt_no_space.py covering the
classifier, the advisory warning and the exit codes.
* Fix Python 3.9 breakage and Windows disk-full detection in the out-of-disk guard
Found by running the guard across the whole supported interpreter range
(requires-python is >=3.9,<3.15) and a spoofed [Linux, WSL, macOS, Windows] x
[NVIDIA, AMD, CPU] host matrix.
- TemporaryDirectory(ignore_cleanup_errors = True) is 3.10+, so the previous
commit raised TypeError at install time on 3.9 and turned a working install
into a hard failure. Replaced with a scratch_dir() contextmanager built on
mkdtemp plus rmtree(ignore_errors = True), which behaves the same on every
supported version.
- getattr(exc, "winerror", None) crashed on 3.9. urllib's HTTPError is an
OSError that proxies unknown attributes to a wrapped file object and raises
KeyError, which getattr does not swallow, so any mirror 404 during an install
would have blown up inside the classifier. Read it defensively instead.
- Classify Windows disk-full by winerror as well as errno. CPython's
PC/errmap.h maps ERROR_DISK_FULL (112) to ENOSPC but has no case for
ERROR_HANDLE_DISK_FULL (39), which arrives as EINVAL, so a Windows
os.replace() onto a full disk read as an ordinary failure and fell through to
the source build.
Tests cover both winerror codes, a non-disk winerror, and HTTPError alone and
wrapped in a PrebuiltFallback. 116 simulation cases pass on 3.9 through 3.14.
* Classify quota, flattened Windows and validate-install out-of-disk for PR #7420
- EDQUOT counts as out of space: a quota'd home has free blocks this user
cannot have, so the source build is just as doomed. Reported separately so
df does not mislead. Confirmed end to end with a real kernel EDQUOT: the
installer went from 6 retries then a source build (exit 2) to exit 4.
- Match the flattened Windows disk-full text. copytree stringifies each
per-file OSError, and OSError.__str__ returns early on winerror, so the
text reads [WinError 112] and never [Errno 28]. Captured on a real NTFS
volume. Markers are bracketed so WinError 112 does not match WinError 1120.
- --validate-install now exits 4 on a full disk. It caught PrebuiltFallback
and exited 2 before the classifier ran, and setup.sh answered 2 by deleting
the GPU build that had just succeeded and starting a CPU rebuild that needs
more of the space that ran out. Both halves are needed: the call site only
tested nonzero.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in the llama.cpp out-of-disk guard
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
- start_training forwards resume_source_run_id to _start_training_impl, which
reads it. Without it every start raised NameError.
- Restore main's anchor in the load-marker order test: the file now has an
earlier `if config.is_gguf:`, so indexing the first one compared the wrong
branch.
- The two diffusion tests that reach diffusers now skip when it is absent,
matching the CPU repo-test env.
- The UI smoke finds nav rows that live in the sidebar's More flyout.
* fix(studio): support hostname-based enterprise proxies
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): strip userinfo from proxy fetch targets
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* ROCm/AMD CI coverage: arch-table parity, native-Linux lib prepend, RDNA4 grouped_mm, discovery-based shell suite
Three merged ROCm fixes shipped without tests, and the CI wiring that
would have run them was gated on files the fixes do not touch.
Tests added (113):
tests/studio/install/test_rocm_arch_table_parity.py (27)
diffs the four duplicated gfx -> AMD pip-index tables across
install.sh, install.ps1, studio/setup.ps1 and install_python_stack.py,
plus the GPU-name -> arch tables and the torch 2.11 pin allowlist.
tests/studio/install/test_rocm_native_linux_lib_dirs.py (26)
covers #7233: system-ROCm lib dirs prepended ahead of bundled
libggml-hip, the /dev/kfd + not-WSL + libhsa gate, the opt-out env
var, root resolution order, and source parity between the two copies.
studio/backend/tests/test_grouped_mm_rdna4_fallback.py (46)
covers #7292: registration on the CUDA dispatch key, grouped and
ungrouped numerics, bias/dtype promotion, and the Linux HIP<7.13 +
RDNA4 name gate, executed from the shipped source rather than a copy.
tests/studio/test_ci_shell_suite_coverage.py (14)
fails if either shell runner goes back to a hardcoded list or skips
a file without a recorded reason.
CI wiring:
studio-backend-ci.yml: add install.sh / install.ps1 to the path filter
(the suites it runs assert against those two files, so install-only
changes -- the shape most AMD/ROCm routing fixes take -- skipped it),
and replace the 13-file hardcoded shell list with directory
discovery. That list had fallen seven files behind, including
test_strixhalo_wsl_reroute.sh, the only shell coverage of the ROCm
WSL reroute, which had never run on a PR.
tests/run_all.sh: same discovery loop so local and CI agree.
* Test review fixes: assert on outcomes, not on the code under test
Self-review of the previous commit found four tests that passed for the
wrong reason.
1. The arch-table parity test pinned expected gfx ids copied out of the
shipped tables, which enshrined three upstream inaccuracies as
correct: RX 9070 (non-XT) is gfx1201 not gfx1200, RX 7800 XT is
gfx1101 not gfx1100, and PRO V710 is gfx1101 not gfx1102 per AMD's
ROCm compatibility matrix. The expectation is now the AMD pip index
leaf -- the thing the tables exist to produce, and what a wrong
answer costs the user. The three known drifts are listed explicitly
with a test asserting they stay cosmetic, i.e. that the wrong and
right ids still map to the same wheel index. That test turns red the
day one of them starts routing users to the wrong wheel.
2. The RDNA4 device-name test extracted the regex from worker.py and
then matched with it, so it could not fail. Widening the pattern --
the dangerous edit, since it forces the slow Python mm fallback onto
RDNA3 users -- would have been silently accepted. It now reads the
live pattern and checks it against fixed cases, plus asserts the
name match stays guarded by `not _lin_arch` and that the name is
lowercased before matching.
3. The CI-coverage test matched a verbatim line of studio-backend-ci.yml,
so reindenting the step would fail the build while a real regression
to a hardcoded list could slip past a reformat. It now parses the
YAML, finds the step by name, and asserts on the glob plus the
absence of individual filenames. The path-filter test likewise reads
the parsed trigger instead of scanning raw text.
4. A set comprehension in the parity helper had a ternary whose branches
were identical.
Mutation-tested: widening the RDNA4 regex, desyncing one copy of the
name table, dropping install.sh from the path filter, and re-skipping
the ROCm WSL shell suite each fail at least two tests. Verified on
Linux (WSL Ubuntu 24.04) with CI's torch pin: 86 + 48 pass.
* Fix three wrong gfx ids in the GPU-name arch tables
The name -> gfx tables disagreed with AMD's ROCm compatibility matrix on
three entries. Corrected against the "Radeon GPU" list at
rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html:
RX 9070, RX 9070 GRE gfx1200 -> gfx1201 (Navi 48, same die as the XT)
RX 7800 XT, RX 7700 XT gfx1100 -> gfx1101 (Navi 32, not Navi 31)
PRO W7700 gfx1100 -> gfx1101
PRO V710 gfx1102 -> gfx1101 (Navi 32, not Navi 33)
No wheel changes for anyone: gfx1200/gfx1201 both resolve to gfx120X-all
and gfx1100/gfx1101/gfx1102 all resolve to gfx110X-all, in all four copies
of the index-family map. That collapse is why the errors survived being
copied into six places -- the leaf-level tests could not see them.
It was not purely cosmetic, though. install.sh's second copy feeds
"Tip: set UNSLOTH_ROCM_GFX_ARCH=<arch>", so a 7800 XT user following the
printed advice exported gfx1100 and made a wrong id authoritative for
every later run. It would also have become a real misroute the moment AMD
split a family across index leaves, as they already do for gfx1151/gfx1150.
Fixed in all six places, which is two more than the table's own "kept in
sync with" comments claim exist:
install.sh _infer_amd_gfx_arch_from_gpu_name
install.sh case "$_gpu_disp_mkt" (banner + env tip; undocumented)
studio/setup.sh
install.ps1
studio/setup.ps1
studio/install_python_stack.py
Ordering is preserved: the gfx1102 arm still precedes gfx1101 in the shell
copies so "RX 7700S" cannot fall onto the "RX 7700" glob, and the
PowerShell copies keep the (?!S) lookahead.
Test changes:
- test_rocm_arch_table_parity.py gains _AMD_DOCUMENTED_ARCH, exact gfx
ids transcribed from AMD rather than from the tables. Agreement between
six copies proves nothing when all six were transcribed from the same
mistake, so the ground truth has to come from outside. Verified it
catches the bug: against the pre-fix tables it fails 6 tests.
- The parity check now covers all six copies. It had four; the two
install.sh copies were being treated as one, and
_WIN_GPU_NAME_ARCH_TABLE was not checked at all.
- test_rocm_support.py's TestGfxArchNameFallback pinned two of the wrong
ids as expected values; updated, and extended with a 9060 XT and a
7900 XTX case so each RDNA3/4 die is represented.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard against unregistered copies of the GPU-name arch table
Counting the copies by hand is what let them drift: the in-code "kept in
sync with" comments claimed four, the arch-id fix found six, and scanning
the tree turns up a seventh.
TestNoUnregisteredArchTable rediscovers the copies from the source tree
instead of trusting a hand-maintained list. A table line is one that names
a card and gives its arch; real tables score 9-17 such lines and the only
other hits in the repo are two single-line prose comments, so the
three-line threshold is not load-bearing. A companion test asserts the
scan still finds the known copies, so the heuristic cannot go blind and
pass by finding nothing.
The seventh copy is tests/_zoo_rocm_spoof.py, the fixture other ROCm tests
build their fake AMD host from. It states the mapping backwards (gfx ->
the name torch should report), which makes it an independent witness: it
had gfx1101 -> RX 7800 XT and gfx1201 -> RX 9070 XT right while all six
installer copies were wrong, and nothing compared the two. Now they are
round-tripped against each other.
RX 6700 XT is pinned as a known divergence rather than normalised. AMD's
compatibility matrix documents no consumer RX 6000 card and no gfx1031 at
all, the installer arm is commented "gfx103X family", and gfx1031 appears
only as an index-family key, never as a value a name table emits. With no
external source to correct against, changing shipped behaviour would be
guesswork. A test fails if the divergence ever disappears, so the
exemption cannot go stale.
Also adds the reverse of the AMD-matrix check: a documented card that
matches no arm anywhere is a silent CPU fallback rather than a wrong id.
This cannot detect hardware nobody transcribed, which would need a live
fetch of AMD's matrix and a non-hermetic suite; the docstring says so
rather than implying coverage that is not there.
Verified on Linux: 478 passed, plus all five new guards mutation-tested
to confirm each fails when its invariant is broken.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Docstring said six copies; the list under it now has seven
* tests: run discovered shell tests with bash, not sh
tests/run_all.sh discovered tests/sh/ instead of listing files, but still
invoked each one with sh. Every file there declares a bash shebang, and on
Debian/Ubuntu /bin/sh is dash: test_apt_distro_prompt.sh,
test_studio_home_node_dir.sh and test_with_llama_cpp_dir_link_behavior.sh
fail on bashisms under dash and pass under bash. The old hand-written list
happened to name only dash-clean files, so switching to discovery is what
surfaced it. Backend CI already used bash, so this was a local-only break.
Guarded by a new test asserting both runners invoke tests/sh/ with bash.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Krackan Point (Radeon 860M/840M) routed to the gfx1150 wheel index
The GPU-name tables map 860M/840M and the Ryzen AI 7 350 / AI 5 340 CPU
strings to gfx1150, but Krackan Point is gfx1152. AMD's own lemonade table
(src/cpp/server/system_info.cpp) maps both Krackan iGPUs to gfx1152.
Unlike the three ids already fixed here, this one is not wheel-neutral:
repo.amd.com publishes gfx1150 and gfx1152 as separate index leaves with
separately built torch wheels, so these laptops were installing wheels
built for a different LLVM target. gfx1152 was absent from the codebase
entirely, so it needed the index-family maps, the torch 2.11 floor lists
(same _grouped_mm bug as gfx1150/1151), the Strix reroute set and the
Windows arch allowlist as well as the seven name tables.
The parity test added in this PR did not catch it because its AMD-matrix
expectations stopped at 890M/880M. Added the APU rows, so the case that
actually changes a wheel is now covered: reverting the tables fails 9
tests naming 860M, 840M and Krackan.
gfx1153 (Ryzen AI 5 430 era) is left alone; AMD publishes no gfx1153
wheel family, so there is nothing to route it to.
Verified: bash -n on both shell installers, PowerShell AST parse on both
.ps1 files, python ast.parse on all touched modules, install suite 1334
passed with no new failures against main, shell suite 20 files.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add gfx1152 to unified-memory classifiers, make parity allowlist set-based
Krackan Point (gfx1152, Radeon 860M/840M) is the third RDNA 3.5 APU and
shares one GPU/system-RAM pool exactly like Strix Point (gfx1150) and
Strix Halo (gfx1151), but only the installers knew about it. The two
runtime classifiers still had two-element arch sets, so a Krackan laptop
got the 0.90 discrete headroom factor on a shared pool and ran llama.cpp
without GGML_CUDA_ENABLE_UNIFIED_MEMORY.
- worker.py _rocm_classify_unified_memory: add gfx1152 to the arch set,
and 860m/840m to the device-name fallback. The NVIDIA GeForce 840M
cannot collide there: the function is only reached under _hw.IS_ROCM.
- llama_cpp.py _amd_apu_wants_unified_memory: add gfx1152 to the arch set.
- Tests for both, including the :sramecc-:xnack- suffix form.
TestGfx211AllowlistParity compared four hardcoded allowlist strings, so
adding gfx1152 to all four installers correctly turned three assertions
red without any installer actually disagreeing with another. Each test
now extracts the set its installer holds and compares it to one EXPECTED
constant. Order and spacing are free, membership is not, and the next
leaf is a one-line edit instead of four.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Resolve the drift between PR #6763 and current main:
- deletion: main moved cached-model deletion into hub/services/models/deletion.py,
so the PR's Images/Video in-use guards move there too as _diffusion_blocks_delete
and _video_blocks_delete, keeping main's fail-closed 503 contract.
- llama_keepwarm: take main's rewrite, re-apply the PR's image/video inference
suffixes so a generation in flight blocks an idle unload.
- routes/training: keep main's sidecar-swap 409 and resume_source_run_id, run
start_training in the worker thread the PR's unload hook needs.
- model picker: main renamed components/assistant-ui/model-selector ->
features/model-picker/... and rewrote pickers.tsx, so the PR's picker work is
ported onto main's version (task/catalog props, task gating of hub + cached +
local rows, single-device expanderGpuGb, fine-tuned section hidden when scoped)
rather than reverting main's pinned-models and per-model-config work.
- images/video pages: imports repointed at the new model-selector path.
- tests: delete-guard tests retargeted at the deletion service.
Typecheck, i18n parity and model-catalog checks pass.
* fix(studio): honor run settings on initial model load
When loading a model from the gear-icon run-settings page, Context Length
and KV Cache Dtype were ignored if the user clicked Load before blurring
the context field, or before React flushed staged config into the store.
- Add NumericValueInput.commit() to flush a focused draft on Load
- Pass effectiveLoadConfig from model-config-page to onRun
- Prefer selection.config in performLoad for all load knobs
- Preserve meta.forceReload from the config-page reload path
Fixes#7346
* fix(studio): flush NumericValueInput draft when Load blurs first
Clicking Load blurs the context field before handleRun runs, so commit()
returned the stale value prop. Keep draft in a ref and parse it even when
the input is no longer focused.
* fix(studio): preserve Auto context when Load is clicked without edits
NumericValueInput.commit() now returns null unless the user actually
changed the field, so GGUF Load/Save no longer pins the displayed native
context into customContextLength when Auto was left untouched.
* fix(studio): clear NumericValueInput dirty state after blur commit
After a normal blur commit, reset dirtyRef so a later Load cannot replay a
stale draftRef when the user changed context via Reset or the slider.
* test(studio): pin NumericValueInput Auto/dirty contracts for #7346
Lock Codex P1/P2: commit returns null unless dirty, blur clears dirtyRef,
and handleRun only promotes a non-null committed context.
* fix(studio): keep same-click context draft after blur (#7346)
Blur can commit and clear dirtyRef before Load's onClick; stash that
committed value for one imperative commit() so typed context is not lost.
* chore: refresh PR head for #7351
* fix(studio): handle context commit edge cases
* chore: refresh PR head
* test(studio): guard invalid context drafts
* style(studio): format context draft guard
* test(studio): exercise same-click model config loads
* fix(studio): drop stale blur pin when the typed context equals the shown value
NumericValueInput cached every blur commit in lastBlurCommittedRef, even when
the draft equalled the current value and no onChange was dispatched. Because the
displayed value never changed, the useEffect([value]) clear never fired, so a
later Reset or external edit that leaves the shown value unchanged could not drop
the cache and the next commit() replayed it into an override that Reset had
removed. Only cache the blur result when it actually dispatched onChange
(final !== value); when final === value the parent is already current and there
is nothing to bridge. Add a Playwright regression that re-types the shown context
and asserts no override is stored.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: commit every same-click numeric draft before staging the load config
The run-settings Load/Reload button flushed only the GGUF Context Length draft
imperatively before building the load config. Max Seq Length (non-GGUF), GPU
Layers and MoE Layers on CPU (GGUF) are the same NumericValueInput and stage
their typed value only on blur, so editing one and clicking Load in the same
gesture staged the load from a still-stale parent config and dropped the value
the user just typed.
Wire an imperative commit handle through those inputs too and fold every
committed draft into the effective config, recomputing the non-GGUF load-time
max sequence length from the committed draft.
* fix(studio): recompute fixed-layer context pin and drop stale blur cache on every render
Two run-settings edge cases on the model-config page:
1) pinFixedLayerContext was computed from the render-time config, before a
same-click GPU Layers draft is committed in handleRun. Typing a positive
fixed-layer value on an auto-fit GGUF and clicking Reload therefore built
the runtime config with customContextLength: null, so a later fresh load
sent the native context with fixed layers (the OOM the pin exists to
avoid). Recompute the pin from the committed effectiveConfig.
2) NumericValueInput cleared its blur bridge only on a value change. A real
edit (final !== value) that Reset then reverts to the same shown number
nets value back unchanged, so the effect never re-ran and the stale pin
survived into the next Load/Save, replaying the override Reset removed.
The bridge is only valid across the single synchronous same-click gesture
that set it, so clear it on every settled render instead.
Add source-contract regressions for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix reasoning-only Qwen3.6 completions in Studio
* Address reasoning-only review findings
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: scale menu, toast, chat and composer icons with the UI font size
Glyphs that sit beside scaled labels now follow the preference: the
shared --icon-size token (nav, settings tabs, chat action bars, code
block actions), classed svgs inside dropdown, select, context, menubar,
popover and command surfaces, toasts, the chat thread and both
composers, and the composer pill glyph slot. Sonner toast text is
unpinned from its injected 13px. Hit targets, paddings and surface
geometry stay fixed and every value is identity at the default size.
* Studio: icons scale at half the UI font size rate; cover review gaps
Icons now follow the preference at half the rate of the text, matching
the logo lockup: base + (setting - 16) / 2. The menu specific rules
that outranked the scoped block (app-user-menu, unsloth-plus-menu,
unsloth-tick) carry the scale too, which also restores the plus menu's
intended 1.15rem glyph base at the default size. From review: closed
select triggers join the scoped surfaces so their chevron tracks the
label, sonner action button labels scale at full text rate alongside
the title and description, and the unused built-in sonner loader gets a
defensive size override.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: icons match the text scale below the default, half rate above
Piecewise icon scaling: below the 16px default icons follow the UI font
size at the full text rate, above it they move at half the rate so
glyphs stay slightly smaller than the text. Written as min(full, half)
since the smaller branch is correct on each side. Applies to the shared
--icon-size token, the scoped menu, toast, chat and composer overrides,
and the menu rules that outrank them.
* Studio: cap icons at their default size above the 16px setting
Below the default icons still match the text scale; above it they now
keep their default size instead of growing at half rate, so enlarged
text dominates and glyphs read slightly smaller than the text. The
curve is min(full rate, base).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: icons above the default scale at half rate, not capped
A 16px glyph at setting 20 renders 18px, as if the setting were 18:
above the default icons move at half the rate of the text, below it
they match the text scale. The curve is min(full rate, half rate).
* Studio: standard icons render at the UI font size itself
One shared --ui-icon-size token replaces the per-base curves for every
glyph with a 16px or larger base: icons match the UI font size below
the default and grow at half the change above it, so setting 12 gives
12px icons, 16 gives 16px and 20 gives 18px, slightly smaller than the
enlarged text. Sub 16px glyphs keep their proportions through the same
curve as a factor. This also slims the previous 18px to 21px icon bases
down to the font size at the default setting.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: icon scale review fixes for ticks, comboboxes and art glyphs
From review: thinking ticks keep their own size inside plus menus (the
important menu rule now excludes them), combobox popups and triggers
join the scoped surfaces, 24px size-6 art glyphs such as attachment
tile icons go back to proportional scaling instead of the uniform
token, branch picker 36px chevrons scale proportionally beside their
counter, and buttons that default un-classed icons to size-4 get the
shared token (xs buttons keep their pinned small icons). Sonner cancel
labels already scale: sonner renders cancel with data-button set, so
the existing override reaches it.
* Studio: keep the toast close glyph compact
The button icon fallback matched Sonner's close button, whose unclassed
12px X then rendered at the shared icon size inside its fixed control.
Exclude data-close-button from the fallback.
* Studio: use text-ui-11 for the new chat settings sheet caption
The raw px guard caught a text-[11px] added on main; raw px text
ignores the UI font size preference.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat(studio): save load settings in chat presets
Presets previously stored only sampling params (temperature, top_p, etc.).
Extend them with an optional loadConfig blob that captures context length,
KV cache dtype, speculative decoding, and GPU layer knobs from the current
runtime when saving.
- Apply loadConfig when switching presets or hydrating on startup
- Show a short summary under the preset controls
- Prompt to reload when a model is already loaded
Fixes#7347
* fix(studio): persist preset loadConfig and capture GGUF context
Add ChatPresetLoadConfig to the chat settings API schema so presets with
load settings no longer 400 on save. Capture effective GGUF context from
ggufContextLength when customContextLength is cleared after auto-mode load.
* fix(studio): address Codex review on preset load settings
Coalesce default maxSeqLength/speculative/gpu knobs when capturing presets,
no-op apply for legacy presets without loadConfig, preserve GPU pin on apply,
and stop replaying stale loadConfig during settings hydration.
* Remove unused getOrderedPresets import
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Fix ROCm wheel-index test: extract the gfx-arch probe helpers get_torch_index_url now calls
get_torch_index_url gained a gfx-arch probe on the ROCm path (Strix reroute
work) and now calls _ensure_rocm_probe_env, _probe_amd_gfx_arch,
_infer_linux_amd_gfx_arch and friends. The unit test in
tests/sh/test_get_torch_index_url.sh sources a curated subset of install.sh
functions, and that list was never updated, so those helpers were undefined
in the harness. On the ROCm path the gfx probe hit an undefined function,
the branch silently fell through to the CPU wheel index, and every ROCm
assertion failed (9 failures: all ROCm versions resolved to /whl/cpu).
Extract the six missing helpers so the ROCm branch runs end to end. All 49
assertions pass. Adds a comment noting these must stay in sync with
install.sh.
* Keep the ROCm wheel-index test hermetic: redirect the /opt/rocm prefix
Extracting _ensure_rocm_probe_env pulled its absolute-path host probe into the
harness: it appends /opt/rocm/bin to PATH and runs the real host rocminfo, and
version detection reads /opt/rocm/.info/version. On a host with ROCm installed
that leaks the host GPU into the minimal-PATH test, so the no-GPU and
CUDA-visible-device assertions could select a host ROCm wheel index instead of
their expected CPU result, making the test host-dependent.
Redirect the whole /opt/rocm prefix to an empty temp dir in the same sed pass
that stubs /usr/bin/nvidia-smi, so the probes stay hermetic. All 49 assertions
pass and the generated harness contains no real /opt/rocm path.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Studio: register text-ui tokens with tailwind-merge so cn keeps them
Stock tailwind-merge classifies text-ui-* as a text color, so cn() dropped
the size class whenever a color utility followed it in the same call. The
element then fell back to the unscaled 16px root font, which made hub tabs
and capability pills look oversized at small UI font sizes. Extend the
merge config so text-ui-* and leading-ui-* resolve as font-size and
line-height groups, and cover the failure in the contract and Playwright
regression tests.
* Studio: rename the Models page to Model hub
Page heading, sidebar navigation label in all locales, and the chat
download toasts that point at the tab.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
Downgrades a headless-Chromium renderer crash in the voice model-picker step to a warning plus page recovery on macos-14, where CheckMediaAccessPermission can kill the tab. Linux and Windows strict smoke jobs keep hard crash coverage and any live-page failure stays a hard fail.
The slim whisper bundle is ggml-less and links the ggml runtime out of the
installed llama.cpp prebuilt, so each whisper release pins a paired llama tag.
The gate required an exact tag match, but llama fork tags are
b<upstream_build>-mix-<ggml_commit> and the build number tracks upstream llama
and fork PRs that live outside ggml. When llama republishes a newer build with
the same ggml commit (a frequent event), the installed llama advances past the
whisper pin and curated dictation goes unavailable until whisper is republished,
even though the ggml runtime is ABI-identical.
Key the pairing gate on the ggml commit after -mix- instead of the full tag, in
all three comparison sites (slim_pairing_for_artifact,
_slim_release_incompatibility, resolve_selection). requires_ggml_sonames stays
the real per-file ABI gate, and a genuine ggml skew still fails closed. Tags
without a -mix- marker fall back to exact matching.
* fix(install): show detected distro in sudo apt Accept prompt
Make the package-install elevation prompt name the detected distro and
state that packages come from official apt repos, so users know we are
not installing a tarball outside their package manager (#6207).
* fix(install): avoid case/;; inside $() for bash 3.2
macOS CI uses bash 3.2, which misparses case arms inside command
substitution and fails install.sh at the apt distro helper. Use a
plain subshell so the Accept? prompt still works everywhere.
* fix(studio): opt-in source-build GPU smoke validation (#5854)
Gap 1 (empty CUDA arch -> CPU) already landed in #6481. Wire gap 2: after a
GPU source build, optionally run the same staged llama-server smoke test as
the prebuilt path, then CPU-fallback on failure. Gated by
UNSLOTH_LLAMA_STAGED_VALIDATION (default off) to avoid Blackwell JIT stalls.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(install): normalize staged validation env in setup.sh (#7322)
Strip and lowercase UNSLOTH_LLAMA_STAGED_VALIDATION before the shell
gate so values like True and surrounding whitespace match the Python
staged_validation_enabled() helper.
* Rebuild visual server after staged-validation CPU fallback (#5854)
Mirror the primary source-build path by best-effort building
llama-diffusion-gemma-visual-server after smoke-failure CPU fallback.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: pin torchcodec for torch 2.10 and warn on ABI mismatch
Add unsloth[audio] extra with torchcodec>=0.10.0,<0.11.0 and emit a
clear warning when installed torchcodec minors disagree with torch
(unslothai/unsloth#7225).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(packaging): address Codex review on torchcodec/torch 2.10 compat (#7299)
- Postpone annotations so import_fixes loads on Python 3.9
- Align TORCH_TORCHCODEC matrix with upstream (2.9: 0.8/0.9, 2.8: 0.6/0.7)
- Fix mismatch hint upper bound (<0.11.0) and gate audio-torch210 suggestion
- Split audio extra per torch minor; gate torch210 pin behind python>=3.10
- Bundle audio-torch210 only in *-torch2100 install extras
* fix(security): refresh openai CRITICAL scan baseline hashes (#7299)
openai package code drift reopened five CRITICAL findings in the
extras pip-scan-packages shard (C2 loop body hashes + IMDS/network
evidence). Update the reviewed allowlist evidence/hashes so CI gates
on new findings only, not benign SDK churn.
* chore: retrigger CI after baseline refresh (#7299)
* chore: touch scan baseline comment to retrigger security audit (#7299)
* Guard torchcodec version parsing so bad version strings cannot break import
* Bundle audio pin into intel-gpu-torch210 and guard the mismatch warning
* Tighten comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio): persist connection model selections server-side
Remote Studio clients could see saved connections but not their enabled
model lists because models lived only in browser localStorage.
Store models and available_models in llm_providers and sync them through
the providers API so alternate clients inherit the same catalog state.
Fixes#7281
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Hydrate external connections on chat startup (#7281)
Extract provider sync logic into sync-external-providers.ts and call it
from chat-page on mount so persisted model selections appear in the
Connected picker without opening Settings → Connections first.
* fix(studio): backfill connection models and preserve local options (#7298)
Address Codex P2 on remote connection persistence:
- Backfill localStorage model selections to /api/providers when backend
rows still have empty models_json (legacy upgrades)
- Carry promptCacheTtl and openaiContainerTtlMinutes through startup sync
- Await hydratePersistedSettings before syncing on ChatPage mount
Contract tests: 7 passed; npm run typecheck passed.
* Tighten comments
* Tighten comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio): show chat sidebar menu on touch devices
Recents/Pinned chat row actions were hidden until hover, so iPad users
could not open the kebab menu to delete chats. Reveal actions on coarse
pointers using the same pattern as hub model rows.
Fixes#7276
* Fix coarse-pointer sidebar row action visibility (#7276)
Move the touch-device override into index.css after .sidebar-row-action so
it wins the cascade. Arbitrary Tailwind media utilities on the element had
equal specificity and were overridden by the base rule.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope coarse-pointer sidebar actions to chat rows (#7276)
Only chat kebabs/unpin buttons that reserve touch padding get
sidebar-touch-reveal, so project/run/nav rows stay hover-revealed.
* Tighten comments
* Reserve full kebab hit area on coarse-pointer unpinned rows
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* feat(studio): offer MLX-supported optimizers on Apple Silicon
The training form's optimizer dropdown only listed CUDA/bitsandbytes optimizers (adamw_8bit, paged variants, torch fused). On Apple Silicon the MLX trainer supports a different set (adamw, adam, lion, muon, sgd, adafactor) and remaps every bitsandbytes/torch name to plain AdamW, so the dropdown misrepresented what actually runs.
Offer the MLX optimizer list when the device is a Mac, and derive the displayed value so the control is never blank: the shared CUDA default and the other bitsandbytes/torch options render as AdamW (exactly how the MLX backend normalizes them), while any other value is shown as-is so an unrecognized or non-canonical imported optimizer is never mislabeled. Non-Mac behavior is unchanged. The run-summary optimizer label now resolves from both lists.
* feat(studio): show an MLX-appropriate optimizer tooltip on Apple Silicon
The optimizer tooltip described "8-bit variants" and recommended "Fused" for vision models, neither of which is offered when training runs on MLX. On Apple Silicon, show a tooltip that matches the MLX optimizer set and notes that Lion typically needs a lower learning rate than AdamW.
Copy-only: no change to the selected optimizer or the learning rate, and the non-Mac tooltip is unchanged. The new string is added to the English locale; other locales fall back to English until translated, matching how new keys are handled elsewhere.
* fix(studio): label Mac CUDA-alias optimizers as AdamW in the run summary
On Apple Silicon the run-configuration summary looked up the stored optimizer name directly, so a run that kept a CUDA/bitsandbytes default such as adamw_8bit was labeled "AdamW 8-bit" even though the picker shows "AdamW" and the MLX backend runs plain AdamW. Mirror the training form's derivation so those aliases are labeled AdamW in the summary too.
Display-only: no change to the stored or submitted optimizer, and non-Mac summaries are unchanged.
* feat(studio): disable LoftQ and sequence packing on Apple Silicon
Neither LoftQ nor sequence packing is supported on MLX — the backend rejects LoftQ and the trainer silently forces packing off — yet the training form still offered both on Apple Silicon.
Disable the LoftQ LoRA-init option (greyed and unclickable, with an inline "Not supported on Apple Silicon" note) and the "Enable packing" checkbox (greyed, with a tooltip explaining why), matching how the unsupported "Enable streaming" control is presented. Clearing effects reset a stale loftq/packing value to its default on Mac so the disabled controls never submit it. Non-Mac behavior is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add Voice settings tab (dictation, dictionary, read aloud)
New Voice tab in Settings, placed just before About:
- Dictation: microphone picker, browser STT engine, recognition language,
and an inline mic test with a live transcript
- Dictation dictionary: entries rewrite matching speech to their exact
spelling and casing, applied in both dictation paths
- Recent dictations: last 20 final transcripts with copy and clear, so
text can be recovered if it lands in the wrong place
- Read aloud: optional button on assistant responses with two engines,
curated system voices (novelty and legacy voices filtered, quality
ranked, capped at 20) or the TTS audio model loaded in Unsloth via
/audio/generate (e.g. Orpheus), plus speed, pitch, volume and preview
Settings persist in localStorage (unsloth_voice_settings) and are read
at call time so changes apply without reloading the runtime. Adds en
keys plus the tab label for ja, zh-CN and pt-BR.
* Studio: drop the single option STT engine select, rename TTS option
The STT engine dropdown only had one entry, so it added noise without
giving a real choice. The engine row can come back once local STT
models land. Also renames the TTS engine option Unsloth TTS model to
Load TTS model to make the action clearer.
* Studio: harden Voice settings against edge cases found in simulation
Simulated the feature across Chromium, Firefox and WebKit plus node
level unit runs and backend contract checks. Fixes from the findings:
- Dictionary rewrite used a replacement string, so entries containing
dollar patterns corrupted transcripts (A$$AP became A$AP, $& injected
the match). Switched to the callback form of String.replace
- Persisted voice settings now validate types on hydration: non string
micDeviceId, dictationLanguage and ttsVoiceURI, and non boolean
ttsEnabled fall back to defaults instead of flowing into the UI
- Dictionary entries are trimmed, capped at 120 chars and re-sanitized
on hydration
- The Test dictation panel now falls back to the default microphone
when the saved device is unplugged, matching the composer adapter
Test coverage: 46 unit assertions (dictionary regex edge cases across
unicode, word boundaries and injection, voice curation for simulated
macOS, Windows and Linux voice inventories, corrupt storage merge),
13 backend contract checks against /audio/generate on an isolated
instance, and 60 browser assertions across the three engines covering
rendering, degradation without SpeechRecognition, curation in a real
DOM, dictionary persistence with unicode and dollar entries, the
no-model preview error path and corrupt localStorage recovery.
* Studio: address Voice settings review feedback
Verified each review comment before acting. Confirmed and fixed:
- Editing a dictionary entry was broken in two ways: the store trimmed
on every keystroke so spaces could not be typed, and clearing the
field deleted the entry and unmounted the input mid edit. Updates now
keep the raw value and a blur commit trims or removes the entry
- The unplugged mic fallback checked instanceof DOMException, but a
cross browser probe showed Firefox and WebKit throw
OverconstrainedError objects that are not DOMExceptions, so the
fallback never fired there. Matching on the error name now
- When the browser ended a dictation test on its own (silence timeout),
the mic stream stayed open. All recognition end paths now stop the
tracks and save the transcript through a single finalize path
- The studio TTS audio element now releases its WAV data URL as soon as
playback ends, fails or is cancelled
- Allow microphone now reports insecure contexts (no mediaDevices)
accurately instead of claiming access was blocked
- Voice tab copy moved into i18n keys per src/i18n/AGENTS.md, so locale
overlays can translate it; en is the baseline and parity passes
- unsloth_voice_settings added to the Reset all local preferences key
list so voice preferences obey the reset
- Non default microphones note that the system default is used when the
browser speech engine cannot bind a specific device, since browsers
without the start(track) overload ignore the argument silently
Re-ran the full simulation set after the changes: 46 unit assertions,
13 backend contract checks and 60 browser assertions across Chromium,
Firefox and WebKit all pass, plus a dedicated browser probe for the
dictionary editing behavior.
* Studio: use the chat mic icon in Voice settings for consistency
The Voice tab and its buttons used the hugeicons Mic02 glyph while the
chat composer uses a custom filled mic. Extract that composer icon into
a shared lib/mic-icon component, drop the duplicate inline copies in
thread.tsx and shared-composer.tsx, and use it for the Voice tab icon
and the tab's mic buttons so the microphone looks the same everywhere.
* Studio: address second round of Voice settings review feedback
Verified each new comment against the current code first. One item was
already fixed in the previous round (recording transcripts when the
browser ends a dictation test on its own). Confirmed and fixed:
- The microphone row showed a picker with generic names when browsers
enumerate unlabeled devices before permission, leaving no way to
grant access from the row. It now branches on whether labels are
visible and shows Allow microphone otherwise
- Compare chat dictation ignored the selected microphone. It now opens
the chosen device with the same fallback rules as the main adapter,
passes the track to recognition where supported and releases the
stream when recognition ends
- Closing the Voice tab cancelled the shared speechSynthesis even when
read aloud was playing a chat message. Cleanup now only cancels when
the tab owns an active preview
- Double clicking Start test could race two recognizers and leak the
first stream. A starting flag set before the getUserMedia await makes
start reentrancy safe
- Turning off the read aloud setting mid playback removed the only stop
control. The stop button now renders whenever a message is speaking
- When an engine lacks the start(track) overload, both dictation paths
now release the selected device stream before retrying with the
default microphone instead of holding it open
- Read aloud support no longer requires Web Speech synthesis: the
Unsloth TTS engine only needs audio playback, so it stays available
in WebViews without speechSynthesis, with a clear error if the system
engine is chosen there
Not addressed here: cancelling in flight backend TTS generation on
stop. The route runs generation in a worker thread without a
cancellation path, which is shared pre existing behavior with audio
chat generation and belongs in a backend change.
All suites re-run green: 46 unit, 13 backend contract and 60 browser
matrix assertions across Chromium, Firefox and WebKit, plus probes for
the unlabeled device branch and the double click race.
* Studio: drop empty and duplicate voiceURIs so the Voice tab never renders a crashing Select item
* Studio: guard dictation mic lifecycle in Voice test and Compare composer
Release a microphone opened after the component unmounts, and stop Compare
dictation on a permission or security failure instead of silently recording
from the default device, matching the main chat adapter.
* Studio: fix dictation and read-aloud lifecycle edge cases in Voice settings
- Join final dictation chunks with a space so recorded transcripts do not merge words
- Ignore a stale recognizer onend so a quick stop then restart is not torn down
- Use previewingRef so a double click on TTS preview does not orphan the first request
- Keep the read-aloud stop control visible when a new run starts while a message is spoken
- Stop the dictionary remove button from deleting an adjacent entry on a blur then click race
* Studio: trim redundant Voice settings comments
* Studio: fix Voice preview and Compare dictation edge cases
- Only cancel the shared speechSynthesis for a system-voice preview, so stopping
a Studio preview no longer stops an unrelated chat read-aloud
- Release the Studio preview audio and its WAV data URL on normal completion
- Iterate every finalized result in Compare dictation so batched phrases are kept
- Cap persisted recent dictations to the last 20 on hydration
* Studio: use clipboard fallback for recents and release failed preview audio
- Copy recent dictations via the copyToClipboard helper so the execCommand
fallback works in Safari and insecure http LAN contexts
- Release the Studio preview audio when play() rejects, not just on ended/error
* Studio: add local speech-to-text dictation engine
Add an offline dictation engine that transcribes with a local faster-whisper
model, alongside the existing browser (Web Speech) engine. The browser engine
streams audio to Apple or Google speech services and needs internet; the new
engine runs on the server, works offline, and drives any chat model without
evicting it (it loads in the backend process, separate from the model
subprocess). It also gives Firefox dictation, which has no Web Speech support.
Backend: a lazily-loaded, kept-warm faster-whisper sidecar and three routes
under /api/inference/audio (stt/status, stt/load, transcribe). faster-whisper
is torch-free, so this does not disturb the existing model stack.
Frontend: a Dictation engine setting (browser or local model), a curated model
picker with sizes, and MediaRecorder capture posted to the transcribe route.
The model warms automatically when the engine is selected, with live status.
* Studio: stream local STT transcription as you speak
Local dictation showed nothing until you stopped, because the whole clip was
transcribed once on stop. Now the growing recording is re-transcribed on a
fast pass every second and emitted as live interim text, with an accurate
final pass on stop. Partial recordings decode fine, and the model refines
earlier words as more audio arrives.
Adds an interim flag to the transcribe route (beam 1, no VAD) for the fast
preview pass; the final stop uses the accurate path.
* Studio: make local dictation stop instant and reliable
Stopping local dictation waited for a final network transcription before the
session ended, so the stop button did not flip and a second click ended the
session early and dropped the text. Now stop commits the live transcript
immediately, releases the mic at once, and ignores a second stop while
finalizing. Previews run more often so the committed text is current.
* Studio: record local dictation in short clips for reliable streaming
Re-transcribing a growing buffer every second got slower as it grew, flooded
the backend, showed stale words, and could leave the stop button stuck waiting
on a backlog. Record short independent clips instead and transcribe each once,
appending the text as you speak. Work per clip is bounded, so stopping is
prompt (with a hard timeout as a safety net) and long dictations stay smooth.
* Studio: dictate then transcribe once on stop, ChatGPT style
Local STT dictation streamed by re-transcribing the growing clip, which
was quadratic and saturated the backend (multi-second lag), and stop only
halted the recorder without releasing the mic, so it kept recording. Record
the microphone continuously, release it the instant the user stops, and
transcribe the whole clip once. Stopping is immediate and the transcript
lands in about a second. Also add the tiny model for the fastest option.
* Studio: surface dictation and read-aloud failures instead of failing silently
- Compare dictation reports microphone and speech-recognition errors via toast,
reusing the main chat adapter's describeMediaError and describeSpeechError
- Read-aloud toasts genuine model or synthesis failures while ignoring cancellations
* Studio: ChatGPT-style recording bar for dictation
Clicking the mic now drops the composer into a dedicated recording bar
with a live waveform, a discard (X) and a confirm (tick), instead of a
plain stop button. The tick stops recording and transcribes the clip;
the X throws the recording away and keeps whatever text was already in
the composer. The model adapter taps the mic with an analyser to drive
the waveform, and the router tracks the live session so the X can cancel
it without transcribing.
* Studio: transcribe dictation while speaking, ChatGPT layout
Match ChatGPT's recording layout: the bar now renders in place of the
input with the left plus button kept, the waveform in the middle, and
the discard and confirm buttons together on the right.
Cut the post-confirm delay by transcribing in the background as the user
talks. The audio is split at natural pauses (voice-activity detection off
the same analyser that drives the waveform) and each clip is transcribed
as it is cut, so confirming only has to finish the short final tail. The
model is also warmed when recording starts so the first run never pays a
cold load.
* Studio: ChatGPT waveform, hide tools while dictating, faster STT
Make the recording UI read like ChatGPT: the waveform is now a dense row
of round dots that rise into thin centered bars, and while dictating only
the plus button shows, with the mode badge and tool toggles hidden so the
bar is just the waveform and controls.
Speed up transcription: decode greedily (beam_size=1), which is several
times faster on CPU with negligible accuracy loss on short dictation
clips, and cap background segments at 6s so the final tail after confirm
stays short.
* Studio: finish ChatGPT voice bar and low-latency STT
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: full-width waveform with a timer that freezes on stop
Use the full-width waveform for the recording bar: brighter, bigger bars
that advance on a fixed cadence (keeping peaks between advances) so they
glide instead of racing by, inset from the composer edges. Keep a visible
timer and the green confirm button, matching the ChatGPT reference, and
freeze the timer and waveform the moment the user confirms.
* Studio: fix multilingual local dictation
* Studio: speed up dictation and release local STT
* Studio: harden dictation finalization and STT decoding
* Studio: restore Firefox dictation fallback
* Studio: add dictation history manager
* Studio: manage speech model downloads
* Studio: remove em dash from voice model label
* Studio: move dictation history into Voice
* Studio: source local STT from Unsloth Whisper models
Point the dictation STT sidecar and its Model Hub download entries at
Unsloth's Hugging Face Whisper repos (small, large-v3-turbo, large-v3)
and run them through Transformers, so Studio only ever downloads
Unsloth-uploaded weights. Drop faster-whisper and the Systran/mobiuslabs
repos; keep the Model Hub as the only download path via local_files_only,
and keep PyAV for audio decoding.
Device selection uses float16 on CUDA and float32 on MPS and CPU, since
Whisper's decoder is unstable in float16 on MPS and repeats tokens.
Shorten the model picker labels to name plus download size and update the
STT tests for the new backend.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: smooth dictation waveform and keep pill height
* Studio: align STT model dropdown width and tidy voice copy
* Studio: guide to local engine when browser dictation is offline
* Studio: clarify voice section and STT model copy
* Studio: keep STT warm with training-aware eviction
* Harden STT lifecycle and browser compatibility
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model discovery test lint
* Harden cross-browser microphone errors
* Harden cross-browser microphone errors
* Surface voice test recognition errors and fall back to Studio TTS
- Voice test now toasts non-abort speech-recognition failures instead of
ending silently, matching the main and Compare dictation paths.
- Read-aloud routes to the backend model when the runtime lacks Web Speech
synthesis (audio-only WebView), so it no longer errors immediately.
* Fix reviewed STT lifecycle races
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix read-aloud fallback controls
* Guard read-aloud stop when deleting a non-speaking message
aui.message().stopSpeaking() throws unless this message is the one being
read aloud, so calling it unconditionally rejected the delete handler before
the message was removed. Only stop speech when this message is speaking.
* Cap recent dictation transcript length before persisting
Recent dictations only limited entry count, so a long transcript stored the
full text in the persisted voice settings and a few could exceed the
localStorage quota, throwing synchronously from the uncaught dictation cleanup
path. Truncate each entry on save and on hydration, matching the dictionary cap.
* Studio: keep dictation mic clickable and guide to local model
Register the dictation adapter unconditionally so the mic stays enabled
for any engine and starts working right after switching to the local
model on an already-open thread.
When the browser engine cannot run (Firefox, Brave, non-secure origins),
clicking the mic shows a toast that points to the local speech-to-text
model instead of leaving a disabled button. The toast stacks its action
below the text with a fully rounded button.
* Studio: add bottom padding below the dictation guidance toast button
* Studio: increase bottom padding under the dictation toast button
* Studio: add bottom padding inside the dictation toast button
* Studio: add five Whisper defaults and custom model search
Add private UnslothAI Tiny and Base mirrors to the curated local STT choices while keeping Small as the default. Let users search or paste a Transformers-compatible Whisper repository and validate it end to end.
Keep short dictations in one clip to avoid repeated padded encoder work, then split longer recordings near Whisper's 30-second boundary.
Update hidden model filters and tests, including the CPU-only CI runtime stub for PyAV.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: use public Unsloth Whisper repositories
Point the Tiny and Base dictation defaults to the public unsloth repositories and remove the private mirror references from model filtering and tests.
* Studio: update Whisper download sizes
Reflect the cleaned public Tiny and Base repositories in the curated model labels.
* Studio: right-align STT model size, fix dropdown wheel scroll, refresh sizes
- Show the download size on the right of each model row so long names
like Whisper Large v3 Turbo no longer hide it
- Update curated Whisper sizes to the safetensors weights actually
downloaded: Tiny 151 MB, Base 290 MB, Small 967 MB
- Drive the model list scroll from a wheel handler so the mouse wheel
scrolls it inside the Settings dialog, not just the scrollbar
- Add a search icon and shorten the placeholder to Search model
* Studio: do not search when a dictation model is picked, shrink repo label
- Treat the filled-in model text as a selection, not a query, so choosing
a model no longer kicks off a Hugging Face search
- Make the repository line under each model name smaller
* Studio: tighten dictation model and local engine descriptions
* Studio: keep model display on pick instead of the query, shrink row text
- Guard the combobox input so selecting a model shows its name and does
not echo the typed query back or start a search
- Map the item label to the friendly display so picks fill the field
- Reduce the model name and size text in each row
* Studio: show only the model name in the dictation field, shrink size label
- Drop the download size from the search field; the name alone is shown
once a model is selected, with sizes kept in the dropdown list
- Reduce the size label text in each row
* Studio: clarify the dictation model description
* Studio: drop Hugging Face from the dictation model description
* Studio: move the dictation dictionary to its own Manage subpage
- Replace the inline entry list with a Manage row, matching Dictation
history, so a long dictionary no longer crowds Voice settings
- Add a DictationDictionaryView subpage that holds the entry editor
* Studio: match STT field font, use best voice for System default
- Bump the dictation model field text to text-sm so it matches the
engine dropdown next to it
- Resolve the System default read-aloud voice to the top curated voice
instead of the browser default, which is a robotic legacy voice on macOS
* Studio: rerank read-aloud voices and drop duplicate voice entries
- Rank by vendor quality, then the user's locale, then a preferred list of
natural voices, so the best voice leads instead of the first alphabetically
- Collapse voices that macOS reports twice under one name and language
* Studio: fold dictionary and recents into the dictation section
- Drop the separate Dictation dictionary and Recent dictations headings;
their Manage rows now sit under Dictation, split by the row divider
- Shorten the custom spellings description
* Studio: add search and sort to dictation history
- Filter saved dictations by text with a search field
- Sort by newest, oldest, or A to Z; show a no-matches message
- Keep Clear all available regardless of the current filter
* Studio: settle cancelled STT loads before training and fix dictation review items
Wait for a cancelled STT load to exit and release its memory before
reporting it freed for training, so the loader cannot still be inside
from_pretrained()/.to(device) holding VRAM when the training subprocess
starts. A load that finishes before observing the cancel now gets
unloaded so the memory is actually reclaimed.
Clear the accelerator cache before the CPU fallback in load() so a failed
CUDA/MPS load does not strand reserved VRAM once the sidecar is marked
CPU-resident.
Send the saved Hugging Face token when polling STT download progress so a
gated or private repo resolves and shows the correct Load/Downloaded
state instead of reporting missing.
Mark the composer Dictate button as type="button" so clicking it does not
also submit the draft when the composer already has text or attachments.
* Studio: pin dictation settings per session and close STT startup races
Capture the STT model and language when a dictation session starts and
pass them to every queued segment and the warm-up load, so changing the
model or language mid-recording no longer transcribes the same clip with
the wrong model or a model that is not downloaded.
Check the local runtime at the top of transcribe(), before the model
cache lookup and the bounded audio decode, so a server missing PyTorch or
Transformers returns 501 up front instead of decoding a long clip first.
Treat the training startup window as active for STT device selection.
start_training frees VRAM in before_spawn but only assigns _proc later, so
a concurrent STT load could take the GPU that was just cleared. A startup
flag now reports training active from the free until the process is live,
forcing those loads to CPU; a finally clears it on every exit.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: stub the STT runtime check in transcribe orchestration tests
transcribe() now verifies the local runtime up front, so the unit tests
that exercise transcription orchestration must treat the runtime as
present to keep passing where PyTorch, Transformers, and PyAV are not
installed. Stub ensure_stt_available in the shared fixture and restore
the real check in the availability and load-rejection tests.
* Harden custom Whisper dictation models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add whisper.cpp dictation engine with per-engine downloads and history rework
Engines
- New GGML STT sidecar that runs a managed whisper-server subprocess with
idle unload, plus a pinned static build script (scripts/build_whisper_cpp.sh)
- Dictation engine picker now offers Browser, Local transcription
(whisper.cpp), and Local transcription (Transformers)
- Both local engines serve the same five curated Whisper models and download
them directly with byte-level progress reported by /audio/stt/status
- Models auto load on selection and when their download finishes
- Unload and training admission account for both engines
Benchmarks (Apple Silicon, greedy, warm, same checkpoints)
- whisper.cpp transcribes 2.4x to 5x faster than Transformers and loads in
about 0.45s vs 0.86s for Whisper Small
- whisper.cpp GGUF path is unchanged by the Transformers addition
(load 0.445s -> 0.444s, short clip 0.391s -> 0.347s, long 1.197s -> 1.129s)
Voice settings UI
- Plain curated model select replaces the searchable combobox
- Single download progress bar with transfer rate for both engines
- Dictation history now stores every dictation with Show more pagination,
a top Clear history action, and links back to the chat it was spoken into
- Archived chats dialog gets the same pagination
- Delete dialog offers deleting a dictation together with its chat
Tests: 88 backend STT tests pass, including new snapshot download coverage.
Frontend typecheck, lint, i18n parity, and production build pass.
* Merge local engines into one option and source GGML models from unslothai
Engine selection
- The dictation engine dropdown is back to two choices: Browser and Local
transcription. The selected model decides the backend: curated ids run
GGML checkpoints through whisper.cpp, searched Hugging Face repositories
run safetensors through Transformers
- Model picker lists the curated models and searches Hugging Face for other
Whisper repositories, validating them before selection. The trigger is a
plain button so the selection never renders inside a text input
- /audio/stt/status accepts a model query param so downloaded state works
for custom repositories; the engine param on load, transcribe, and
download routes is derived from the model everywhere
Model source
- Curated GGML checkpoints now download from the Unsloth-hosted
unslothai/whisper-*-GGUF repositories (one repo per model) instead of
ggerganov/whisper.cpp; cache lookups, progress totals, and in-flight blob
tracking are per-model
Fixes
- Voice settings and dictation history were not persisting: the quota-safe
localStorage wrapper was declared after the store that uses it, so the
persist storage factory failed silently. Every settings write also threw
mid-click, which kept the model picker popover from closing on selection
- is_model_downloaded now verifies config, preprocessor config, and real
weight files instead of trusting an offline snapshot lookup, so a partial
download left by an aborted fetch shows the Download button instead of
failing to load
- Removed whisper.cpp mentions from user-facing text: the ready status
shows Loaded instead of the runtime name, picker rows show the source
repository, and runtime error messages say local transcription runtime
Verified with automated browser sessions and live API checks: selection
closes the picker with no page errors, persisted settings hydrate on
reload, a stale partial snapshot triggers download then loads on MPS and
transcribes, and curated models download from the unslothai repos. 88
backend STT tests, typecheck, lint, i18n parity, and build pass.
* Skip the duplicate source line for custom models in the STT picker
A custom repository's display name is its id, so search results and the
appended current selection rendered the same string twice. The source
line now only renders when it differs from the name; curated rows keep
their name, unslothai source repository, and download size.
* Verify every shard of a sharded checkpoint in the downloaded check
A snapshot holding one of N shards (or a corrupt shard index) passed the
downloaded check and then failed at load. When model.safetensors.index.json
exists, every shard in its weight map must now be present. Found by
simulation; covered by a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Rename stale _starting references in the pump resilience tests
The startup flag on TrainingBackend was renamed to _spawn_in_progress but
two tests added alongside it still asserted on the old name, failing the
Python 3.11 to 3.13 CI jobs.
* Make the selected model row clearly highlighted in the STT picker
The current selection was a faint background tint. It now uses the accent
background with a medium weight name. Two line rows use a small corner
radius; single line custom repo rows keep the pill shape.
* Address review feedback on STT snapshot checks, VRAM release, and dictation UX
Verify snapshot completeness in the load preflight so a partial download
fails before the audio is decoded, for curated and custom repos alike.
Drop the failed accelerator traceback before the CPU retry so the cache
clear can actually release that memory. Keep unloading the GGUF sidecar
after cancelling an in-flight Transformers load; both engines can hold
memory at once. Allow Auto language with English-only .en checkpoints,
matching the backend which sends no forced language. Keep the discard
button usable while a transcription is pending so a slow or hung request
cannot trap the composer in dictation mode. Stop linking Compare and
settings test dictations to the unrelated active single chat thread.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move the CPU retry out of the exception handler
On Python 3.10 the interpreter exception state keeps its own reference
to the traceback, so dropping it from the caught exception was not
enough to release the failed accelerator load during the retry. Leaving
the handler before clearing the cache works on every supported version.
* Address review feedback on session handoff, chat pinning, and server lifetime
Starting a dictation from a second entry point now cancels the session
it replaces, so the old recording cannot keep the microphone open or
save a transcript with no discard button pointing at it. The linked
chat is pinned when recording starts, so switching threads while a
transcription finalizes cannot relink the transcript to the newly
opened chat. whisper-server is now bound to Studio's lifetime like the
other long-lived children: PDEATHSIG on Linux, the parent job object on
Windows, and pid adoption so the shutdown sweep reaps it; before this
it survived a Ctrl+C exit as an orphan still holding the model.
* Remove the dictation mic test from Voice settings
The composer dictate button covers the same check, so the test row, its
transcript panel, the unsupported fallback row, and their strings and
search entry are gone.
* Studio STT: gate GGUF whisper-server on training and fix dictation retry and dictionary edits
GGUF (whisper.cpp) sidecar:
- Launch whisper-server with --no-gpu while training is active, mirroring the Transformers sidecar's CPU device choice, so a mid-training dictation cannot reclaim the VRAM training just freed.
- Report is_loading() during whisper-server startup so training VRAM admission accounts for the accelerator memory it is about to bind.
- Require PyAV in is_available() so /audio/stt/status reports the engine unavailable when uploads cannot be decoded, instead of loading fine and then 501ing at transcription.
- Reject a missing model before decoding audio, matching the Transformers download preflight.
Voice settings:
- The download Retry button now restarts the download; the sidecar error is sticky until a new start(), so re-polling alone never cleared it.
Dictation dictionary:
- Tabbing from an emptied entry to its remove button no longer commit-splices the row first, which shifted indices and deleted the wrong entry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio STT: fix curated GGUF whisper filenames to match hosted repos
The unslothai/whisper-*-GGUF repos host the checkpoint as whisper-<id>.bin,
not ggml-<id>.bin, so every curated dictation download and cached-path
lookup 404'd and the whisper.cpp engine could never load a model. Point
GGML_STT_MODELS at the real filenames and guard the naming with a test.
* Studio STT: validate a custom dictation repo before downloading it
The Transformers STT engine accepts an arbitrary owner/model repo, but the
download route handed it straight to snapshot_download, pulling a possibly large
non-Whisper repository into the shared HF cache. Confirm the repo is a Whisper
checkpoint first with the existing metadata-only validate_remote_model (no
weights); curated ids short-circuit and the GGUF engine (curated-only) is
unaffected. A non-Whisper repo now 422s before any download.
* Studio STT: preempt a still-loading GGUF server for training admission
A whisper-server still in its startup window binds accelerator memory but has no
loaded_model yet, so training admission could miss it and launch into an OOM.
Make the GGUF startup cancellable (cancel_pending_load signals an abort event and
terminates the starting process without the load lock; _wait_for_server observes
it and raises SttLoadCancelledError; wait_for_load_to_settle blocks on the lock
until the killed server is reaped), and always fold the GGUF sidecar into the
resident-STT summary so a resident Transformers model cannot mask a loading GGUF
server. free_stt_model_for_training now cancels an in-flight load and waits for it
to settle before training claims the memory.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio STT: fall back to Transformers when whisper-server is absent
A curated dictation model (including the default small) hard-pinned the GGUF
engine, but standard installs do not ship whisper-server, so every recording
501'd instead of using the Transformers engine that serves the same checkpoint
-- the GGUF sidecar's own documented contract. Add _resolve_serving_stt_engine:
a GGUF request for a curated id (the only ids GGUF accepts, all Transformers-
servable) downgrades to Transformers when whisper-server is unavailable, applied
consistently to download, load and transcribe (not unload, which targets a
specific engine). The Voice tab likewise falls back to the Transformers status so
the model is not shown unavailable and download is not blocked.
* Studio STT: hide custom Whisper caches from the legacy model pickers
The legacy /cached-models (and /cached-gguf) routes called is_hidden_model with
only the owner/model id, which cannot reach the config-based Whisper check, so a
downloaded custom (non-curated) Whisper checkpoint was still offered as a chat
model. Pass the cached snapshot path so _path_is_whisper_model inspects the repo
config and hides it, matching the discovery route.
* Studio STT: hide GGUF dictation repos, lock-free status, unload fallback, split training eviction
- Hide the curated GGUF dictation repos (unslothai/whisper-*-GGUF) from the chat
model inventory and pickers, backend and frontend. Only their Transformers
safetensors companions were hidden; the GGUF repos use a different org and a
-GGUF suffix and carry a raw .bin with no whisper config.json, so they leaked
into chat pickers.
- Make the GGUF sidecar loaded_model/device accessors lock-free, mirroring the
Transformers sidecar. transcribe() holds self._lock across the whole inference
call, so /audio/stt status polls and training admission previously blocked
behind an in-flight transcription.
- stt_unload resolves through the serving resolver: a "gguf" pick on a host
without whisper-server is served by the Transformers fallback, so unload must
target that engine or the resident model is never freed. Unload also attempts
every engine even if one raises, so a failure freeing one backend no longer
skips the other.
- free_stt_model_for_training frees the Transformers and GGUF sidecars under
independent exception boundaries so a failure unloading one no longer skips
the other before training claims the memory.
Adds tests/test_stt_review_fixes.py covering all four.
* Studio STT: resolve Auto dictation language for the model engine + snapshot process liveness
- The model dictation adapter sent the raw setting (the literal "auto") to the
backend, while the browser engine resolves Auto via resolveDictationLanguage.
A batch of non-English voice notes came back mostly English on Auto. Add
resolveModelDictationLanguage: only the literal "auto" is resolved to a
concrete locale, gated so it becomes a language the model AND Whisper can
honor (mirroring the backend's known-whisper-languages set); an explicit
language, or a locale Whisper cannot honor, stays unchanged/auto-detect. Wire
it into both adapter call sites.
- GgmlSttSidecar._process_alive() read self._process twice; a concurrent
unload() nulls it under the lock while loaded_model/device read lock-free, so
a null between the two reads called None.poll(). Snapshot once. Adds a
deterministic regression test.
* studio: tighten comments and docstrings in the dictation modules
* studio: harden dictation model downloads, GGML readiness, and recording paths
Address review findings on the STT dictation feature:
- build_whisper_cpp.sh refuses to delete a whisper.cpp tree under a custom
Studio home unless it carries the Studio ownership marker, matching the
setup.sh policy, and marks trees it creates
- _snapshot_is_complete validates every shard of a sharded PyTorch
(pytorch_model.bin.index.json) checkpoint like the safetensors path, and
requires tokenizer assets (tokenizer.json or vocab.json + merges.txt)
- custom-repo downloads pin the revision resolved at validation time and
restrict snapshot_download to the model/tokenizer/config/preprocessor file
classes Studio loads
- the GGML sidecar holds its port reservation until just before spawning
whisper-server and only accepts readiness from a responder that both looks
like whisper.cpp's server and belongs to the still-running managed child,
probing twice, so mic audio cannot be posted to a foreign local process
- the recording adapter transcribes every non-empty segment; the RMS meter
only shapes segment boundaries and can no longer discard quiet speech
- Compare-pane dictation can cancel a pending transcription on second click,
with the button relabeled while finalizing
- localStorage quota recovery halves the dictation history until the save
fits, so small histories shrink too
- the System default TTS voice resolves to the platform default voice
- new dictation UI imports go through the chat and hub feature barrels
Regression tests cover the build-script gate, sharded PyTorch and tokenizer
completeness, revision pinning and allow patterns, and the whisper-server
readiness probe.
* Fix STT download and voice picker follow-ups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add dictation button regression coverage
* Studio: prebuilt whisper.cpp via the shared llama.cpp install core, slim bundles paired to the llama prebuilt (#7294)
* Studio STT: add prebuilt whisper.cpp (whisper-server) installer
New install_whisper_prebuilt.py downloads a per-platform whisper-server
bundle published by the unslothai/whisper.cpp prebuilt CI into the managed
whisper.cpp dir (build/bin/whisper-server) so local dictation needs no
compiler. Mirrors install_node_prebuilt.py / install_llama_prebuilt.py:
host + backend detection, sha256 pins (whisper_prebuilt_pins.json) as the
trust anchor, staging + install lock + atomic swap, traversal-safe extract,
co-located shared libs (RUNPATH=$ORIGIN), an UNSLOTH_WHISPER_PREBUILT_INFO.json
marker with idempotent "already matches", and exit codes 0/1/2/3. Not wired
into setup yet; the pins ship empty so every asset fails closed until the
first fork release is published and its digests are reviewed in.
* Studio STT: install prebuilt whisper.cpp during setup and update
Add a fail-open whisper.cpp block to setup.sh after the llama.cpp section so
`unsloth studio update` (and a fresh install) fetch the prebuilt whisper-server
into the managed whisper.cpp dir the sidecar discovers. It skips a user-set
WHISPER_SERVER_PATH/UNSLOTH_WHISPER_CPP_PATH, honors UNSLOTH_SKIP_WHISPER_INSTALL,
forwards the resolved ROCm gfx, and never aborts setup: a busy install keeps the
existing runtime, and an unavailable prebuilt stays quiet (source build is opt-in
via UNSLOTH_WHISPER_FORCE_COMPILE) since Transformers STT and browser dictation
remain. Register UNSLOTH_WHISPER_PREBUILT_INFO.json as Studio-owned evidence.
* Studio STT: harden whisper-server child env + WSL ROCm detection
- Sidecar spawns whisper-server with a scrubbed child env that prepends the
binary dir (co-located GPU libs) to the loader path, and on WSL2 ROCm loads
the system HIP first (HSA_ENABLE_DXG_DETECTION=1) so a bundle's bare-metal HIP
does not segfault on /dev/dxg. Secret-bearing vars are dropped from the child.
- find_whisper_server_binary now requires an executable, not just a file.
- Installer rocm probe passes HSA_ENABLE_DXG_DETECTION and falls back to
/opt/rocm/bin/rocminfo so a WSL ROCm host is not misdetected as CPU-only;
gfx parsing skips the gfx000 CPU agent and generic ISA lines.
- Tests for the child env (secret scrub, lib dir, WSL HIP precedence), the
executable check, and the WSL rocm detection.
* Studio STT: in-app whisper.cpp prebuilt update stack + ship pins in the wheel
Mirror the llama.cpp update stack for the whisper.cpp prebuilt so Studio can
detect and install a newer whisper-server release from inside the app:
- backend/utils/whisper_cpp_freshness.py: read UNSLOTH_WHISPER_PREBUILT_INFO.json
and compare the installed release against the newest unslothai/whisper.cpp
release. Whisper tags are v<upstream>-unsloth.<N>, so is_behind compares a
(major, minor, patch, serial) key with a strict downgrade guard; 24h cache;
fail-open.
- backend/utils/whisper_cpp_update.py: run install_whisper_prebuilt.py to fetch
and atomically swap the newest bundle, unloading the warm GGUF sidecar first.
- backend/routes/whisper.py mounted at /api/whisper (update-status + update).
- pyproject: add whisper_prebuilt_pins.json to studio package-data so the
installer's trust anchor ships in the wheel (it is a data file, not a .py
module, so package discovery alone does not include it; node_prebuilt_pins.json
is listed for the same reason). Without this a pip-installed wheel had no pins
and the prebuilt install aborted to Transformers STT.
Adds test_whisper_cpp_freshness.py (version parser, is_behind matrix + downgrade
guard, marker layouts, stale decision, fail-open).
* Studio STT: verify whisper prebuilts via the release checksum index, like llama.cpp
Re-align the whisper.cpp prebuilt installer to install_llama_prebuilt.py's trust
model: instead of a committed whisper_prebuilt_pins.json, verify every download
against the release's own whisper-prebuilt-sha256.json checksum index, fetched
from the same GitHub release.
- parse_release_checksums / fetch_release_checksums / expected_sha256_for replace
the pins layer. The index is validated for schema/component and that its
release_tag matches the resolved release; an asset absent from it, a release
that does not publish it, or a manifest sha256 that disagrees with it all fail
closed to a source build.
- resolve_release_tag now resolves the newest published release at runtime (or an
explicit --published-release-tag), matching llama and the freshness check;
removed the pinned-default and the UNSLOTH_WHISPER_ALLOW_UNVERIFIED opt-in.
- Delete studio/whisper_prebuilt_pins.json and drop its pyproject package-data
entry (nothing to ship now, same as llama which has no committed pins).
- Adds test_install_whisper_prebuilt_checksums.py (index parser, fail-closed on
uncovered asset, tampered-manifest guard, newest-release resolution).
This is a same-origin checksum (integrity, not authenticity), identical to the
llama.cpp installer; pair releases with GitHub artifact attestations for provenance.
* Resolve whisper prebuilt release via the download host (no GitHub API)
Mirror install_llama_prebuilt.py's fast path: resolve the release tag from
the releases/latest redirect and fetch the manifest + checksum index from
constructed releases/download URLs, so the common install path makes zero
api.github.com calls (unauthenticated api.github.com is capped at 60 req/hour
per IP; the download host is not). Fall back to the GitHub API only on a 404,
malformed asset, or tag mismatch.
* Studio STT: coverage-aware whisper prebuilt selection via a shared core
whisper's select_artifact returned the first os/arch/backend manifest match and
ignored the SM-coverage fields the release manifest already carries, so a
Blackwell B200 (sm_100) was served cuda12-legacy (sms 50-61) -- runnable only via
forward PTX JIT. install_llama_prebuilt.py on the same host correctly picks
cuda13-newer.
Extract the coverage-aware selection into a shared, component-agnostic core under
studio/backend/utils/prebuilt/ (selection + GPU host-capability detection), lifted
from llama's linux_cuda_choice_from_release / _artifact_covers_sms / _sm_range and
generalised over a normalised artifact. whisper's HostInfo now records the GPU
compute caps + driver CUDA version (honoring CUDA_VISIBLE_DEVICES), and
select_artifact routes CUDA/ROCm through the shared selector: every visible SM
must be covered, the tightest-covering profile wins (Blackwell-aware runtime-line
ordering), ROCm matches the gfx target exactly, and an uncovered GPU falls back to
the CPU bundle. CPU/Metal/Vulkan keep first-match. The resolver JSON, exit codes,
and "already matches" contract are unchanged.
On the B200 the installer now resolves cuda13-newer, matching llama.
* Studio STT: gate whisper CUDA selection on the on-disk runtime, like llama
The prebuilt CUDA bundles are dynamically linked and intentionally do NOT ship
libcudart/libcublas -- they load the same runtime the host already has. So the
driver's advertised CUDA version is only an upper bound: a cuda13 bundle still
needs cuda13 runtime libraries present on disk. Port llama's on-disk runtime
scan (detected_linux_runtime_lines / detected_windows_runtime_lines) into the
shared core and intersect it with the driver-compatible lines in
select_cuda_attempts. A host with a cuda13 driver but only cuda12 runtime (e.g.
torch-cuda12) now correctly gets a cuda12 bundle instead of an unloadable cuda13
one; a host with no CUDA runtime at all falls back to CPU.
Fixes a glob bug in the port (any(Path(d).glob(p) for d in dirs) tests generator
truthiness, not a match) that made every major report present; add a real
filesystem test that exercises the scan.
* studio: harden shared prebuilt core to full llama parity
Apply the review findings on the shared coverage-aware prebuilt-consumer
core so whisper.cpp selection is exactly equivalent to the llama.cpp path.
hosts.py: port llama's CUDA_VISIBLE_DEVICES handling. A GPU hidden by an
index/UUID selector now reports has_usable_nvidia False instead of staying
usable, via supports_explicit_visible_device_matching plus the physical /
explicit-match branches, and _select_visible_rows now matches rows the way
llama does (index or UUID, gpu- prefix optional) and skips unmatched tokens
rather than keeping all rows. Adds the Linux /proc/driver/nvidia/gpus
fallback and has_physical_nvidia. Adds parse_macos_version.
runtime_libs.py: the Linux on-disk scan now requires the exact libcudart /
libcublas SONAME (libcudart.so.13), not a libcudart.so.13* glob, so a bare
versioned file without the SONAME symlink no longer counts as loadable.
Hardens the ldconfig parse against an empty left-hand side.
selection.py: fix the Blackwell/torch reordering so it keys on the covering
runtime lines (falls through to the torch preference when the covering lines
were filtered out), matching linux_cuda_choice_from_release. Corrects the
compatible_runtime_lines_for_driver docstring: the bundles do not ship the
CUDA runtime, so the driver version is only an upper bound and the caller
must intersect with the on-disk scan.
install_whisper_prebuilt.py: enforce a macOS artifact's min_os (new
HostInfo.macos_version) so a bundle that cannot load on the host OS version
is dropped. Keep resolver stdout to only the JSON line by leaving logs on
stderr in --resolve-prebuilt mode, and map an unexpected probe failure to
prebuilt_available False instead of a traceback.
Tests: new host-probe suite for the visible-device logic, exact-SONAME
runtime-scan cases, macOS min_os filtering, resolver stdout-only-JSON,
exit-code mapping, and the repo key.
* studio: fix whisper prebuilt selection + launch parity gaps from review
A parallel review surfaced integration defects where the whisper path could
select or launch a bundle that cannot run on a concrete host. Each is fixed to
match install_llama_prebuilt.py.
macOS min_os: the manifest labels macOS requirements as macos-<version>
(e.g. macos-14.0), which the version parser could not read, so the guard was a
no-op and a macOS-13 host would install the macos-14 Metal bundle. Strip the
platform prefix before parsing.
ROCm gfx detection: _detect_rocm_gfx returned the first gfx token and ignored
HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES. Since exact
ROCm matching treats that token as the active GPU, a mixed APU + dGPU host
(gfx1151 + gfx1100) with HIP_VISIBLE_DEVICES=1 installed the wrong archive. Route
through a shared pick_rocm_gfx_target (lifted from llama) that parses per-GPU
sections and honors the visibility vars (empty / -1 -> no AMD GPU).
--rocm-gfx override: recording the arch without setting has_rocm left the host on
its CUDA/CPU path so the ROCm bundle was never picked. --rocm-gfx now implies
has_rocm and clears NVIDIA state, like llama's _apply_host_overrides.
CUDA launch env: a CUDA bundle ships the ggml CUDA backend but not
libcudart/libcublas, and the sidecar launch env exposed only the bundle dir, so
on a host whose CUDA runtime lives only in the PyTorch wheels the selection would
gate cuda usable but the server could not load it. Add the CUDA-from-PyTorch
runtime dirs to the child loader path for CUDA bundles (bundle dir still first),
mirroring binary_env.
Also normalize a manifest artifact's supported_sms defensively (parity with
llama's parser) and document that blackwell_min_toolkit_for_caps is retained for
the Phase B llama Windows path.
Not changed (verified parity, not defects): Linux/Windows min_os is enforced
nowhere in llama (macOS only); the resolver is optimistic about the checksum
index and the install path verifies.
* studio: tighten prebuilt-core code comments
* studio: lift shared prebuilt installer core out of the whisper installer
* studio: reuse the llama.cpp prebuilt installer machinery for whisper
* studio: unify llama and whisper prebuilt installers on a shared descriptor core
* studio: consolidate prebuilt installer tests into the shared core suite
Grow tests/studio/install/test_prebuilt_core.py from 62 to 164 tests so every
component-agnostic behavior runs against both descriptors: the full seven
profile CUDA release matrix (multi-GPU, on-disk runtime gating, shuffle
stability, missing SM metadata, dotted SM normalization, no-driver fallback
policy), the ROCm gfx family matrix, macOS min_os gating and its helper,
backend resolution incl. cpu-fallback precedence and Intel-mac auto detect,
checksum-index non-object and plain-lookup cases, the tar symlink/hardlink
extraction guards moved from the llama suite, and the compute-cap, visible
device, runtime-line and Blackwell helper value tables moved verbatim from
the llama characterization suites.
Delete only tests whose exact behavior the master now asserts for the same
component: 40 pure-alias helper cases in test_selection_logic.py (replaced by
value-identical master tables plus an alias-identity pin), 6 extraction moves
and the master-absorbed zip-symlink case in the llama logic suite, 3 routing
twins in test_rocm_support.py already pinned byte-for-byte in
test_selection_logic.py, the 2 Blackwell helper tables in the backend resolve
suite, 28 whisper logic tests and 10 whisper checksum tests re-asserted by
the master whisper parameterization. Wrapper wiring pins, the llama release
plan dialect, fingerprints and every llama-only behavior stay untouched.
* studio: dedupe sidecar and update helpers into the backend prebuilt package
* studio: chain whisper.cpp prebuilt updates onto the llama.cpp update flow
* studio: consume paired slim whisper prebuilts via the llama ggml runtime
* studio: serve every whisper backend from slim prebuilts
* studio: drop the whisper fat per-accelerator selection chain
unslothai/whisper.cpp releases are slim-only from v1.9.1-unsloth.2: one
ggml-less bundle per os/arch, paired to the llama.cpp prebuilt that provides
every ggml backend. Delete the whisper-side fat CUDA/ROCm/metal/vulkan
selection glue; keep slim selection + pairing, link_ggml_runtime, and one
legacy shape, the published fat CPU bundle of an explicitly pinned pre-slim
release. Exit 2 now reads as prebuilt unavailable (whisper never source
builds); setup already treats it that way.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Wire libomp runtime DLL alongside ggml in slim whisper installs
llama's clang-built windows-arm64 ggml-base.dll imports
libomp140.aarch64.dll, shipped in the llama bundle but not a system DLL.
Without it next to whisper-server.exe the loader fails with
STATUS_DLL_NOT_FOUND before main. MSVC x64 links vcomp140.dll from
System32 and Linux ggml uses system libgomp.so.1, so only windows-arm64
was affected. The empty-runtime guard still requires a real ggml
library; libomp alone is not a pairing.
* studio: drop whisper-side fat-selection support structure
Slim whisper bundles are selected per os/arch only; all accelerator
capability comes from the installed llama.cpp prebuilt, whose installer
already did the coverage-aware selection. Remove the machinery that only
existed to pick among fat per-accelerator whisper bundles:
- prebuilt_core: delete the generic CUDA/ROCm coverage selection
(select_cuda_artifact, select_rocm_artifact, ArtifactView adapters,
detected_cuda_runtime_lines, the exact-SONAME linux probe) that no
shipped component routes through; llama keeps its own selection chain
and whisper shadows select_artifact with the slim-only version.
select_artifact is now a plain os/arch/backend first-match.
- install_whisper_prebuilt: drop the HostInfo CUDA fields
(compute_caps, driver_cuda_version, torch_runtime_line) and the torch
runtime probe that populated them; nothing reachable reads them, and
the resolver payload sources runtime_line from the artifact.
- whisper_cpp_update: delete the standalone start_update job worker;
whisper applies only run as the chained phase of the combined
llama+whisper update. The status payload keeps its job field (idle).
- routes/whisper: drop the progress logger that could never fire.
- tests: remove tests of the deleted paths and tests duplicating the
descriptor-parameterized core suite or the llama freshness suite.
Contracts unchanged: resolver JSON keys, exit codes, marker fields,
pairing logs, and the pinned pre-slim fat CPU escape hatch.
* Address review feedback on the whisper prebuilt update and install paths
- Pin the chained whisper phase to the release the freshness check
offered, so the download-host latest pointer cannot reinstall an
older build in a loop
- Wire the whisper prebuilt install into setup.ps1 (Windows setup
previously skipped it entirely)
- Treat a non-executable server or missing wired ggml libraries as a
broken install instead of reporting already matches
- Keep whisper sidecar reloads out of the job-level reload flag and
resync chat state after a partial chained update that unloaded llama
- Repoint home and profile vars for the whisper-server subprocess at a
managed scratch dir and drop credential-store pointers
- Clear the prebuilt marker before the opt-in source build overwrite
- Write the prebuilt marker with explicit utf-8 encoding
* Tighten comments in the whisper prebuilt consumer
* Harden the Windows whisper setup phase and the chained update edges
- setup.ps1: honor WHISPER_SERVER_PATH / UNSLOTH_WHISPER_CPP_PATH /
UNSLOTH_SKIP_WHISPER_INSTALL, run the custom-home ownership guard
before the atomic install, and forward the release-tag pin and ROCm
hints like setup.sh
- sidecar: a cpu-selected install launches whisper-server with --no-gpu
(slim wiring links every llama backend, so the flag is what keeps a
deliberate CPU choice off the GPU)
- chained update: leave whisper unpinned on macOS (the llama phase can
walk back there, and a newest-tag pin could be an impossible pairing
on every retry) and treat installer exit 2 as kept-existing-runtime
instead of failing the combined job
- job.to_tag now comes only from the llama phase, so a whisper-only
round cannot report a llama update that never ran
* Fix slim whisper runtime follow-ups
* Address remaining whisper update reviews
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address remaining prebuilt update reviews
* Fix remaining chained update reviews
* Fix remaining whisper runtime review edges
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
---------
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
Adds a configurable Hugging Face model download cache location to Unsloth Studio, selectable from Settings, with per-cache download manifests, scoped deletion, and read-only inventory of previously selected caches.
* Studio: drive UI font size through a typography scale, not the root font size
Follow up to #7355. The preference now writes --ui-font-scale
(selected / 16) and a data-ui-font-size attribute on the root instead
of mutating the root font size, and the applier clears any stale inline
root font-size left by older builds. Because the rem base never moves,
every layout-only rem-to-px conversion from #7355 is reverted to its
original form: the spacing, radius and container tokens, sidebar and
thread widths, grid tracks, calc margins and hub.css dimensions match
pre-#7355 main again, which also restores rem-based accessibility
scaling for users with a larger browser default font size.
Typography scales through tokens in index.css, all exact at 16px:
- The named Tailwind sizes (--text-xs through --text-4xl) multiply
their defaults by the scale, so standard utilities scale
- One token per design px size (--text-ui-8 ... --text-ui-34) replaces
every arbitrary text-[Npx] class; leading-ui-* mirrors the exact
line heights and the numeric --leading-3..10 scale as well
- CSS font-size and line-height declarations multiply by the scale
- Chart labels scale through a .recharts-text rule; streamdown and
react-flow px text is re-based via scaled overrides; KaTeX's 1px
layout trick stays fixed by design
- The logo lockups keep their half-rate behavior via the scale var
- The explicit Code font size remains unmultiplied
Keeps the #7355 behavior fixes: color chip min width, voice select
min/max widths, and the select and dropdown menus scrolling an inner
viewport so their corners stay rounded. The whitespace-password and
IME rename guards that merged alongside are preserved.
* Studio: contract and Playwright coverage for the UI font size scale
test_ui_font_scale_contract.py pins the mechanism (scale var written,
root font size never mutated, tokens scaled, code font size not
multiplied, the Radix select viewport owning scroll state) and guards
against new raw pixel typography, with a documented allowlist for the
recharts fontSize props covered by the stylesheet override and the
offscreen clipboard textarea.
playwright_ui_font_scale.py drives the real appearance controls: root
font size fixed at 12/16/20, text and line height scale by size/16,
sidebar width invariant, explicit code font size stays fixed, an
overflowing dictation select scrolls its Radix viewport by keyboard
and wheel, and the default restores exactly. Wired into the UI smoke
workflow against the second studio boot.
The thinking-compact and descender contracts move back to the rem and
token forms now that layout values no longer need px pinning.
* fix(dataprep): don't emit a degenerate chunk for empty text
smart_chunk_text feeds empty / whitespace-only text (which tokenizes to
zero tokens) into the single-chunk branch, which unconditionally returns
one chunk. That yields a lone-EOS "document" (input_ids=[eos]) or, when
the tokenizer has no eos_token_id, a zero-length input_ids=[] — an
invalid sample that breaks a downstream collator/trainer.
load_from_file already guards against this with a ValueError, but
chunk_text, smart_chunk_text and load_from_files do not, so batch-loading
a directory that contains an empty file silently injects garbage rows.
Return no chunks when the tokenized text is empty, so empty inputs
contribute nothing instead of a degenerate sample. load_from_file keeps
its explicit ValueError (its guard runs first).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard empty/whitespace text before tokenizing in raw_text
Real BPE/SentencePiece tokenizers emit tokens for spaces and newlines, so the len(tokens)==0 check let whitespace-only documents through as a degenerate lone-EOS sample. Guard on text.strip() before tokenizing (mirroring load_from_file), and raise in load_from_files when every file is empty so return_tokenized mode never falls back to a text-column dataset. Test now uses a whitespace-preserving tokenizer and covers both return_tokenized modes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: make UI font size scale all text without moving layout
The UI font size setting changes the root rem base, so only rem sized
text reacted. Hundreds of px text classes, px font sizes in CSS, and
chart labels stayed fixed, while rem based padding, widths and radii
wrongly grew.
Convert all text sizes to rem so every font follows the setting, and
pin spacing, radius, container widths, sidebar and thread widths to px
so layout no longer follows the rem base. Library styles (streamdown,
react-flow) are re-based via overrides. All conversions are exact at
the default 16px root, so the default rendering is unchanged.
* Studio: keep logo at fixed size and fit tight controls at large UI fonts
The logo lockups (sidebar wordmark with beta badge, onboarding wizard)
are branding and now keep px sizes at any UI font size.
Two controls clipped their text at the largest setting: the appearance
color chips (fixed w-24) and the voice tab selects (fixed w-56). Both
use min widths now, so they keep the default look at 16px and only
grow when the text needs the room.
* Studio: keep dropdown corners rounded when the menu scrolls
A scrolling dropdown lost its rounded corners on the scrollbar side:
WebKit paints the surface square when the rounded element itself hosts
the scrollbar, which shows up in the desktop app whenever a menu
overflows, for example at larger UI font sizes.
Dropdown menu and select content now clip with overflow hidden and
scroll an inner viewport instead. The surface padding insets the
scrollbar clear of the curve, so corners stay rounded in every engine.
Submenus are unaffected since sub content is portaled.
* Studio: scale the logo lockups at half the UI font size rate
Rather than pinning the logo, the sidebar lockup (sticker, wordmark,
beta badge) and the onboarding lockup now follow the UI font size at
half the rate of the change: size = base + (root - 16px) / 2, written
as calc((base - 8)px + 0.5rem). A 4px font size change moves the logo
by 2px, and the default 16px root renders the exact base sizes.
* Studio: address review feedback on leading, grid tracks and select scrolling
Numeric leading utilities (leading-3 through leading-10) derive from
--spacing, so pinning spacing to px also froze their line-heights while
the paired text sizes now scale. Define them as rem theme tokens so
line-height follows the UI font size again; values are identical at the
16px default.
Convert the grid tracks the rem-to-px codemod missed (rem followed by
an underscore escaped the word boundary): the response details label
column and the on-device folder rows.
Make the Radix select viewport the bounded scroller instead of a
wrapper div, so Radix's scroll handling and the browser scroll the same
element. Restore the app's thin scrollbar with an inline style, which
beats the scrollbar hiding stylesheet Radix injects at runtime.
* Studio: cap voice select widths and update CI contracts
* installer: fix Linux AMD GPU detection + actionable ROCm-less warning
The rocminfo/amd-smi-less fallback in _has_amd_rocm_gpu keyed on a
/gpu_id/ line inside each KFD node's properties file, but gpu_id is a
separate sibling sysfs file and never appears in properties. The guard
never matched, so the fallback missed every AMD host without ROCm
tooling (e.g. a fresh CachyOS/Arch box) and reported 'no GPU detected'
despite vendor_id 4098 being present in the KFD topology.
Detect via vendor_id == 4098 directly: the KFD CPU node reports
vendor_id 0, so any 4098 node is an AMD GPU, while NVIDIA's KFD nodes
report 4318 and stay excluded.
Also rework the 'ROCm version could not be determined' warning into an
actionable message (install the ROCm/HIP SDK; Arch/CachyOS:
rocm-hip-sdk) so ROCm-less users know the concrete next step instead of
silently landing on CPU-only PyTorch.
* tests: replace the FNR==1 KFD invariant with the per-line vendor_id check
The FNR==1 reset guarded the old paired gpu_id+vendor_id awk against
cross-node state leakage. The new detection is a single atomic
vendor_id==4098 line condition, so there is no per-node state to reset;
assert the new invariant instead (single-line vendor match, and no
/gpu_id/ pattern, which never matched inside properties).
tests/studio/install/test_rocm_support.py: 344 passed, 2 skipped.
* installer: mirror the KFD vendor_id fix in setup.sh + honest CPU-fallback summary
Codex P2 follow-ups:
- studio/setup.sh carried the same dead gpu_id-inside-properties awk, so a
host install.sh now routes to ROCm still failed setup's independent AMD
re-probe and got a CPU llama.cpp. Use the same per-line vendor_id 4098
check.
- When the AMD GPU is detected but the torch index stays CPU, the summary
printed the old false diagnosis (gpu none / "No GPU detected"). Gate both
on _has_amd_rocm_gpu and say what actually happened: AMD GPU present, no
usable ROCm, CPU fallback.
- Structure test asserting setup.sh's KFD awk stays in sync with install.sh.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep KFD-only AMD hosts on the CPU fallback (Codex P2s)
The KFD-topology fix makes _has_amd_rocm_gpu / _setup_amd_detected true on hosts that expose an AMD GPU to the kernel but ship no rocminfo/amd-smi. Detection alone does not mean ROCm is usable or that the gfx arch is known, and two downstream paths wrongly assumed it did:
- studio/setup.sh forwarded --has-rocm with no gfx, so install_llama_prebuilt found no per-gfx bundle and dropped to a HIP source build (slow, or a hard failure without build deps) instead of the CPU prebuilt these hosts used to get. Now --has-rocm is forwarded for a gfx-unknown host only when hipcc is present; otherwise it keeps the CPU prebuilt.
- install.sh get_torch_index_url selected a generic rocmX.Y index whenever the ROCm version was readable, but the Strix reroute only learns gfx from rocminfo/amd-smi, so a Strix KFD-only host landed on the broken _grouped_mm wheels. Now, when neither rocminfo nor amd-smi is present (gfx unknowable), it stays on CPU with a hint to install them.
Detection and the improved diagnostics are unchanged; only the routing for gfx-unknown KFD-only hosts is made safe. Adds tests for both gates.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden KFD-only fallback: probe gfx, accept versioned hipcc (Codex P2s)
Follow-up to the previous commit's two guards:
- install.sh: the KFD-only torch guard tested only 'command -v rocminfo/amd-smi', so a host where those binaries exist but do not enumerate the GPU (gfx unreadable) slipped through and, with hipconfig/rocm-core present, still got a generic rocm index -- breaking Strix. Now it actually reads the gfx (rocminfo, then amd-smi list / static --asic, the same probe the reroute uses) and falls back to CPU whenever the arch is unreadable, not just when the binaries are absent.
- studio/setup.sh: the hipcc gate missed a HIP toolchain installed only under a versioned prefix (/opt/rocm-*/bin/hipcc), which the source build at setup.sh:1663 does support, so such hosts were dropped to the CPU prebuilt unnecessarily. The gate now also accepts /opt/rocm-*/bin/hipcc.
Tests updated to assert the gfx-read (not binary-presence) gate and the versioned hipcc path; full test_rocm_support.py green (347 passed). Verified the gfx probe by execution: rocminfo-with-no-gfx now routes to CPU, amd-smi fallback still resolves gfx.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor UNSLOTH_ROCM_GFX_ARCH before the CPU fallback for PR #7314
Seed both the gfx-unknown guard in get_torch_index_url and the Strix reroute
from UNSLOTH_ROCM_GFX_ARCH before probing rocminfo/amd-smi, so a host that
names its arch reaches the correct rocm index instead of being forced to CPU
(or to the generic wheels) when the runtime probes can't enumerate the GPU.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Probe gfx with visibility masks cleared for PR #7314 (Codex P2)
rocminfo/amd-smi honor ROCR/HIP_VISIBLE_DEVICES, so a container that masks the
GPU (e.g. ROCR_VISIBLE_DEVICES=-1) would make the gfx probe read nothing and
force CPU torch, even though the KFD-based AMD detection is env-independent and
hipconfig can still supply the ROCm version. Clear the visibility masks for the
rocminfo/amd-smi arch probe only (the Strix reroute keeps them for per-GPU index
selection), so a masked/container host keeps its ROCm route.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Re-probe gfx unmasked in the Strix reroute when a mask hides all agents for PR #7314 (Codex P2)
* Remove leftover conflict marker from the test merge
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Report an explicit CPU pin instead of a ROCm misdiagnosis for PR #7314 (Codex P3)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trigger the reroute re-probe on a set-but-empty visibility mask for PR #7314 (subagent review)
* Guard the ROCm version chain against set -e when no source exists for PR #7314 (simulation find)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Preserve the inferred-gfx reroute for KFD-only hosts (Codex P2)
The gfx-unknown CPU guard in get_torch_index_url fired before the
runtime-less reroute could run: with the KFD topology fix,
_has_amd_rocm_gpu is true on KFD-only hosts, so the reroute's
'! _has_amd_rocm_gpu' gate never let _infer_linux_amd_gfx_arch route
them to AMD per-arch wheels, regressing inferable boxes (PCI/cpuinfo/
lspci) from arch-specific PyTorch to CPU-only.
- Factor the override->rocminfo->amd-smi gfx probe (masks cleared)
into _probe_amd_gfx_arch, shared by the guard and the reroute gate
so the two can't disagree on what 'readable' means.
- Reroute gate now also fires when the GPU is detected but the probe
is empty (KFD-only). Deliberate CPU fallbacks (old/unreadable ROCm
version) all had a readable gfx and stay excluded.
- The guard defers to the reroute (no false 'installing CPU-only
PyTorch' promise) only when inference yields a supported family;
otherwise the actionable CPU warning is unchanged.
Executed tests: KFD-only host reroutes to repo.amd.com per-arch wheels
and exports UNSLOTH_ROCM_GFX_ARCH for setup.sh; readable-gfx CPU
fallback stays un-rerouted; undetected-GPU reroute unchanged; the
guard's three inference outcomes covered. Suite: 375 passed, bash -n
clean on both scripts.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix two false diagnostics on the KFD-only paths (Codex P3s)
1. get_torch_index_url: with UNSLOTH_ROCM_GFX_ARCH set on a KFD-only
host that has no ROCm version sources, the no-version endpoint
printed 'falling back to CPU-only PyTorch' even though the reroute
(gated on the override) then installs the per-arch wheels. When the
override maps to a wheel family, defer with an accurate message;
an unmappable override keeps the CPU warning since the reroute
can't route it either.
2. Runtime-less reroute: the KFD-only branch reached the warning
'ROCm runtime not visible (/dev/kfd, rocminfo, amd-smi)' although
/dev/kfd is exactly what detected the GPU. The diagnostic now
distinguishes KFD-visible/tooling-blind hosts from truly
runtime-invisible ones.
Executed tests: supported override defers without the false CPU
warning, unsupported override and readable-gfx no-version hosts keep
it; KFD-only reroute emits the KFD wording, undetected-GPU reroute
keeps the original. Version sources are shimmed so the tests hold on
dev boxes with a real hipconfig. Suite: 376 passed, bash -n clean.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(install): infer Strix gfx when ROCm runtime is absent
When /dev/kfd and rocminfo are missing on Linux (e.g. Arch/CachyOS Strix
Halo), route to AMD per-arch wheels via cpuinfo/lspci inference instead
of CPU-only PyTorch. Mirrors install.ps1 Windows behavior and fixes
studio update via install_python_stack.py (unslothai#7301).
* Map Radeon 8065S to gfx1151 in the Linux gfx inference (Codex P2)
install.sh _infer_amd_gfx_arch_from_gpu_name missed 8065S, so a Strix Halo host that only exposes 'AMD Radeon 8065S' via lspci (no Ryzen AI Max branding in /proc/cpuinfo) was left on CPU torch. setup.sh and setup.ps1 already list 8065S -> gfx1151. Added it, and widened the cpuinfo regexes (install.sh and install_python_stack.py) from Radeon 80[0-9]0S to 80[0-9][05]S to match the 80X5S naming, consistent with the display-side check already in install.sh. Tests cover the 8065S name and the cpuinfo-only case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate the Linux gfx inference out of WSL without the ROCDXG runtime for PR #7305
On WSL /proc/cpuinfo and lspci still see the host APU, so a standalone
'unsloth studio update' could infer gfx1151 and install per-arch ROCm wheels
into a WSL env whose ROCDXG bridge (librocdxg) was never bootstrapped, i.e. one
that cannot expose the GPU. Skip the cpuinfo/lspci inference on WSL unless
librocdxg is present; an explicit UNSLOTH_ROCM_GFX_ARCH override still wins.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address Codex review on PR #7305 (WSL runtime gate, Linux mirror, arch guard)
- install.sh _infer_linux_amd_gfx_arch: skip the cpuinfo/lspci inference on WSL
unless librocdxg is present (the ROCDXG bridge), mirroring the Python fix, so a
WSL box whose ROCm bootstrap was skipped keeps the CPU fallback instead of
installing AMD wheels that cannot reach the GPU. The explicit UNSLOTH_ROCM_GFX_ARCH
override still returns first, so it stays authoritative.
- install.sh: guard the inferred-gfx reroute on x86_64|amd64. ROCm torch wheels are
not published for arm64, so an inferred/overridden gfx no longer pushes an arm64
host to the AMD arch index (get_torch_index_url returns CPU there).
- install_python_stack.py _amd_arch_index_url: honour UNSLOTH_AMD_ROCM_MIRROR on
Linux (the same var install.sh uses) instead of the Windows mirror var, so a
mirrored/air-gapped Linux 'unsloth studio update' reaches the index install.sh
chose. Windows still delegates unchanged; both default to repo.amd.com.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scan all AMD display controllers in the lspci fallback for PR #7305 (Codex P2)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): keep inferred AMD wheels from being overwritten
After a successful inferred-gfx install, skip the generic pytorch.org
ROCm reinstall so readable ROCm userland without /dev/kfd cannot undo
the per-arch repair (Codex P1 on #7305). Also merge latest main.
* Only take the inferred-gfx install when the runtime sees no GPU for PR #7305 (Codex P1)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Isolate three updater tests from the host cpuinfo for PR #7305 (Strix dev box leak)
* Gate the reroute on invisible ROCm and forward the inferred gfx to setup.sh for PR #7305 (Codex P2s)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Require AMD PCI display evidence for cpuinfo inference; honor gfx override with visible ROCm for PR #7305 (Codex P2s)
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
* Faster safetensors weight loading on unified-memory (integrated) GPUs
On unified-memory GPUs (AMD APUs / "Strix Halo", NVIDIA GB10 "Spark", Intel
iGPUs) the GPU shares the system memory pool. PyTorch's fast pinned-DMA
host->device path does not recognize the Rust-allocated, mmap-backed buffers
that safetensors hands back, so a direct safetensors GPU load
(`safe_open(..., device=<cuda>)`) drops onto a slow per-tensor copy that, on
unified memory, additionally triggers page-attribute changes and page faults.
Wrap `transformers.modeling_utils.safe_open` so that, when transformers asks it
to load a shard directly onto a CUDA/HIP device, the shard is opened on CPU and
each tensor is `.clone()`-d into a normal torch allocation before `.to(device)`.
This restores the fast DMA path. Data, dtype and final device are unchanged, so
outputs are bit-identical -- only *how* the bytes reach the GPU changes.
Strictly gated to integrated/unified-memory GPUs via the standard `is_integrated`
device property (every visible device must be integrated): a hard no-op on
discrete NVIDIA/AMD GPUs, CPU, XPU and MLX, where the pinned-DMA path already
works. Only intercepts `framework="pt"` CUDA-device targets; CPU / disk-offload
loads are left untouched. Accuracy-neutral, idempotent, opt out with
UNSLOTH_DISABLE_UMA_CLONE_LOAD=1 (force the gate for tests with
UNSLOTH_FORCE_UMA=1/0).
This is the AMD/universal-UMA counterpart to the NVIDIA DGX Spark work in #5945
(which deliberately left the H2D clone-then-move out): gating on `is_integrated`
covers AMD Strix Halo, Intel iGPUs and Spark-class parts alike.
Verified on an AMD Radeon 8060S (gfx1151, Strix Halo) Windows ROCm box with
in-process, ordering-cancelled A/B benchmarks:
- H2D mechanism (safe_open device=0 vs cpu->clone->.to(0)): 2.08x faster
(1.076s -> 0.518s for a 988MB bf16 shard)
- full `from_pretrained`: 1.56x faster (1.552s -> 0.996s), saving 0.555s --
matching the H2D delta exactly
- max|logit diff| stock vs patched == 0.0 (bit-identical), generate + a LoRA
train step both verified
The absolute/relative win grows with bf16/fp16 weight volume (the same trick is
reported as ~2.3-2.75x on NVIDIA GB10 Spark for larger models).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: evaluate the integrated-GPU gate lazily, not at import (Gemini review)
patch_unified_memory_safetensors_load() called
is_integrated_unified_memory_gpu() at install time, and the gate queries
torch.cuda.get_device_properties() for every visible device -- initializing
the CUDA context during `import unsloth` on every CUDA machine (discrete
included). That (a) breaks fork-based multiprocessing, (b) runs BEFORE
patch_dgx_spark_memory_config can set PYTORCH_CUDA_ALLOC_CONF on Spark,
defeating that patch's expandable_segments config in the very environment
this PR targets, and (c) charges a CUDA context to CPU-only imports.
The gate now runs lazily inside the wrapper, ordered AFTER the
framework/device check so non-CUDA loads never trigger the property query;
a CUDA-target safe_open means the caller is initializing CUDA anyway, and
the gate is lru-cached so it is evaluated once. The wrapper installs
unconditionally (opt-out and idempotency unchanged) and passes through when
the gate is off.
Tests: install-time no-eval guarantee (gate raises if called during
install), wrapper passthrough with the gate off, all previous gating /
passthrough / CUDA correctness tests kept -- 16/16 pass. Verified on the
N1X (WSL2): module exec + patch install leave torch.cuda.is_initialized()
unchanged; CPU loads pass through; forced CUDA-target loads intercept and
land bit-identical on the GPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Compress PR comments to essentials (comment-only; AST-verified)
Docstrings and the _utils hook comment trimmed to their load-bearing
content (lazy-gate rationale, gating scope, opt-out env). AST dumps
with normalized docstrings are identical before/after for all three
files; the module's 16 unit tests pass unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: tighten the UMA-load import comment (no code change)
* Tighten and trim code comments
* Drop unused is_integrated_unified_memory_gpu import from _utils.py
The UMA hook only needs patch_unified_memory_safetensors_load(); the
gate symbol is imported and used from ._uma_safetensors directly, so the
hoisted alias here was dead and tripped the import-hoist safety-net lint.
* Scope the UMA loader docstring to CUDA/HIP direct-device loads
The module text claimed Intel iGPU coverage, but the gate and device check
are CUDA/HIP only, and the clone path only wraps safe_open calls that carry
a CUDA device. State the actual scope and name the deliberate exclusions
(Intel XPU, CPU-open + .to() flows like bnb/HQQ) until they can be validated
on real hardware. Comment-only change.
* Tighten UMA safetensors loader comments
Trim the inline comments in the UMA clone-then-move path and the
_utils.py install site to be shorter and clearer. No code changes.
* uma: fall back to the direct move when the clone cannot allocate
The clone-and-move fast path transiently doubles one tensor's CPU
footprint while the mmap source and the CUDA destination are live. On a
UMA box with little free shared memory a large tensor could OOM where
the stock direct safe_open path would have loaded it. Both move sites
now go through a helper that catches the allocation failure and falls
back to the direct (slow but allocation-free) move, so the load always
succeeds; a genuine non-memory error re-raises identically from the
fallback.
Added a test that forces the clone to fail and verifies the wrapper
still lands tensors on the device with intact values (17 tests pass on
a real GPU).
* tests: track the moved pass-through inheritance in the gguf order check
Main moved the llama_extra_args pass-through inheritance out of the
GGUF branch into _resolve_inherited_extra_args, which runs before it,
so the source-order assertion's "if request.llama_extra_args is None"
anchor no longer exists inside the branch and the check failed after
the main merge. The test now asserts the same property in the current
shape: inheritance before the GGUF branch (a carried --no-mmproj still
shapes the hub guard's companion requirement), and marker, hub guard,
unload in order within the branch. Full file passes (32 tests).
* tests: anchor the inheritance order check on the call, not the definition
source.index("_resolve_inherited_extra_args(") matched the function
definition, which always precedes the endpoint, so the ordering
assertion was vacuously true. Anchoring on "= _resolve_inherited_
extra_args(" pins the first call site inside the load endpoint (line
4505), which is the statement whose position relative to the GGUF
branch the test is meant to guard. 32 tests pass.
* tests: align the gguf order test with main
Main fixed the stale ordering assertion in PR 7252; adopting its
version verbatim removes this file from the branch diff entirely and
avoids a conflict on the next main merge. 32 tests pass.
* uma: tighten comments
* Relicense UMA safetensors module and test under AGPL-3.0
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(install): route Strix to AMD gfx index on ROCm 7.14
When ROCm 7.3+ caps to the generic pytorch.org rocm7.2 index (or the
Radeon repo is unavailable), gfx1150/gfx1151 hosts were left on
torch 2.11+rocm7.2 instead of AMD's arch-specific wheels. Broaden the
Strix reroute in install.sh and studio/install_python_stack.py so
`studio update` repairs the same path as fresh installs (unslothai#7280).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* install.sh: route Strix (gfx1151/gfx1150) to the AMD arch index on rocm7.2, add PCI hint
Two Linux install fixes for AMD Strix Halo / Strix Point:
1. #7264: Strix reverts to rocm7.2. Modern ROCm (7.3+) caps to the generic
rocm7.2 index and the Radeon repo can be unavailable, so gfx1151/gfx1150
landed on a non-arch-specific build (torch 2.11+rocm7.2) instead of
repo.amd.com/rocm/whl/gfx<arch> (torch 2.11+rocm7.13, AMD's real Strix
fixes). The reroute to that arch index only fired on rocm7.1; broaden it to
rocm7.2 too. Only acts when a gfx1151/gfx1150 is actually detected, so other
arches on rocm7.2 pass through unchanged.
2. Rows about Strix not detected -> CPU-only: when no GPU is detected but an AMD
display GPU is on the PCI bus, print a targeted hint (ROCm kernel stack /
/dev/kfd missing) instead of only the generic docs pointer. Purely
additive diagnostic; does not change the torch index decision.
* Address review on PR #7293: gate PCI hint on ROCm-detection failure, fix test marker, use read builtin
- Only show the 'ROCm cannot see the GPU' hint when _has_amd_rocm_gpu fails;
a detected-but-too-old ROCm (rocminfo works, wheels need 6.0+) has its own path.
- Update test_previous_torch_pin.sh to the stable 'Strix Halo / Strix Point:'
marker after the heading reworded (the old grep broke the ordering assert).
- _amd_gpu_present_via_pci: read builtin instead of spawning cat twice per
device, and guard /sys/bus/pci/devices existence.
* install.sh: reroute Strix on any generic index older than the arch build
Generalize the Strix reroute from the hardcoded rocm7.1/rocm7.2 match to a
version compare against the arch index's own build (rocm7.13):
- backwards: rocm6.0-6.4 and rocm7.0 now reroute (were silently missed)
- forwards: any future intermediate rocm7.x below 7.13 reroutes; rocm7.13+
is left alone so a generic index that already carries the fix is not
downgraded to the arch build
_rocm_index_below does an integer major.minor compare (so rocm7.2 < rocm7.13);
non-rocm, arch (gfx), and unparseable URLs return false, so NVIDIA/CPU and the
arch index itself are untouched. Reroute still fires only for gfx1150/gfx1151.
* install.sh: tighten _amd_gpu_present_via_pci comment (no code change)
* install.sh: match the index leaf in the Strix version reroute (#7293 review)
Address two review points on the rocm-version reroute:
- Parse the final path segment (_torch_index_leaf) instead of grepping the whole
URL. A custom mirror whose base path holds its own rocm token (e.g.
.../rocm7.13/cache/rocm7.2) previously matched the base and skipped the reroute;
now it compares the leaf (rocm7.2) like the nearby index-family logic. Renamed
the helper to _rocm_leaf_below and switched the case selector to $_torch_index_leaf.
- Replace the stale test_strix_override_only_fires_on_rocm71 (which passed by
matching the new rocm7.13 comment) with an executed test that runs _rocm_leaf_below
and asserts rocm6.0-7.12 reroute while rocm7.13+/gfx/cu leaves do not.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: keep gfx probes non-fatal under set -e (#7293 review)
The Strix reroute now matches every rocm* index, not just rocm7.1, so its gfx
detection runs on all AMD installs. Each `_gfx_all=$(rocminfo|amd-smi | grep -oE
gfx...)` returns 1 when grep finds no match, which under set -euo pipefail aborts
the installer before the next fallback runs (e.g. rocminfo present but emitting no
gfx token). Append `|| true` to the three probes, matching the display block that
already guards this. Add an executed regression test (shimmed rocminfo/amd-smi)
that fails if any probe becomes fatal again.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
pre-commit-ci[bot]