a05c58b6b read the arch list back out of the Dockerfile with
"$(dirname "$0")/Dockerfile", but build.sh already does
cd "$(dirname "$0")" near the top. The dirname is therefore applied
twice, so every invocation by a path other than ./build.sh from inside
docker/ died before reaching docker build:
$ bash wt_r5748/docker/build.sh
sed: can't read wt_r5748/docker/Dockerfile: No such file or directory
EXIT=2
set -euo pipefail turns the sed failure into an abort, so this broke the
whole script rather than just the banner it was meant to print.
Use a bare filename, which is what the rest of the script already does
(the docker build context below is a bare "."). Verified from the
workspace root, from an absolute path, and from docker/ itself: all
three now print
arch list 7.5;8.0;8.6;8.9;9.0;10.0;12.0+PTX
Two bugs that compound.
The sync backgrounds a GitHub refresh child and the parent exits immediately,
firing its `trap finalize EXIT` (Colab intro strip plus categorized view rebuild)
while the child is concurrently cp -a'ing refreshed notebooks into the same tree
and rewriting the same state file. Six identical fresh-container boots reported
cleaned 337/311/316/277/297/360 notebooks, and one of them published a
categorized view holding 176 of 359 notebooks because both processes tore down
and rebuilt the symlink farm at once. The lost writes are permanent: 222 to 309
recorded hashes no longer matched the file on disk, so those notebooks were
treated as user-edited and skipped by every later strip, which is where 10 of the
23 notebooks still carrying the Colab intro came from.
Keep the refresh detached, which is the whole point of it, and fix the ordering
instead. One exclusive flock covers a whole invocation so the child cannot start
until the parent has exited, the parent runs the finalize explicitly before it
forks so the order holds even where flock is missing, the finalize is run-once,
and the child re-arms it only when the refresh actually copied something.
The strip itself only inspected cells[0], which missed 23 of the 433 shipped
notebooks: 21 put the Colab badge in cells[0] and the sentence in cells[1]
(Advanced_Llama3_2_(3B)_GRPO_LoRA, Falcon_H1-Alpaca, gpt-oss-(20B)-GRPO and
friends), and 2 (NeMo-Gym-*) wrap the sentence in a single-line HTML comment.
Scan the leading markdown block instead, stopping at the first code cell so it
can never reach prose between code cells, and match the closed single-line
comment form. The strip stays idempotent and leaves the content signature of all
433 notebooks unchanged, so the boot refresh does not re-copy and re-strip them
forever.
Measured on the rebuilt image: ten consecutive fresh-container boots all report
cleaned 536 notebook(s) and view 359 notebooks in 26 folders, 0 of 433 notebooks
retain the Colab intro (was 23), 0 recorded hashes mismatch (was 222 to 309), and
a second boot on the same volume is a no-op.
The pip shim fronts pip/uv inside the notebook kernel so an install cell cannot
replace the baked cu128 stack, but _KEEP only covered torch/vLLM/unsloth. Across
the 433 shipped notebooks that left the training half wide open:
trl 382 pin an older release, 378 of them ending the install cell with
`pip install --no-deps trl==0.22.2`, against a baked trl 0.24.0
torchao 273 reinstall it and 2 pin 0.15.0, replacing 0.17.0+cu128
torchcodec 92 reinstall it and 26 pin 0.5 or 0.7.0, replacing the
0.11.0+cu128 wheel the Dockerfile pairs with torch 2.11
datasets 254 reinstall it, observed falling from 4.3.0 to 3.0.0
peft 225 reinstall it, observed falling from 0.19.1 to 0.14.0
accelerate 225 reinstall it
hf hub 240 reinstall it and tokenizers 64, both version-locked to
transformers and shipped in matched copies inside every sidecar
So every notebook run mutated the stack the image was validated with, while the
shim printed that it was keeping the baked versions.
The membership criterion is "replacing this invalidates the tested stack or
breaks unsloth", not "a notebook mentions it": snac, causal-conv1d, mamba-ssm,
omegaconf, protobuf, sentencepiece and the rest still install normally.
Verified in the rebuilt image by running the Gemma3 (270M) install cell verbatim:
trl 0.24.0, peft 0.19.1, datasets 4.3.0, accelerate 1.14.0, torchao 0.17.0+cu128,
transformers 5.14.1 and huggingface_hub 1.24.0 are all unchanged afterwards, the
requested transformers pin is still recorded for the sidecar, and a package the
image does not bake still installs.
The existing shim tests used peft as their "unprotected package" sentinel, so
they move to snac.
The image runs unslothai/notebooks unchanged by refusing a notebook's
transformers pin and activating a baked sidecar on sys.path instead. Selection
was a pure ceiling (smallest baked version >= the request) and ignored that vLLM
is version-locked to transformers, so two of the four baked sidecars could not be
imported by the baked vLLM 0.26.0 at all:
4.57.6 ImportError: Support for Transformers v4 is deprecated and was removed
in vLLM v0.24.0
5.3.0 ImportError: cannot import name 'ALLOWED_LAYER_TYPES' from
transformers.configuration_utils
Those two are exactly the ones the common pins select. 241 notebooks pin
4.48/4.52.3/4.55.4/4.56.1/4.56.2/4.57.x and land on the 4.57.6 sidecar, 13 pin
5.2.0/5.3.0 and land on the 5.3.0 sidecar. All 254 died at
`from unsloth import FastModel`, before the first model cell. Pointing
UNSLOTH_TF_SIDECAR_ROOT at an empty directory and changing nothing else turned
Gemma3 (270M) and Gemma3 (1B) GRPO into clean 22/22 and 25/25 passes.
Put a floor in front of the ceiling. Which versions clear the floor is measured,
not hardcoded: the build imports vllm.transformers_utils.config under every
candidate sidecar, deletes the ones that raise, and records the lowest survivor.
That is the vLLM module which reads the transformers API, it reproduces both
failures, and it imports without a GPU, which matters because the build host has
none. A request below the floor is clamped up to the lowest eligible sidecar,
the closest version to the notebook's pin this image can actually run; a request
above every sidecar still falls through to the baked transformers.
Measured on the rebuilt image: sidecars 5.5.0 and 5.10.2 survive, floor 5.5.0,
tf-sidecars drops from 250M to 123M, and all 13 distinct transformers pins found
across the 433 shipped notebooks now reach `from unsloth import FastModel`.
Gemma3 (270M) runs end to end exactly as shipped, 22 of 22 cells, loss 4.09 down
to 0.85 over 10 steps.
The banner printed before the build hardcoded a second copy of the CUDA
arch list, and it had already drifted: it showed
8.0;8.6;8.9;9.0;10.0;12.0+PTX
while the Dockerfile builds with
7.5;8.0;8.6;8.9;9.0;10.0;12.0+PTX
so anyone reading build.sh's output was told Turing is not covered when
in fact it is. The echo does not feed the build, so no image was ever
wrong; only the report was.
Read the value back out of the Dockerfile instead of repeating it. The
sed anchors on an optional-leading-whitespace assignment, so the
Dockerfile's explanatory comment mentioning the same variable is not
matched, and head -n1 takes the builder-stage ENV. Verified to yield
7.5;8.0;8.6;8.9;9.0;10.0;12.0+PTX against the current Dockerfile.
Comment-only pass over the PR's own files. No executable line changes.
- Dockerfile / Dockerfile.studio: drop the decorative stage banner rules, the
stale "5)" / "6)" step numbering, and the entrypoint pre-flight list that
restated (and had drifted from) entrypoint.sh's own accurate header. Cut the
llama.cpp asset bullet list that repeats fetch_llama_prebuilt.py's docstring
and the structlog rationale already spelled out at the install site.
- entrypoint.sh / studio_launch.sh: fold the section banners into the
explanation lines that follow them.
- docker-publish.yml: remove the comment rule lines around the job headers.
- validate_studio_features.py: same for the numbered section headers.
- smoke_test.py: drop the stale "~125M params" note on a 1B model.
- unsloth_branding.py, unsloth_nb_view.py, unsloth_nb_pip_magic.py,
colabTitle.ts: remove comments that restate the adjacent line.
Two defects found by running the built image rather than reading it.
1. Every unsloth_cli subcommand that touches the studio backend died on
import. `unsloth list-checkpoints` on the published image:
ModuleNotFoundError: No module named 'structlog'
and the same for train / export / chat, since all four import
studio.backend.core.*. structlog is a studio backend requirement, not
an unsloth[huggingface] one, so nothing in the base install pulled it
in. Added it to the base venv, and added a build-time
`from studio.backend.core.export import ExportBackend` so a future
missing dependency in that closure fails the build instead of the
user's first CLI invocation. That guard has to live in the LAST
builder verification block: the closure also needs starlette, which
only arrives with vLLM two stages later.
2. flashinfer-jit-cache was pinned to a literal 0.6.6 while vLLM 0.26.0
resolves flashinfer-python 0.6.14. flashinfer raises at import when
the two disagree, and that exception is thrown inside the vLLM
EngineCore, so Unsloth's GRPO fast_inference path fails at engine
start with no earlier warning. A literal pin drifts again on the next
vLLM bump, so the version is now read back from the resolved
flashinfer-python, and the build proves `import flashinfer` works.
Verified on the rebuilt image: flashinfer-python 0.6.14 with
flashinfer-jit-cache 0.6.14+cu128, structlog 26.1.0, the export backend
importable, and `unsloth list-checkpoints` exiting 0.
tests/python/test_docker_llama_cuda_backend.py gains two static cases
pinning both: the jit-cache version must be derived rather than literal
and the build must import flashinfer, and the base venv must ask for
structlog with the CLI reachability guard present.
The v5 branch popped logits_to_keep and num_logits_to_keep unconditionally,
so an explicit caller value was discarded before generate() ever saw it.
v5 injects logits_to_keep=1 itself, but that injection is guarded by
`"logits_to_keep" not in model_kwargs`, which makes it a default rather
than an override: a value the caller passed is honored and must not be
dropped.
The cost of dropping it is not a no-op. Measured on a LoRA Qwen2-VL under
transformers 5.14.1, with the model's forward hooked so the validator still
sees the real signature: asking for logits_to_keep=0 (the whole sequence)
reached forward as 1 and returned logits of shape (1, 1, 151936); with the
value preserved it reached forward as 0 and returned (1, 88, 151936).
Deleting the pops outright would be wrong in the other direction. An
explicit num_logits_to_keep raises from _validate_model_kwargs on plain,
PEFT, text and vision models alike, because v5 renamed it away, and
logits_to_keep raises on the 12 of 79 image-text-to-text architectures
whose top-level forward does not take it. So each key is now stripped only
when _unsloth_generate_accepts_kwarg says this model would reject it, which
is the same predicate the validator uses. Under PEFT, self inside the
wrapper is the object the validator later runs against, so the check has no
false negatives there.
Also softened the comment above the branch. Unsloth 2026.7.5 does
pre-inject on a LoRA Qwen2-VL under 5.14.1 and generation succeeds, so
"pre-injecting makes the strict validator raise on PEFT models" overstates
it. Skipping the injection on v5 is still right: it is redundant, and the
arch walk can select a key the top-level model rejects.
tests/test_generate_kwarg_gate.py gains four cases covering a preserved
supported value, a stripped unsupported one, untouched neighbours, and the
absence of the unconditional pop.
The portable llama.cpp bundle loads libggml-cuda.so with dlopen, links it
against libcublas, and does not ship libcublas. The CUDA runtime base
image only carries libcudart, and the only libcublas in the image is
torch's wheel copy under site-packages/nvidia/cublas/lib, which was not
on the loader path. So the CUDA backend failed to load, and llama.cpp
said nothing about it: `--list-devices` printed an empty list and every
GGUF request ran on the CPU.
Measured in the built image on a B200 with gemma-4-E2B-it UD-Q4_K_XL:
1.6 tok/s from llama-cli and 4.2 tok/s from llama-server. With the fix,
the same image and model report `CUDA0: NVIDIA B200` and run at 229 tok/s
and 193 tok/s. Studio's GGUF chat and the GGUF export path go through the
same bundle, so both were affected.
The venv loader config already existed for torchcodec, so cublas/lib
joins it there rather than on LD_LIBRARY_PATH: ld.so.conf.d is consulted
after DT_RUNPATH, which keeps llama.cpp resolving its own $ORIGIN libs
first. cu13/lib comes along for the arm64 bundle's layout.
A silent 140x slowdown deserves a build-time gate, so the layer after the
fetch runs ldd over libggml-cuda.so, installs the cublas major the bundle
actually asks for when it is missing, and fails the build on anything
still unresolved. The amd64 bundle wants libcublas.so.12 and torch
already provides it; the arm64 bundle is CUDA 13, and deriving the major
from ldd keeps that leg honest without hardcoding either. libcuda.so.1 is
exempt: nvidia-container-toolkit injects the driver stub at
`docker run --gpus`, so it is never resolvable at build time. ldd needs
no GPU, so the build stays host-independent.
tests/python/test_docker_llama_cuda_backend.py pins the loader entry, the
guard, the driver-stub exemption and the ordering.
Three conflicts, all where main rewrote code this branch had also touched:
- .github/workflows/studio-backend-ci.yml path filter: kept both sides,
so docker/** still triggers Backend CI and main's install.sh,
install.ps1 and scripts/** triggers come along too.
- The same workflow's shell-test step and tests/run_all.sh: took main's
directory discovery over this branch's hand-written file lists, which
had drifted. tests/studio/test_ci_shell_suite_coverage.py passes.
- studio/install_llama_prebuilt.py: took main's delegation of
_os_error_messages and is_busy_lock_error to prebuilt_core and kept
this branch's is_cross_device_error, which the EXDEV copy-and-remove
fallback still calls. BusyInstallConflict is already the prebuilt_core
class here, so the delegated isinstance check is unchanged.
tests/studio/install/test_rocm_rdna_routing.py errors out on CPU-only CI,
taking Repo tests (CPU) with it, all 12 cases with
OSError: libhipblas.so.2: cannot open shared object file
AttributeError: module 'torch._C' has no attribute '_cuda_getCurrentRawStream'
The spoof presents torch as a Radeon card, which flips
torch.cuda.is_available() to True and sets torch.version.hip. bitsandbytes
gates its backend on exactly that:
if torch.cuda.is_available():
from .backends.cuda import ops as cuda_ops
so a bitsandbytes imported afterwards walks into the CUDA/ROCm path against a
CPU-only wheel and dies reading torch._C._cuda_getCurrentRawStream. It reaches
the test because unsloth_zoo imports it eagerly, guarded by except ImportError,
which neither OSError nor AttributeError satisfies.
Import it in the spoof instead, while is_available() is still False, so the CPU
path is cached in sys.modules before torch is rewritten. Placed in the shared
apply(), ahead of the first mutation and inside the idempotence guard, so the
ROCm spoof that layers on top gets it too.
Co-authored-by: danielhanchen <unslothai@gmail.com>
Follow-up to #7435, which fixed _smart_apt_install. Three sites were left.
studio/setup.sh: the WSL GGUF build-deps block is the pre-#7435 install.sh
pattern verbatim. It probes with 'test -r /dev/tty', assumes REPLY=y when that
fails, and then runs the elevated apt-get with stdin open. Its own guard
comment says a password is needed on WSL, so this is exactly the scenario from
issue #7307, and install.sh runs setup.sh in the same install. Give it the same
treatment: a real open probe, -n -k with stdin closed on the headless path, and
the manual command plus the existing _SKIP_GGUF_BUILD degradation on failure.
The helper is defined locally because setup.sh runs as its own process.
install.sh autostart prompt: still used 'test -r /dev/tty' and printed the
question before checking, leaving a dangling prompt in container logs. Reuse
_can_read_tty and move the printf inside the branch.
install.sh interactive escalation: a sudoers denial, a wrong password or an apt
error aborted on the bare message while the headless branch printed what to run
by hand. Make both symmetric.
Co-authored-by: danielhanchen <unslothai@gmail.com>
* studio: fix Backend CI red on main from an ambiguous ordering anchor
test_load_marker_precedes_hub_guard_and_unload fails on main, so every
open PR against the repo inherits the failure.
Root cause. #7239 (a7761e174) reworked the GGUF GPU-pool validation in
_load_model_impl from "if config.is_gguf and effective_gpu_ids is not
None:" to a bare "if config.is_gguf:", placed earlier in the function
than the GGUF load branch. The test anchors on
source.index("if config.is_gguf:"), a first-match search, so it silently
re-anchored onto the GPU-pool statement. #7251 (95f42bcce) then restored
the assertion "= _resolve_inherited_extra_args(" before
"if config.is_gguf:" against a tree where that anchor already pointed at
the wrong statement, and main went red. Checking out 95f42bcce and
running the suite reproduces the same single failure.
The code is correct. _resolve_inherited_extra_args still runs before the
GGUF load branch and before the hub-download guard that consumes
extra_llama_args for require_mmproj, so the guarantee #7251 protects is
intact; only the assertion is wrong.
Fix. Assert that guarantee behaviourally instead of by source offsets.
The new test drives _load_model_impl over a vision GGUF with a stored
--no-mmproj from a previous same-model load and captures the
require_mmproj the hub guard is called with: inherited --no-mmproj gives
False, nothing to inherit gives True, and an explicit request list wins
over the stored one both ways. Moving the resolution call after the
guard makes the inherited case report True and the test fails, so it
detects the reorder the old assertion was meant to catch, without
depending on how many "if config.is_gguf:" statements the endpoint has.
The surviving marker-before-guard-before-unload assertion had the same
ambiguous anchor for its slice start, silently widening the slice past
the GPU-pool block. It now slices from the "if config.is_gguf:" nearest
above the in-flight marker, which pins the load branch.
The structlog test stub gains a get_logger factory so routes/inference.py
is importable when structlog is absent.
34 pass in tests/test_gguf_load_cache_reuse.py (was 32 pass, 1 fail);
350 pass across it plus test_llama_cpp_mmproj_fallback.py and
test_llama_cpp_mtp_detection.py. A full backend run before and after is
identical apart from this test going from fail to pass.
* studio/tests: repair a pre-existing bare structlog stub before importing routes
* studio/tests: tighten the comments on the new load-ordering coverage
* Tighten comments on the load-ordering coverage for PR #7442
The Agents tab added in #7303 sets its avatar initial and its two status
pills with raw px utilities (text-[11px], text-[10px]). Those ignore the UI
font size preference, so the text stays fixed while the rest of Settings
scales, and tests/studio/test_ui_font_scale_contract.py fails on main.
Swapped for the existing tokens in index.css, which are the same sizes
multiplied by --ui-font-scale: text-ui-11 and text-ui-10.
Co-authored-by: danielhanchen <unslothai@gmail.com>
The Agents tab landed with three raw px text utilities, so its avatar initials
and the two status pills ignore the UI font size preference and stay fixed while
the rest of the dialog scales.
Swap them for the existing text-ui-11 / text-ui-10 tokens, which is what the rest
of the frontend already uses (149 and 128 call sites respectively).
This is what test_no_raw_pixel_text_utilities guards, so Repo tests (CPU) has been
red on main since the tab was added, and every open PR inherits the failure.
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Fix GGUF tool chat server recovery
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cover MTP precedence and loosen the replay assertion for PR #7424
Add a regression test for the MTP branch of the tool-loop respawn retry: the
file-wide _make_backend stub forces _maybe_recover_from_mtp_crash to False, so
nothing exercised the case where an MTP crash reload is already claimed and an
ordinary same-config respawn must not run on top of it. Cover both the next
tool-loop request and the final synthesis pass.
Replace the whole-payload equality assertions with a field-wise check. Comparing
the full dict pins max_tokens to the value derived from the dead server's
effective context, so a later fix that rebuilds server-derived defaults after a
respawn would read as a test failure rather than an improvement.
Document that the one-retry budget is per model request, not per chat turn.
* Recover from prefill-time deaths and stop respawn racing the MTP reload
Two gaps in the tool-loop respawn retry, both reproduced before fixing.
A child that exits during prefill has already accepted the socket, so httpx
raises ReadError, WriteError or RemoteProtocolError rather than ConnectError.
Those all arrive before the response opens, which is exactly the window where a
replay is safe, but the helper only caught ConnectError and gave up. Widen the
catch to NetworkError plus RemoteProtocolError. Timeouts stay excluded on
purpose: they mean the server is slow, not dead, and retrying one would spend
the 20 minute first-token budget twice. Windows resets connections where Linux
refuses them, so this also covers the common Windows presentation.
_maybe_recover_from_mtp_crash returns False both when the crash is not an MTP
crash and when an MTP-free reload is already in flight. Callers read that as
permission to respawn, so _respawn_if_dead replayed the crashing MTP kwargs and,
by replacing the process, made the in-flight reload abort on its own newer-load
check. Skip the respawn while that reload owns the corpse. The guard lives in
_respawn_if_dead so the plain chat path gets it too.
Regression tests for both, including a guard against retrying prefill timeouts.
* Release the MTP single-flight claim when the reload never starts
_mtp_runtime_fallback_in_progress is claimed before the reload thread exists, and
only that thread's finally clears it. Two statements ran in between with no unwind
path: re-reading _last_load_kwargs, which an unload can null underneath us, and
Thread.start(), which raises under the thread exhaustion that is exactly the
pressure killing llama-server in the first place. Nothing else ever resets the
flag, so a failure there latched it for the life of the process.
That was survivable before, since respawn ignored the flag. It is not now: the
guard added in db78184be keys off the flag alone, so a latch would silently
disable auto-respawn for every later model, including plain non-MTP ones. Read
the kwargs and process once before claiming, and release the claim if the thread
cannot start.
Restore the whole-payload equality assertions. Comparing field-wise was meant to
leave room for rebuilding server-derived defaults on replay, but the payload is
built once before the retry and re-sent unchanged, so the looser check only
dropped seven real keys and added a vacuous seed comparison.
Also correct the docstring: llama-server flushes its 200 at slot start, so a
death during decode arrives with the response already open. The pre-header window
this covers is an upload still in flight or a request waiting behind busy slots.
* Confirm the child exited before spending the retry
A closing llama-server can beat its own exit status: the socket error arrives while
poll() still reports the process running. _respawn_if_dead then took the alive
branch, handed back the stale _healthy, and the caller read that as a successful
respawn and spent its single retry on the same corpse. When that retry failed,
attempt was no longer 0, so no respawn ever happened and the turn died, with a log
line claiming a respawn that had not occurred. The window matters most for the
pre-header ReadError and RemoteProtocolError shutdowns the retry now covers.
Wait a bounded second for the exit status before calling the child alive. The same
race is already conceded in _maybe_recover_from_mtp_crash, whose recovery thread
polls for 5s because the error can arrive a beat early; 1s here because this runs
on the request path, and a genuinely live server, including one a concurrent caller
has just respawned, still returns promptly.
* Tighten the recovery comments
* Harden the respawn path around concurrent unloads and replacements
Two problems with the reap grace loop, both found by review.
Skip the grace when the server was already replaced. A caller queued on
_respawn_lock behind someone else's respawn woke holding the healthy replacement,
could not tell it from the child its own request had used, and waited out the full
grace. That sleep is under the lock, so the waits serialised: four concurrent
generations cost roughly three grace periods before any retry began. Capture the
process before taking the lock and return early once it has been swapped.
Do not respawn a server that is being torn down on purpose. unload_model() sets
_cancel_event and only clears _last_load_kwargs after the kill, so a request losing
its connection mid-unload could watch that deliberate exit through the grace loop,
read the stale kwargs and load the model straight back; a model switch landing
during the wait was reverted the same way. Re-check the cancel flag and the process
identity under _serial_load_lock before capturing the replay kwargs, matching what
the MTP-crash reload already does.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the respawn comments
* Do not charge the reap grace to a server that is still serving
The grace loop added for the not-yet-reaped race waits on poll(), which for a
live child never returns, so every transient transport error paid the full
_RESPAWN_REAP_GRACE_S. That sleep is held under _respawn_lock, so the cost
serialised: measured 1002 ms for one caller and 8.02 s for eight concurrent ones,
against 0 ms on main. A working install pays this, not a broken one.
A llama-server's listening socket dies with the process, so a loopback connect
separates the two cases in microseconds. Probe it first and return immediately
when the port still accepts; fall through to the grace only when the port is
gone, which is the case the grace exists for. Back to 0.7 ms for one caller and
0.00 s for eight.
Cross-checked on real hardware over Qwen3.5-2B, Llama-3.2-1B, Gemma-3-4B with
mmproj and Qwen3-30B-A3B: decode throughput within noise of main (-0.06%, -3.71%,
+2.57%, +0.29%, against a 54-232% spread between rounds of a single run), output
byte-identical on every round, tool-path recovery restored on the three families
whose model calls the tool, and plain-chat recovery still working on all four.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make the respawn lose to a deliberate unload in every window
Two follow-ups on the respawn path, both reproduced first.
Check _cancel_event before the socket fast path. unload_model sets the flag before
it kills, so the child is still accepting when the probe runs; returning the stale
_healthy there aims the retry at a server that is deliberately going away.
Close the unload TOCTOU. The old cancel check sat under _serial_load_lock, which
unload_model never takes, so an unload could land entirely between that check and
load_model and the captured kwargs would restart a model the user had stopped.
Snapshot the kwargs, the flag and a new _unload_epoch together under _lock, the
lock unload does hold, so a teardown is either wholly before the snapshot or
wholly after it. load_model clears _cancel_event on the way in, so the epoch is
the only evidence that survives; when it moves during the reload the replacement
is unloaded again rather than left running.
_lock stays uncontended across load_model, which would deadlock a plain Lock and
block /status for the length of a load. Error-path latency is unchanged: 0.6 ms
for a live server and 0.00 s for eight concurrent callers.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Fix the CPU-only ROCm routing errors and two font-scale UI flakes
Two unrelated causes of red CI on every PR, both reproduced before fixing.
ROCm routing: 12 errors on Repo tests (CPU). The spoof reports an AMD GPU, and
unsloth_zoo pulls in bitsandbytes, which picks a compute backend at import. Once
torch looks like a GPU is present, bnb loads its ROCm/CUDA ops, which a CPU-only
torch cannot satisfy (no libhipblas.so.2, no torch._C._cuda_getCurrentRawStream),
so the child died before printing RESULT. Nothing here tests bitsandbytes, so
import it first, under the honest hardware. Reproduced in a CPU-only torch venv:
11 passed with 12 errors before, 23 passed after. Still 23 passed on a CUDA build.
Font-scale UI: the select-viewport step pressed ArrowDown six times behind fixed
sleeps, but Radix moves focus into the listbox after the content opens, so on a
loaded runner the keys landed on the trigger and nothing scrolled. Wait on the
overflow and press until it moves, bounded at 40. The same fixed-sleep pattern
made open_appearance miss the dialog when the shortcut fired before the app wired
its handler; alternate both chords on a bounded retry and wait for the control the
caller is about to drive.
Both were reproduced locally by running the suite against a real Studio under full
CPU load. Original: 2 of 10 passed, with the exact CI signature 'keyboard did not
scroll the select viewport: 0' five times. Fixed: 10 of 10.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the ROCm routing assertion live on Apple Silicon for PR #7469
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* install.sh: do not assume sudo consent when there is no terminal (#7307 P7)
_smart_apt_install printed an "Accept? [Y/n]" prompt, and when /dev/tty was
unreadable it set REPLY=y and escalated anyway. Every sudo call in that branch
redirects stdin from /dev/null, so on any host where sudo needs a password the
install died on sudo's own error rather than the actionable message the no-sudo
path already prints. Containers, CI and locked-down corporate machines hit this.
Probe with `sudo -n true` first. If there is no terminal to prompt on and sudo
would need a password, exit with the missing packages and the exact command to
run, matching the no-sudo path. Passwordless sudo still escalates unattended,
which is the one case where that is legitimate, and says so in the log.
With a readable /dev/tty the behaviour is unchanged, and the prompt now only
prints when something can actually answer it.
Extend tests/sh/test_apt_distro_prompt.sh to drive the real function across all
four TTY/sudo combinations, rewriting /dev/tty to a fixture path the same way
the existing cases rewrite /etc/os-release. Against the old install.sh five of
these assertions fail. Register the file in studio-backend-ci.yml's shell suite,
which did not run it before.
* install.sh: probe the real tty and the real sudo commands (#7307)
Codex review follow-ups on the no-TTY sudo escalation guard.
`test -r /dev/tty` only reads the device node's permission bits. Inside
containers and systemd units those bits look fine while open() fails with
ENXIO, so the guard still fell through to a prompt nobody could answer.
_can_read_tty() does a real open. The subshell is load-bearing: in dash a
failed redirection on the special builtin `:` exits the script.
`sudo -n true` proves only that `true` is allowed. Under a command-specific
rule like `NOPASSWD: /usr/bin/apt-get` it is the wrong question in both
directions. _sudo_runs_unattended() asks the sudoers policy about the exact
argument vectors we are about to elevate, via `sudo -n -l --`, which checks
without running and fails instead of prompting.
Tests cover both: a NOPASSWD-on-trivia-but-not-apt-get sudoers stub, and a
readable-but-unopenable /dev/tty faked with a unix socket (skipped where the
platform cannot produce that shape).
* install.sh: test sudo by running it with -n, not by asking sudo -l
Codex follow-up. `sudo -n -l -- apt-get ...` answers authorization, not
authentication: on a host where apt-get is permitted but still carries the
PASSWD tag, list mode exits 0 while the actual run needs a password, so the
guard reported unattended and the escalation died exactly as #7307 described.
Inferring the answer from list output means parsing for `!authenticate`, which
is human-readable text that varies by sudo version. Drop the inference. In the
no-terminal branch, run the real commands with `sudo -n`: -n never prompts, so
it cannot block on a closed stdin, and its exit status is the question we were
trying to answer. If it is refused, print the actionable manual command as
before. The terminal branch is unchanged: prompt, then plain sudo, which may
ask for a password because someone is there to type it.
The test stub now models sudo properly (-n refuses and runs nothing when a
password is needed) instead of special-casing the probe's argv.
* install.sh: require a real NOPASSWD rule, and stop blaming the password for apt failures
Two review findings on the headless escalation branch.
A cached authentication timestamp from an earlier, unrelated elevation made
`-n` succeed for a PASSWD-tagged apt-get, so packages installed with nobody
having answered the prompt. Add `-k` so the probe ignores the timestamp and
only a real NOPASSWD rule counts as passwordless. Per sudo(8), `-k` alongside
a command ignores the cached credentials for that invocation and "will not
update the user's cached credentials", so an interactive session elsewhere
does not have to re-authenticate afterwards.
A nonzero status from the elevated apt-get was reported as "likely needs a
password" even when sudo had authenticated fine and apt itself failed on a bad
repository, a dpkg lock or a network outage. sudo returns the command's own
exit status when the command runs, so the two cases are not distinguishable
from the status alone. Report both possibilities and point at the real error.
tests/sh/test_apt_distro_prompt.sh: teach the sudo stub about -k, add a cached
mode, and assert both behaviours. The three new assertions fail against the
previous commit.
* install.sh: an unreadable answer at the consent prompt declines
_can_read_tty proves the device opens, not that anyone is there to answer. A
read that hits EOF still fell back to REPLY=y and escalated, so the branch that
does have a terminal kept the behaviour this change removes from the branch
that does not. A drained or half-closed terminal reached it.
Default to n instead, which is what the post-install autostart prompt at the
bottom of this file already does on the same condition. Enter still means yes:
that is a successful read of an empty line, not a failed read.
tests/sh/test_apt_distro_prompt.sh: add an eof tty fixture, which opens
normally and returns EOF immediately. Both new assertions fail against the
previous commit.
* install.sh: tighten the escalation comments, and correct the exit-status claim
Comment-only. The earlier note said a nonzero status from the elevated apt-get
was not distinguishable from the status alone; sudo(8) is more specific than
that. sudo exits 1 on an authentication or configuration failure and passes the
command's own status through when the command runs, while apt-get(8) returns
100 on error, so the two usually are distinguishable. sudo also exits 1 when
the command cannot be executed, which is why the message still states both
causes rather than naming one.
* install.sh, tests: tighten the comments added by this branch
Comment-only pass over the branch's own comments in both files. Same intent,
fewer lines: drop restatement, keep the parts a reader cannot derive from the
code (why test -r is the wrong probe, why the subshell around the redirection
is load-bearing under dash, what -k buys over -n, and why a nonzero status
does not by itself name the cause).
Verified to touch nothing but comments and blank lines.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* studio: shard export checkpoint loads across all visible GPUs
Export checkpoint loading always used unsloth's from_pretrained default of
device_map="sequential", which stacks the whole model on GPU0. On a multi-GPU
host this OOMs GPU0 while the other GPUs sit empty, so a GGUF export that would
comfortably fit across the machine fails with CUDA out of memory (#7053).
Add _multi_gpu_device_map_kwargs(): when the CUDA/ROCm host exposes more than
one visible GPU and get_device_map resolves to "balanced" (the same policy the
inference loader already uses), pass device_map="balanced" to every
from_pretrained in load_checkpoint. In every other case -- single GPU, CPU,
MLX, or any probe failure -- it returns {} so the loader default is untouched.
Fixes#7053
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/save: reach the UUID/MIG fallback, release sharded models before quantize
Two review fixes on the multi-GPU export sharding:
1. UUID/MIG CUDA_VISIBLE_DEVICES masks resolve to no numeric ids, so the
len(visible) > 1 gate skipped get_device_map entirely and large exports on
those hosts still stacked onto GPU0. An empty id list now routes to
get_device_map(None), whose visible-count fallback exists for exactly this
case; a genuinely GPU-less host still resolves "sequential" and keeps the
loader default.
2. The compressed (FP8/NVFP4) export freed GPU memory before its llm-compressor
subprocess only for single-device models -- a plain .to("cpu") is invalid on
an accelerate-dispatched model, so a multi-GPU-sharded checkpoint stayed
resident on every GPU while the subprocess loaded a second copy. The release
is factored into _offload_model_for_quantize_subprocess /
_restore_model_after_quantize_subprocess: dispatched all-GPU shards get their
accelerate hooks removed, move to CPU, and are re-dispatched over the
recorded hf_device_map afterwards. Maps with cpu/disk targets (already
offloading) and quantized models are left alone, as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/save: budget merged tensors per device, restore hooks if CPU offload fails
Two review fixes on the multi-GPU export path:
1. The LoRA-merge save path budgeted every merged tensor against GPU0
(get_device_properties(0) + unqualified memory_allocated()). A merged tensor
lives on the GPU of its source layer, so for a model sharded across GPUs
(the device_map="balanced" this PR enables) GPU1+ could OOM as their weights
accumulated while only GPU0's headroom was checked. Budget against W's own
device via a per-device cache; single-GPU behavior is unchanged (W on GPU0).
2. _offload_model_for_quantize_subprocess removed the accelerate hooks and then
moved a dispatched model to CPU; if that move raised (host RAM too small for
the sharded checkpoint) the model was left hookless and half-moved, breaking
later exports in the same worker. It now re-dispatches (or, for the
single-device path, moves back) on a failed move before aborting the offload.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/save: release sharded models before the torchao reload too
The portable torchao FP8/INT8 export freed the in-memory model only when every
parameter sat on one device, then reloaded a second copy with
device_map="auto". A checkpoint loaded through the new multi-GPU export map is
accelerate-dispatched across several GPUs, so that single-device gate never
fired and the original stayed resident on every GPU during the reload -- an OOM
for exactly the models large enough to have needed the sharded load.
It now uses the same _offload_model_for_quantize_subprocess /
_restore_model_after_quantize_subprocess pair as the compressed export, which
removes the accelerate hooks, moves to CPU, and re-dispatches over the recorded
hf_device_map afterwards. Those helpers are extended to XPU as well, since
torchao also runs on Intel GPUs and the path they replace covered both.
* studio/save: release quantized and cpu-spilled shards before quantize reloads
Two cases the release helper skipped outright, both of which leave GPU memory
held while the compressed subprocess or the torchao device_map="auto" reload
allocates a second copy:
- Quantized models. ExportBackend.load_checkpoint loads 4-bit by DEFAULT, so the
common Studio export hit the is_loaded_in_4bit guard and kept a quantized shard
on every visible GPU. They are now attempted like any other model: transformers
refuses .to() for some bitsandbytes builds, but that refusal raises before
anything moves, so the existing recovery path restores the model and returns
None -- best-effort where the stack allows it, old behaviour where it does not.
- Maps that spill to CPU. Any non-GPU target disqualified the whole model even
though the GPU-mapped modules were still resident and are exactly what needs
reclaiming. A cpu spill is safe to move (those weights are already in host RAM)
and is now released; only disk/meta targets are still skipped, because
accelerate keeps those parameters off the model and moving would try to
materialize the whole checkpoint. An all-CPU map is skipped as a no-op.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix multi-GPU offload for PEFT exports and fall back when sharding OOMs (#7215)
The dispatch branch of _offload_model_for_quantize_subprocess never ran for a
PEFT model: the wrapper proxies _hf_hook, so remove_hook_from_submodules raised
AttributeError and the bare except returned None. Studio always loads adapters,
so the new balanced map turned the offload off (0 percent freed against 91.8 on
the sequential path it replaces).
- resolve the real dispatch root before removing or replaying hooks
- snapshot and replay hooks, tensor placements and instance forwards; a plain
re-dispatch rebuilds hooks against the post-PEFT tree (395 to 1379) and drops
the fused kernels accelerate captured into _old_forward before unsloth patched
- drop the accelerator side of tied_params_map so the offload actually frees
- pass skip_keys on the fallback dispatch_model
- log the swallowed exception instead of returning None silently
- guard _unsloth_save_torchao_with_given_config like its two siblings
- retry the export load once on the loader default when the balanced map OOMs,
which happens when a training or chat job already owns the other GPUs
Measured on 4x B200 with Qwen3-0.6B: 89.9 percent freed bf16 and 79.7 percent
4bit under balanced, logits bit-identical, hooks and placements restored
exactly, 184 Params4bit round-tripped unchanged including nested state2.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the original offloaded until the torchao copy is released, and retie shared weights (#7215)
Two follow-ups from review of 8b6b4ca0b.
_unsloth_save_torchao_with_given_config restored the original inside a finally
that ran as soon as from_pretrained returned, so the original and the quantized
copy were both resident while the copy was still being saved. The restore now
sits in an outer finally that covers saving and releasing quantized_model, which
is what the two sibling paths already do.
The dispatch replay did not preserve tied embeddings. A CPU round trip repoints
every tensor and accelerate's tied_params_map is keyed on the old pointer, so
replaying the hooks produced two independent parameters. Reproduced on a tied
Llama: lm_head picked up its own storage, the embedding was duplicated in VRAM,
and an update to one no longer reached the other. The snapshot now records tied
groups (named_parameters(remove_duplicate=False), since the default hides one
half of every pair) and re-ties them after placements are restored.
Verified: tie preserved, no extra storages, live CUDA storage census identical
before and after, updates propagate again, logits bit-identical, and the 4 GPU
invariants unchanged at 89.9 percent freed bf16 and 79.7 percent 4bit.
* Keep meta tensors out of tie groups, restore accelerate move guards, retry CPU spills (#7215)
Four follow-ups from review of a58f1086b.
Meta tensors all report storage pointer 0, and accelerate parks every
CPU-offloaded parameter on meta, so grouping by pointer collapsed them into one
fake tied group. Reproduced with a balanced map that spills two blocks to CPU:
18 meta parameters in a single group with shapes 64x64, 32x64 and 128x64, which
the retie step would have overwritten with the first one. Meta and null-pointer
tensors are now skipped, and the retie also checks shape.
remove_hook_from_submodules deletes the to/cuda/xpu wrappers dispatch_model
installs to stop a caller moving an offloaded model. The snapshot now records
and replays those alongside forward and _old_forward.
The single-device retry only matched OOM, but a balanced map that spills to CPU
is refused by bitsandbytes with a plain ValueError saying modules were dispatched
to the CPU or the disk (transformers quantizers/quantizer_bnb_4bit.py:128), with
no memory wording. That is now retryable too, which matters because Studio loads
4-bit by default and busy secondary GPUs are exactly when balanced spills.
The torchao path dropped the quantized copy at the end of the try, so a failure
in save_pretrained left it resident while the original was restored. The del
moved into the finally, ahead of the restore.
Four regression tests added; suites now 25 and 9.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Retry exports whose multi-GPU load silently offloads to CPU, and clear the failed torchao traceback (#7215)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments for PR #7215
* Keep gradients across the export offload and release the failed torchao copy (#7215)
* Tighten comments for PR #7215
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
* Add fast fast_inference GRPO smoke test for the vLLM LoRA rollout path
Covers the vLLM >= 0.25.0 LoRA collision path (unsloth#7283, fixed in
unsloth-zoo#919) with all seven attention and MLP projections as LoRA targets so
both fused families (qkv_proj, gate_up_proj) are exercised. Kept tiny: the
ungated unsloth/Qwen2.5-0.5B-Instruct, max_steps=1 (the collision triggers on the
first rollout), short prompts/completions, and enforce_eager=True to skip CUDA
graph capture. Runs in ~89s cold and ~37s on a warm torch.compile cache.
Wrapped as a pytest test that skips without CUDA and still runs as a script; a
length-based reward gives non-zero GRPO advantages; asserts the vLLM engine is
attached at load and still bound on the trainer. Heavy imports are deferred into
the test so CPU-only collection stays import-free.
Co-authored-by: JoshuaL3000 <joshua.jian.ern.liew@intel.com>
* Assert GRPO metrics and pin seed in fast_inference test
Switch to unsloth/Qwen3-0.6B, disable vLLM torch.compile
(compilation_config=0) and run 3 steps so the updated LoRA adapter is
re-synced into vLLM on every step, not just loaded once.
Pin GRPOConfig(seed=...), which TRL forwards to vLLM SamplingParams, so
the run is reproducible, and assert per-step metrics (loss, grad_norm,
completion length, reward, reward spread, kl) instead of only checking
that train() returned. Verified across seeds 42/123/2024/7.
* Correct the seed comment and drop the pytest return
GRPOConfig(seed=...) does not reach vLLM SamplingParams: TRL's
generation_kwargs carries no seed key. Reproducibility comes from the
Trainer's set_seed pinning the global RNG the colocated sampler draws
from, so describe that instead.
Returning a value from a test triggers PytestReturnNotNoneWarning, which
pytest intends to make an error; the value was unused.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* guard llama.cpp prebuilt against out-of-disk instead of doomed source build
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review comments on out-of-disk guard
* keep reusable installs and Windows parity in the out-of-disk guard
* preserve the ENOSPC cause when re-raising fallback errors
* catch out-of-disk before the attempt loop and accept all llama-server layouts
* Fix out-of-disk detection gaps and false positives for PR #7420
Follow-ups found while testing the guard against a real ENOSPC (LD_PRELOAD
shim returning errno 28 under a path prefix, real network, real release):
- hydrate_source_tree retried the next mirror after an ENOSPC and only raised
on the last URL. Both source fallbacks 404 for the published mix commit, so
the reported cause was HTTP 404 and the run fell through to the source build
exactly like before the guard. Stop at the first environment-fatal error.
- The 5 GB preflight rejected hosts that install fine. A full CUDA install
peaks at 0.87 GB, the largest published bundle is 0.77 GB and macOS is
0.01 GB, so at 3 GB free the install succeeded before and exited 4 after,
with the source-build fallback suppressed too. It is now advisory, and a
real ENOSPC still exits 4. This also drops the case where an install
matching an older release plan was rejected before its reuse check.
- ENOSPC raised inside shutil.copytree arrives as shutil.Error with errno
None and no __cause__ or __context__, so it was never classified. That path
covers the hydrated source tree, the runtime overlay and the activation
fallback copy.
- _causal_chain followed __context__ even when __suppress_context__ was set,
so `raise ... from None` over an unrelated ENOSPC reported disk full and
wrongly suppressed the source build.
- TemporaryDirectory now ignores cleanup errors: an rmtree failure on the way
out replaced the in-flight SystemExit and lost EXIT_NO_SPACE.
- setup.sh skips the arm64 CPU last resort after exit 4; it re-ran the same
disk-rejected installer and buried the hint under a second error dump.
- The in-app updater turns exit 4 into a readable message instead of
"installer exited 4" plus a log tail.
Adds tests/studio/install/test_llama_prebuilt_no_space.py covering the
classifier, the advisory warning and the exit codes.
* Fix Python 3.9 breakage and Windows disk-full detection in the out-of-disk guard
Found by running the guard across the whole supported interpreter range
(requires-python is >=3.9,<3.15) and a spoofed [Linux, WSL, macOS, Windows] x
[NVIDIA, AMD, CPU] host matrix.
- TemporaryDirectory(ignore_cleanup_errors = True) is 3.10+, so the previous
commit raised TypeError at install time on 3.9 and turned a working install
into a hard failure. Replaced with a scratch_dir() contextmanager built on
mkdtemp plus rmtree(ignore_errors = True), which behaves the same on every
supported version.
- getattr(exc, "winerror", None) crashed on 3.9. urllib's HTTPError is an
OSError that proxies unknown attributes to a wrapped file object and raises
KeyError, which getattr does not swallow, so any mirror 404 during an install
would have blown up inside the classifier. Read it defensively instead.
- Classify Windows disk-full by winerror as well as errno. CPython's
PC/errmap.h maps ERROR_DISK_FULL (112) to ENOSPC but has no case for
ERROR_HANDLE_DISK_FULL (39), which arrives as EINVAL, so a Windows
os.replace() onto a full disk read as an ordinary failure and fell through to
the source build.
Tests cover both winerror codes, a non-disk winerror, and HTTPError alone and
wrapped in a PrebuiltFallback. 116 simulation cases pass on 3.9 through 3.14.
* Classify quota, flattened Windows and validate-install out-of-disk for PR #7420
- EDQUOT counts as out of space: a quota'd home has free blocks this user
cannot have, so the source build is just as doomed. Reported separately so
df does not mislead. Confirmed end to end with a real kernel EDQUOT: the
installer went from 6 retries then a source build (exit 2) to exit 4.
- Match the flattened Windows disk-full text. copytree stringifies each
per-file OSError, and OSError.__str__ returns early on winerror, so the
text reads [WinError 112] and never [Errno 28]. Captured on a real NTFS
volume. Markers are bracketed so WinError 112 does not match WinError 1120.
- --validate-install now exits 4 on a full disk. It caught PrebuiltFallback
and exited 2 before the classifier ran, and setup.sh answered 2 by deleting
the GPU build that had just succeeded and starting a CPU rebuild that needs
more of the space that ran out. Both halves are needed: the call site only
tested nonzero.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in the llama.cpp out-of-disk guard
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Add Agents settings tab for unsloth start
Adds a Settings > Agents tab documenting the `unsloth start` command:
quickstart, supported agents with click-to-copy commands, model
selection, common options, remote Studio setup, argument pass-through,
and a dry-run preview. Agent CLIs found on PATH are badged as installed.
Also removes the "New" badge from the System and Chat tabs.
* Use official brand logos for agents, invert Ollama and OpenRouter in dark mode
Claude Code and OpenAI Codex now use the Anthropic and OpenAI logos from
the provider-logos registry; agents without an official asset keep the
monogram tile. Also inverts the Ollama and OpenRouter logos in dark mode
so their monochrome marks stay visible.
* Title Agents tab "Agents (unsloth start)" and move it below Connections
The in-tab header now reads "Agents (unsloth start)" while the sidebar
label stays "Agents". Reorders the tab to sit below Connections.
* Address review: guard PATH detection, fix copy timeout, OS-aware remote snippet
- Only probe agent PATH in the desktop app on a loopback backend, so
Installed badges are not driven by a remote server's environment.
- Show the "none found" note only when detection actually ran and
returned empty, not when the call failed.
- Share one copy hook that resets its timeout on rapid clicks and clears
it on unmount.
- Render the Remote Studio snippet with PowerShell syntax on Windows.
- Note that --no-launch can still load a model when --model is set.
- Drop unused quickstart translation keys.
* Use client OS for remote commands, fix copy a11y and model wording (#7303)
- Pick the remote snippet shell from the client platform, not the server deviceType
- Single-line the model examples so they paste in POSIX, PowerShell and cmd
- Split the pass-through block into independent one-command copies
- Derive detection visibility instead of clearing state in the effect
- Announce copy success to assistive tech
- Correct the quickstart/model copy: bare start uses the loaded model
* Agents tab: flag the Codex row when the loaded model is not GGUF
* Agents tab: target the active Studio server, wrap narrow rows, index the tab's search terms
* Agents tab: build copied commands from the browser-reachable Studio and show the key placeholder
* Agents tab: index the intro for agent-name searches and keep long commands inside the panel
* Agents tab: omit --api-key so the CLI can replay a saved key for the base
* Agents tab: label the indexed heading rows and fall back to the active desktop API base
* Agents tab: name every supported agent in the indexed intro for PR #7303
* Tighten the agents tab comments for PR #7303
* Build the agents tab example commands from the active Studio base for PR #7303
* Take the agent command shell from the Studio host for PR #7303
* Pick the command shell from where the CLI runs for PR #7303
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Settings: match dialog fills to the app shell surfaces
Tabs use the sidebar fill and the content pane uses the page fill, so
both track the active palette in light and dark.
* Pair the tab column fill with the sidebar foreground
Custom themes set --foreground but not --sidebar, so search result rows
could land white on white. Track the sidebar token instead.
* feat(studio): add drag and drop sources to create project
Files dropped on the create-project dialog upload to the new project's
sources as soon as it exists, so a project can start with context instead
of needing a second trip to the Sources tab.
The sidebar and projects page dialogs now reuse NewProjectDialog rather
than each keeping their own copy, and the OCR / caption ingest overrides
move to a shared helper so every upload path sends the same settings.
* fix(studio): harden project source drops
Drops are not filtered by the `accept` attribute the way the picker is, so a
folder or an image would stage and then fail server-side with a confusing
per-file error. Unsupported entries are now refused up front with one message.
Cancel bypassed the dialog's reset, so a discarded name and its staged files
came back on reopen and uploaded into the next project created. Every close
path now goes through one handler.
Long filenames lost their extension in _sanitize_filename and were then
rejected as an unsupported type; the stem is trimmed instead. Adds backend
tests for the project scope, the sanitizer and path stripping.
* fix(studio): address second review pass on source drops
A drop landing on the panel while uploads run was not cancelled, because
pointer-events-none took the panel out of hit testing and nothing else on the
page cancels a file drop. The browser would navigate to the file and kill the
uploads in flight. Drag defaults are now cancelled even while disabled, and the
files are ignored instead.
Name, size and mtime can match for two genuinely different files, so a skipped
duplicate now says so rather than disappearing.
A slow upload could resolve after the dialog unmounted and still navigate,
pulling the user off the page they had moved to. Post-upload work is gated on
the component still being mounted.
* fix(studio): make source drops safe under StrictMode replay
The mount sentinel was only cleared in effect cleanup, so StrictMode's
setup/cleanup/setup replay left it false for good and every create in a dev
build stopped short of closing the dialog or navigating. It is now set on
setup as well.
The pending-sources marker was consumed inside a useState initializer, which
React replays, so the discarded pass ate the flag and the project opened on
Chats. Reading is now a peek and the marker is dropped in an effect.
Identical bytes under two names collapse to one document server-side, which
looked like both files had been added. The upload loop now tracks returned
document ids and says when files were merged.
* fix(studio): guard the route and storage around staged uploads
The sidebar's dialog lives in the root layout and never unmounts on a route
change, so the mount check alone could not stop a slow upload from navigating
the user back to the new project. The route is captured when create is pressed
and compared afterwards, and callers get that answer so the sidebar can still
move a chat while leaving the user where they are.
Reading the vision-pass overrides went straight at localStorage, which throws
outright where storage is blocked. That happened before the upload loop, so a
project was created and every staged source was lost. It now falls back to the
backend defaults, matching loadOptionalBool in the chat runtime store.
* fix(studio): support hostname-based enterprise proxies
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): strip userinfo from proxy fetch targets
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* ROCm/AMD CI coverage: arch-table parity, native-Linux lib prepend, RDNA4 grouped_mm, discovery-based shell suite
Three merged ROCm fixes shipped without tests, and the CI wiring that
would have run them was gated on files the fixes do not touch.
Tests added (113):
tests/studio/install/test_rocm_arch_table_parity.py (27)
diffs the four duplicated gfx -> AMD pip-index tables across
install.sh, install.ps1, studio/setup.ps1 and install_python_stack.py,
plus the GPU-name -> arch tables and the torch 2.11 pin allowlist.
tests/studio/install/test_rocm_native_linux_lib_dirs.py (26)
covers #7233: system-ROCm lib dirs prepended ahead of bundled
libggml-hip, the /dev/kfd + not-WSL + libhsa gate, the opt-out env
var, root resolution order, and source parity between the two copies.
studio/backend/tests/test_grouped_mm_rdna4_fallback.py (46)
covers #7292: registration on the CUDA dispatch key, grouped and
ungrouped numerics, bias/dtype promotion, and the Linux HIP<7.13 +
RDNA4 name gate, executed from the shipped source rather than a copy.
tests/studio/test_ci_shell_suite_coverage.py (14)
fails if either shell runner goes back to a hardcoded list or skips
a file without a recorded reason.
CI wiring:
studio-backend-ci.yml: add install.sh / install.ps1 to the path filter
(the suites it runs assert against those two files, so install-only
changes -- the shape most AMD/ROCm routing fixes take -- skipped it),
and replace the 13-file hardcoded shell list with directory
discovery. That list had fallen seven files behind, including
test_strixhalo_wsl_reroute.sh, the only shell coverage of the ROCm
WSL reroute, which had never run on a PR.
tests/run_all.sh: same discovery loop so local and CI agree.
* Test review fixes: assert on outcomes, not on the code under test
Self-review of the previous commit found four tests that passed for the
wrong reason.
1. The arch-table parity test pinned expected gfx ids copied out of the
shipped tables, which enshrined three upstream inaccuracies as
correct: RX 9070 (non-XT) is gfx1201 not gfx1200, RX 7800 XT is
gfx1101 not gfx1100, and PRO V710 is gfx1101 not gfx1102 per AMD's
ROCm compatibility matrix. The expectation is now the AMD pip index
leaf -- the thing the tables exist to produce, and what a wrong
answer costs the user. The three known drifts are listed explicitly
with a test asserting they stay cosmetic, i.e. that the wrong and
right ids still map to the same wheel index. That test turns red the
day one of them starts routing users to the wrong wheel.
2. The RDNA4 device-name test extracted the regex from worker.py and
then matched with it, so it could not fail. Widening the pattern --
the dangerous edit, since it forces the slow Python mm fallback onto
RDNA3 users -- would have been silently accepted. It now reads the
live pattern and checks it against fixed cases, plus asserts the
name match stays guarded by `not _lin_arch` and that the name is
lowercased before matching.
3. The CI-coverage test matched a verbatim line of studio-backend-ci.yml,
so reindenting the step would fail the build while a real regression
to a hardcoded list could slip past a reformat. It now parses the
YAML, finds the step by name, and asserts on the glob plus the
absence of individual filenames. The path-filter test likewise reads
the parsed trigger instead of scanning raw text.
4. A set comprehension in the parity helper had a ternary whose branches
were identical.
Mutation-tested: widening the RDNA4 regex, desyncing one copy of the
name table, dropping install.sh from the path filter, and re-skipping
the ROCm WSL shell suite each fail at least two tests. Verified on
Linux (WSL Ubuntu 24.04) with CI's torch pin: 86 + 48 pass.
* Fix three wrong gfx ids in the GPU-name arch tables
The name -> gfx tables disagreed with AMD's ROCm compatibility matrix on
three entries. Corrected against the "Radeon GPU" list at
rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html:
RX 9070, RX 9070 GRE gfx1200 -> gfx1201 (Navi 48, same die as the XT)
RX 7800 XT, RX 7700 XT gfx1100 -> gfx1101 (Navi 32, not Navi 31)
PRO W7700 gfx1100 -> gfx1101
PRO V710 gfx1102 -> gfx1101 (Navi 32, not Navi 33)
No wheel changes for anyone: gfx1200/gfx1201 both resolve to gfx120X-all
and gfx1100/gfx1101/gfx1102 all resolve to gfx110X-all, in all four copies
of the index-family map. That collapse is why the errors survived being
copied into six places -- the leaf-level tests could not see them.
It was not purely cosmetic, though. install.sh's second copy feeds
"Tip: set UNSLOTH_ROCM_GFX_ARCH=<arch>", so a 7800 XT user following the
printed advice exported gfx1100 and made a wrong id authoritative for
every later run. It would also have become a real misroute the moment AMD
split a family across index leaves, as they already do for gfx1151/gfx1150.
Fixed in all six places, which is two more than the table's own "kept in
sync with" comments claim exist:
install.sh _infer_amd_gfx_arch_from_gpu_name
install.sh case "$_gpu_disp_mkt" (banner + env tip; undocumented)
studio/setup.sh
install.ps1
studio/setup.ps1
studio/install_python_stack.py
Ordering is preserved: the gfx1102 arm still precedes gfx1101 in the shell
copies so "RX 7700S" cannot fall onto the "RX 7700" glob, and the
PowerShell copies keep the (?!S) lookahead.
Test changes:
- test_rocm_arch_table_parity.py gains _AMD_DOCUMENTED_ARCH, exact gfx
ids transcribed from AMD rather than from the tables. Agreement between
six copies proves nothing when all six were transcribed from the same
mistake, so the ground truth has to come from outside. Verified it
catches the bug: against the pre-fix tables it fails 6 tests.
- The parity check now covers all six copies. It had four; the two
install.sh copies were being treated as one, and
_WIN_GPU_NAME_ARCH_TABLE was not checked at all.
- test_rocm_support.py's TestGfxArchNameFallback pinned two of the wrong
ids as expected values; updated, and extended with a 9060 XT and a
7900 XTX case so each RDNA3/4 die is represented.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard against unregistered copies of the GPU-name arch table
Counting the copies by hand is what let them drift: the in-code "kept in
sync with" comments claimed four, the arch-id fix found six, and scanning
the tree turns up a seventh.
TestNoUnregisteredArchTable rediscovers the copies from the source tree
instead of trusting a hand-maintained list. A table line is one that names
a card and gives its arch; real tables score 9-17 such lines and the only
other hits in the repo are two single-line prose comments, so the
three-line threshold is not load-bearing. A companion test asserts the
scan still finds the known copies, so the heuristic cannot go blind and
pass by finding nothing.
The seventh copy is tests/_zoo_rocm_spoof.py, the fixture other ROCm tests
build their fake AMD host from. It states the mapping backwards (gfx ->
the name torch should report), which makes it an independent witness: it
had gfx1101 -> RX 7800 XT and gfx1201 -> RX 9070 XT right while all six
installer copies were wrong, and nothing compared the two. Now they are
round-tripped against each other.
RX 6700 XT is pinned as a known divergence rather than normalised. AMD's
compatibility matrix documents no consumer RX 6000 card and no gfx1031 at
all, the installer arm is commented "gfx103X family", and gfx1031 appears
only as an index-family key, never as a value a name table emits. With no
external source to correct against, changing shipped behaviour would be
guesswork. A test fails if the divergence ever disappears, so the
exemption cannot go stale.
Also adds the reverse of the AMD-matrix check: a documented card that
matches no arm anywhere is a silent CPU fallback rather than a wrong id.
This cannot detect hardware nobody transcribed, which would need a live
fetch of AMD's matrix and a non-hermetic suite; the docstring says so
rather than implying coverage that is not there.
Verified on Linux: 478 passed, plus all five new guards mutation-tested
to confirm each fails when its invariant is broken.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Docstring said six copies; the list under it now has seven
* tests: run discovered shell tests with bash, not sh
tests/run_all.sh discovered tests/sh/ instead of listing files, but still
invoked each one with sh. Every file there declares a bash shebang, and on
Debian/Ubuntu /bin/sh is dash: test_apt_distro_prompt.sh,
test_studio_home_node_dir.sh and test_with_llama_cpp_dir_link_behavior.sh
fail on bashisms under dash and pass under bash. The old hand-written list
happened to name only dash-clean files, so switching to discovery is what
surfaced it. Backend CI already used bash, so this was a local-only break.
Guarded by a new test asserting both runners invoke tests/sh/ with bash.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Krackan Point (Radeon 860M/840M) routed to the gfx1150 wheel index
The GPU-name tables map 860M/840M and the Ryzen AI 7 350 / AI 5 340 CPU
strings to gfx1150, but Krackan Point is gfx1152. AMD's own lemonade table
(src/cpp/server/system_info.cpp) maps both Krackan iGPUs to gfx1152.
Unlike the three ids already fixed here, this one is not wheel-neutral:
repo.amd.com publishes gfx1150 and gfx1152 as separate index leaves with
separately built torch wheels, so these laptops were installing wheels
built for a different LLVM target. gfx1152 was absent from the codebase
entirely, so it needed the index-family maps, the torch 2.11 floor lists
(same _grouped_mm bug as gfx1150/1151), the Strix reroute set and the
Windows arch allowlist as well as the seven name tables.
The parity test added in this PR did not catch it because its AMD-matrix
expectations stopped at 890M/880M. Added the APU rows, so the case that
actually changes a wheel is now covered: reverting the tables fails 9
tests naming 860M, 840M and Krackan.
gfx1153 (Ryzen AI 5 430 era) is left alone; AMD publishes no gfx1153
wheel family, so there is nothing to route it to.
Verified: bash -n on both shell installers, PowerShell AST parse on both
.ps1 files, python ast.parse on all touched modules, install suite 1334
passed with no new failures against main, shell suite 20 files.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add gfx1152 to unified-memory classifiers, make parity allowlist set-based
Krackan Point (gfx1152, Radeon 860M/840M) is the third RDNA 3.5 APU and
shares one GPU/system-RAM pool exactly like Strix Point (gfx1150) and
Strix Halo (gfx1151), but only the installers knew about it. The two
runtime classifiers still had two-element arch sets, so a Krackan laptop
got the 0.90 discrete headroom factor on a shared pool and ran llama.cpp
without GGML_CUDA_ENABLE_UNIFIED_MEMORY.
- worker.py _rocm_classify_unified_memory: add gfx1152 to the arch set,
and 860m/840m to the device-name fallback. The NVIDIA GeForce 840M
cannot collide there: the function is only reached under _hw.IS_ROCM.
- llama_cpp.py _amd_apu_wants_unified_memory: add gfx1152 to the arch set.
- Tests for both, including the :sramecc-:xnack- suffix form.
TestGfx211AllowlistParity compared four hardcoded allowlist strings, so
adding gfx1152 to all four installers correctly turned three assertions
red without any installer actually disagreeing with another. Each test
now extracts the set its installer holds and compares it to one EXPECTED
constant. Order and spacing are free, membership is not, and the next
leaf is a one-line edit instead of four.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
_load_model_impl contains more than one `if config.is_gguf:`, so
source.index() returned the earlier one, which belongs to a different check
than the branch the assertion is reasoning about. The inheritance call sits at
line 4543, the earlier branch at 4508 and the branch holding the load marker at
4567, so the comparison read 186995 < 185014 and failed on main.
The branch is now located from the load marker itself, which is the landmark
the rest of the test already relies on, so the assertion compares the
inheritance call against the branch that actually guards it. The slice used by
the following assertions is anchored the same way, which also tightens them:
they previously searched from the earlier branch to end of file.
The invariant is unchanged and still has teeth: moving the inheritance call
after the branch makes the assertion fail.
Co-authored-by: danielhanchen <unslothai@gmail.com>
* fix(studio/colab): survive ipykernel OutStream close() during startup
Unsloth Studio crashed at server startup on Colab with:
Unsloth Studio failed to start: 'OutStream' object has no attribute
'watch_fd_thread'
Root cause:
- Colab's ipykernel OutStream is created with watchfd=False, so it never
gains a watch_fd_thread. The OutStream.close() in the affected ipykernel
versions joins that thread unconditionally and raises AttributeError
(ipython/ipykernel#867).
- _setup_server_disk_logging() replaces sys.stdout/sys.stderr with a tee.
That changes the console object identity, so Colab's absl logging handler
(which captured the original OutStream and whose close() deliberately skips
sys.stdout/sys.stderr) no longer treats it as the live console.
- run_server builds uvicorn.Config(...), whose configure_logging runs
logging.config.dictConfig -> logging.shutdown, closing every existing
handler. The absl handler then calls close() on the orphaned OutStream and
the AttributeError propagates out of uvicorn.Config and aborts startup.
Fix:
- Before installing the tee, harden the displaced console streams' close() so
only the ipykernel#867 AttributeError is swallowed; a healthy close() runs
unchanged and any other error still propagates. The buggy close() raises
before it nulls pub_thread, so the stream stays fully usable.
- Give _TeeStream its own close() that flushes the log copy and forwards
close() to the wrapped console stream best-effort, so a handler that
captured the tee cannot crash startup either.
Add regression tests reproducing the exact path (an absl-style handler closing
a watchfd=False OutStream stand-in during logging.shutdown) and asserting the
tee/console path survives and keeps logging.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Show the Colab login password in the shareable link card
* Tighten Colab card comments for PR #7404
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make the Colab tunnel URL clickable and emphasise the password
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Narrow the console close() hardening to the watch_fd_thread AttributeError
* Put the Colab password on its own line so selection excludes the label
* Keep the Colab password as plain selectable text
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: reset quantized KV cache to f16 when flash-attn-off fallback fires
Studio force-enables --flash-attn on for GGUF launches. On a hard startup
or first-decode crash it retries via _with_flash_attn_off, which flipped FA
off but left --cache-type-k/-v untouched. A quantized KV cache (q8_0, q4_0,
q4_1, q5_0, q5_1, iq4_nl) requires flash attention in llama.cpp, so the retry
itself aborted at init with 'V cache quantization requires flash_attn' instead
of recovering.
Reset any quantized --cache-type-k/-v to f16 in the FA-off fallback path so
the retry can actually launch. Non-quantized types (f16, bf16, f32) run fine
without flash attention and are left unchanged. Handles long and short flag
forms and both space and equals syntax, rewriting in place to preserve list
length. Adds pytest coverage.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: FA-off fallback resets only the quantized V cache and drops env-only V cache
Only the V cache requires flash attention in llama.cpp; a quantized K cache
runs fine without it. Restrict the FA-off crash-recovery reset to the V axis
(main and draft) so a memory-constrained config keeps its quantized K cache
instead of risking an OOM on the recovery. Also drop an inherited quantized V
cache set purely through the environment (LLAMA_ARG_CACHE_TYPE_V /
LLAMA_ARG_SPEC_DRAFT_CACHE_TYPE_V) at the FA-off retry sites, which the argv
rewrite cannot reach, so the child falls back to the f16 default rather than
aborting.
* Studio: normalize underscore V-cache aliases in the FA-off fallback
llama.cpp rewrites '_' to '-' for any '--' long option before matching,
so a pass-through --cache_type_v q8_0 enables a quantized V cache just
like --cache-type-v. The FA-off crash-recovery reset only matched the
hyphenated spelling, so the underscore alias slipped through and the
retry still aborted with "V cache quantization requires flash_attn".
Canonicalize the flag name the same way before matching (short flags and
the type value are untouched).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio): honor run settings on initial model load
When loading a model from the gear-icon run-settings page, Context Length
and KV Cache Dtype were ignored if the user clicked Load before blurring
the context field, or before React flushed staged config into the store.
- Add NumericValueInput.commit() to flush a focused draft on Load
- Pass effectiveLoadConfig from model-config-page to onRun
- Prefer selection.config in performLoad for all load knobs
- Preserve meta.forceReload from the config-page reload path
Fixes#7346
* fix(studio): flush NumericValueInput draft when Load blurs first
Clicking Load blurs the context field before handleRun runs, so commit()
returned the stale value prop. Keep draft in a ref and parse it even when
the input is no longer focused.
* fix(studio): preserve Auto context when Load is clicked without edits
NumericValueInput.commit() now returns null unless the user actually
changed the field, so GGUF Load/Save no longer pins the displayed native
context into customContextLength when Auto was left untouched.
* fix(studio): clear NumericValueInput dirty state after blur commit
After a normal blur commit, reset dirtyRef so a later Load cannot replay a
stale draftRef when the user changed context via Reset or the slider.
* test(studio): pin NumericValueInput Auto/dirty contracts for #7346
Lock Codex P1/P2: commit returns null unless dirty, blur clears dirtyRef,
and handleRun only promotes a non-null committed context.
* fix(studio): keep same-click context draft after blur (#7346)
Blur can commit and clear dirtyRef before Load's onClick; stash that
committed value for one imperative commit() so typed context is not lost.
* chore: refresh PR head for #7351
* fix(studio): handle context commit edge cases
* chore: refresh PR head
* test(studio): guard invalid context drafts
* style(studio): format context draft guard
* test(studio): exercise same-click model config loads
* fix(studio): drop stale blur pin when the typed context equals the shown value
NumericValueInput cached every blur commit in lastBlurCommittedRef, even when
the draft equalled the current value and no onChange was dispatched. Because the
displayed value never changed, the useEffect([value]) clear never fired, so a
later Reset or external edit that leaves the shown value unchanged could not drop
the cache and the next commit() replayed it into an override that Reset had
removed. Only cache the blur result when it actually dispatched onChange
(final !== value); when final === value the parent is already current and there
is nothing to bridge. Add a Playwright regression that re-types the shown context
and asserts no override is stored.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: commit every same-click numeric draft before staging the load config
The run-settings Load/Reload button flushed only the GGUF Context Length draft
imperatively before building the load config. Max Seq Length (non-GGUF), GPU
Layers and MoE Layers on CPU (GGUF) are the same NumericValueInput and stage
their typed value only on blur, so editing one and clicking Load in the same
gesture staged the load from a still-stale parent config and dropped the value
the user just typed.
Wire an imperative commit handle through those inputs too and fold every
committed draft into the effective config, recomputing the non-GGUF load-time
max sequence length from the committed draft.
* fix(studio): recompute fixed-layer context pin and drop stale blur cache on every render
Two run-settings edge cases on the model-config page:
1) pinFixedLayerContext was computed from the render-time config, before a
same-click GPU Layers draft is committed in handleRun. Typing a positive
fixed-layer value on an auto-fit GGUF and clicking Reload therefore built
the runtime config with customContextLength: null, so a later fresh load
sent the native context with fixed layers (the OOM the pin exists to
avoid). Recompute the pin from the committed effectiveConfig.
2) NumericValueInput cleared its blur bridge only on a value change. A real
edit (final !== value) that Reset then reverts to the same shown number
nets value back unchanged, so the effect never re-ran and the stale pin
survived into the next Load/Save, replaying the override Reset removed.
The bridge is only valid across the single synchronous same-click gesture
that set it, so clear it on every settled render instead.
Add source-contract regressions for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The gguf order fix that landed on main dropped the only assertion
covering the prerequisite that llama_extra_args inheritance runs before
the GGUF branch: the inherited value (a carried --no-mmproj) shapes the
hub guard's require_mmproj, so a future reorder could reject a load
over an mmproj download the inherited arguments would disable. The
comment also misattributed the inheritance site to
_guard_chat_load_against_training.
The assertion is restored anchored on the call form
"= _resolve_inherited_extra_args(", which pins the endpoint's call site
(the bare name would match the function definition, which always
precedes the endpoint, making the check vacuous), and the comment now
names the real inheritance site. 32 tests pass.
* Studio desktop: fix loading-toast overlap and typing lag on model load
- Toaster: on desktop, offset toasts below the ~34px custom window titlebar
(top 46 when isTauri) so they no longer cover the min/max/close controls.
Web is unchanged (top 12).
- Model load: the 2s load poll wrote loadProgress state every tick, which
re-renders the whole chat page during "Starting model" (cheap in Chrome,
janky in the desktop WebView2 -> laggy typing). That state is only read by
the dismissed-toast inline status, so gate all four poll branches to write
it only when the inline view is live; while the toast is up it updates via
Sonner alone.
* Studio desktop: fix HTML canvas preview, download, and panel offset
- CSP: add frame-src for localhost/127.0.0.1 so the desktop webview can
frame the backend-served artifact preview. default-src 'self' (no
frame-src) blocked it -> "127.0.0.1 refused to connect"; web is
same-origin so it already worked.
- Download: route the canvas Download button through the native save
dialog (downloadFile) instead of a blob-anchor click, which the Tauri
WebView2 silently drops.
- Nudge the artifact panel down 8px so its top edge/shadow isn't tucked
under the window top bar.
* Studio desktop: add HTML filter for native canvas save dialog
Canvas Download saves .html via save_native_file, but save_filter() had no
html/htm case, so the native dialog fell back to the JSON/CSV/etc filter and
could block saving/browsing the .html export. Add an HTML filter and include
html/htm in the catch-all. Addresses Codex review on #7391.
* Studio desktop: unblock canvas preview in dev shell + clear header fade
- Preview: the app CSP frame-src fix wasn't enough in the tauri dev shell.
The preview endpoint sets its own frame-ancestors response header, which
only allowed 'self' tauri://localhost http://tauri.localhost -- so the
Vite dev origin (http://localhost:5173) was blocked and the frame stayed
"refused to connect". Extend the allowlist with http://localhost:* and
http://127.0.0.1:* (the endpoint only renders postMessage'd HTML in a
no-same-origin sandbox, so it exposes no server resource).
- Shadow: the artifact panel toolbar sat under the full-width
chat-header-fade; lower the panel top (mt 80->90px) so the controls clear
the fade.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio recipes: full-height canvas and in-app maximize control
- Recipe editor fills its container (drop the outer padding and the fixed
75vh height); the canvas reaches the window edges
- Viewport controls: the fit button now reads as center (it always
fit/centered); add an expand-to-full-view button that collapses the
sidebar and maximizes the canvas in-app, toggling back to restore
* recipe studio: exit full view when leaving the editor tab
Addresses review: the Exit full view control lives inside the editor
canvas, which unmounts on the Easy/Runs tabs. Clear maximized (and restore
the sidebar) when activeView leaves "editor" so those views aren't left
stuck under the fixed full-view overlay.
* recipe studio: keep full view below titlebar and off the sidebar state
* Fix reasoning-only Qwen3.6 completions in Studio
* Address reasoning-only review findings
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: scale menu, toast, chat and composer icons with the UI font size
Glyphs that sit beside scaled labels now follow the preference: the
shared --icon-size token (nav, settings tabs, chat action bars, code
block actions), classed svgs inside dropdown, select, context, menubar,
popover and command surfaces, toasts, the chat thread and both
composers, and the composer pill glyph slot. Sonner toast text is
unpinned from its injected 13px. Hit targets, paddings and surface
geometry stay fixed and every value is identity at the default size.
* Studio: icons scale at half the UI font size rate; cover review gaps
Icons now follow the preference at half the rate of the text, matching
the logo lockup: base + (setting - 16) / 2. The menu specific rules
that outranked the scoped block (app-user-menu, unsloth-plus-menu,
unsloth-tick) carry the scale too, which also restores the plus menu's
intended 1.15rem glyph base at the default size. From review: closed
select triggers join the scoped surfaces so their chevron tracks the
label, sonner action button labels scale at full text rate alongside
the title and description, and the unused built-in sonner loader gets a
defensive size override.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: icons match the text scale below the default, half rate above
Piecewise icon scaling: below the 16px default icons follow the UI font
size at the full text rate, above it they move at half the rate so
glyphs stay slightly smaller than the text. Written as min(full, half)
since the smaller branch is correct on each side. Applies to the shared
--icon-size token, the scoped menu, toast, chat and composer overrides,
and the menu rules that outrank them.
* Studio: cap icons at their default size above the 16px setting
Below the default icons still match the text scale; above it they now
keep their default size instead of growing at half rate, so enlarged
text dominates and glyphs read slightly smaller than the text. The
curve is min(full rate, base).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: icons above the default scale at half rate, not capped
A 16px glyph at setting 20 renders 18px, as if the setting were 18:
above the default icons move at half the rate of the text, below it
they match the text scale. The curve is min(full rate, half rate).
* Studio: standard icons render at the UI font size itself
One shared --ui-icon-size token replaces the per-base curves for every
glyph with a 16px or larger base: icons match the UI font size below
the default and grow at half the change above it, so setting 12 gives
12px icons, 16 gives 16px and 20 gives 18px, slightly smaller than the
enlarged text. Sub 16px glyphs keep their proportions through the same
curve as a factor. This also slims the previous 18px to 21px icon bases
down to the font size at the default setting.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: icon scale review fixes for ticks, comboboxes and art glyphs
From review: thinking ticks keep their own size inside plus menus (the
important menu rule now excludes them), combobox popups and triggers
join the scoped surfaces, 24px size-6 art glyphs such as attachment
tile icons go back to proportional scaling instead of the uniform
token, branch picker 36px chevrons scale proportionally beside their
counter, and buttons that default un-classed icons to size-4 get the
shared token (xs buttons keep their pinned small icons). Sonner cancel
labels already scale: sonner renders cancel with data-button set, so
the existing override reaches it.
* Studio: keep the toast close glyph compact
The button icon fallback matched Sonner's close button, whose unclassed
12px X then rendered at the shared icon size inside its fixed control.
Exclude data-close-button from the fallback.
* Studio: use text-ui-11 for the new chat settings sheet caption
The raw px guard caught a text-[11px] added on main; raw px text
ignores the UI font size preference.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Swap the lucide CircleOff icon on the Run automatically permission mode
for the Hugeicons AI Security 03 glyph, matching the app's existing
Hugeicons usage. A small lucide-compatible wrapper lets it drop into the
option list. Icon-only change, no behavior change.
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
* feat(studio): add DoRA support to studio
* fix: added use_dora fast encoder LoraConfig and gated use_dora on AdapterMethod
* fix(studio) serverside normalization for use_dora=true - add note documenting use_dora is silently dropped on diffusion
* fix: dora button disabled on mac, add preflight guard on GGUF lora export, mismatch now correctly falls through to existing error instead of silently no-opping
* Studio: add dora to the WizardState LoRA variant union for consistency
* Reject --use_dora on the MLX (Apple Silicon) CLI path
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* feat(studio): save load settings in chat presets
Presets previously stored only sampling params (temperature, top_p, etc.).
Extend them with an optional loadConfig blob that captures context length,
KV cache dtype, speculative decoding, and GPU layer knobs from the current
runtime when saving.
- Apply loadConfig when switching presets or hydrating on startup
- Show a short summary under the preset controls
- Prompt to reload when a model is already loaded
Fixes#7347
* fix(studio): persist preset loadConfig and capture GGUF context
Add ChatPresetLoadConfig to the chat settings API schema so presets with
load settings no longer 400 on save. Capture effective GGUF context from
ggufContextLength when customContextLength is cleared after auto-mode load.
* fix(studio): address Codex review on preset load settings
Coalesce default maxSeqLength/speculative/gpu knobs when capturing presets,
no-op apply for legacy presets without loadConfig, preserve GPU pin on apply,
and stop replaying stale loadConfig during settings hydration.
* Remove unused getOrderedPresets import
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio/colab): restore iframe embed via serve_kernel_port_as_iframe
Colab's output sanitizer often strips custom <iframe> tags from
IPython.display.HTML without raising, leaving a blank cell even though
display() succeeded. The kernel-port helper is the supported embedding
path and registers the proxy correctly.
- Prefer serve_kernel_port_as_iframe; keep raw HTML iframe as fallback
- Always show the clickable link card via show_link() so the proxy URL
is visible even when iframe embedding fails
- Add regression tests for embed ordering and URL truncation
Fixes#7344
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/colab): harden iframe embed fallbacks per Codex review
Guard show_link so a display failure cannot skip embedding, and only use
serve_kernel_port_as_iframe when get_colab_url returned a real Colab proxy
URL so localhost/colabtools environments still get the HTML iframe path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/colab): stop opening Colab proxy URLs in a new tab (#7349)
Colab *.prod.colab.dev proxy hosts are session-scoped and return HTTP 404
when opened as a top-level tab or from another device. Replace the
clickable Open button for those URLs with an in-notebook ready card, keep
serve_kernel_port_as_iframe for the UI, and point users at
start(cloudflare=True) for a real shareable / new-window link.
* fix(studio/colab): use kernel iframe on real Colab when eval_js fails (#7349)
Gate serve_kernel_port_as_iframe on COLAB_RELEASE_TAG + google.colab import
instead of a successful proxyPort URL. When eval_js fails and get_colab_url
falls back to localhost, real Colab notebooks still embed via the kernel helper
(port-only). colabtools without COLAB_RELEASE_TAG keeps the HTML iframe path.
Thanks @mfielding92 for the runtime diagnosis.
* Mock top-level google package in Colab embed tests
* test(studio/colab): mock top-level google package in Colab tests
Patching only sys.modules["google.colab"] fails when no google namespace
is installed: import google.colab resolves the parent first and returns
False in _is_colab_runtime(). Add a shared helper that mocks both google
and google.colab for deterministic tests across environments.
* Tighten comments in Colab embed helpers and tests
* fix(studio/colab): default Cloudflare on Colab with durable login credentials
Colab proxy iframes often load an empty document even when the kernel helper
appends the frame, leaving users unable to reach Studio to change the bootstrap
password and blocking start(cloudflare=True).
On real Colab runtime:
- Default cloudflare to True (pass cloudflare=False to opt out)
- Finalize the random admin password and print credentials in the notebook
- Persist credentials across cell re-runs after interrupt
- Show Cloudflare link before login credentials; skip blank proxy iframe when ready
- Reuse main._IS_COLAB for runtime detection (not COLAB_RELEASE_TAG alone)
- Only trust serve_kernel_port_as_iframe on real Colab; colabtools falls back to HTML
- Keep embedding when the link card display fails
Addresses Codex review feedback on #7349 and @mfielding92's catch-22 report.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/colab): skip credential finalize when cloudflare=False
Only call _finalize_colab_admin_password() when opening a Cloudflare
tunnel. start(cloudflare=False) should not clear the bootstrap-password
gate or show a login card that references a missing tunnel link.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/colab): drop stale cached Colab credentials after password change
On a Colab rerun the finalize path redisplayed the cached first-run
password whenever the bootstrap gate was already cleared. If the admin
changed the password through the app, that cached copy no longer
authenticates, so the notebook printed dead credentials. Validate the
cached password against the current stored hash before redisplaying and
drop the cache when it no longer matches.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Fixes#7244
The Studio per-model config dropdown only surfaced bf16, q8_0, q5_1,
and q4_1 even though llama.cpp already accepts q4_0, q5_0, iq4_nl, and
f32. Add the missing options to KV_CACHE_DTYPES and align API field
descriptions with the backend _valid_cache_types set.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Daniel Han