* Studio: show an actionable message when the GGUF runtime is missing
Selecting a GGUF model with no llama-server installed surfaced a generic
"Invalid model" in the UI, because validate_model's catch-all discarded the
real cause. Add LlamaServerNotFoundError (a RuntimeError subclass) raised by the
GGUF preflight in ModelConfig.from_identifier, and catch it in the validate
route so users get an actionable message: run `unsloth studio setup` to
download the prebuilt llama.cpp runtime. Other validation failures keep the safe
generic message. Adds a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: also map missing GGUF runtime to a 400 in load_model
validate_model already surfaces the actionable 'install the runtime'
message for LlamaServerNotFoundError; load_model fell through to the
generic 500 'Failed to load model'. Catch it there too so a GGUF load
without llama-server gives the same install hint instead of a 500.
* Trim comments for PR #6327
* Studio: fix stale validate test after #6398 and surface missing GGUF runtime on /load
- test_other_runtime_errors_do_not_get_gguf_message: after merging #6398,
validate_model surfaces a RuntimeError's own message, so a plain RuntimeError
no longer returns "Invalid model". Assert it does not receive the GGUF
install message instead (the prior assertion was stale after the main merge).
- Raise LlamaServerNotFoundError (not a plain RuntimeError) at the backend
load-time missing-binary branch, after diffusion routing, so /load returns the
actionable 400 like remote validation, instead of a generic 500.
- Share LLAMA_SERVER_NOT_FOUND_DETAIL between the from_identifier preflight and
the load-time raise so the message stays in sync.
- Add a propagation regression test for the non-tensor load path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Reap Studio child processes when the parent dies abnormally
Standalone `unsloth studio` launches orphaned cloudflared and llama-server when
the parent exited without running the cooperative shutdown path (terminal-window
close, Task Manager End Task, SIGKILL): the children reparented to init and kept
running, leaving an authenticated Cloudflare tunnel up for days.
Add utils/process_lifetime.py: a parent-owned Windows Job Object
(JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, children auto-inherit) plus Linux
PR_SET_PDEATHSIG, behind a best-effort helper that mirrors the desktop app's
windows_job.rs. initialize_parent_lifetime() runs at the top of run_server;
long-lived spawns (cloudflared, llama-server, RAG embedder, llama.cpp updater)
get the PDEATHSIG preexec, multiprocessing workers are adopted into the job, and
_graceful_shutdown plus atexit gain a terminate_all() backstop sweep. The
cooperative shutdown path is otherwise unchanged.
Verified on Linux: killing the parent now reaps cloudflared and llama-server
within ~2s instead of orphaning them.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test: add real Windows kill-on-job-close integration test
Spawn a parent that installs the job and a child that inherits it, terminate
the parent, and assert the child is reaped. Skipped off Windows. Also make the
liveness probe Windows-safe (os.kill(pid, 0) terminates on Windows).
* Fix Win64 handle truncation in the Job Object calls
Set explicit argtypes so the 64-bit job/process handles are not marshaled as
c_int (which truncated them on Win64, failing AssignProcessToJobObject). Assert
install success in the Windows integration test.
* Bind multiprocessing workers to parent death; harden the sweep
Review follow-ups:
- Multiprocessing workers (inference/export/training/data-recipe/Xet) cannot be
given a preexec_fn by the parent, so adopt_pid alone left them orphanable on a
Linux SIGKILL. They now bind themselves with PR_SET_PDEATHSIG at startup via
bind_current_process_to_parent_lifetime(), wired into the shared
run_without_native_path_secret entrypoint and the Xet child entry.
- Wire the previously-missed data-recipe worker through adopt_pid.
- terminate_all now honors its timeout: SIGTERM, wait, then SIGKILL the
survivors, so cooperative children can exit cleanly.
- Track adopted pids with a /proc starttime identity and add forget_pid, so the
shutdown sweep never signals a recycled pid.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Michael Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: scale export GGUF size estimates from the real model size
The Export page showed hardcoded, model-independent GGUF quant size
labels (Q8_0 ~8.2 GB, BF16 ~14.2 GB, ...) calibrated for an ~8B model.
For a 35B MoE model like Qwen3.6-35B-A3B (67 GiB bf16, Q8 ~34 GiB) the
picker wrongly reported Q8 ~8.2 GB. Only the displayed estimate was
wrong; the actual export via save_pretrained_gguf was always correct.
Add GET /api/models/export-size, which returns a model's MoE-aware
fp16/bf16-equivalent size and total params using the existing
estimate_fp16_model_size_bytes (safetensors -> config -> local -> vllm).
The result is memoized and degrades to nulls so a size hint can never
break the Export page.
The Export picker now scales each quant from that size
(bytes ~= fp16_bytes * bits_per_weight / 16, GiB units to match the
model selector), and renders no size when it is unknown rather than a
misleading fixed number. The Est. size summary in the page and dialog
is restored now that the value comes from the backend.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio export-size: address review feedback
- Run the size estimate off the event loop with asyncio.to_thread so a slow
Hugging Face request cannot stall other API or SSE endpoints.
- Cache only successful estimates; a transient failure (offline, gated before
credentials) is no longer pinned as unavailable until restart.
- Forward the HF token so private and gated models can be sized, and refetch
when the token changes.
- Clamp the size formatter index so sub-1-byte values cannot pick an
out-of-range unit.
* Studio export-size: address second review pass
- Send the HF token in an X-HF-Token header instead of the query string, so
it never lands in URLs, logs, or browser history.
- Key the estimate cache by model id only (the fp16 size is token independent),
so HF tokens are never retained in the cache.
- Restrict local-path sizing to known Studio roots (outputs/exports/cache/home)
so an authenticated caller cannot trigger a scan of an arbitrary directory.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio export-size: fix CI (import-hoist + isolated-load test stubs)
- Import ExportSizeResponse from models.models in routes/models.py instead of
re-exporting it through models/__init__.py, so the import-hoist lint does not
flag a newly added but un-loaded re-export (models/__init__.py is unchanged).
- Add Header and ExportSizeResponse to the stubbed fastapi / models.models in
test_export_absolute_paths.py, which loads routes/models.py in isolation.
* Studio: validate export-size local path before filesystem access
CodeQL flagged the export-size local-path guard as path injection: the
user-provided model path was resolved and stat-ed before it was checked
for containment under a Studio data root. Decide containment by lexical
normalization (normpath/abspath/expanduser, no filesystem access) and
only touch the filesystem once the path is proven to sit under a trusted
root, so an unvalidated value never reaches a filesystem call. Add a
direct containment unit test (under-root, root itself, missing, /etc,
and '..' traversal).
* Studio: trim export-size comments to be more concise
Shorten docstrings and comments on the export-size endpoint, helpers, tests,
and frontend size utilities; drop comments that just restate the code. Verified
code-identical (comments only) via AST/TS-compiler check. No behavior change.
* Studio: harden export-size local-path handling
Address review feedback on the export-size endpoint's local sizing:
- Resolve symlinks and re-verify containment in _is_sizable_local_path so a
symlink inside a Studio root can't point the sizer outside it.
- Re-validate the resolved LoRA base before sizing, so a crafted adapter
whose base_model points outside the roots can't redirect the scan.
- Skip nested checkpoint-*/global_step* snapshots when summing local weight
sizes so a run dir's intermediate checkpoints don't inflate the estimate.
- Size the checkpoint directory for full fine-tune checkpoint exports (whose
base may be a local/custom path), keeping base-model sizing for adapters.
Adds tests for the adapter-base escape, symlink escape, and nested-checkpoint
exclusion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Harden model fetching: consent gate for trust_remote_code
Add a load-path consent gate that scans a model's auto_map repository code
before it executes and blocks CRITICAL/HIGH findings unless the user pins
approval of that exact code version. Capability detection stays code-free,
reading raw config.json instead of AutoConfig.
- Scan config.json and tokenizer_config.json auto_map, nested local helpers,
and external owner/name--module repos; fail closed on partial downloads.
- Gate inference, training, and export workers, including the MLX path and a
LoRA's base model, and report requires_trust_remote_code from the raw config
so chat and auto-load surface the dialog.
- Verify trusted-org auto-enable against the Hub with the request token and key
the verdict cache by token; reject local-path and spoofed names.
- Add a consent dialog showing the flagged file, line, and surrounding code.
- Thread hf_token through the scan and load paths for gated repos.
* Address review: token handling, tokenizer/LoRA scan coverage, rollback
- Send the HF token for remote-code scans in the POST body, not the URL, so it
never lands in a log or browser history.
- Collect tokenizer_config.json auto_map files directly instead of relying only
on the repo file listing.
- Resolve a LoRA's base model for the validate flag and the scan endpoint so the
dialog scans the code the workers actually gate.
- Pass the request token to the training YAML trusted-org auto-enable.
- Resend a previously approved fingerprint when rolling back to a custom-code
model after a failed switch.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Consent UX: drop legacy chat toggle, fix decline copy, purge declined downloads
The per-model consent dialog is now the single approval path for custom
(auto_map) code in chat, so three leftovers from before it existed are removed:
- Remove the "Enable custom code" switch from Chat Settings and stop persisting
trust_remote_code, so a previously saved blanket-on cannot linger and load a
model without going through per-version review. The flag stays as an internal
YAML/preset default (e.g. first-party auto-enable); the load path still gates
every custom-code load on a fingerprint only the dialog produces.
- Reword the decline message and the auto-load toast to describe approving the
model's code from the dialog, not a missing settings toggle.
- On decline, purge the repo the scan downloaded so untrusted code is not left
on disk. A new /api/models/discard-remote-code endpoint deletes only a
metadata-only cache entry the scan created; it refuses local paths, loaded
models, and any repo with weight files cached, so a model the user already had
or pre-downloaded is always left untouched. The frontend only calls it when
the scan reported created_by_scan.
Adds discard-endpoint tests (delete metadata-only, refuse on weights/gguf,
refuse local, no-op when not cached) and a created_by_scan payload assertion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Export: remove the user-facing trust remote code toggle
The Export page kept a "Trust remote code" switch (default on) next to the HF
token field. Like chat, custom (auto_map) code should be approved per model
through the load-time review dialog, not a persistent blanket switch, so the
toggle is removed. The export load path already routes through the same consent
dialog: an HF source now starts with trust_remote_code off and only enables it
when the user approves the scanned code in the dialog (a local checkpoint the
user exported stays trusted by default). With the dialog unreachable and no
approval, an HF source loads with trust_remote_code off, which fails closed
rather than running unreviewed code.
* Block loads of repos with unsafe files using Hugging Face's security scan
The trust_remote_code consent gate covers one load-time RCE vector (a repo's
auto_map Python). It does not cover the other: a malicious pickle inside a weight
file (pytorch_model.bin, *.pkl, *.dat) deserializes during from_pretrained even
with trust_remote_code False, so a repo with a normal config plus a poisoned
pickle slips past the existing gate.
Add a metadata-only malware gate that uses Hugging Face's own scan (picklescan +
ClamAV), read via model_info(securityStatus=True).security_repo_status. It never
downloads, opens, or unpickles the flagged files; it only reads the Hub's verdict
and surfaces the flagged file names. New evaluate_file_security runs
unconditionally (independent of trust_remote_code) in every load path (inference,
training SFT/MLX, export), blocking the load when a file is flagged
unsafe/suspicious/malicious. The /remote-code-scan preflight and the validate
endpoint also report the result so the consent dialog opens as a hard block (no
override) listing the flagged files, even for a repo with no custom code.
Policy: hard block with no user override; fail open when the scan is unavailable
(offline/unscanned) so legitimate loads are not broken; no first-party exemption
(a poisoned pickle in a compromised trusted repo still blocks); local paths and
GGUF are skipped (no Hub scan, non-pickle format). Blocking does not gate on
scansDone, since that is often false for clean repos and a file already flagged
unsafe is unsafe regardless.
Adds test_file_security.py covering the block/allow/fail-open/skip matrix.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: scan list-form tokenizer auto_map, gate unsafe files on all load paths
Fixes from a 10-reviewer pass on the model-fetching hardening:
- The remote-code scanner skipped tokenizer auto_map encoded as a [slow, fast]
list (transformers' standard tokenizer shape, e.g.
{"AutoTokenizer": ["owner/repo--tokenization_x.Slow", null]}). External
tokenizer code in that form was never fetched, scanned, or fingerprinted, so an
AutoTokenizer(trust_remote_code=True) load could run it. _auto_map_refs now
flattens string, list, and nested values. Adds a regression test.
- Compare-mode chat loads and background auto-load only gated on
requires_trust_remote_code, so a repo flagged unsafe by the Hub scan but with no
custom code skipped the hard-block dialog. Both now also gate on
requires_security_review, matching the main chat path.
- The /remote-code-scan and /validate routes collapsed a LoRA adapter to its base
before the malware scan, so unsafe files in the adapter repo itself were missed
in the pre-load review (the workers already scan both). Both routes now run the
file-security scan over the adapter and the base.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Require approval for all HIGH remote code, fail closed when unscannable
Tighten the load-time security gates based on review:
Consent gate
- HIGH-severity auto_map code now requires explicit, per-version approval for
every repo, including first-party unsloth/nvidia. The org is no longer a
blanket bypass: a compromised first-party repo with HIGH code still warrants
review. CRITICAL stays a hard block; clean code still loads after the consent
prompt.
- Fail closed when auto_map code is present but cannot be fully fetched or
listed to scan (gated, offline, transient, or a repo-listing failure that
could hide an imported helper). We cannot fingerprint code we cannot see, so
this is a non-approvable block, retryable once the repo is reachable.
- Scan auto_map from every config that can carry one (model, tokenizer, image
and feature processor, processor, video processor), not just config.json and
tokenizer_config.json, so a custom-processor model is not missed. The file
list is the single source of truth in remote_code_scan and is pinned to the
transformers filename constants by a guard test.
- Distinguish a genuine 404 (config truly absent) from a transient error: only
the latter forces a scan, so a repo with no config is correctly a no-op.
Malware gate
- Scan a remote repo even when its name ends in .gguf; only local paths skip the
Hub scan, so a repo cannot dodge the scan by naming itself "*.gguf".
- Correct the docstring: a file already flagged unsafe blocks regardless of
scansDone; the only fail-open path is an unavailable scan.
Coverage
- Resolve a remote LoRA adapter's base model (not just local directories) so the
base, where the code and weights actually execute, is scanned in validate,
the scan route, and the training and export workers.
- Gate the embedding training path (FastSentenceTransformer) with the malware
and consent checks, matching the other load paths.
Tests updated and added for each change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope malware gate to the load-path vector; stop false-blocking first-party models
Follow-up hardening from a second review pass + a broad live model matrix
(unsloth/* , nvidia/* , third-party, and the eicar malware repo).
Malware / unsafe-file gate
- Scope the block to the actual RCE vector: a root-level file in a code-executing
format. from_pretrained deserializes weight files at the repo ROOT, so a flag is
only a load-path pickle vector there. Two exclusions, because neither is loaded:
inert formats (safetensors is tensor-only, gguf is non-pickle, configs/text/
images) and files in subdirectories. This keeps eicar blocked (its *.pkl/*.dat/
eicar_test_file sit at the repo root) while no longer false-blocking legitimate
first-party repos: nvidia/Nemotron-H-8B-Base-8K ships root safetensors plus NeMo
pickle checkpoints under nemo/ that the loader never touches, and the Hub flags
both; the gate previously hard-blocked it.
- Unknown / future non-"safe" levels now fail closed (block) instead of being
silently allowed, so Hub schema drift cannot introduce a bypass; in-progress
("pending"/"scanning"/"error") levels stay non-blocking to avoid false blocks.
Consent gate
- Ignore a STALE own-repo auto_map target that is absent from the repo listing (an
older config pointing at a file the repo no longer ships) instead of failing the
whole repo closed as unscannable. The present .py are still fully scanned, which
is the stronger coverage, and a file that is not there cannot execute. This
unblocks first-party models like unsloth/PaddleOCR-VL (its tokenizer_config.json
names processing_ppocrvl.py while the repo ships processing_paddleocr_vl.py). A
referenced .py that IS present but cannot be fetched, and a repo-listing failure,
still fail closed.
Remote LoRA base resolution
- Distinguish a genuine 404 (not a LoRA / repo absent -> None) from a transient
error: the transient case is retried once, then logged as a WARNING (a missed
base is scanned by neither gate) rather than silently skipped.
Discard endpoint
- Treat .onnx and .ckpt as weights so a repo whose only heavy artifact is one of
those is never eligible for the declined-download purge.
Tests added for each: load-path scoping (safetensors/subdir/Nemotron-H shapes,
unknown-level fail-closed, pending non-block), stale own-repo auto_map ref, remote
LoRA transient retry, and the empty-config-list (all-404 -> []) semantics.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make LoRA-base transient-warning test robust to logging backend
Assert on the logger object directly instead of capsys, so the test does not
depend on whether the real structlog logger or the module-stub logger is active
(which varies with test collection order).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Allow a repo with auto_map but no executable code (e.g. GGUF) instead of blocking
A config can declare an auto_map yet the repo ship NO executable .py -- most
commonly a GGUF repo whose config.json carries an auto_map copied from the original
model (e.g. unsloth/Llama-3_1-Nemotron-Ultra-253B-v1-GGUF references
modeling_decilm.py, which the GGUF-only repo does not contain). A GGUF model loads
through llama.cpp, which never executes auto_map, and transformers cannot run a file
that is not present, so there is nothing to scan and trust_remote_code is a no-op.
The fail-closed change treated this empty result the same as "code is present but we
could not fetch it" and hard-blocked the load. Distinguish the two: repo_remote_code_files
now RAISES RemoteCodeUnscannable when code is present but cannot be fully fetched or
listed (offline / gated / transient / a present .py that 404s / a listing failure),
and returns an empty dict only when the listing succeeded and the repo genuinely ships
no executable .py. The consent gate blocks on the exception (fail closed) and allows the
empty case as a no-op. Real unscannable code still hard-blocks; eicar and CRITICAL/HIGH
custom code are unaffected.
Verified against all 37 unsloth/*Nemotron* models (two GGUF repos were false-blocked,
now load) and the existing matrix (eicar still blocks; DeepSeek-OCR / NVLM-D-72B still
prompt approvable consent). Tests updated to expect the raise for unscannable cases and
added for the no-executable-code no-op.
* Ignore vestigial auto_map in GGUF repos (llama.cpp never runs it)
A GGUF repo's config.json is often copied verbatim from the original
transformers model, auto_map and all, but a GGUF load goes through
llama.cpp which never executes auto_map, so the config is inert. Treat
a direct .gguf reference, and a repo that ships .gguf weights with no
.safetensors, as having no remote code so the consent flow is never
triggered. A mixed repo with both .gguf and .safetensors is still gated,
since the safetensors variant would load through transformers where
auto_map does run. The check sits behind the existing auto_map-present
gate so normal models pay no extra repo listing.
* Add scanner-result copy to the remote-code consent dialog
Make the consent dialog state the scan outcome in plain language for
every model. When the static scan finds nothing, reassure the user with
'Our automatic scanner did not flag any worrying files, but please
double check.' (shown only for the clean, approvable case). When the
scan flags custom code or unsafe files, label the list with 'Our
automatic scanner flagged issues including:'. The Hugging Face
attribution for unsafe files stays in the dialog description.
* Close GGUF-suffix consent bypass for repo ids ending in .gguf
The .gguf short-circuit in _config_has_auto_map skipped the scan for any
model name ending in .gguf, including a bare two-segment repo id like
'evil/model.gguf'. Such a repo can still ship safetensors plus auto_map
Python that transformers would execute, so skipping the scan was an
asymmetric bypass (file_security already scans those repos). Restrict the
short-circuit to genuine direct GGUF file references via
_is_direct_gguf_file_ref: a local .gguf path, or a remote repo_id plus
filename (three or more segments). A two-segment repo id named *.gguf now
falls through to the config scan and _is_gguf_repo file inspection, so it
only skips consent when it actually ships .gguf weights and no safetensors.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align consent dialog body with the title and fix narrow-width overflow
The scan results (the 'Our automatic scanner...' label, finding/unsafe
cards, and the clean-scan reassurance) sat at the dialog's left padding
while the title and description were indented past the status icon, so
the body did not line up under the description. Move the title,
description and results into one column to the right of the icon so they
share a left edge, and let that column fill its width so the description
no longer wraps early.
Also stop a wide code snippet from pushing the dialog off-screen on
narrow viewports: AlertDialogHeader is a grid with place-items-center,
which sized the content row to its content; give the row w-full so it
fills the track, and add min-w-0 down the results chain so the snippet
scrolls inside its card instead of widening the dialog. Verified aligned
and contained from mobile portrait through ultrawide.
* Treat a repo as GGUF-only only when it ships no transformers weights
_is_gguf_repo excluded only .safetensors, so a repo with a .gguf and a
pytorch_model.bin (or .pt/.pth/.h5/.msgpack/.onnx/.ckpt) and no
safetensors was treated as GGUF-only and skipped the consent scan, even
though transformers can load that weight set and execute the repo's
auto_map code. Require the absence of ANY transformers-loadable weight
before treating the repo as a llama.cpp-only GGUF load. A genuine
GGUF-only repo (only .gguf) is still inert; a mixed repo with any pickle
or safetensors weight is gated. Adds a regression test across all the
non-safetensors weight formats.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Block flagged subdir weight shards referenced by a root index
The malware gate treated every subdirectory file as non-loadable, but
from_pretrained deserializes a subdir shard a root index references
(pytorch_model.bin.index.json -> shards/...-00001-of-00002.bin). Read the
root weight indexes and block a flagged subdir pickle the weight_map
points at; a flagged subdir pickle no index lists (NeMo nemo/*.distcp)
stays non-blocking, and an inconclusive index lookup fails closed.
* Pass hf_token to the export checkpoint load
ExportBackend.load_checkpoint scanned with hf_token in the worker but
loaded the weights unauthenticated, so a gated/private checkpoint passed
preflight then 401'd at from_pretrained. Add hf_token to load_checkpoint
and forward token to every from_pretrained branch; the worker passes the
command's hf_token.
* Scope created_by_scan to every HF cache the discard searches
created_by_scan used get_cache_path (active HF_HUB_CACHE only) while
/discard-remote-code deletes across active, legacy, and default caches. A
repo the user already had in a legacy/default cache was marked
scan-created and deleted on decline. Check all three caches for the repo
dir before declaring the scan created it.
* Scan the full .py closure of external auto_map repos
An auto_map cross-repo ref (owner/name--module.Class) only had its entry
file downloaded, but transformers also fetches that file's relative
imports from the same repo, so a dangerous helper.py was left outside the
scanned fingerprint. List each external repo's .py and scan the whole set
(plus the referenced entry files); fail closed if the repo cannot be
listed or fetched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fail closed when a weight index cannot be fully read
_indexed_shard_paths treated a partial result as definitive: if one weight
index read cleanly but another failed transiently, it returned the shard
paths it did see. A flagged subdirectory pickle listed only by the index we
could not read would then be classed as "not a load input" and skipped,
re-opening the very fail-open this guard was added to close.
Return None whenever any index read is inconclusive, even if another read
cleanly, so the caller blocks the already-flagged subdir pickle. A repo that
ships no index files raises EntryNotFoundError for each (never inconclusive)
and still returns an empty set.
* Match cached repos case-insensitively in the created_by_scan guard
_repo_in_any_hf_cache resolved casing only against the active cache and then
probed every cache with an exact directory name. A case-variant already
present in a legacy or default cache (models--Unsloth--Foo for a scan of
unsloth/foo) was missed, so the repo was marked created_by_scan and deleted
on decline -- but discard_remote_code_download deletes case-insensitively,
so that delete would hit the user's pre-existing cache entry. Detect
case-insensitively too, mirroring the deletion path.
* Skip remote-code and security review for selected GGUF variants
validate_model ran the trust_remote_code and Hugging Face security-scan
preflight against the repo even when the selected artifact is a .gguf. A
GGUF loads through llama.cpp, which never executes the repo's auto_map
Python and never deserializes root pickle weights, so repo-level Transformers
artifacts (a config.json with auto_map, or an unsafe pytorch_model.bin next
to the .gguf in a mixed repo) are inert for that load. Gating the GGUF on
them is a false positive. Run both preflights only for non-GGUF loads.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope the malware gate to actual load roots and serialized files
Two fixes to evaluate_file_security so it neither misses a load-path pickle nor
false-blocks an inert file:
- Honor subdirectory load roots. Spark-TTS / BiCodec call from_pretrained on the
snapshot's LLM subdirectory, so a flagged pickle directly under it is a
root-level load artifact there. A new load_subdirs parameter (set from the
model's audio type via security_load_subdirs) reclassifies those files relative
to the load root and looks for weight indexes under it, so a flagged shard in
that subdir is no longer skipped as "not root-level".
- Exempt source files. A root .py is never deserialized by from_pretrained;
executable repo code runs only through auto_map, which the remote-code consent
gate scans. Flagging a Python helper here would false-block a repo that merely
ships a build or train script.
* Scan a LoRA adapter and base as one consent unit, and gate MEDIUM code
A LoRA load runs both the adapter's and the base's repo code. The consent gate
scanned them separately and pinned one fingerprint per repo, so an adapter that
shipped its own auto_map code was either never shown in the dialog (which only
saw the base) or impossible to approve with the base's fingerprint.
evaluate_remote_code_consent_for_targets now scans all of a load's repos as a
single combined unit and pins ONE fingerprint over the union of their code, so
approving the load approves every repo's code together. evaluate_remote_code_consent
becomes a thin single-target wrapper, and an unscannable target fails the whole
load closed.
Also gate MEDIUM findings: like HIGH they now block pending pinned approval, so a
direct API caller cannot run flagged code by setting trust_remote_code=True
without consenting. Only a clean scan loads without a fingerprint.
* Preflight a LoRA load's adapter and base as one combined consent scan
scan_model_remote_code rewrote a LoRA adapter to its base and scanned only the
base for remote code, so the dialog never surfaced an adapter's own auto_map
code. Scan the adapter and base together through
preflight_remote_code_consent_for_targets, which pins one combined fingerprint
the worker gate accepts. The malware preflight is also scoped to each target's
load subdirectories.
* Apply combined consent and subdir-aware malware scan in load workers
Each load worker (inference, export, training) evaluated remote-code consent
once per target with a single shared fingerprint, so a LoRA adapter that ships
its own auto_map code could not be approved by the base's fingerprint. They now
scan the adapter and base together via evaluate_remote_code_consent_for_targets,
which pins one combined fingerprint over the union of their code. The malware
scan in each worker is also scoped to the model's load subdirectories so a
flagged pickle under a from_pretrained load subdir is not missed.
* Report a consistent trust_remote_code requirement after a model loads
validate_model reports requires_trust_remote_code from the YAML default OR the
raw auto_map, but the load, already-loaded, and status responses reported only
the YAML default. A custom-code model approved and loaded via auto_map was then
reported as not requiring trust_remote_code, so the frontend stored false and a
later retry or rollback sent trust_remote_code=false and failed.
A shared resolver reports the same requirement for a loaded model (a value
stored at load time, else the trust_remote_code the load used, else the YAML
default, else the raw auto_map check), and the load response persists it so the
status and already-loaded paths stay consistent. The selected-GGUF security
review is also scoped to the model's load subdirectories.
* Run the consent gate on training resume and for YAML-only trust_remote_code
Three frontend gaps left a model loading without the trust_remote_code it needs:
- The shared consent helper returned early when the scan found no auto_map and no
unsafe files, dropping a requirement that comes from a model's Studio YAML
default (e.g. GLM-4.7-Flash). It now grants the caller's requirement with an
empty pin instead of sending trust_remote_code=false.
- Resume-from-history called startTraining directly with no consent gate, so a
resumed run whose model needs custom code (or an old run with no approved
fingerprint) hit the worker block with no dialog. It now runs the same gate as
a fresh start.
- HF export passed requiresTrustRemoteCode=false for every HF source, so a
YAML-only model could not flip the flag before export. It now signals the
requirement for HF sources.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cover both LoRA repos in validate, report GGUF as inert, purge all declined repos
Three follow-on gaps from the combined adapter+base consent work:
- validate_model resolved requires_trust_remote_code from the base alone, so a
LoRA adapter that ships its OWN auto_map code (with a plain base) was reported
as not needing trust_remote_code and the consent dialog never opened. It now
checks the [adapter, base] target set, matching the scan route and the workers
(which already gate both) and the security review already running over both.
- The already-loaded, loaded, and status responses for a selected GGUF reported
requires_trust_remote_code from the model's YAML default. A GGUF loads through
llama.cpp, which never executes the repo's auto_map Python, so the requirement
is inert for that load. They now report False, matching validate_model (which
already skips both gates for GGUF) so a status refresh cannot flip the flag
back on.
- The remote-code scan downloads both the adapter's and the base's config, but
created_by_scan tracked only the primary, so a base the scan was first to pull
into the cache was left on disk when the user declined. The scan now reports
scan_created_repos (every repo it newly cached) and the decline cleanup purges
each; created_by_scan stays for older clients. The frontend falls back to the
primary flag when the list is absent.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scan the repo the load fetches, purge external code on decline, harden consent pins
Six follow-on hardening fixes from a fresh review pass over the gate:
- The malware gate scanned the literal "Spark-TTS-0.5B/LLM" alias, but the trainer
downloads it as unsloth/Spark-TTS-0.5B and loads LLM/, so the alias 404'd and
failed open, missing a flagged LLM/ pickle. evaluate_file_security now resolves
the alias to the repo the loader fetches and scans LLM/ as a load root.
- security_load_subdirs relied only on tokenizer detection, which fails on an
unresolved alias or offline; it now also honors the Studio YAML audio_type
default, so a BiCodec LLM/ load root is not missed.
- The remote-code scan downloads external auto_map repos (owner/name--module.Class),
but the decline cleanup tracked only the model/adapter/base, leaving the external
untrusted code cached. The scan now enumerates external auto_map repos and reports
the ones it created in scan_created_repos, so a decline purges them too.
- External auto_map refs failed the whole load closed on a stale or mis-derived
dotted ref (sub.mod.py vs the real sub/mod.py) even though the actual file was
present and scanned. They now drop such refs when the repo listing is real, exactly
like the own-repo path; an empty/incomplete listing still fetches and fails closed.
- The combined consent fingerprint keyed code by the raw target string, so the scan
endpoint's canonicalized casing and a worker's raw user input produced different
pins for identical code, rejecting a valid approval. Hub repo ids are now folded to
lowercase in the key (local paths stay case-sensitive), so the pin tracks the code.
- Export threaded hf_token into the weight load but not into detect_audio_type /
is_vision_model, so a gated multimodal base 404'd in detection and fell through to
the text loader. Both probes now use the same token.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Thread the token through check-vision and guard the gate's parallel sites
The /check-vision endpoint classified a model without the hf_token, so a gated or
private vision model 404'd in the probe and was reported as a plain text model --
the same dropped-token shape as the export probes, at a sibling site. It now passes
the token like the neighboring /check-embedding endpoint.
Add deterministic consistency guards (tests/test_security_gate_consistency.py) that
enumerate the gate's parallel sites mechanically instead of relying on a review to
spot a missed sibling: every is_vision_model / is_embedding_model / detect_audio_type
caller under routes/ and core/ must thread the token, every GGUF response must report
trust_remote_code via the resolver or False (never the raw YAML default), and every
load worker that runs the malware or consent gate must resolve the LoRA base. A new
site that drops the token or mis-reports the requirement now fails CI directly.
* Narrow the LLM alias rewrite and make audio detection token-aware
Three fixes from the confirmatory review, one a regression from the previous round:
- _load_scan_target rewrote EVERY remote repo ending in "/LLM" to unsloth/<parent>,
so a real third-party repo named "<owner>/LLM" was scanned as unsloth/<owner>
while the loader still fetched the real repo -- a fail-open hole introduced when
the Spark-TTS alias handling was added. It now rewrites only a registry-known
bicodec alias; every other "/LLM" repo is scanned as itself.
- detect_audio_type cached results under the bare model name, so an unauthenticated
probe of a gated/private repo cached None and poisoned a later authenticated call
with the token. The cache is now keyed by (normalized_name, token_fingerprint),
matching the vision cache.
- The training fallback /check-vision call dropped the hf_token, misclassifying a
gated/private VLM when the config endpoint failed. It now passes the token, like
the getModelConfig call it falls back from; checkEmbeddingModel takes the token too.
Extend the consistency guards: every capability cache must be keyed by a tuple
including the token, so a cache re-declared as Dict[str, ...] fails CI.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Document the broad .py scan as deliberate and enforce it with a test
The remote-code scanner scans every .py in a repo once an auto_map exists, not
just the auto_map entry's static import closure. This is intentional: the entry
module can reach a sibling via an absolute import, importlib, or exec, none of
which a static relative-import closure follows, so closure-only scanning would be
a real bypass of a load-time RCE gate. The broad scan never under-scans; the cost
is that an unrelated benign script can over-block, which is the safe failure
direction (HIGH stays approvable; only CRITICAL hard-blocks).
Spell this out at both the local and remote scan sites so the choice reads as
deliberate, and add a test asserting an unrelated, never-imported .py is still
scanned -- so a future narrowing to the static closure fails CI.
* Purge a declined remote LoRA adapter the scan downloaded
scan_model_remote_code probed the created-by-scan state AFTER resolving the base,
but get_base_model_from_lora_identifier downloads a remote adapter's own
adapter_config.json, so the adapter looked already-cached and was dropped from
scan_created_repos. On decline the adapter -- including the auto_map .py the
preflight fetched -- was left on disk, defeating the "untrusted code is not left
on disk" guarantee for the adapter itself.
Snapshot the primary's cache state BEFORE base resolution and use it when marking
the adapter scan-created; on any probe error treat it as pre-existing so a decline
never deletes it. The base and external repos are unaffected (their configs are not
downloaded before their own probe). Add a test that models the mid-scan download
side effect, which the prior static-stub tests did not.
* Clear remote-code approval when the training model changes
Switching the training model from an approved custom-code model to a clean one
kept the previous model's trust_remote_code=true and approved fingerprint in the
store: setSelectedModel reset visionImageSize on a true switch but not the
remote-code approval. The clean model then trained with trust_remote_code=true,
which bypasses the compiler and disables fused cross-entropy.
Reset trustRemoteCode and approvedRemoteCodeFingerprint on a true model switch.
The new model's own YAML default is re-applied by loadAndApplyModelDefaults, and a
custom-code model still re-opens the consent dialog before training starts, so the
only change is that a clean model no longer inherits a stale approval.
* Trim verbose comments across the model-fetching hardening changes
Condense the explanatory comments and docstrings introduced across the
trust_remote_code consent gate, the malware/unsafe-file gate, the remote-code
scanner, the load workers, the model routes, and the security frontend into
fewer, tighter lines while preserving every security rationale (fail-open vs
fail-closed direction, the deliberate broad-scan anti-bypass note, the
empty-vs-unscannable distinction, stale-ref handling, and the alias-rewrite
spoof guard).
Comments and docstrings only. No code, logic, identifiers, or test behaviour
changed; verified comment-only via the AST/TypeScript checker (40/40), with the
backend test suite and frontend tsc green.
* Do not cache transient audio-detection failures
detect_audio_type cached _detect_audio_from_tokenizer's result
unconditionally, so a transient read failure (network error or 5xx,
returned as None) poisoned the cache and the later successful probe never
ran. Mirror the vision cache: _detect_audio_from_tokenizer now returns
(audio_type, definitive) and the caller caches only definitive results.
A read that succeeds with no audio tokens, or clean 404s for every
tokenizer path, stays a cacheable None; only a genuine transient failure
(connection error, timeout, 5xx, malformed body) skips the cache so the
next call retries.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: pin CUDA_DEVICE_ORDER=PCI_BUS_ID and list GPUs at startup
On a mixed-GPU host, Studio could load a model onto a different physical
GPU than the one it selected. The free-VRAM probe numbers GPUs via
nvidia-smi (PCI-bus order), but CUDA defaults to FASTEST_FIRST ordering,
so a selected index written into CUDA_VISIBLE_DEVICES resolved to the
wrong card. Example: 5090 + RTX PRO 6000, the picker chose the emptier
RTX PRO 6000 (nvidia-smi index 1) but CUDA read index 1 as the 5090.
Pin CUDA_DEVICE_ORDER=PCI_BUS_ID at import (before any CUDA context is
created) in both the Studio entrypoint and the hardware module, so torch,
nvidia-smi, and CUDA_VISIBLE_DEVICES share one index space. setdefault
keeps an explicit user override intact. Child processes inherit it via
os.environ.
Also list every detected CUDA GPU with its index at startup instead of
naming only device 0, matching nvidia-smi -L and making the selected
index unambiguous on multi-GPU hosts.
* Studio: make CUDA_DEVICE_ORDER tests exercise module import and respect user override
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: guard full _print_cuda_device_list body and fix test PYTHONPATH trailing separator
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: stop the llama.cpp update banner flickering and show the download size
The banner animated in and out with a motion opacity + scale + translate
transition. That transform/opacity transition promotes a GPU compositing
layer whose first and last frame can flash for a moment on real displays,
which reads as a flicker on appear and again on dismiss/snooze. Drop the
animation and render the banner as a plain conditional mount: it appears
and leaves cleanly with nothing to flash.
Also surface the download size. update-status now reports the size of the
prebuilt that Update would fetch (the latest-release asset matching this
host's bundle), and the banner shows it as whole MB next to the no-restart
note, so the cost of the update is clear before clicking.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Show llama.cpp update size for upstream and source-build installs
update_download_size_bytes now accepts the upstream ggml-org ubuntu-/win-
asset suffixes and falls back to the marker's binary_repo, so the size
resolves for CPU/ROCm prebuilts (the fork publish repo only carries the
app-* and macOS bundles). The source-build update path now populates
update_size_bytes from the resolved asset, matching the marker path.
Both fail open to null. Adds regression tests for the upstream and
source-build size lookups and the route field round-trip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add 'Load on selection' toggle to configure load options before loading
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: seed staged speculative decoding from the standing default
* Studio: address PR review for load-on-selection staging
* Studio: handle direct GGUF staging and stale-stage edge cases from load-on-selection review
* Studio: cancel replaced staged downloads and keep staged pick on load failure
* Studio: centralize staged-download cancel and guard staged-load restore
* fix: address staged GGUF load review
* fix: honor staged GGUF load metadata
* fix: clarify load-on-selection tooltip
Keep the load-on-selection hint visually anchored to the control and make the on/off behavior explicit without changing the broader deferred-load flow.
* Studio: reset orphaned staged knobs on abandon and cap Max Tokens to staged context
* Studio: remove dead code and cancel staged download when loading a different model
* fix: surface staged model in run settings before deferred load
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* Studio: add shared Xet-primary download helper with HTTP stall fallback
Xet is the fast default transport in huggingface_hub, but a stalled Xet
transfer hangs with no progress and no exception, and a blocked native thread
cannot be killed. The safetensors inference path already recovers (subprocess
watchdog + respawn with HF_HUB_DISABLE_XET=1); the GGUF and training paths do
not. Add a reusable helper that the in-process paths can adopt.
utils/hf_xet_fallback.py:
- DownloadStallError (moved here from core/inference/orchestrator.py, which now
imports it; behavior unchanged, still a RuntimeError subclass).
- get_hf_download_state / start_watchdog: a no-progress watchdog built on the
sparse-aware hub.utils.hf_cache_state helpers; fires only while a .incomplete
is present and the on-disk byte total is unchanged for stall_timeout.
- hf_hub_download_with_xet_fallback: cached files short-circuit; otherwise the
download runs in a spawn child (own process group) supervised by the watchdog.
On a stall it kills the child, makes the partial safe for HTTP via
prepare_cache_for_transport, and respawns once with HF_HUB_DISABLE_XET=1. Cancel
and deterministic errors (auth/missing/disk) propagate without a fallback.
Tests cover the watchdog state machine, the transport decision logic, and a
regression lock that HF_HUB_DISABLE_XET is honored in a fresh interpreter.
* Studio: route GGUF Chat-Mode downloads through the Xet->HTTP fallback
The GGUF load path (_download_gguf main+shards, _download_companion_gguf for
mmproj/MTP) called a bare blocking hf_hub_download with no recovery, so a Xet
stall hung the Chat-Mode load with no fallback. Route those three calls through
hf_hub_download_with_xet_fallback: Xet stays primary, HTTP is used only if Xet
stalls, per-file so finished shards stay cached. The existing _cancel_event is
threaded through, the Cancelled sentinel is preserved, and companions stay
best-effort (a terminal stall is swallowed to None). Cached files short-circuit
in the helper with no subprocess, so the fast path is unchanged.
The two offline mmproj tests are repointed from huggingface_hub.hf_hub_download
to the new call boundary (the helper) since the download now goes through it.
* Studio: recover a stalled training model-load via Xet->HTTP respawn
Training runs in a spawn subprocess and FastModel.from_pretrained downloads
internally, so the download cannot be wrapped per-file like GGUF. Instead the
worker now watches the HF cache during the model-load phase (emitting
model_load_started / model_load_completed and a stall event), and the parent
recovers a stall by terminating the worker and respawning it once with
HF_HUB_DISABLE_XET=1.
worker.py: set HF_HUB_DISABLE_XET=1 before any HF import when the parent passes
disable_xet (respawn), and wrap trainer.load_model with start_watchdog.
training.py: plumb disable_xet through the config; track the model-load window;
on a first-load stall arm a one-shot respawn (handled on the exiting pump thread,
so no pump self-join) that preserves the DB run row (history is not duplicated)
and re-runs the load over HTTP. A second stall, or a stall outside model-load,
surfaces as a normal error. W&B init happens after model-load, so a pre-load
respawn cannot duplicate it; the dataset is re-formatted in the new worker.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add gated test-only fault-injection hook for the Xet stall path
UNSLOTH_HF_XET_FORCE_STALL=1 makes the Xet download attempt write a partial
blob and hang, so the no-progress watchdog and the HTTP fallback can be
exercised end to end against a real repo (never set in production). Used to
verify recovery on real models: a forced Xet stall on a 5.37GB Qwen3.5-35B-A3B
shard triggered the watchdog and the HTTP retry downloaded the correct file
(sha256 verified).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten Xet-fallback comments and consolidate its tests
Trim docstrings and inline comments across the Xet->HTTP fallback code to
the non-obvious why (spawn-not-thread, killpg-not-getpgid, the sparse-partial
HTTP-resume hazard); drop comments that merely restate the code. Verified
comment-only with an AST signature check.
Merge the three helper-level test files (watchdog, transport policy, and the
HF_HUB_DISABLE_XET regression lock) into tests/test_hf_xet_fallback.py, and
prefer the real structlog over a bare stub so test collection order cannot
leak an incomplete module to others that log at import.
Full backend suite: 3455 passed, 14 pre-existing flash-attn failures only.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: log transformers version-switching decisions and stop swallowing MLX activation failures
Two logging gaps in dynamic transformers version switching (issue #6103):
1. get_transformers_tier returned a tier with no trace of why. Add an
info log at each decision point naming the model and the trigger
(which substring matched, or which config check fired), so a model
landing on the wrong tier is diagnosable.
2. The MLX fast-path in run_training_process activated the transformers
version inside a bare 'except Exception: pass', silently swallowing
failures while the non-MLX path reports them. A missing or broken
version venv (e.g. Gemma-4 needing 5.5.0) left no trace and only a
confusing downstream crash. Extract a small _activate_transformers_version_or_warn
helper that logs a warning on failure while keeping the non-fatal
fall-through, and call it from the MLX path.
Adds tier-selection logging tests and helper warn/silent tests.
* Studio: clarify path-prepend log, warn on venv version mismatch, log per-package install progress
Completes the remaining logging items of #6103 in studio/backend/utils/transformers_version.py:
- activate_transformers_for_subprocess: the early "Activated transformers X.X.X" line was misleading because at that point only the venv directory has been prepended to sys.path, not imported. It now says it prepended the venv to sys.path and notes the loaded version is confirmed later by "Subprocess loaded transformers ...".
- _venv_dir_is_valid: a detected version mismatch is logged at warning instead of info, since it immediately triggers a full venv wipe and reinstall that should be visible in the logs.
- _ensure_venv_dir: log each package as it starts installing with an N/M progress counter, so a slow runtime install is not mistaken for a hang (pip/uv output is piped and only surfaced on error).
Adds tests covering all three behaviours; pre-existing unused imports are left untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make tier log-capture tests independent of import order
The new issue #6103 caplog assertions in test_transformers_version.py
relied on the module-level sys.modules.setdefault("loggers", stub)
winning the import race. In a full backend pytest run another module
(for example test_log_filter_no_truncation, collected earlier) imports
the real loggers first, so the setdefault is a no-op and
transformers_version.logger becomes a structlog/stdout logger that
caplog cannot capture -- the tier, activation, venv-mismatch and
install-progress log assertions then fail even though the line was
emitted.
Bind a real stdlib logger to transformers_version.logger for the
duration of each test via an autouse fixture, so the module logs through
logging and caplog captures them regardless of collection order.
* Studio: log local checkpoint tier decisions and warn on MLX inference activation
- get_transformers_tier: the local config.json fast path returned a tier
without logging it, so local checkpoints stayed opaque while HF ids were
traceable. Log each decision there too, with a caplog regression test.
- inference worker: the MLX path swallowed _activate_transformers_version
failures with a bare except, the same gap issue #6103 fixed for training.
Warn instead, keeping the non-fatal fall-through.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: make lifespan shutdown resilient to a dead default executor
On an abrupt shutdown (closing the Windows console window, or interpreter
teardown racing uvicorn's graceful stop) the event loop's default thread-pool
executor can already be shut down by the time the FastAPI lifespan shutdown
runs. The first post-yield statement was an unguarded
`await asyncio.to_thread(terminate_hub_downloads)`, so executor.submit raised
`RuntimeError: cannot schedule new futures after shutdown`. That raise
propagated up through every nested merged_lifespan __aexit__, aborted the rest
of the cleanup (DEVICE reset, compiled-cache clear), and surfaced as
"Application shutdown failed. Exiting."
Extract the post-yield cleanup into utils/lifespan_shutdown.run_lifespan_shutdown
and guard each step independently. On the to_thread RuntimeError, fall back to
running the (already best-effort, quick) terminate inline on the loop thread so
shutdown still completes cleanly. The helper is dependency-injected and free of
the heavy backend import graph, so it is unit-tested in isolation.
Add tests/test_lifespan_shutdown.py (4 cases: dead-executor survival, normal
path, terminate error, clear error). Validated on windows-latest and
ubuntu-latest runners on Python 3.13.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: only retry terminate inline when scheduling fails, not when the body raises
Address review feedback on the shutdown helper: the previous
`except RuntimeError` after `asyncio.to_thread(terminate_downloads)` could
not tell a dead-executor scheduling failure from a RuntimeError raised by
terminate_downloads itself, so a body-side RuntimeError on a healthy executor
ran the cleanup a second time inline.
Schedule via loop.run_in_executor and await separately: a dead default executor
raises synchronously at submit time (inline fallback), while a body exception
only surfaces when the future is awaited (logged, never retried). Add a
regression test that a body RuntimeError runs terminate exactly once.
* Studio: run terminate cleanup with a copied context (parity with asyncio.to_thread)
Simulation across the executor-state x exception x DEVICE matrix surfaced the one
behavioural difference from the original implementation: asyncio.to_thread copies
the caller's contextvars into the worker thread, while a bare
run_in_executor(None, fn) does not. Restore exact parity by scheduling
ctx.run(terminate_downloads) from a contextvars.copy_context(), so the refactor
is a behavioural no-op apart from the intended dead-executor recovery. Add a
regression test asserting the copied context is visible to terminate_downloads.
* Studio: tighten comments in lifespan_shutdown helper and tests
Condense the verbose docstrings/comments to the non-obvious rationale and drop
the self-evident ones. Verified comment-only with comment_tools.py check
--strip-docstrings (code signature unchanged); tests and sims still green.
* Studio: address review nits on lifespan shutdown helper
Annotate hw_module as types.ModuleType, reword the schedule/await comment
(inline fallback runs, it does not retry), and add a test for the public
loop.shutdown_default_executor() path (the 'Executor shutdown has been called'
RuntimeError that real uvicorn shutdown takes, distinct from the submit-time
'cannot schedule new futures after shutdown').
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: show llama.cpp version and GPU specs in the About panel
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: run the hardware endpoint off the event loop
* Studio: show the full llama.cpp release tag (incl -mix-<sha>) in the About panel
* Studio: order About-panel GPUs by visible ordinal and skip the llama.cpp probe during updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix hardware info refresh and endpoint scope
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* Studio: enable stdio MCP servers on a loopback bind
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address codex review on stdio MCP loopback gate
* Studio: fix banner URL and preserve stdio MCP env opt-in on network binds
* Studio: scope loopback to exact aliases and honor force-disable on run_server reuse
* Studio: cover force-disable across a public re-bind and fix a stale test comment
* Studio: keep stdio MCP off on Colab loopback launches
* Studio: set tool policy before server startup
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
The update banner is driven by update_available from
/api/llama/update-status. Prebuilt (marker) installs decide this via
freshness.is_behind, which is mix-aware: a release at the same upstream
base build but with a new -mix-<sha> suffix counts as behind.
Source-build installs went through _source_build_status, which compared
only the numeric base build (installed_build < latest_build). When a new
prebuilt shared the installed upstream base (our usual mix re-tag at the
same base), it returned update_available=False and the banner never
showed. This is the common macOS case, where a failed prebuilt fetch
falls back to a source build that lacks the mix patches.
Make the source-build path mix-aware to match the marker path: same base
plus a -mix-<sha> tag now offers the update (and displays the mix tag),
a bare same-base rebuild does not, and the downgrade guard and unknown
-version fallback are unchanged.
* Studio: derive training output dir from model basename for local-drive models
A LoRA/QLoRA run started from a model loaded by absolute path (common when
models live on a non-system drive, e.g. G:\modelsAI\...\gemma-4-12B-it) seeded
the default output dir with that full path. resolve_output_dir then raised
"path escapes root ... is not under the studio outputs folder", so training
could not start from a model stored off the system drive.
Add default_run_dir_name(): Hugging Face repo ids keep their namespace
(org/model becomes org_model), while local paths collapse to their final
component so an absolute source path can no longer leak into the output dir.
Use it at the three worker derivation sites and add a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cap run dir name length and drop redundant resolve
Apply PR review feedback: length-cap the auto-generated output dir component so an unusually long model name stays under the filesystem name limit, and drop the redundant double resolve_output_dir at the embedding site so all three derivation sites match.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: keep llama-server discovery from crashing on an access-denied candidate
_find_llama_server_binary probed candidates with Path.is_file(), which raises
PermissionError (WinError 5) when a path exists but is momentarily inaccessible
(antivirus lock, an install replace in flight, an elevated-install ACL),
aborting model validation. Treat a denied-but-present path as the real binary
so discovery returns it; absent paths still skip.
* Retry a transiently locked binary instead of returning a denied path
Returning a still-denied path only moved the PermissionError to the next
is_file() (probe_server_capabilities). Retry briefly so a transient lock
clears and discovery returns an accessible path; on a persistent lock return
nothing rather than a path downstream cannot stat.
* Studio: do not fall back to another llama-server when a pinned one is locked
A denied LLAMA_SERVER_PATH made discovery skip the explicit pin and run a
lower-priority managed or PATH binary, so a load could silently use a stale or
incompatible server. Split the probe into a file/absent/denied status: when the
pinned path exists but stays access-denied, warn and stop rather than falling
back to a different executable.
* Studio: never downgrade past a denied pinned or managed llama-server
Extend the no-fallback rule beyond LLAMA_SERVER_PATH: a present-but-denied
UNSLOTH_LLAMA_CPP_PATH or managed ($STUDIO_HOME/llama.cpp, ~/.unsloth/llama.cpp)
binary now reports temporarily-unavailable instead of silently launching a
lower-priority legacy or PATH server. Shared _scan_pinned/_unavailable helpers;
legacy in-tree and PATH stay genuine fallbacks (a denied candidate there just
continues).
* Studio: let diffusion asset lookup use a locked llama-server path for its dir
DiffusionGemma does not run llama-server; _find_diffusion_assets only needs the
install dir to find the adjacent llama-diffusion-gemma-visual-server. The
no-fallback rule returning None on a transiently locked llama-server therefore
hid an available visual-server and raised 'runner not found'. Add an
include_denied option so diffusion lookup gets the locked path (its dir is all
it needs), while inference keeps the no-denied-path, no-downgrade behavior.
* Studio: report a locked llama-server as temporarily unavailable, not missing
When the pinned/managed binary stays access-denied through the retries, discovery
returns None and load_model raised 'binary not found', a terminal error that
points users at reinstalling rather than retrying a transient AV/install lock.
Reuse include_denied to detect the locked path and raise a distinct
temporarily-unavailable, retry message instead.
* Studio: GGUF preflight treats a locked llama-server as present
The pre-download preflight (and so /api/inference/validate) used the default
discovery, which returns None for a transiently access-denied binary, so it
raised 'binary not found' for a binary that merely needs the lock to clear. Use
include_denied so the existence check counts a locked binary as present; the
load itself still reports a still-locked binary as temporarily unavailable.
list_gguf_variants() keys files by _extract_quant_label(), which only captured the base quant token. Repos that ship the same base quant at multiple bits-per-weight (e.g. byteshape/Qwen3.6-35B-A3B-MTP-GGUF with three IQ4_XS files at 3.53/3.97/4.19 bpw) collapsed into a single row and Studio summed their sizes (~48 GB).
Extend the regex to capture an optional trailing -<N>(.<N>)?bpw modifier so each flavor produces a unique label. Round-trips through _find_local_gguf_by_variant and _download_gguf since both sides use the same extractor.
Fixes#5728.
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* fix(studio): keep local GGUF vision on llama-server
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): lower local GGUF vision log level
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): find GGUF companions from variant dirs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* Studio: fix Downloaded model list disappearing and order it by last download
The chat model picker scan for cached GGUF and safetensors models aborted
whenever an auxiliary Hugging Face cache dir (such as ~/.cache/huggingface/hub)
was unreadable, returning an empty list. That hid the Downloaded section and
let already downloaded models appear under Recommended. Isolate each cache
probe so an inaccessible directory is skipped instead of failing the scan.
Also order Downloaded newest-first using cached blob mtimes (multi-quant repos
group by their most recent quant), keep the section visible while searching,
and make the per-quant downloaded check per-snapshot and mmproj aware so a
Recommended quant is never falsely marked downloaded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden gguf-variants scan and dedupe by newest timestamp
Guard f.stat() per file so a broken symlink or unreadable file in a
snapshot no longer aborts the downloaded check early, and match quant
labels case-insensitively. When the same repo is present in multiple
caches with equal size, keep the newest last_modified so Downloaded
ordering reflects the most recent copy.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: apply cache-scan guards to sibling endpoints found in review
Extend the inaccessible-cache guard and mmproj/stat hardening to the
parallel HF cache code paths flagged in review:
- list_local_models and the Hub inventory scan now skip an unreadable
auxiliary cache instead of returning 500.
- The GGUF download-progress endpoint excludes mmproj adapters and
guards f.stat() so one bad file does not zero a repo's progress.
- The offline snapshot scanner guards its is_dir() probes.
- The chat-only picker no longer renders a blank list when a search
matches only cached non-GGUF models.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: allow absolute save_directory in export paths to prevent cross-drive copy failures
The GGUF export pipeline (and all other export flows) forced every
save_directory through resolve_export_dir(), which always resolved
the path under exports_root() — typically ~/.unsloth/studio/exports/
on the system drive (C: on Windows).
When a user selected an output directory on a different drive (E:):
1. The absolute path was rejected at the Pydantic validator level.
2. Even if it got through, resolve_export_dir would re-resolve it
under C:\Users\.unsloth\studio\exports\.
3. After GGUF conversion completed on E:, the relocation step would
try to move/copy the finished files to C:, causing:
- WinError 17 (cross-drive move failure when shutil.move falls
through to a cross-filesystem copy)
- WinError 112 (disk full on C:)
Fix both layers:
- _validate_save_directory: accept absolute paths (they represent an
explicit user choice of output location).
- resolve_export_dir, resolve_output_dir, resolve_tensorboard_dir:
return absolute paths as-is instead of forcing them under the
default root. Keep the existing safety checks (null bytes, '..'
segments) and fall through to resolve_under_root for relative paths.
Fixes: https://github.com/unslothai/unsloth/issues/6082
* refactor: centralize user path validation into _resolve_user_path helper
Addresses code review feedback: the null-byte, '..', and absolute-path
checks were duplicated across resolve_output_dir, resolve_export_dir,
and resolve_tensorboard_dir. Extract a single _resolve_user_path helper
that all three delegate to.
No behavioral change — pure consolidation.
* fix: address code review — contain destructive cleanup and scope absolute paths
Address all review feedback from gemini-code-assist:
1. P1: destructive subdirectory cleanup (export_gguf)
The flattening loop in export_gguf previously rmtree'd every
subdirectory under abs_save_dir. When targeting an existing user
directory on a different drive (#6082), this could nuke unrelated
subdirectories. Now snapshot existing subdirectories before the
export and only clean up dirs created during this run.
2. P2: keep scan/read endpoints contained
Only resolve_export_dir accepts absolute paths (export is a write
path where user picks location). Reverted resolve_output_dir and
resolve_tensorboard_dir to use resolve_under_root directly — these
are used by scan/read/training endpoints that must stay contained
under their respective roots.
3. Centralization feedback
Removed the _resolve_user_path helper since it's no longer needed
with the narrowed scope. resolve_export_dir has the absolute path
logic inline with a clear docstring.
* fix: skip pre-existing subdirs in GGUF flatten loop and clean stale export intermediates
Two issues caught in code review (chatgpt-codex-connector):
1. The flattening loop moved ALL .gguf files from ALL subdirectories
into abs_save_dir, including pre-existing unrelated user subdirs.
Now skip pre-existing subdirs entirely unless they are known
export-owned intermediates (model/, model_gguf/).
2. After a failed export, known export-owned subdirectories (model/,
model_gguf/) were snapshotted as pre-existing on retry and never
cleaned up. These are now always cleaned up regardless, since they
are known intermediates created by the export pipeline.
* fix: separate write vs read export paths, guard same-dir rmtree
Three issues caught in code review (chatgpt-codex-connector):
1. P1: scan endpoint containment
resolve_export_dir was changed to accept absolute paths, but it's
also used by scan/read endpoints (routes/models.py) that must stay
contained under exports_root(). Split into:
- resolve_export_dir: contained, used by scans
- resolve_export_write_dir: accepts absolute paths, used by export
backend only
2. P1: same-directory rmtree
When a non-PEFT checkpoint's gguf_dir resolves to the same path as
abs_save_dir (user selected the checkpoint's gguf output as their
export directory), shutil.rmtree(gguf_dir) would delete the user's
chosen output directory. Now skip relocation when both paths resolve
to the same location.
3. P1: pre-existing subdir flatten loop
Reverted _EXPORT_OWNED_SUBDIRS logic — 'model/' and 'model_gguf/'
are common directory names in shared model folders and don't prove
export ownership. Now only clean up subdirs that didn't exist before
the export started.
* fix: remove dead _EXPORT_OWNED_SUBDIRS and fix _export_details for absolute paths
Two fixes from review comments:
1. Remove unused _EXPORT_OWNED_SUBDIRS declaration (leftover from
previous iteration that was intentionally removed).
2. _export_details now returns the full absolute path when the export
target is outside exports_root(), instead of truncating to basename.
Users who export to E:\ can now see the full destination path in
the success dialog.
* fix: use unique tmp dir for GGUF intermediates to avoid overwriting user dirs
When exporting to an absolute destination that already contains a
model/ subdirectory (e.g. a shared models folder), the hard-coded
model_save_path would overwrite files in that unrelated directory.
Use _tmp_model_<uuid> as the intermediate path instead, so user
directories are never touched. The tmp dir is created as a new subdir
of abs_save_dir and cleaned up by the flatten loop after GGUF files
are relocated.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF local export paths for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address GGUF export follow-ups for PR #6088
* Clean GGUF temp dirs on export failure for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust export path tests for PR #6088
* Fix/adjust export path review findings for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust home export path handling for PR #6088
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
The post-install path cleared only the in-memory freshness caches and then
re-primed the 24h disk cache with a forced GitHub refresh. When that refresh
cannot reach GitHub, latest_published_release falls back to the last-good disk
value, so a still-fresh same-base mix tag cached before the swap (b9596-mix-aaa
vs the just-installed b9596-mix-bbb) is replayed and the prebuilt reads as
behind, surfacing a false update banner that points back at the build that was
just replaced.
Give reset_caches a drop_disk option and use it on the update path: with the
disk cache gone, an offline post-install refresh leaves latest as None and the
banner fails open (off) instead of lingering on the stale same-base value. The
no-arg form stays in-memory only. Adds regression coverage for the drop, the
default no-op, and the fail-open vs stale-replay contrast.
IOReport energy counters can reset (sleep/wake, power gating), making a poll
delta negative. Return None for a negative total so the monitor shows -- for
that poll instead of a bogus negative wattage; it self-corrects next poll.
* Studio: show Apple GPU temperature and power in the GPU monitor (macOS)
The GPU monitor on Apple Silicon always showed -- for Temperature and
Power: the MLX branch of get_gpu_utilization() hardcoded None because
ioreg's AGXAccelerator PerformanceStatistics carries neither metric.
Add utils/hardware/apple.py, mirroring macmon's no-sudo approach:
- Temperature: average of the AppleSMC "Tg*" float keys via the
AppleSMCKeysEndpoint user client (ctypes/IOKit, macOS 14+).
- Power: IOReport "Energy Model" group, "GPU Energy" channels; each
poll diffs the energy counter against the previous poll's sample, so
the value is the average wattage over the polling window. The first
poll only sets the baseline and returns None.
Both readers latch to None on first failure and never raise, so
non-Mac platforms and locked-down hosts keep the previous behavior.
* Sample IOReport with the subscribed channels descriptor for PR #6187
IOReportCreateSubscription writes the channel descriptor that later samples
must use; sampling with the original requested group can return no Energy
Model entries on hosts that normalize the channel set, leaving power_draw_w
null after the baseline. Use the subscribed descriptor (matching macmon) and
fall back to the requested channels if the OS leaves it unset.
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <23090290+danielhanchen@users.noreply.github.com>
* Studio: llama.cpp update banner redesign, About tab license info, inline system prompt editing, naming cleanup
- Redesign the llama.cpp update banner to match the chat composer surface
(borderless rounded card, composer shadow, Hellix Medium title), rename
actions to Update and add a 15 minute Remind me later snooze
- Keep the banner up until the user explicitly acts on it; drop the
outside click dismissal
- Add a Settings > General > Notifications toggle to disable the banner
for training-only setups (on by default)
- Rename the Help settings tab to About and add a License section
(Unsloth Studio AGPL-3.0, Unsloth Core Apache-2.0) linking to the
license files in this repo
- Make the run settings system prompt box an inline editable textarea;
the popup editor opens when the prompt overflows the box
- Pointer cursor on the preset dropdown chevron
- Dark mode toasts use the chat composer surface color
- Replace standalone Studio with Unsloth in user facing strings; keep
Unsloth Studio, LM Studio, Fine-tuning Studio, Recipe Studio and CLI
commands unchanged
* Studio: open the system prompt popup on box click, balance banner padding
- The system prompt box opens the Edit System Prompt dialog on click,
matching the pencil action
- Slightly more bottom padding on the llama.cpp update banner so the
spacing reads even next to the action pills
* Studio: replace unsloth studio update with the installer commands in update guidance
- The unsloth studio update command no longer works, so the About tab
update section now shows the one-line installer (curl or irm) for
PyPI and unknown installs, and git pull plus the local installer for
checkouts
- Add a short note that unsloth studio update is no longer supported
- Link the Installation, Updating and Windows install docs pages
- The package update banner now copies the platform installer command
instead of unsloth studio update
* Studio: rounder account menu, inline system prompt box with popup from the label
- Account menu corners go from 14px to 18px via a specific override,
since list menus pin border-radius globally
- llama.cpp banner bottom padding 22px
- System prompt is an inline editable textarea again; clicking the
System Prompt label opens the popup editor, and an overflowing
prompt opens it on box click
* Studio: show the standard install commands in the About update section
- Both one-line install commands (MacOS/Linux/WSL and Windows
PowerShell) are always shown, labeled like the docs, since running
them again updates an existing install
- Drop the unsloth studio update deprecation note
- Add the Mac install guide to the docs links
* Studio: clearer platform toggle and layout in the About update section
- Section heading is Update
- Platform picker is a pair of pill buttons, MacOS / Linux and Windows,
and only the selected platform's install command is shown
- Intro reads: To install or update Unsloth
- Local update heading separates checkout guidance from the standard
install command
* Studio: report GitHub branch instead of dev for source checkouts
A source checkout not on an exact release tag now shows
GitHub <branch> (e.g. GitHub main) as the Studio version in About.
Detached or unusual HEADs still fall back to dev.
* Studio: tighten the About update section copy and toggle styling
- Platform toggle buttons are borderless pills
- Shorter local update wording and restart note
- Docs links read Mac and Windows
* Studio: tighten line spacing in the sidebar account button
* Studio: fix vanishing compact MCP icon on hover, single line pill tooltips
- Compact caret pills (MCP, RAG) keep their icon on hover for inactive
pills too; the off switch hover rules hid the icon while compact mode
hid the X, leaving an empty slot
- Compact icon tooltips and single line compact tooltips render as full
pills; wrapped tooltips keep the 9px corners. TooltipContent measures
line count in a ref callback since Radix mounts portal content
without re-rendering the wrapper
- 1px gap between the name and Unsloth lines in the sidebar account
button
* Studio: Projects hover plus button, align recents with the label
- Hovering the Projects nav item reveals a plus button that opens the
New project dialog, with the same circular hover treatment as the
chat row actions
- Recent chat titles start at the same x as the Recents label
- The system prompt overflow lock only engages for a non-empty prompt
with a laid-out box, so a mis-measure cannot turn clicks into the
popup
* Clip system prompt overflow inside the rounded box
Wrap the inline system prompt textarea in a rounded overflow-hidden
surface so scrolled text and the scrollbar stay inside the box. The
focus ring moves to the wrapper via focus-within.
* Add updating progress bar to llama banner and shorten settings copy
While an update is applying, the banner action row becomes an
indeterminate progress bar that keeps animating under reduced motion,
matching the other loading indicators. Settings descriptions across
General, Profile, Appearance, Chat, Connections, API, and About are
trimmed without losing meaning.
* Address review: desktop update note, server platform detection, zh-CN keys
The About tab no longer shows terminal install commands in the desktop
app, where the bundled backend updates through the built-in updater;
it shows a short note and the docs links instead.
fetchDeviceType now sends the auth token to /api/health, which only
reports the server platform to authed callers, and caches only a
server-reported value. Copied install commands then match the host
platform rather than the browser when they differ (WSL, SSH).
zh-CN gains translations for the new notification and license keys,
the renamed About tab title, and the desktop update note.
* Real download progress for llama.cpp updates, prompt and sidebar polish
The update worker now streams the installer output and parses its
download percent lines into job progress, exposed via the update-status
API. The installer emits finer non-tty milestones when
UNSLOTH_PROGRESS_PERCENT_STEP is set; the worker requests 5 percent
steps. The banner renders a determinate bar from the reported fraction
and falls back to the sweep until the first percent arrives.
Also removes the focus ring on the inline system prompt box and
slightly shrinks the Projects hover plus icon.
* fix(rocm): stop overwriting ROCR_VISIBLE_DEVICES in apply_gpu_ids
ROCR_VISIBLE_DEVICES uses HSA agent-level indexing, not physical GPU
indices. Setting it to a bare integer breaks multi-GPU ROCm systems
where the parent already set ROCR_VISIBLE_DEVICES=0,1: narrowing to
1 causes torch.cuda.is_available() to return False in the training
worker, producing a misleading 'no HIP accelerator' error even on a
correctly configured ROCm host.
HIP_VISIBLE_DEVICES is sufficient for GPU selection on ROCm.
Leave ROCR_VISIBLE_DEVICES inherited from the parent environment.
* test(rocm): update apply_gpu_ids test to assert ROCR_VISIBLE_DEVICES is not overwritten
* Studio: auto Cloudflare tunnel for 0.0.0.0 launches
Binding Studio to 0.0.0.0 for remote access often leaves the raw
http://<ip>:<port> URL unreachable (https-vs-http, blocked high ports,
closed cloud security groups). On a wildcard bind, auto-start a free
cloudflared quick tunnel and show its https://*.trycloudflare.com URL in
the startup banner:
Secure link access via Cloudflare: https://<random>.trycloudflare.com
- new studio/backend/cloudflare_tunnel.py: find or download+cache the
cloudflared binary (per-OS/arch GitHub release, safe .tgz extract),
start the tunnel, parse the URL, tear it down. Stdlib only; best-effort
and non-fatal throughout (a missing binary or offline box never blocks
or slows startup).
- run_server starts the tunnel for 0.0.0.0 only (skips loopback, api-only
and Colab), prints the line in the banner, and _graceful_shutdown stops
the child so it never orphans.
- --cloudflare/--no-cloudflare flag (default on) on `unsloth studio` and
`unsloth studio run`, forwarded through the re-exec into run_server.
- tests for the helper, the CLI flag forwarding, and the run.py defaults.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio cloudflare: send a User-Agent on the cloudflared download
GitHub's CDN can 403 the default Python-urllib User-Agent on release asset
downloads. Set an explicit UA and pin it with a test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio cloudflare: address review (opt-out for subcommands, tunnel teardown)
- reject --no-cloudflare placed before a subcommand (it would not reach the
subcommand), mirroring the --parallel guard
- register the tunnel before waiting for its URL so a shutdown during the wait
stops cloudflared instead of orphaning it
- tear the server + children down if `unsloth studio run` startup aborts
(health timeout, model-load error, Ctrl+C) before the wait loop
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: offer the in-app llama.cpp update for source-build (markerless) installs
Source-build installs have no UNSLOTH_PREBUILT_INFO.json marker, so freshness
reported supported=False and the Update button never showed (notably on macOS,
where the fork shipped no prebuilt before b9585 and setup fell back to a source
build). When an install has no marker but an official prebuilt now exists for
the host, surface the update and let one click swap it in place.
- install_llama_prebuilt.py: published_repo_for_host() (the setup.sh host->repo
rule in Python) and a --resolve-prebuilt mode that reports whether a prebuilt
exists for this host without downloading.
- llama_cpp_update.py: markerless branch in get_update_status/start_update,
version-suppressed so source builds already newer than latest are not nagged;
fail-open throughout.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: run llama update detection off the event loop, expose source_build
The markerless source-build check probes the host and reads GitHub, so run
get_update_status and start_update in a worker thread to keep the API
responsive. Expose source_build in the status response so the banner can label
the source-build switch.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the llama-route auth stub out of sys.modules for the rest of the suite
test_llama_route.py replaced sys.modules['auth.authentication'] with a
bare stub at collection time and never restored it, so every later test
importing create_access_token got the stub: 17 failures across
test_desktop_auth, test_middleware, test_openai_tool_passthrough and
test_rag_preview on all four Backend CI Python versions. Import the
real module when its deps are available and only stub in minimal envs,
popping the stubs after the standalone route load either way.
* Studio: address review on the source-build update path
- published_repo_for_host: route CPU-only Windows to ggml-org too (mirrors
setup.ps1; the fork ships no win-cpu bundle), macOS always the fork.
- markerless detection compares/display the upstream llama_tag, not a possible
fork wrapper release_tag, so a source build is not wrongly judged newer.
- do not offer when there is no resolvable install root (a pinned
LLAMA_SERVER_PATH outside a managed dir): an apply would not take effect.
* Ignore version probes in the update tests' subprocess capture
The status polls in these tests trigger the new source-build detection,
which shells out to llama-server --version through the same patched
subprocess.run. On slow runners that probe lands after the installer
call and clobbers the single captured argv, failing the flag
assertions (seen on the 3.10/3.11 Backend CI jobs). Skip probe calls
in all three fakes so only the installer invocation is captured.
* Skip markerless re-detection while the update job is swapping the tree
On a source-build install the frontend polls update-status every 3s
during an apply, and each poll ran _source_build_status, which execs
the very llama-server binary the job is concurrently replacing. On
Windows that exec can hold the exe long enough to fail the installer's
os.replace; everywhere it is a per-poll subprocess spawn for a status
the poller does not read (it only consumes job progress). Gate the
markerless branch on the job not running; the marked path is probe-free
and still returns the live job state.
* Studio: tighten source-build update root, repo routing, and downgrade guard
Only manage a markerless install when the active binary lives under a
resolvable llama.cpp root (marker dir, UNSLOTH_LLAMA_CPP_PATH it sits in,
or a llama.cpp ancestor); a pinned LLAMA_SERVER_PATH or a PATH/system
binary is left alone so an apply cannot install where it would not take
effect. Gate start_update on the same suppression as detection so a
direct POST cannot downgrade a source build newer than the latest
prebuilt. Route Linux hosts with AMD tooling (rocminfo/amd-smi/hipconfig/
hipinfo) to the fork in --resolve-prebuilt, matching setup.sh, so a HIP
source build is not offered an upstream CPU prebuilt.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cover inactive env root and pinned llama.cpp checkout in update root tests
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Adds an in-app "Update llama.cpp" banner and button to Unsloth Studio. When the installed prebuilt is behind the latest published release, a non-invasive banner appears; clicking Update downloads the latest prebuilt for this host and swaps it in place in the background, with no restart.
Detection reuses the freshness check from #5529. The update re-runs install_llama_prebuilt.py the same way setup.sh and setup.ps1 do after #5963: it forwards the published repo and the AMD gfx target derived from the install marker, and does not pass the removed --simple-policy or the arm64-only --cpu-fallback.
While the installer swaps binaries the backend enters a maintenance state (flag set under the serial load lock, active server unloaded) so a concurrent load cannot start a server from a half-swapped binary; the next load uses the new build. The banner also handles refused responses and jobs started in another tab so it never sticks on "Updating...".
Verified end to end on an NVIDIA B200: installed b9493, detected the update, applied it, and confirmed the binary at the same path advanced to b9585 in the same process. Hermetic backend tests and the frontend type-check pass.
* Studio: support separate-file MTP GGUF drafters (Gemma 4)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix review findings for separate-file MTP drafters
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: pair local MTP drafters by name and include them in reload dedup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: manage --model-draft in extras and reject MTP/ copies as models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix Studio Python, Gemma 4 Unified sidecar, and worker crash messages
* Clean up Gemma 4 sidecar test patch contexts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Polish inference worker crash message
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address transformers tier review feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Route Gemma 4 assistant models to transformers 5.10
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: training survives a non-writable HF datasets cache
A shared HF datasets cache can contain subtrees owned by another user
(for example populated by an earlier root-run job). datasets then dies
with "[Errno 13] Permission denied: ..._builder.lock" while locking
the cached builder and the training run fails. load_dataset in the
training worker and trainer now goes through a wrapper that catches the
EACCES and rebuilds the dataset in a Studio-owned cache under
cache_root()/hf-datasets, logging the fallback.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope the HF_DATASETS_CACHE override to the fallback load
* Route non-streaming dataset preview loads through the cache-safe wrapper
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix Windows installer winget msstore certificate failure
`winget install` was invoked without `--source winget`, so winget also
queried the msstore source. When msstore fails certificate pinning
(error 0x8a15005e, "The server certificate did not match any of the
expected values") winget aborts and demands `--source`, so the Python
(and uv) install fails even though the package exists in the winget
source.
- Pass `--source winget` to all winget install calls (Python x2, uv).
Both packages live in the winget source, so this is strictly correct
and skips the failing msstore round-trip entirely.
- Add a python.org fallback (Install-PythonFromPythonOrg) that downloads
the official installer and runs it silently per-user (no admin/UAC)
when winget is unavailable or fails for any reason. Mirrors the
existing uv -> astral.sh fallback so Python installs without manual
steps. Resolves the latest 3.13.x from python.org with a pinned
fallback, and selects the amd64/arm64/x86 installer per architecture.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Pin remaining setup.ps1 winget calls to --source winget
Two winget invocations in studio/setup.ps1 still queried all sources and
could hit the same msstore certificate-pinning failure (0x8a15005e) that
broke the Python install in install.ps1:
- `winget show Nvidia.CUDA --versions` (CUDA Toolkit version probe)
- `winget install ... ShiningLight.OpenSSL.Dev` (OpenSSL dev for llama-server)
Every other winget call in this file already passes `--source winget`
(Git, CMake, VS Build Tools, CUDA install, Node.js, and setup.ps1's own
Python 3.12 install), so these two were stragglers. Both packages live in
the winget source; pinning it makes setup robust to an unhealthy msstore
source, matching the rest of the file.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stop amd-smi GPU probe from popping a DiskPart UAC prompt
On Windows, AMD GPU detection in install.ps1 and studio/setup.ps1 runs
`amd-smi list` / `static --asic` / `version`. amd-smi (shipped in
System32 by the Adrenalin driver) auto-elevates to read GPU/APU memory
details, surfacing a confusing DiskPart UAC prompt mid-install. The
Studio backend already documents and circuit-breaks on this in
studio/backend/utils/hardware/amd.py, but the installers did not.
Add an Invoke-AmdSmiNoElevate helper (both scripts) that runs amd-smi via
Start-Process under __COMPAT_LAYER=RunAsInvoker so it cannot auto-elevate
(no prompt), with a 30s timeout (matching amd.py) so a flaky amd-smi
cannot stall the install for minutes. On failure/timeout the existing WMI
name -> gfx fallback still resolves the arch, so detection is unchanged on
working hosts.
Verified on a Strix Halo (Radeon 8060S / gfx1151) box: the prompt is gone
and the probe is bounded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add experimental ROCm-on-WSL setup helper for Strix Halo (gfx1151)
install.sh already routes gfx1151 (Radeon 8060S / Strix Halo) to the
repo.amd.com/rocm/whl/gfx1151 wheels once a ROCm runtime is present, but
it does not install AMD's driver/ROCm stack -- a large, admin-gated
prerequisite. scripts/install_rocm_wsl_strixhalo.sh automates the Linux
side on a dedicated Ubuntu 24.04 WSL2 distro: ROCm 7.2 (wsl usecase), the
rocr4wsl HSA runtime, a librocdxg build, env setup, and a PyTorch gfx1151
GPU smoke test. A hard preflight refuses to run until the Adrenalin
>=26.3.1 driver is actually present, so it cannot half-install.
Procedure adapted from AMD's ROCm-on-WSL docs and community gfx1151 notes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Detect AMD GPUs by name so native Windows gets a GPU llama.cpp
The gfx-arch inference from the WMI GPU name was gated behind $HasROCm,
which the hipinfo/amd-smi probe leaves false on the common Windows case
(Adrenalin driver only, no HIP SDK -- and amd-smi often cannot read the
arch without elevation). So an AMD GPU was detected by name but never
mapped to a gfx target, --rocm-gfx was not forwarded, and studio setup
fell back to a CPU llama.cpp build.
Un-gate the inference (install.ps1 + studio/setup.ps1) so it runs whenever
an AMD GPU name is available. The inferred gfx is forwarded as --rocm-gfx,
which makes install_llama_prebuilt.py download the matching lemonade-sdk
ROCm prebuilt (e.g. llama-bNNNN-windows-rocm-gfx1151-x64.zip) -- a
GPU-accelerated llama.cpp that bundles its own ROCm runtime, so it runs
with just the Adrenalin driver. PyTorch's ROCm wheels still require a
confirmed HIP SDK ($HasROCm), so this only affects llama.cpp / inference
and never pulls broken ROCm torch.
Also broaden the name->arch table to every family lemonade ships Windows
assets for: gfx120X (RDNA 4), gfx110X (RDNA 3), gfx1151/gfx1150
(RDNA 3.5), and gfx103X (RDNA 2). Unknown names still fall back to CPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Suppress amd-smi DiskPart UAC prompt in the Python install/runtime paths
The earlier PowerShell guard covered install.ps1 / setup.ps1, but the
Python installer (install_llama_prebuilt.py detect_host,
install_python_stack.py ROCm probes) and the Studio backend monitor
(amd.py) also shell out to amd-smi on Windows, where it auto-elevates and
pops the same DiskPart UAC prompt mid-install / at runtime.
Inject __COMPAT_LAYER=RunAsInvoker into the amd-smi subprocess env on
Windows so it runs un-elevated (no prompt). Callers already tolerate an
empty/failed result and fall back to WMI / name detection (installer) or
the existing circuit breaker (amd.py). Gated to Windows so Linux/macOS
amd-smi behaviour is unchanged.
- install_llama_prebuilt.py: handled centrally in run_capture (covers
detect_host's `amd-smi list` and the version probe).
- install_python_stack.py: new _amd_smi_env() helper on its 3 raw
subprocess.run amd-smi calls.
- amd.py: merge RunAsInvoker into the existing child env.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Tighten AMD GPU name->arch patterns to avoid mismatches
The W9[0-9]{3} and RX 90[0-9]{2} patterns added for RDNA 4 were
speculative and over-broad: W9xxx would also match old GCN FirePro
W9100/W9000 cards (wrong gfx1201 -> a lemonade gfx120X download that
fails validation), and RX 90[0-9]{2} was redundant with the explicit
9070/9060 entries. Drop both; keep only confirmed RDNA 4 SKUs. Unmatched
AMD names still fall back cleanly to CPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fetch the llama.cpp validation model via huggingface_hub
The prebuilt validation downloads a tiny GGUF test model from huggingface
via bare urllib. On Windows / proxy setups where the server sends an
incomplete TLS chain, urllib cannot complete the Amazon CA chain (it does
no AIA intermediate fetching) and fails with CERTIFICATE_VERIFY_FAILED, so
a perfectly good GPU prebuilt is rejected and the installer falls back to a
CPU source build.
Route the validation-model download through huggingface_hub
(hf_hub_download) -- the same mechanism Studio uses for model downloads,
which completes the chain where urllib cannot -- keeping the direct URL as
a fallback. This lets the lemonade ROCm prebuilt validate and install on
cert-restricted machines (verified: hf_hub_download succeeds where urllib
returns CERTIFICATE_VERIFY_FAILED).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Guard the remaining raw amd-smi version probe via run_capture
A ROCm-version detector in install_llama_prebuilt.py called amd-smi version through a raw subprocess.run that bypassed run_capture's Windows RunAsInvoker guard, so it still triggered the DiskPart UAC prompt during setup. Route it through run_capture like the other amd-smi calls.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Forward --rocm-gfx even when the ROCm runtime is unconfirmed
setup.ps1 forwarded --rocm-gfx (and picked the windows-hip llama.cpp
prebuilt) only inside `if ($HasROCm)`. On Adrenalin-only hosts (amd-smi
present but no HIP SDK, so $HasROCm stays false) the gfx arch was
name-inferred but never forwarded, so install_llama_prebuilt.py saw
has_rocm=False and installed the CPU build -- even though the lemonade
gfx1151 GPU prebuilt runs fine there (it bundles its own ROCm runtime;
verified: llama-cli --list-devices -> ROCm0: AMD Radeon 8060S, 69 GB).
Forward --rocm-gfx whenever a gfx arch is known (it is authoritative and
implies ROCm in install_llama_prebuilt.py), and treat a known gfx arch as
windows-hip in the existing-install mismatch check. --has-rocm stays gated
on the confirmed-runtime signal.
Verified on Radeon 8060S / gfx1151: the installer now selects, validates,
and installs llama-b1286-windows-rocm-gfx1151-x64.zip (ROCm DLLs present)
instead of the CPU build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Install AMD ROCm PyTorch on name-inferred gfx hosts (enables Train/Export)
setup.ps1 picked the AMD ROCm PyTorch wheels only inside `if ($HasROCm ...)`.
On Adrenalin-only hosts (amd-smi present but no HIP SDK, so $HasROCm is
false) the gfx arch was name-inferred but the ROCm-wheel branch never ran,
so the host got torch+cpu. With CPU torch, torch.cuda.is_available() is
False, so the Studio backend sets CHAT_ONLY=True and hides Train/Export.
Un-gate the ROCm PyTorch index resolution on a known gfx arch (mirrors the
llama.cpp --rocm-gfx fix). AMD's per-arch Windows wheels
(repo.amd.com/rocm/whl/<gfx>) bundle the ROCm runtime, so they work without
a HIP SDK; a failed install still falls back to CPU.
Verified on Radeon 8060S / gfx1151: torch 2.11.0+rocm7.13.0 installs and
torch.cuda.is_available() -> True, device "AMD Radeon(TM) 8060S Graphics",
GPU matmul OK -> CHAT_ONLY=False -> Train/Export enabled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Force amd-smi un-elevated process-wide in the Python installers
Guarding individual amd-smi call sites kept missing some (install_python_stack.py's probe loop and its Windows GPU re-check), so the DiskPart UAC prompt kept reappearing. Set __COMPAT_LAYER=RunAsInvoker process-wide at the top of install_python_stack.py and install_llama_prebuilt.py on Windows so every amd-smi subprocess (current and future) runs un-elevated with no per-call guard. Safe: these scripts only spawn amd-smi/rocminfo/hipinfo probes and pip/uv. setup.ps1 keeps per-call guards because it also spawns winget installers that need elevation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix Invoke-AmdSmiNoElevate exit code on PS 5.1 + RX 7700S arch match
Start-Process -PassThru leaves the returned process object's .ExitCode
$null after WaitForExit on Windows PowerShell 5.1, so the helper set
$LASTEXITCODE to $null and every caller's `if ($LASTEXITCODE -eq 0 ...)`
was always false -- the amd-smi GPU / gfx-token / ROCm-version detection
branch was effectively dead (masked only because the un-gated WMI
name->gfx inference still ran). Reproduced on PS 5.1.26100.
Rewrite the helper to use [System.Diagnostics.Process]::Start with a
ProcessStartInfo (UseShellExecute=false), whose .ExitCode is reliable,
with async stream reads (ReadToEndAsync) to avoid a pipe-buffer deadlock
and WaitForExit(timeout) to bound a flaky amd-smi. __COMPAT_LAYER=
RunAsInvoker (inherited via the process env) still suppresses the
auto-elevation / DiskPart prompt. Also drops the temp files and the
empty-ArgumentList edge case. Verified: exit code propagates
(7 -> $LASTEXITCODE=7), output captured, env restored.
Also fix the gfx1100 name pattern `RX 7700(?! S)` -> `RX 7700(?!S)` so the
spaceless retail name "RX 7700S" is correctly excluded (it belongs to the
gfx1102 row). Both found by PR review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address PR review follow-ups (install.sh table, update path, tests, WSL)
From the multi-agent PR review:
- install.sh: sync the AMD name->arch table with install.ps1 / setup.ps1
(the bash table had drifted to the old narrow patterns). Adds RDNA 2
(gfx103X), workstation PRO W SKUs, and more Strix Halo/Point names, and
orders gfx1102 before gfx1100 so the spaceless retail name "RX 7700S"
resolves correctly (bash case has no negative lookahead). AMD-ROCm-only:
the name inference stays gated behind _has_amd_rocm_gpu(), so NVIDIA /
CPU / macOS are unaffected.
- setup.ps1: the "dependencies up to date" fast path skipped the torch
reinstall, so an existing user who had CPU torch (installed before
ROCm-wheel support) stayed stuck in CHAT_ONLY. Now, when an AMD gfx arch
is known AND the installed torch is CPU-only, don't skip -- force the
dependency pass so the ROCm wheels install.
- scripts/install_rocm_wsl_strixhalo.sh: resolve the real /opt/rocm dir
instead of hardcoding ROCM_VER for LD_LIBRARY_PATH / the librocdxg
symlink (breaks if amdgpu-install lays ROCm under a patch-version dir);
add a LIBROCDXG_REF pin knob and a "verified against" freshness header.
- tests/studio/install/test_pr5940_followups.py: cover _hf_resolve_url_parts,
_fetch_validation_model_bytes (hf path + urllib fallback), run_capture's
Windows-only amd-smi RunAsInvoker injection, and install.ps1 vs setup.ps1
name-table parity (catches future drift). 14 tests, all passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix DiskPart UAC prompt: skip amd-smi on Windows without a HIP SDK
On Windows, amd-smi re-initialises the ROCm runtime on every invocation
(even `amd-smi version`) and, on hosts without a working HIP runtime
(consumer APUs/dGPUs with only the Adrenalin driver), elevates a child
process at runtime -- popping a UAC/DiskPart prompt. amd-smi's own
manifest is asInvoker, so __COMPAT_LAYER=RunAsInvoker cannot suppress
that runtime elevation (verified: even `amd-smi version` hangs and
times out with RunAsInvoker set).
Replace the ineffective RunAsInvoker-only approach with a real gate:
only spawn amd-smi on Windows when a HIP SDK is detectable (hipinfo
present, so amd-smi runs un-elevated) or the user opts in with
UNSLOTH_ENABLE_AMD_SMI=1. The gfx arch is already resolved from WMI
name inference (forwarded via --rocm-gfx), so ROCm wheel + lemonade
llama.cpp selection is unaffected. Linux/macOS amd-smi never elevates
and is untouched (no regression). RunAsInvoker is kept as harmless
belt-and-suspenders for tools that DO use manifest elevation.
Applied consistently across:
- studio/backend/utils/hardware/amd.py (runtime GPU polling)
- install.ps1, studio/setup.ps1 (install-time detection)
- studio/install_llama_prebuilt.py (prebuilt arch probe + version)
- studio/install_python_stack.py (ROCm version + arch probe)
Verified live on AMD Radeon 8060S (gfx1151), native Windows: fresh
install detects the GPU, installs ROCm torch (torch.cuda.is_available()
True), launches Studio with no DiskPart prompt, and inference, tool
calling, web search, LoRA finetuning, and GGUF export all run on the GPU.
Tests: add 6 _amd_smi_allowed() gating tests + PowerShell-installer gate
assertions; update the three amd-smi monitoring tests to opt in (they
mock amd-smi as available). Full suite: 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: helpful WSL message when the GPU isn't exposed to ROCm
In WSL, an AMD GPU's ROCm-on-WSL runtime is only available with a recent
Adrenalin driver AND a distro AMD supports (currently Ubuntu 24.04). When
neither is in place, GPU detection (rocminfo/_has_amd_rocm_gpu) finds
nothing and we silently fall back to CPU.
Add an actionable hint in the CPU-fallback path, shown only on WSL and
only AFTER detection has already failed -- so it is forward-compatible:
the moment a driver/distro DOES expose the GPU (e.g. if AMD later adds
Ubuntu 26.04 support), detection succeeds and the hint never fires. The
message:
- notes a GPU is plumbed in (/dev/dxg) but no ROCm runtime is exposed,
- lists the two prerequisites (Adrenalin driver + Ubuntu 24.04),
- if the distro is not 24.04, says AMD may not support it yet,
- tells the user to `wsl --install Ubuntu-24.04` and re-run,
- links AMD's ROCm-on-WSL guide + the experimental Strix Halo helper.
Verified live: on Ubuntu-24.04 the hint shows (version-warning omitted)
and the CPU install completes; on Ubuntu-26.04 the extra "this distro may
not be supported" line appears and points to 24.04.
Also fix the experimental scripts/install_rocm_wsl_strixhalo.sh: AMD's
repo.radeon.com/amdgpu-install/ is indexed by unified installer version
(30.30, 31.30, ...), NOT ROCm version, so the hard-coded
amdgpu-install/7.2.0/ path 404'd. Scan the installer dirs newest-first
for a noble .deb matching the target ROCm major.minor (ROCm 7.2 ->
30.30.x/amdgpu-install_7.2.x), falling back to the newest available.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* WSL: fix shortcut collision + pin ROCm-on-WSL driver reqs from AMD docs
Two WSL-related fixes informed by AMD's official ROCm-on-WSL docs and
field reports for Strix Halo / Ryzen AI Max+ (Radeon 8060S, gfx1151):
1. Shortcut collision (real bug). install.sh's WSL branch wrote
"Unsloth Studio.lnk" to the SAME Desktop / Start Menu folder as the
native-Windows installer (install.ps1 New-StudioShortcuts). Running
install.sh in WSL therefore silently retargeted the native shortcut at
the WSL launcher (wt.exe -> wsl.exe), so the desktop/start-menu icon
stopped launching native GPU Studio. Now the WSL shortcut uses a
DISTINCT name -- "Unsloth Studio (WSL - <distro>).lnk" -- and fetches
the Unsloth .ico to %LOCALAPPDATA%\Unsloth Studio so it shows the
proper icon. Native and WSL shortcuts now coexist.
2. Precise ROCm-on-WSL prerequisites. Research (AMD radeon-ryzen WSL
compatibility matrix, gianni.rosagallina.com Feb-2026 guide,
ROCm/ROCm#4952/#5509/#6022) confirms WSL GPU on Strix Halo requires
AMD Adrenalin Edition >= 26.1.1 (26.2.2+ is the first production
ROCDXG/WSL release) + ROCm 7.2.1 + Ubuntu 24.04; an older driver does
not inject the ROCm/DXG runtime into /usr/lib/wsl/lib, so rocminfo sees
only the CPU. install.sh's WSL hint and the experimental
install_rocm_wsl_strixhalo.sh header/preflight now state the exact
driver version (was a guessed ">=26.3.1"), bump ROCM_VER to 7.2.1, link
AMD's radeon-ryzen docs, and document the known librocdxg caveat that
usable VRAM is currently capped at the .wslconfig memory setting.
bash -n clean; install test suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: hint when the AMD driver is too old for ROCm-on-WSL
Adds a detect-and-guide hook for the optional WSL-GPU path. An AMD GPU on
native Windows can also be used inside WSL2, but only with AMD Adrenalin
Edition >= 26.2.2 (the first production ROCDXG/WSL release). Native Windows
GPU works with any recent driver, so this is purely about enabling the WSL
path.
We intentionally do NOT auto-install the driver: AMD referrer-gates driver
downloads (scripted curl/Invoke-WebRequest are blocked) and does not publish
Adrenalin via winget, so no installer can reliably fetch it -- and silently
swapping a live display driver is risky. Instead we point the user at AMD's
official download page (one click), after which the existing WSL detection
lights up automatically.
- install.ps1: new Show-AmdWslDriverHint -- when an AMD GPU is present and the
installed driver predates the 26.2.2 release (DriverDate < 2026-02-01),
print a concise tip with the AMD download URL. Handles DriverDate as either
a CIM DateTime or a WMI string. Suppress with UNSLOTH_SKIP_AMD_DRIVER_HINT=1.
- install.sh (WSL hint): add the direct Adrenalin 26.2.2 download URL and note
that AMD downloads are referrer-gated (open in a browser).
Verified: hint fires on a Sept-2025 driver, auto-suppresses on >= 2026-02-01;
install.ps1 parses; install.sh bash -n clean; suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.ps1: refresh shell icon cache after creating the shortcut
After writing the Desktop / Start Menu .lnk, nudge Explorer to refresh
its icon (ie4uinit.exe -show). Without this, a stale icon cache can show
a blank shortcut icon until the next explorer restart -- most visible
when a shortcut of the same name was rewritten (e.g. a native install
followed by a WSL install, which previously shared the name; now they use
distinct names, but the cache nudge makes the icon appear immediately
regardless). Best-effort and wrapped in try/catch so it never fails the
install. The bundled unsloth.ico itself is valid (verified it renders).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* setup.ps1: don't silently CPU-build llama.cpp on an AMD GPU
For AMD, GPU acceleration comes from the lemonade ROCm prebuilt (it bundles
the ROCm runtime, no HIP SDK needed) and is the preferred/default path. The
source-build fallback is CPU-only -- a HIP/ROCm *source* build would need the
full HIP SDK + ROCm clang toolchain, which the prebuilt exists to avoid.
Previously, if an AMD-GPU host ever fell through to the source build (e.g. the
prebuilt could not be downloaded), it printed "building llama.cpp (CPU-only,
no NVIDIA GPU detected)" and quietly produced a CPU binary -- masking the lost
GPU acceleration. Now that case emits a loud [WARN] explaining the GPU prebuilt
is the AMD path and how to restore it (re-run / check network / set
UNSLOTH_LLAMA_RELEASE_TAG), so AMD never silently degrades to CPU.
No behavior change on the happy path: AMD still gets the GPU prebuilt (verified
on gfx1151: ggml-hip.dll bundled, ~80% GPU compute during inference). NVIDIA
(CUDA source build) and CPU-only hosts are unchanged.
setup.ps1 parses; install suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: remove shared llama.cpp build, kill lock-holders, match WSL shortcut
Three gaps found by running a real uninstall on a native-Windows + WSL host;
all fixes are scoped to Unsloth-owned paths and no-op on the other pathways
(env/custom-root, NVIDIA/AMD/CPU, Mac) so nothing else regresses.
uninstall.ps1:
- Remove the default-mode SHARED llama.cpp build + cache. setup.ps1 installs
them at ~/.unsloth/llama.cpp and ~/.unsloth/.cache -- SIBLINGS of studio,
not under it -- so deleting <studio> left hundreds of MB behind. Now removed
explicitly, then ~/.unsloth is dropped ONLY if empty (never nukes unrelated
content). No-op in env/custom mode (llama.cpp nests under the custom root,
removed already) and when absent. UNSLOTH_LLAMA_CPP_PATH (user-owned) is kept.
- New _StopProcessesLockingRoots: _StopStudioProcesses only matched the venv
unsloth/python/studio exe, so it missed (a) llama-server.exe under llama.cpp
and (b) an orphaned multiprocessing python fork that ran from the SYSTEM
python but loaded a venv DLL (bitsandbytes) -- on Windows an open DLL handle
blocks the directory delete, leaving a half-removed install. The new helper
kills any process whose image path OR loaded module is under a target root
(module scan scoped to python/unsloth/llama-server names; vendor-agnostic).
- _RemovePath now retries (transient post-kill handle release).
uninstall.sh:
- Remove the default-mode ~/.unsloth/llama.cpp + ~/.unsloth/.cache; rmdir
~/.unsloth only if empty.
- WSL Windows-side shortcut cleanup now matches by TARGET (any
"Unsloth Studio*.lnk" whose target launches wsl.exe), covering both the
legacy "Unsloth Studio.lnk" and the new "Unsloth Studio (WSL - <distro>).lnk"
-- and never removes a native-Windows shortcut (which launches wscript.exe).
uninstall.ps1 parses; uninstall.sh passes sh -n and bash -n.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.ps1: invalidate Win11 Start Menu tile cache after creating shortcut
The Start Menu shortcut kept showing a blank/generic icon even after the
Explorer icon-cache rebuild, because Windows 11's StartMenuExperienceHost
keeps its OWN pre-rendered tile-icon cache
(%LOCALAPPDATA%\Packages\Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy\
TempState\TileCache_*.bin + StartUnifiedTileModelCache.dat), separate from
Explorer's iconcache_*.db. ie4uinit and an explorer.exe restart do not touch
it, and they don't recycle the host -- so a rewritten same-name shortcut keeps
showing the first-rendered (often the generic wscript ">") tile until the host
restarts on its own.
Fix: after creating the shortcut, drop only the Start Menu RENDER caches
(TileCache_* + StartUnifiedTileModelCache.dat) and stop StartMenuExperienceHost
(Windows auto-relaunches it), so the tile re-resolves the real icon via the
shell image factory. start2.bin (the user's pinned layout) is deliberately
preserved. Guarded by Test-Path (Windows 10 has no such host -> skipped) and
wrapped in try/catch so it can never fail the install. Windows-only
(install.ps1); no effect on Linux/macOS/Studio.
Verified live: rendering the shortcut via IShellItemImageFactory::GetImage (the
API StartMenuExperienceHost uses) returns the Unsloth sloth icon, color-matched,
after this invalidation -- previously it returned the generic script tile.
install.ps1 parses; install suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ROCm-on-WSL for AMD Strix Halo (gfx1151): auto-setup + runtime enablement
Make Unsloth Studio set up ROCm-on-WSL automatically for AMD Strix Halo
(Radeon 8060S / gfx1151) and use the GPU at runtime, validated end-to-end
on a Ryzen AI Max+ PRO 395 (ROCm 7.2.1 + librocdxg + Adrenalin Apr-2026):
rocminfo enumerates gfx1151, torch.cuda True, ~85.8 GB UMA pool.
Every change is a strict no-op for all other configs (NVIDIA/CUDA,
discrete + native-Linux AMD ROCm, macOS/MLX, Windows, CPU-only, non-Strix
WSL) and can never abort the installer.
- scripts/install_rocm_wsl_strixhalo.sh: rewrite to the validated recipe.
Fixes that would have broken a working box: drop the /usr/lib/wsl/lib
preflight (a working ROCDXG host has only d3d12/dxcore there); remove the
obsolete rocr4wsl step (gone from the 7.2.1 repo; would hard-fail and also
rips out the standard hsa-rocr ROCDXG needs); dynamic librocdxg soname
(was hardcoded 1.1.0; build is 1.2.0); direct apt-repo install; Windows
SDK auto-discovery; persist env to /etc/profile.d + ~/.bashrc; idempotent.
- install.sh: _maybe_bootstrap_rocm_wsl auto-offers/runs the helper when it
detects a Strix Halo APU in WSL (/dev/dxg) with no ROCm runtime, then
loads the env so detection routes to the gfx1151 wheels. Fast-path when
already configured. Fix an inaccurate WSL hint line.
- studio/backend/main.py + worker.py: set HSA_ENABLE_DXG_DETECTION=1
in-process before torch (gated on /dev/dxg AND librocdxg.so), so the
worker uses the GPU even when launched outside a login shell. Mirrors the
existing BNB_ROCM_VERSION injection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: clean up ROCm-on-WSL artifacts + Start Menu tile cache
- uninstall.sh: remove the ROCm-on-WSL helper artifacts -- the librocdxg
build clone (~/.unsloth/librocdxg, which otherwise blocks the empty-dir
rmdir of ~/.unsloth), the throwaway smoke-test venv, the persisted env
(/etc/profile.d/unsloth-rocm-wsl.sh) and the ~/.bashrc block. The system
ROCm userspace is a shared prereq like CUDA and is kept by default;
UNSLOTH_UNINSTALL_ROCM=1 removes it too. No-ops on macOS / non-Strix Linux.
- uninstall.ps1: invalidate the Win11 Start Menu tile cache after removing
the shortcut so its tile disappears promptly (mirrors install.ps1),
preserving start2.bin.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: accurate AMD ROCm messaging (HIP SDK optional, not required)
The Windows installer printed "HIP SDK not found - GPU-accelerated training
unavailable" / "ROCm wheels require the HIP SDK" whenever the HIP SDK was
absent. That is misleading: for a detected AMD GPU arch (gfx1151 etc.),
setup.ps1 installs AMD's bundled-runtime ROCm PyTorch wheels (repo.amd.com)
which ship their own ROCm runtime and do NOT need the HIP SDK -- verified
end-to-end (torch 2.11.0+rocm7.13.0, cuda True, QLoRA training on GPU) on a
Radeon 8060S with no HIP SDK installed.
Gate the GPU-detection + rocm-step messages on a detected gfx arch: when one
is known, state that GPU PyTorch uses bundled-runtime wheels and the HIP SDK
is optional; only when the arch is unknown fall back to the HIP-SDK hint.
Behavior (torch routing) is unchanged; this is messaging only. No-op for
NVIDIA/CUDA, HIP-SDK-present, and CPU paths (they hit earlier branches).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: fix /opt/rocm data-loss + make WSL shortcut create/remove interop-robust
Two fixes from the 3-reviewer regression audit + live testing on a
systemd-enabled WSL distro (interop disabled):
F1 (data-loss, install_rocm_wsl_strixhalo.sh): the /opt/rocm symlink-repair
could force-delete a pre-existing REAL ROCm install. The guard only checked
that /opt/rocm is a real directory, not that it is the stray librocdxg stub.
Now it only touches /opt/rocm when it is NOT a real install (no bin/rocminfo,
bin/hipcc, or .info/version present), and MOVES it aside (rocm.unsloth-stub-bak)
instead of deleting it, so a wrong guess can never lose data.
WSL interop robustness (install.sh + uninstall.sh): both relied on
`command -v powershell.exe`, which is true even when WSL interop cannot EXECUTE
it (on systemd distros powershell.exe fails with "Exec format error"). Result:
the WSL shortcut silently failed to create (install) and to remove (uninstall).
- uninstall.sh: test that powershell.exe actually runs; if not, remove the
"Unsloth Studio (WSL...).lnk" files directly via drvfs (/mnt/<drive>), which
works without interop. The name is WSL-install-specific, so a native install's
"Unsloth Studio.lnk" is never touched.
- install.sh: when the shortcut cannot be created, warn with the manual launch
command + how to re-enable interop, instead of failing silently.
No behavior change on the interop-on path. The regression audit otherwise found
no regressions on Linux/Mac/Windows/CPU/NVIDIA install paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.sh: fast-path fully restores ROCm-on-WSL env when the drop-in is gone
Reinstall regression found by uninstall->reinstall testing: after a Studio
uninstall that removed /etc/profile.d/unsloth-rocm-wsl.sh but KEPT the shared
ROCm (the default), a non-login reinstall hit the bootstrap fast-path
(librocdxg present) and its else-branch only set HSA_ENABLE_DXG_DETECTION --
NOT PATH/LD_LIBRARY_PATH. So rocminfo was not on PATH, GPU detection failed,
and the installer fell back to CPU-only PyTorch.
Fix: when librocdxg is present but the env drop-in is missing, restore the
FULL env inline (HSA + TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL + PATH +
LD_LIBRARY_PATH) so rocminfo is found and detection routes to the GPU, and
recreate /etc/profile.d/unsloth-rocm-wsl.sh so future shells and the Studio
worker get it too. No change to the env-present fast-path or any other host.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: clear Explorer icon cache so shortcut icons aren't blank
Root cause of the persistent blank Desktop + Start Menu icons: Explorer caches
each shortcut's icon in iconcache_*.db and does NOT re-read the .ico when a
same-name .lnk is recreated across reinstalls. The .ico and .lnk are correct
(the shell renders them non-blank via IShellItemImageFactory; the .ico has real
image data at 16/32/48/128 px), but the stale cache entry wins. The previous
fix only ran a weak `ie4uinit -show` + the Start Menu tile-cache clear -- it
never invalidated Explorer's icon cache, so the desktop icon stayed blank.
Fix (native install.ps1 New-StudioShortcuts AND the WSL shortcut path in
install.sh):
- ie4uinit -ClearIconCache (thorough; replaces -show as the primary refresh)
- SHChangeNotify(SHCNE_ASSOCCHANGED) to force a live desktop/taskbar refresh
WITHOUT restarting explorer
- keep the Win11 Start Menu tile-cache invalidation (and add it to the WSL
shortcut path too, preserving start2.bin)
Non-disruptive (no explorer restart). install.ps1 parses clean; install.sh
passes bash -n + dash -n; the heredoc-generated WSL PowerShell parses clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: per-item SHChangeNotify(UPDATEITEM) reliably fixes blank icons
The blank Desktop/Start Menu shortcut icons are a stale Explorer PER-ITEM icon
cache: when a same-name .lnk is recreated across reinstalls, Explorer caches the
previously-resolved (often generic "white page") icon for that item and won't
re-extract the .ico on its own. The .ico and the .lnk's IconLocation are correct
(every icon API renders the sloth) -- only Explorer's cached display is stale.
The previous refresh (ie4uinit -ClearIconCache + a GLOBAL SHCNE_ASSOCCHANGED
broadcast) does NOT recover a stale item -- confirmed by reproduction. The
reliable, NON-disruptive fix (no explorer restart) is a PER-ITEM
SHChangeNotify(SHCNE_UPDATEITEM, SHCNF_PATHW, <lnk path>) for each created
shortcut, which forces Explorer to re-read that exact item's icon.
Verified end-to-end: deliberately staled a shortcut to the generic icon, ran the
installer's exact new refresh code, and the sloth icon recovered with NO explorer
restart (confirmed by capturing the live desktop via PrintWindow).
Applied to both native install.ps1 (New-StudioShortcuts) and the WSL shortcut
path in install.sh. Still clears the on-disk icon cache (ie4uinit) and the Win11
Start Menu tile cache (preserving start2.bin).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: remove leftover llama.cpp .staging root so ~/.unsloth is cleaned
The llama.cpp atomic-install staging root (install_llama_prebuilt.py
INSTALL_STAGING_ROOT_NAME=.staging) is a sibling of the llama.cpp install
dir (~/.unsloth/.staging in default mode). It is normally pruned after a
successful activate, but an interrupted or retained build can leave a
<name>.staging-XXXX tree behind. The uninstallers removed llama.cpp and
.cache but not .staging, so the final empty-dir cleanup of ~/.unsloth failed
and the directory lingered. Reproduced on WSL (Ubuntu-24.04) where an empty
llama.cpp.staging-XXXX dir kept ~/.unsloth alive after uninstall.
Remove ~/.unsloth/.staging in both uninstall.sh and uninstall.ps1. No-op in
env/custom mode (staging nests under the custom root removed already) and
when absent. Cross-platform fix (the staging logic is platform-agnostic).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: WSL-absent hint + fix here-string lint false positive
install.ps1: in the AMD WSL-ROCm driver hint, detect when wsl.exe is absent
and add a one-line "wsl --install -d Ubuntu-24.04" pointer so a Strix Halo
user with no WSL yet gets an actionable next step (the hint previously assumed
an Ubuntu-24.04 distro already existed). Best-effort, informational only.
test_rocm_support.py: test_no_here_strings did a crude substring check that
false-positived on the conda-style block marker
printf '# <<< Unsloth ROCm-on-WSL (gfx1151) <<<' -- a string literal written
into the /etc/profile.d drop-in, also used as a sed delimiter pair by
uninstall.sh, not a here-string. Strip quoted spans before the check so the
lint still catches a real here-string operator but ignores quoted literals.
install.sh remains POSIX-clean (sh -n / dash -n / bash -n all pass).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer: address PR review comments (gfx1150 mapping, amd-smi opt-out, WSL bootstrap, SDK path, make)
Apply the valid bot review findings on #5940; reject the ones that don't hold.
Fixed:
- AMD name->gfx table (setup.ps1 + install.ps1): Radeon 890M and Ryzen AI 9 HX
370/375 are Strix POINT (gfx1150), not Strix Halo (gfx1151). Move 890M / HX 37x
/ AI 9 HX to the gfx1150 row and drop the bogus HX 38x pattern (no such Strix
Halo SKU). Matches the runtime classifier in worker.py (890M/880M -> gfx1150;
8060S/8050S -> gfx1151). Prevents Strix Point hosts from getting the wrong ROCm
prebuilt/wheels.
- amd-smi opt-out (setup.ps1 + install.ps1): an explicit UNSLOTH_ENABLE_AMD_SMI=
0/false/no/off now wins over the HIP-SDK heuristic, so a host with a HIP SDK
binary but a broken runtime no longer gets the DiskPart/UAC prompt the opt-out
exists to avoid.
- amd-smi warning probes (install_python_stack.py): _has_rocm_gpu and
_detect_amd_gfx_codes now gate amd-smi behind _amd_smi_allowed() (and pass
_amd_smi_env()), closing the last unguarded amd-smi spawn on Windows.
- WSL ROCm bootstrap (install.sh): the "already-usable ROCm?" early return now
requires rocminfo to enumerate the real gfx1151 agent instead of the generic
_has_amd_rocm_gpu (whose broad gfx[1-9][0-9] match accepts a fallback
"gfx11-generic" ISA), so a Strix Halo box missing the ROCDXG bridge is no longer
skipped. The shared helper is untouched (no gfx90a regression).
- install_rocm_wsl_strixhalo.sh:
* Quote-safe Windows SDK discovery: the old for-in-$(ls -d "...Program Files
(x86)/...") word-split on the space and never matched; use find + read loop.
* Add `make` to apt prereqs (cmake only recommends it; minimal images lacked it
and the librocdxg `make -j` build failed).
* Verification requires gfx1151 exactly (not gfx1[0-9]) so a generic ISA or an
unrelated RDNA GPU can't pass while the real GPU is absent.
Reviewed but NOT changed:
- "Forward inferred ROCm arch without HasROCm" (setup.ps1): already correct --
--rocm-gfx is forwarded under `if ($script:ROCmGfxArch)`, not `if ($HasROCm)`.
- "Route inferred arch into install.ps1 torch path": not a bug -- install.ps1
installs CPU torch as a base by design and setup.ps1 swaps in the ROCm wheel for
the inferred arch (gate `($HasROCm -or $ROCmGfxArch) -and cpu`); verified live
the native install ends on torch 2.11.0+rocm7.13.0.
- "$p null guard after Start-Process" (install.ps1/setup.ps1): redundant -- the
amd-smi runner uses [Process]::Start wrapped in try/catch, so a null process
already returns "" with LASTEXITCODE=1 (no uncaught exception).
- "ls -> find for /usr/lib/wsl/lib" (gemini): stale -- that heuristic was removed;
only a comment about it remains.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer(rocm-wsl): auto-install the Windows 11 SDK via winget (fewer manual steps)
librocdxg's build needs the Windows SDK 'shared' headers on the Windows host.
Previously the helper just die()d with "install the Windows 11 SDK and re-run" if
they were missing -- a manual prerequisite that broke the otherwise-seamless
`curl ... install.sh | sh` one-liner on Strix Halo.
Now, when the headers aren't found, the helper installs the Windows 11 SDK on the
Windows host from inside WSL via winget (powershell.exe interop), then
re-discovers them. The SDK installer elevates -> ONE UAC prompt on the Windows
desktop; the headers appear under /mnt/c immediately (drvfs is live, no reboot).
The user already consented to the ROCm-on-WSL setup, so no extra prompt is added
beyond the OS UAC gate.
- New _find_win_sdk (space-safe find of the newest installed SDK 'shared' dir)
and _install_windows_sdk_via_winget helpers.
- winget IDs tried newest-stable first: Microsoft.WindowsSDK.10.0.26100, then
.22621. The presence of the headers (re-check) is the source of truth, not
winget's exit code. </dev/null so winget never consumes a piped `curl|sh` stdin.
- Best-effort + non-fatal: interop-off / no-winget / declined-UAC all fall
through to the existing clear manual-install die(). Opt out with
UNSLOTH_SKIP_WIN_SDK_INSTALL=1.
Removes the last avoidable manual step from the WSL Strix Halo path; only the AMD
Adrenalin driver (AMD referrer-gates the download) remains manual. Verified
_find_win_sdk resolves the spaced "Program Files (x86)" path; bash -n clean; all
winget flags validated against `winget install --help`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer(amd): gate install-time amd-smi probe to fix DiskPart UAC prompt
install_python_stack.py's Windows "AMD GPU detected but ROCm torch missing"
warning probe ran `amd-smi list` whenever amd-smi was on PATH -- and amd-smi
ships in C:\Windows\System32 with the AMD Adrenalin driver -- without the
_amd_smi_allowed() gate that every other amd-smi call site in the file uses.
On Adrenalin-only hosts (no HIP SDK) amd-smi elevates a child at runtime and
pops a UAC/DiskPart prompt that __COMPAT_LAYER=RunAsInvoker cannot suppress
(amd-smi's manifest is asInvoker). The probe also ran before the
ROCm-torch-installed check, so it fired on every Windows AMD install.
Gate it behind _amd_smi_allowed() and pass _amd_smi_env(), matching
_has_rocm_gpu()/_detect_amd_gfx_codes(). When skipped, the only loss is the
best-effort "AMD GPU detected" note on HIP-SDK-less hosts.
Adds a per-function AST regression test asserting every function in
install_python_stack.py that names the amd-smi command and spawns a subprocess
also references _amd_smi_allowed() (flags the pre-fix code; passes after).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* studio(cli): fix `unsloth studio stop` crashing on Windows
`stop` used the POSIX `os.kill(pid, 0)` liveness probe, but on Windows
CPython raises OSError (WinError 87, "The parameter is incorrect") for
*every* pid -- alive or dead. `stop` only catches ProcessLookupError /
PermissionError, so the OSError propagated and the command crashed with
a traceback before ever reaching its (correct) `taskkill /F` path.
Add a cross-platform `_pid_alive(pid)` helper (tasklist on Windows,
signal-0 elsewhere) and use it for both the pre-check and the post-kill
wait loop. The actual kill path is unchanged.
Verified on Windows (Python 3.13): os.kill(pid,0) raises WinError 87 for
both a live and a dead pid; `_pid_alive` returns True/False correctly and
the full stop() flow (alive -> taskkill -> dead -> "stopped") passes
end-to-end against a throwaway process.
Adds tests/studio/test_cli_studio_stop_windows.py (AST guard against a
bare os.kill(pid,0) liveness probe + mock-only _pid_alive behaviour for
the win32 tasklist branch and the POSIX signal-0 branch).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer(amd): fix install.sh name->arch table misrouting Strix Point to gfx1151
The bash name->arch inference table in install.sh placed Strix Point
identifiers (Radeon 890M, "Ryzen AI 9 HX 370/375", "AI 9 HX") in the
gfx1151 (Strix Halo) row, diverging from the install.ps1 / setup.ps1
PowerShell tables which correctly map them to gfx1150. It also carried a
stray "HX 38" token absent from the PowerShell source-of-truth.
Align install.sh with the PowerShell tables:
gfx1151 row: 8060S|8050S|8040S|Strix Halo|Ryzen AI Max|AI Max
gfx1150 row: 890M|880M|860M|840M|Strix Point|Krackan|HX 37|AI 9 HX|...
Impact is low (the bash table only feeds the display label _gpu_disp_gfx
and the "set UNSLOTH_ROCM_GFX_ARCH=..." hint; wheel selection is driven
by the detected ROCm version, not this name string) but a Strix Point
user would otherwise see/copy the wrong gfx arch.
Add a parity test (test_install_sh_name_arch_agrees_with_ps_for_strix_and_non_amd)
that parses install.sh's case table and asserts Strix Halo->gfx1151,
Strix Point->gfx1150, RX 7700S->gfx1102, and NVIDIA/Intel->no match,
cross-checking against install.ps1 (the previous parity test only
compared install.ps1 <-> setup.ps1, missing install.sh).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.ps1: keep prebuilt-llama ownership guard within the test's block window
The AMD additions to the prebuilt-llama.cpp block (the windows-hip vs
windows-cpu existing-install kind validation) pushed the
install_llama_prebuilt.py invocation to ~1999 chars after the
"installing prebuilt llama.cpp bundle (preferred path)" anchor, right at
the edge of the 2000-char window that
test_setup_ps1_prebuilt_llama_cpp_has_ownership_guard slices -- so the
helper string was truncated and the test failed with "substring not
found" (CI: Repo tests (CPU)).
The ownership-guard invariant (Assert-StudioOwnedOrAbsent precedes the
install_llama_prebuilt.py call) was already satisfied; only the proximity
to the anchor regressed. Move the "installing prebuilt..." substep to
immediately before the install (after the existing-install pre-cleanup),
which also reads better (validate/clean existing -> then "installing"),
shrinking anchor->helper from 1999 to 413 chars. Behaviour is unchanged
(console message ordering only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.sh: auto-run Strix Halo ROCm-on-WSL setup by default
`curl -fsSL https://unsloth.ai/install.sh | sh` should make a Strix Halo
(gfx1151) GPU usable inside WSL with no extra commands. Previously the
ROCm-on-WSL bootstrap was opt-in: it required UNSLOTH_ROCM_WSL_AUTO=1 or an
interactive [Y/n] at a TTY, and silently skipped under a pipe (no /dev/tty),
so the piped one-liner never set the GPU up automatically.
Flip it to auto-by-default for the single narrow case the existing guards
allow (WSL + Strix Halo + /dev/dxg + no usable ROCm yet) -- exactly the GPU
setup the user ran the installer for. Opt out with
UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The Tauri desktop app keeps its own consent UI
(only auto-runs when it passes UNSLOTH_ROCM_WSL_AUTO=1). All hardware/OS
guards are unchanged, so non-Strix / non-WSL / NVIDIA / native-Linux / macOS /
CPU paths are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* PR comments: condense to be succinct (comments/docstrings only)
Shorten the verbose explanatory comments and docstrings this PR added across
the installer, scripts, backend shims, CLI, and tests -- tighter, fewer lines,
while preserving every non-obvious "why" (os.kill WinError 87, amd-smi
RunAsInvoker/UAC, /dev/dxg + librocdxg gating, the ROCm-on-WSL bootstrap guard
chain, ownership guards, etc.). No executable code, string literals, messages,
or behavior changed.
Verified comments-only: docstring-normalized AST equality (Python, 9 files),
non-comment token equality (PowerShell, 3 files), comment-stripped diff +
sh -n / bash -n (shell, 3 files). Behavior re-confirmed: get_torch_index_url +
gfx name->arch table 44/44 under dash & bash; rocm_support / pr5940_followups /
cli_studio_stop tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Installer: address PR review (amd-smi opt-out, pipefail, multi-distro, non-root)
Fixes valid findings from the Codex/Gemini PR review:
- install.ps1 / setup.ps1: gate the `amd-smi version` ROCm-version fallback with
$amdSmiAllowed so UNSLOTH_ENABLE_AMD_SMI=0 opt-out is honored (the device
probe was gated but this fallback wasn't), avoiding the DiskPart/UAC prompt.
- install_rocm_wsl_strixhalo.sh: make the post-verification rocminfo summary
best-effort (|| true) so head's early pipe-close under `set -o pipefail` can't
fail the bootstrap after gfx1151 was already enumerated; pin the Windows SDK
`winget install` to --source winget (matches the msstore-cert fix rationale).
- install.ps1: python.org fallback installs the py launcher per-user
(InstallLauncherAllUsers=0, avoids admin), and derives the fallback full
version from the requested minor so a non-default UNSLOTH_PYTHON (e.g. 3.12)
isn't silently replaced with 3.13 when the listing is unreachable.
- install.sh: recreate /etc/profile.d/unsloth-rocm-wsl.sh via `sudo tee` for a
non-root reinstall (a plain redirect failed silently, dropping the ROCm env).
- uninstall.sh: scope WSL Windows-side shortcut removal to the current
WSL_DISTRO_NAME (per-distro name or -d "<distro>" arg) so uninstalling one
distro no longer deletes other distros' launchers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Studio ROCm Windows: fix field-reported issues from Strix Halo testers
Four fixes from PR #5940 field reports (Win11 native, gfx1151):
1. bitsandbytes arch-probe spam: bnb's get_rocm_gpu_arch() runs
hipinfo.exe via subprocess PATH at import; the AMD torch wheel ships
hipInfo.exe in the venv Scripts dir, which is only on PATH for
activated venvs. Every bnb import logged "Could not detect ROCm GPU
architecture: [WinError 2]" ERROR + WARNING (even with the HIP SDK
installed, whose bin dir is not on PATH either). Prepend the Scripts
dir to PATH before bnb imports in main.py, worker.py, and
install_python_stack.py, gated on the file existing (only AMD wheels
ship it). Verified on gfx1151: ROCM_GPU_ARCH now resolves to gfx1151
with zero errors.
2. OOM-guard double-tax on native Windows unified APUs: mem_get_info's
total is the WDDM budget the driver grants HIP (BIOS carve + ~half
of remaining RAM) -- the OS share is already outside it. The 0.80
unified cap on top denied loads that fit (field report: 48.49 GiB
budget -> "38.79 GiB allowed" OOM for a 47.29 GiB load with 48.08
free). Use 1.0 on win32 unified; Linux keeps 0.80, discrete 0.90.
3. "Missing VRAM" confusion: log the WDDM budget vs physical RAM with
the fix (BIOS UMA frame buffer / AMD Software Variable Graphics
Memory) when the grant is under 75% of RAM, so a 48 GiB cap on a
96 GiB box reads as policy, not a Studio bug.
4. llama-server fit-step crash (Qwen3.6-27B-MTP + mmproj, lemonade
gfx1151): --fit defaults to 'on' upstream, so the fit step runs even
when Studio already placed the model via -ngl -1, and aborts in
ggml-cuda.cu on some ROCm hosts. Retry the spawn once with --fit off
when the server crashes during startup and Studio's own VRAM math
had placed the model (never when use_fit or an explicit fit flag was
passed). Also keep the TAIL of crash output in the error log (the
diagnostic line prints last; head-truncation cut exactly that) and
reference the full on-disk log.
Verified live on Radeon 8060S: bnb import clean, Qwen3.5-4B-MTP loads
and generates through the new spawn loop, stub-crash retry appends
--fit off and recovers, fraction probes confirm WDDM overcommit and
sub-1.0-only enforcement on current AMD wheels.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio ROCm Windows: GPU-name fallbacks so nothing depends on amd-smi
amd-smi does not reliably exist on Windows: the HIP SDK never ships a
CLI, inbox Windows Update drivers do not, and only some full Adrenalin
packages drop amd-smi.exe into System32 (field report: fresh Win11 +
Adrenalin + HIP SDK, still no amd-smi anywhere). Make every consumer
work without it:
- install_python_stack._detect_windows_gfx_arch: two new probes after
hipinfo/amd-smi -- (2b) the venv Scripts hipInfo.exe shipped by AMD
torch wheels (drives `studio update` on driver-only hosts), and (4) a
last-resort GPU marketing-name -> gfx table via WMI
(Win32_VideoController), mirroring setup.ps1's $nameArchTable so a
standalone repair resolves the arch with zero AMD tooling installed.
- install_llama_prebuilt._resolve_exe: also probe the venv Scripts dir
so a standalone rerun finds hipInfo.exe without HIP_PATH.
- hardware/amd.py _run_amd_smi: which() guard before spawning --
absence now disables the poller in one step instead of burning the
3-strike circuit breaker on FileNotFoundError; corrected the stale
comment claiming Adrenalin ships amd-smi.
Simulated against the real detection functions on gfx1151: amd-smi
absent, present-but-crashing (exit 1), present-but-hanging (60s sleep
vs 5-10s probe timeouts), and hard opt-out -- all resolve gfx1151, no
exceptions, bounded time. Full adversarial install (broken amd-smi
stub first on PATH + UNSLOTH_ENABLE_AMD_SMI=1, fresh uninstall first):
exit 0, name-table arch inference, lemonade gfx1151 b1292 prebuilt,
torch 2.11.0+rocm7.13.0 cuda_avail=True on the 8060S, Studio boots
healthy and stops cleanly.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: per-attempt llama-server log names + amd-smi test portability
Found by cross-platform simulation of the --fit off retry (Windows +
Linux sandboxes, real load_model with stub servers):
- llama-server log filename now carries the spawn-attempt index. The
retry can respawn within the same epoch second; reusing the name
opened the same file with "w" and truncated the crash log the retry
warning had just pointed the user at (proven with a frozen
time.time: one file, crash evidence gone; with the suffix both
attempts keep their logs). Regression-pinned in
test_llama_cpp_wait_for_health.py.
- test_amd_primary_gpu_with_mock now mocks shutil.which alongside
subprocess.run: the amd-smi absence guard which()-checks before
spawning, so on hosts without a real amd-smi (Linux CI, driver-only
Windows) the subprocess mock was never reached and the test failed.
Surfaced by running the suite in a clean Linux sandbox.
Simulation coverage on both OSes: 67-case platform/edge matrix
(real shipped code blocks under win32/linux/darwin spoofs: OOM-guard
fractions + VGM-hint boundary, bnb PATH-prepend gates, retry
eligibility incl. equals-forms and decoy tokens, GPU-name table
adversarial set, WMI fallback without powershell, monitor absence
semantics), 6-scenario live retry matrix (crash-once/crash-always/
exit-zero/explicit-fit/hang/log-collision) against real llama-server
spawns on Windows and WSL (GPU success legs on the 8060S), and a
3-engine browser matrix (chromium/firefox/webkit) driving the live
backend's health + authed /v1 chat completion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: classify unified-memory via props.is_integrated first
Align the ROCm OOM-guard classifier with PR #5988's UMA gate: consult
hipDeviceProp_t.integrated (props.is_integrated) before the hardcoded
arch set. Strictly additive -- truthy upgrades to unified; 0/absent
falls through to the existing gfx1150/gfx1151 + device-name logic, so
wheels that omit or zero the field cannot downgrade the known APU set.
Extends correct unified-cap treatment to APUs outside that set (e.g.
gfx1103 Phoenix iGPUs) and keeps Studio's two unified-memory consumers
on one driver signal. Verified live on gfx1151 (is_integrated == 1 on
the AMD Windows wheel -> ('gfx1151', True) via the new path).
* AMD detection: probe rocminfo with HSA_ENABLE_DXG_DETECTION and sync setup.sh gfx table
Fleet validation on a Strix Halo WSL2 box showed the system rocminfo
(HSA 1.18, ROCm 7.2.1) only enumerates the GPU over /dev/dxg when
HSA_ENABLE_DXG_DETECTION=1, and that rocminfo can sit at /opt/rocm/bin
off PATH outside login shells. Detection probes that miss either of
these report no GPU on a working ROCDXG host and select the CPU build
even though the lemonade bundle offloads fine (95.7 tok/s measured vs
64.5 CPU on the same laptop). Seed the env (a no-op on bare metal) and
the PATH fallback in install.sh, studio/setup.sh, and the installer's
Linux rocm probe, mirroring what main.py/worker.py already do for the
runtime.
Also sync studio/setup.sh's name->gfx table with install.sh: 890M and
the HX 37/AI 9 HX SKUs are Strix Point (gfx1150, not gfx1151), RX 7700S
must match gfx1102 before the gfx1100 row, and the RDNA2/workstation
rows were missing. New parity test pins the two bash tables together so
they cannot drift again.
* Studio: persist server session logs + native-crash stacks to disk
Field report (Strix Halo, 96 GB UMA carve, WSL and native Windows):
"the studio just terminates without a warning". A native crash in the
GPU runtime kills the process with no Python traceback, and a desktop-
shortcut console closes before anything can be read. The server only
ever logged to the console, so there was nothing to send back.
run_server now tees stdout/stderr to
~/.unsloth/studio/logs/server/server-<ts>-pid<n>.log (console behavior
unchanged; file copy is best-effort), arms faulthandler at the same
file so access violations / SIGSEGV leave a stack trace on disk, and
exports PYTHONFAULTHANDLER=1 so training workers inherit crash dumps
on their captured stderr. Armed before `from main import app` so even
import-time failures leave evidence. Keeps the newest 20 session logs;
opt out with UNSLOTH_STUDIO_NO_FILE_LOG=1. Prints "Session log: <path>"
at startup so users know what to attach.
Verified on this box: a forced real segfault (faulthandler._sigsegv)
leaves the full session output plus "Fatal Python error: Segmentation
fault" and the thread stack in the file while the console shows
nothing; a normal server boot captures the startup banner and serves
health as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* AMD probe: honor a pre-set HSA_ENABLE_DXG_DETECTION value
Match the shell helpers, which use the parameter-default form: a user
who exports HSA_ENABLE_DXG_DETECTION=0 to deliberately hide the GPU
from DXG detection should not have the probe override it.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Adds a self-contained RAG stack to Studio: knowledge bases with chunked indexing, hybrid (dense + lexical) retrieval, and an automatic first-pass context inject into chat. Embeddings run through a local llama-server GGUF backend (default unsloth/bge-small-en-v1.5-GGUF) with a sentence-transformers fallback. The chat tool loop gains a search_knowledge_base tool, a per-turn re-search cap, and source citation, layered on top of the shared ToolLoopController.
* Studio: fix OpenAI- and Anthropic-compatible API spec compliance
* Studio: fix API spec-compliance gaps on passthrough and streaming paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: carry context_length_exceeded through the OpenAI passthrough error path
* Studio: count tool-schema tokens in the Anthropic server-tool stream, and small stream-handling guards
* Studio: guard message_delta usage against None and normalize developer role before proxying
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: honor max_completion_tokens on the external-provider proxy path
* Studio: forward llama-server cached_tokens into OpenAI prompt_tokens_details
* Studio: sanitize messages in count_tokens to match the /v1/messages prompt
* Studio: report max_tokens for truncated tool calls and guard null usage in metadata events
* Studio: drop the request-id middleware (headers aren't declared in either spec)
* Studio: include the required request_id field in Anthropic error bodies
* Studio: honor max_completion_tokens on the audio (TTS / audio-input) paths
* Studio: add the _effective_max_tokens helper and route all max-token sites through it
* Studio: align API compatibility edge cases
* Studio: clarify multi-choice chat support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: clarify logprobs chat support
* Studio: opt the local chat UI into the streaming usage chunk so the context bar and tok/s repopulate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: forward seed to llama-server, and fix Anthropic server-tool stop_reason, tool_result id correlation, and parallel-tool execution cap
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: align OpenAI chat completion spec edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: align backend API compatibility tests
* Studio: honor tool caps and internal stream usage
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: coerce nullable stream usage counts
* Studio: preserve system prompts with developer messages
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: make Helper LLM startup pre-cache opt in
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: fix recipe dataset preview
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Trim and tighten code comments and docstrings across studio/ Python. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent.
* Studio: stop leaking internal exceptions to API clients; harden sandbox path
Security hardening for the FastAPI backend.
Error exposure (CodeQL py/stack-trace-exposure): many route handlers returned
raw caught-exception text to clients via HTTPException detail / response bodies,
which can leak internal filesystem paths and stack detail. Add shared helpers in
utils/utils.py (safe_error_detail, log_and_http_error) that log the full
exception server-side and return a generic message, and sweep the route layer
(inference, models, export, training, datasets, chat_history, providers,
mcp_servers, settings, data_recipe/{jobs,seed,validate,mcp}) to use them.
Intentionally user-facing validation messages, the existing _friendly_error SSE
paths, and upstream-service body passthrough (llama-server / OpenAI) are kept;
absolute server paths echoed in models.py browse/read errors are redacted.
Path injection (CodeQL py/path-injection): serve_sandbox_file already does
basename + realpath containment; add a strict filename allowlist
(^[A-Za-z0-9._-]{1,255}$) before the path is built as defense-in-depth and to
give the analyzer a clear sanitizer.
No behavior change beyond error-message text; status codes preserved.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: keep curated error messages, fix remaining load leak
- inference.py /load non-native path: redact str(e) instead of leaking it
(matched the native branch which already redacted).
- llama_extra_args validation: return the curated, path-redacted message
instead of the generic fallback so users see the offending flag.
- sandbox file serving: allowlist now forbids only separators/control chars
via fullmatch, so generated images like 'loss curve.png' render again
while traversal is still blocked by basename + extension + realpath.
- Add safe_curated_detail() for domain/validation exceptions whose message
is intentionally user-facing; apply it to data_recipe job/validate,
chat conflict, provider test, and MCP probe paths (these were collapsing
to 'An internal error occurred', and 'connection' even mis-mapped to an
upstream-service message). Generic Exception paths keep safe_error_detail.
- log_and_http_error: tolerate stdlib loggers (no structlog kwargs).
- delete_openai_container: log transport errors with exc_info like list/create.
- Drop helper/HTTPException imports this change left unused.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* log_and_http_error: log original error traceback on stdlib-logger fallback
* Tidy error-helper and sandbox comments for PR #6072
* Trim redundant comments in studio error-hardening routes for PR #6072
* Re-trigger CI now that unsloth-zoo #727 is merged (Core pulls zoo main)
* Address PR #6072 review feedback
- inference.py: keep the actionable NativePathLeaseError detail (path-redacted)
instead of collapsing it to the generic message, matching the other curated
validation paths in this file.
- utils.py: log via a single formatted log.error(exc_info=error) call that works
for structlog and stdlib loggers; drop the now-unneeded try/except helper.
- models.py: use Path.name instead of os.path.basename(str(current)).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: enable audio input for Gemma 4 GGUF models
Audio file upload was disabled for Gemma 4 vision+audio GGUFs (e.g.
gemma-4-12b-it-GGUF) even though their mmproj carries an audio encoder
(clip.has_audio_encoder, gemma4ua). Two causes:
- Audio-input detection only matched Gemma 3n's <audio_soft_token>;
Gemma 4 uses <|audio|>, so audio_vlm was never detected.
- The GGUF load/status responses hardcoded has_audio_input=False, so the
flag was dropped even when audio_vlm was detected (affected Gemma 3n
GGUFs too).
Changes:
- Recognize <|audio|> alongside <audio_soft_token> in the llama-server
token probe and the tokenizer-config pattern.
- Read clip.has_audio_encoder from the mmproj as an independent,
model-agnostic signal (read_mmproj_audio_capability).
- Emit the computed has_audio_input on the GGUF load/status responses.
- Tests for the new pattern and the mmproj reader.
* Studio: default chat model and dataset helper to Qwen3.5-4B-MTP
Switch the auto-loaded chat default and the dataset-analysis helper GGUF
from gemma-4-E2B-it to unsloth/Qwen3.5-4B-MTP-GGUF (UD-Q4_K_XL).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Adds studio/backend/utils/datasets/dataset_none_detect.py, a standalone scanner that reports None/empty content turns in alpaca, chatml, sharegpt, and gptoss datasets without modifying data, plus generator and runner scripts under tests/utils. Depends only on the datasets library and is not wired into the package init, so it stays import-light.
* studio: cap training dataset uploads
* studio: clean up failed dataset uploads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: raise upload limits to 500MB
* studio: make upload limit configurable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: stream upload routes
* studio: split recipe upload caps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten upload limit handling
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: import settings router directly
* studio: polish upload cap setting control
* studio: cap settings request bodies
* studio: stub settings route in desktop auth test
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: align project sidebar UX with ChatGPT
* feat: align project sidebar UX with ChatGPT
* feat(chat): load stored project list
* feat(chat): add project sidebar workflows
* fix: stabilize project page navigation
* fix: projects chat loading
* fix: show project chat thread
* style: sidebar project spacing and hover clipping
* style: add expandable project chat history and move-to-project submenu
* feat: polish project sidebar
* feat: persist project sandbox paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: only create sandbox project workspace dir
* feat: add optional project workspace deletion from delete dialog
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: stabilize chat projects CI failures
* fix: polish project chat navigation
* Studio: manage chat history with projects
Group chats into projects with a dedicated projects page and route.
Sidebar shows recents with per-row actions and a vertical more-vertical
menu, and the sidebar scrollbar stays hidden so rows never shift on
hover. Includes chat settings and composer refinements.
* Studio: projects sidebar and breadcrumb polish
Sidebar:
- Remove the Compare nav item.
- Widen the sidebar to match the projects layout.
- Replace the scroll-gated bottom fade with a static fade pinned above
the profile box, so it no longer attaches to Recents or lags the
collapse and expand animation.
Topbar breadcrumb (chat-page):
- On a project landing show "Projects" linking to the projects list.
- Inside a project chat show the project name and chat title, with the
project name linking back to that specific project page.
- Drop the divider between the model selector and the breadcrumb.
* Studio: make project workspace delete test cross-platform
test_chat_project_delete_files_removes_workspace rooted the project under
pytest tmp_path, which resolves to /private/tmp on macOS. The workspace
delete guard refuses paths under the system denylist by design, so the
test passed on Linux CI but failed on macOS.
Add a workspace_projects_home fixture that keeps tmp_path on Linux and
Windows (CI unchanged) and falls back to a home subdir only when the temp
root is on the platform denylist. Derive the workspace path from the
created project so it tracks the projects home.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: satisfy import-hoist check for new path re-exports
documents_root and project_workspaces_root are re-exported from
utils.paths but only referenced as __all__ string literals, which the
import-hoist safety net does not count as a use. It flagged the two newly
added re-exports as unused imports and failed Source lint.
Name-load both via a module-level _REEXPORTED tuple so the check sees
them used. No behaviour change; consumers still import them from
utils.paths.
* fix: avoid projects empty-state flash
* fix: batch chat search indexing
* Studio: polish chat sidebar, run settings, and search
- Use the native OS scrollbar for the chat sidebar, Run settings panel, and chat search list instead of a custom scrollbar
- Highlight the active run in the sidebar and keep chat search available during training
- Stop the training log view from replaying when navigating back to a run
- Rename the chat settings panel to Run settings and align its toggle icon and position
- Tighten heading and sidebar letter spacing and lighten the Train and Recents labels
- Match the search dialog corner style across light and dark and drop the stray border
- Make the MCP Servers section header plain text instead of a link
- Remove a stray .orig backup file
* studio/frontend: restore Compare entry point in the sidebar
The chat-projects sidebar redesign dropped the Compare nav item and moved
it to thread-sidebar.tsx, which is not imported or rendered anywhere. That
left no way for a user to start a new model comparison (enterCompare only
fired from the guided tour and the training handoff), and broke the
Compare/Recipes/Export UI smoke test that clicks [data-tour="chat-compare"].
Re-add the Compare NavItem to the New Chat / Search group, carrying
data-tour="chat-compare" and the same new-comparison navigation as before.
* studio/frontend: use Unsloth green for the fallback profile avatar
Switch the initials-avatar background from blue to #14b789 so the sidebar
and edit-profile avatar match the Unsloth brand colour.
* studio/frontend: turn project breadcrumb into a project switcher dropdown
* studio/frontend: stop project card kebab clicks from opening the project
* studio/frontend: hide project switcher outside projects
* studio/frontend: stabilize project switcher loading
* style: project switcher alignment
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Harden local GGUF detection so a .gguf path is not misrouted to the transformers backend during the brief Windows lock window after llama-server is killed. Catch OSError from stat() and treat the path as the file, while a directory named *.gguf still falls through to the directory scan. Adds regression tests for the lock-window and directory cases.
Follow-up cleanups to the merged AMD ROCm support PR #5301:
1. De-duplicate the torchao Windows-ROCm import stub into a single shared
module (studio/backend/core/_torchao_stub.py); both workers call one
install_torchao_windows_rocm_stub() entrypoint.
2. Align the gfx name/arch comment columns in setup.sh and setup.ps1.
3. Isolate the float16 dtype fallback to AMD without native bf16; NVIDIA
keeps dtype=None so unsloth's own bf16/fp16/FORCE_FLOAT32 detection is
honored.
4. Hoist unconditional stdlib imports (gc, glob, re, subprocess, copy,
types, sys, importlib.metadata) from function bodies to module top
across the PR #5301-touched files; heavy/optional/relative imports stay
lazy.
5. bitsandbytes Windows-ROCm install now uses plain pip (force_pip=True)
instead of UV_SKIP_WHEEL_FILENAME_CHECK, per the AMD hackathon docs.
Also adds scripts/verify_import_hoist.py (a scope-aware LEGB AST resolver
that catches dangling-alias and rename-clash bugs in import-hoist
refactors) and wires it into the Lint CI source-lint job as a self-test
plus a pull_request compare gate.
Daniel Han