* CI: retry transient HTTP timeouts in Studio smoke probes
The post() helper in the Studio inference smoke workflows does a single
urlopen with a 240s timeout against the local Studio server. On shared
runners this sporadically hits TimeoutError while the server is stalled,
failing the whole job for a transport hiccup; the same flake has recurred
across unrelated PRs on Linux and Windows (JSON/images and tool-calling
jobs) and passes on rerun.
Retry the probe up to 3 times on transport-level failures only
(TimeoutError, ConnectionError, non-HTTP URLError), 15s apart. HTTP
status errors still surface immediately, so genuine server failures are
unaffected. post_sse() is left unchanged: it has a 600s budget and has
not flaked.
* CI: retry only short probes so worst case fits the job budget
Some json-images calls pass timeout=600; three attempts there could spend
30 minutes in one step and hit the job's timeout-minutes instead of failing
with the Python error. Retry (3 attempts) only when timeout <= 300s, which
covers the observed flaky 180-240s probes; longer probes keep the pre-PR
single attempt.
* CI: give long smoke probes one capped retry
Round two of bounding the retries: timeout>300s probes previously got a
single attempt, so a transient stall in the 600s JSON-mode probes still
failed on first occurrence. Give them one retry with the attempt timeout
capped at 300s. Worst cases stay inside timeout-minutes: 240s probes
12.5 min, one 600s probe 15.25 min, the Windows JSON job's two long
probes 30.5 min against its 35 minute budget.
* scripts: refresh scan_packages allowlist baseline
Regenerate scripts/scan_packages_baseline.json against the current
resolved dependency set so the blocking pip scan-packages gate matches
what the scanner now finds. Refreshes evidence hashes for benign
findings whose code shifted lines (unsloth-zoo mlx loader, gguf/mlx
test /tmp fixtures) and adds two mainstream-library entries that were
newly surfaced (torch inductor codecache base64+subprocess compile
cache, torch testing common_utils socket import). Stale entries whose
matching code changed and no longer triggers are dropped.
All entries remain CRITICAL/HIGH findings manually judged benign;
matched on (package, file, check, evidence_hash).
* ci(security-audit): re-run scan when the allowlist baseline changes
The security-audit pull_request trigger listed the scanners but not
their allowlist baselines, so a baseline-only edit never re-ran the
scan that consumes it. A refreshed baseline could therefore merge
without CI confirming its evidence hashes match what the scanner finds.
Add scan_packages_baseline.json and scan_npm_packages_baseline.json to
the paths filter so baseline changes are validated on their own PR.
* unsloth start: add --resume to persist and reopen agent sessions
`unsloth start <agent>` launches a coding agent whose home is a throwaway
temp dir wiped on exit, so codex/openclaw/hermes/pi (which relocate their
whole home there) cannot resume a conversation after you quit. opencode and
claude keep their session data in a fixed user dir, so they already resume.
Add an opt-in --resume/--no-resume flag: it routes the launch to the stable
Unsloth agents dir (the same one --no-launch already uses) so the session
survives the exit, never touching the user's own ~/.<agent>. A bare --resume
also reopens the last conversation via the agent's native flag (codex
`resume --last`, opencode/claude/pi `--continue`). The default is unchanged:
a plain launch still uses a temp dir and persists nothing.
Add a dispatch-only `resume` job to the Local Agent Guides CI that drives the
real launch path and asserts the split: codex/pi are wiped without --resume
and persist with it, while opencode/claude persist either way.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* unsloth start: rename --resume to --persist
The session flag collided with agents' own resume flags. `unsloth start
claude --resume <id>` used to forward `--resume <id>` straight to Claude
(which keeps its history in ~/.claude regardless), so a boolean --resume on
unsloth start would have swallowed the session id and turned it into a stray
prompt. Name the persistence flag --persist instead, so every agent's native
resume flag (claude --resume <id>, codex resume, opencode --continue, ...)
still passes through untouched. Behavior is otherwise identical: --persist
keeps a launched agent's session under the Unsloth agents dir, and a bare
--persist reopens the last conversation.
Add a regression test that `--resume <id>` passes through verbatim, and in the
CI resume experiment skip the redundant second pass for opencode/claude (they
persist either way, and a second CPU turn only risks a timeout).
* unsloth start: correct --persist help and drop the buggy auto-resume
Reword the --persist help to be accurate: claude and opencode keep sessions in
the user's own stores and resume regardless, so --persist only stabilizes the
otherwise-ephemeral relocated home of codex/openclaw/hermes/pi. Drop the
bare-launch auto-append of native resume tokens: it errored on a first launch
with no prior session, and was inconsistent between launch and no-launch.
--persist now only keeps the session dir; resume via the agent's own command
(e.g. `unsloth start codex --persist resume`), which now finds it.
In the CI resume experiment, fail the pass when the launched turn exits
non-zero, so a write-then-error is not misread as PERSISTED.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Stabilize floating monitor drag
* Restore floating monitor exit animation
* Harden Windows Studio smoke checks
* Keep API menu badge removed
* Apply no-build-tools env overrides in-script
The runner does not apply step-level env keys containing parentheses,
so ProgramFiles(x86) kept its real value and Find-VsBuildTools still
detected VS through vswhere. Set the overrides inside each pwsh step
instead; child processes inherit them. The resolver step moves to pwsh
because bash cannot export a variable named ProgramFiles(x86).
* Reset chat UI session without a second browser context
macOS runs Chromium with --single-process, where closing the last
context tears down the whole browser, so the shutdown re-login died
with TargetClosedError on new_page. Clear cookies and swap pages
inside the same context instead, opening the replacement page before
closing the old one.
* Keep the no-build-tools Path filtered across session refreshes
install.ps1's Refresh-SessionPath and setup.ps1's Refresh-Environment
rebuild the session Path from the Machine and User registry scopes, so
the process-level filter could be undone mid-install and re-expose
CMake. Filter those scopes in the Prepare step with normalized dir
matching and restore them in cleanup.
* Drop stale localStorage auth tokens before re-login
Auth tokens live in localStorage, not cookies, and the login guest
guard redirects on their mere presence. Remove them during the session
reset so the /login navigation is deterministic instead of relying on
the tolerated redirect bounce.
* Studio: source CPU llama.cpp prebuilts from the unslothai fork
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject unknown Linux CPU arches and keep ROCm-tooling hosts off the CPU prebuilt
* Studio: extend the resolve-prebuilt ROCm-tooling guard to Windows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: let ROCm-SDK-only CPU hosts take the fork CPU prebuilt
* Studio: accept windows-arm64 prebuilt kind and refresh stale fork-routing comments
* Studio: correct stale fork-routing comments and --resolve-prebuilt help
* Refresh stale ggml-org routing comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* version-compat CI: fake CPU training runs for SFT/GRPO/DPO
Adds a runtime layer on top of the patch-run canary: actually runs
trainer.train() for a couple of steps on a CPU-only runner under the CUDA
spoof, wrapping a plain tiny HF model in the Unsloth-patched trainer. Exercises
the real train() loop (collation, generation, the injected
_get_per_token_logps_and_entropies, loss, backward, optimizer) so a TRL or
transformers change that breaks the loop at runtime -- not just the source
structure -- surfaces here. No GPU, no meaningful numerics.
Needs a chain of small CPU shims (eager torch.compile, dynamo suppress, cuda
tensor-alloc redirect to CPU, model.for_training/for_inference equivalents)
documented inline. Does not exercise Unsloth's Triton/GPU kernels (CPU can't).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* cpu fake-train: force adamw_torch + disable dynamo for CPU runner
On a real CPU-build torch runner (GitHub CI) two things bit that a CUDA-build
torch with GPUs hidden masked locally:
- The default optimizer is adamw_8bit (bitsandbytes), whose is_on_gpu() check
dies on CPU tensors. Force optim=adamw_torch in all three configs.
- import unsloth reinstalls the real torch.compile over the eager passthrough,
so the GRPO hot path (chunked_selective_log_softmax) actually compiles and
inductor picks the spoofed CUDA device, crashing on device props
(gcnArchName). Re-apply the eager passthrough after import and flip
torch._dynamo.config.disable so every @torch.compile runs eager at call time.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* cpu fake-train: write checkpoints under pytest tmp_path
Use pytest's tmp_path for each trainer's output_dir instead of a hardcoded
relative temp/ci_* path, so a local pytest run does not leave untracked dirs in
the repo tree and the tests are CWD-independent.
* version-compat CI: disable dynamo at process level for the fake-run job
Set TORCHDYNAMO_DISABLE / TORCH_COMPILE_DISABLE in the fake-run step env so
dynamo/inductor is off before conftest.py's early import unsloth, not only via
the per-test runtime shim. Defense in depth on the GPU-less runner: the GRPO
hot path never compiles regardless of when its functions were decorated.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix PEFT replacement for TRL >= 1.7.0, add missing compute_aux_loss for TRL 1.7.0
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GRPO for TRL >= 1.7.0: PEFT ref-adapter removal and return arity
rl.py: for trl >= 1.7.0, scope the PEFT removal regex to the ref-adapter
block only by anchoring the end on ref_param.data.copy_(param.data), so it
no longer also deletes the following gradient-checkpointing
enable_input_require_grads() block. Neutralize TRL 1.7.0's
`if _is_quantized_model:` bf16 cast the same way the existing
is_loaded_in_4bit cast is handled.
rl_replacements.py: initialize _extra_moe_kwargs before use (it was
referenced before assignment whenever compute_aux_loss was passed) and only
request output_router_logits when the aux loss is actually wanted.
rl_replacements.py: _get_per_token_logps_and_entropies now returns a 3-tuple
(logps, entropies, aux_loss) for trl >= 1.7.0 and a 2-tuple for older TRL,
matching how every TRL call site unpacks the result. Without this, TRL 1.7.x
_generate_and_score_completions unpacks 3 values from a 2-tuple and raises
"not enough values to unpack (expected 3, got 2)".
* Return zero aux_loss placeholder and drop inference-mode aux collection
* GRPO TRL >= 1.7.0: reject router aux-loss opt-in at init; drop zero aux placeholder
Unsloth's optimized GRPO forward cannot compute the MoE router auxiliary loss.
Previously an explicit opt-in (router_aux_loss_coef > 0) returned a fabricated
zero, silently training without the requested load-balancing penalty. Now reject
it at trainer init with a clear NotImplementedError, and return None (not zero)
for the aux slot of TRL's 3-tuple. Default stays off (coef 0), so the common
path is unaffected.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* GRPO hidden-states fallback: free ModelOutput before chunked log-softmax
The old/ref logprob fallback binds the full ModelOutput (which holds every
layer's hidden_states when output_hidden_states=True) and kept it alive across
chunked_hidden_states_selective_log_softmax, an avoidable OOM on large models.
Extract logits then del outputs in both the text and VLM branches.
* Version-compat CI: proactively catch TRL GRPO breakage
The existing TRL canary is a static symbol/source grep: it verifies symbols
exist but is blind to structural changes (TRL 1.7.0's 2->3-tuple per-token-logps
return arity and restructured PEFT ref-adapter block, which the fix in this PR
addresses, both slipped past it because the methods still existed).
Two additions:
- test_trl_grpo_pinned_symbols.py: extend TRL_TAGS to 1.5/1.6/1.7 and pin the
exact source-string contracts the rl.py / rl_replacements.py transforms depend
on for TRL >= 1.7.0 (PEFT elif ref-adapter block + enable_input_require_grads
survival, if _is_quantized_model, aux_loss_enabled anchor, compute_aux_loss
arity). A future TRL change fails on main a few days before the PyPI release.
- test_trl_grpo_fake_run.py + a version-compat-ci job: fake-CUDA run that drives
the real GRPO/SFT/DPO source-transform patchers against latest + main TRL on a
CPU-only runner (no training) and asserts the generated Unsloth trainer still
satisfies the transform contracts. Catches behavioral regressions the grep
cannot see.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fake-run test: use a normal Version import for the aux gate
* version-compat CI: fix fake-run job gate + torch-absent collection
- Drop the invalid job-level matrix if (matrix is not available in
jobs.<id>.if -> 'Unrecognized named-value: matrix' fails the whole
workflow). Use a single job that runs vs TRL latest always and re-runs
vs TRL main only on schedule/dispatch via a step-level github.event_name
guard. Validated with actionlint.
- Module-level skip the fake-run test when torch is absent so
daily-fresh-fetch (pytest-only, collects tests/version_compat/) does not
crash on the top-level spoof import.
* fake-run test: do not skip on import failure
unsloth/trl are installed in the grpo-fake-run job, so a failing import is the
import-time drift this canary must catch. Keep only the not-installed find_spec
skips; let a real import error fail the test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* GRPO arity gate: regex downgrade + fail loud + CI coverage
The TRL < 1.7.0 per-token-logps return downgrade was an exact-string replace
anchored on the full return line incl. its comment, so a reformat (e.g.
pre-commit) could silently no-op it and ship a 3-tuple to older TRL. Switch to
a regex tolerant of comment/whitespace drift, and raise if the anchor stops
matching (re.subn count != 1) instead of failing silently. Add a monkeypatched
trl_version unit test asserting both arities, since CI only installs TRL >= 1.7.0
and never exercised the downgrade otherwise.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fake-run: give SFT/DPO a real contract, not just ast-parse
The SFT/DPO fake patch runs only checked the generated trainer parses. Also
assert the shared QLoRA _is_quantized_model bf16 cast is neutralized (TRL 1.7's
spelling, present in both sft_trainer and dpo_trainer), so a structural TRL
change to that block is caught for SFT/DPO too, not just GRPO.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* GRPO PEFT ref-adapter removal: lower gate to the TRL 1.4.0 floor
The elif is_peft_model(model) and args.beta != 0.0: ref-adapter block was
introduced in TRL 1.4.0 and is unchanged through 1.7.x, but the removal was
gated at >= 1.7.0, so for 1.4 <= TRL < 1.7 the transform fell through to the
0.27 branch (which matches the older if is_peft_available()... form) and
silently no-oped: a PEFT + beta != 0 GRPO run then computed the KL reference
from the copied ref adapter instead of the base model. Lower the gate to
1.4.0 and keep the 1.7.0-only router aux-loss fail-fast nested. Widen the
pinned-symbol contract test to run from 1.4.0 so the covered versions are
actually exercised.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
The setup.ps1 unit-tests job intermittently fails on the windows-latest
runner with 'No repository with the name PSGallery was found.' when the
default PowerShell Gallery is not registered, so Set-PSRepository throws
before Pester can be installed. Register the default gallery first when it
is missing, then set its policy and install Pester as before.
* Studio: expose full compressed-tensors scheme set in an export formats dropdown
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: multi-select export formats, portable torchao FP8/INT8, GGUF LoRA, source parity
Export page overhaul on top of the formats dropdown:
- Unify merged precision into one sorted multi-select list (16-bit first, then
8-bit, then 4-bit). Drop "vLLM" from labels, add INT8 (W8A8), INT8 (W8A16),
INT4 (W4A16), MXFP4, MXFP8. Quick formats render as toggle pills; the rest live
in a multi-select "More formats" dropdown, so several formats export in one run.
- Add a portable torchao FP8/INT8 save path (Float8WeightOnlyConfig /
Int8WeightOnlyConfig) that needs no NVIDIA GPU to produce and loads in vLLM.
FP8 serializes to safetensors, INT8 to .bin. Wired into save_pretrained_merged
and push_to_hub_merged via a TORCHAO_EXPORT_SCHEMES registry and
_unsloth_save_torchao, parallel to the compressed-tensors path.
- Hide NVIDIA-only compressed-tensors formats when no NVIDIA GPU is present; keep
16-bit and portable FP8/INT8. The backend also rejects a compressed request on
non-NVIDIA hardware so it stays authoritative.
- Relax merged export to non-PEFT models so Local Model and Hugging Face sources
get the same 16-bit / compressed / portable options.
- GGUF: send the whole quant list in one call (merge once, quantize many).
- LoRA: add a GGUF adapter option (convert_lora_to_gguf.py) with an outtype
select (f16/bf16/f32/q8_0/auto), alongside the safetensors adapter.
- Thread the new fields through models, routes, orchestrator, and worker; extend
the export tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: gate export by accelerator with a torch-aware reason; fix export save dir naming
Export runs through Unsloth, which requires a compute accelerator (NVIDIA/AMD/Intel
GPU or Apple MLX) and has no CPU code path, so a bare-CPU host cannot export even
with PyTorch installed. Add export_capability() in utils/hardware that reports
export_supported plus a precise reason so the UI stops showing a generic "no GPU":
- pytorch_not_installed: a --no-torch install (even a physical GPU is unusable)
- no_accelerator: PyTorch present but no supported accelerator (bare CPU)
- mlx_unavailable: Apple Silicon where the MLX stack is missing or too old
Expose the fields on /api/system/hardware and /api/system, and guard the mutating
export routes (load-checkpoint, export/merged|base|gguf|lora) with HTTP 400 and the
reason, leaving read-only endpoints usable so the Export page still renders.
Make core/export/export.py import without PyTorch and without a usable accelerator
(the Unsloth import is caught) so the export worker degrades to a clear message
instead of crashing at import.
Frontend: keep /export reachable on chat-only hosts and gray out the method and
format options with the backend reason (Alert plus disabled MethodPicker) instead
of silently redirecting to /chat, so users see why export is unavailable.
Also fix the export save directory producing "model/null" for Local Model and
Hugging Face sources that have no run/checkpoint, naming the folder from the model id.
* CI: validate Studio export capability gating on Linux, Windows and macOS
Add a small pytest matrix that runs studio/backend/tests/test_export_capability.py
on ubuntu-latest, windows-latest and macos-latest. It confirms, on each real OS,
that hardware.export_capability() reports the right decision and reason
(pytorch_not_installed, no_accelerator, or mlx_unavailable) and that the export
backend imports without PyTorch and degrades to a clear message instead of crashing.
Hosted runners have no GPU/MLX, so this covers the "export unavailable, here is why"
path a Mac/Windows user without an accelerator sees; a real accelerator export is
validated separately. The job installs only a CPU PyTorch plus the backend import
deps (no unsloth, triton, or llama.cpp), so it runs in seconds with no GPU.
* Studio export: address Codex review (source-aware gating, GGUF LoRA token/MLX/guard)
Frontend (export-page):
- Gate LoRA and quantized-model restrictions on the active source. isAdapter /
isQuantized come from the selected checkpoint; in Local Model / Hugging Face
("model") source mode they were stale, so LoRA stayed wrongly enabled for a
direct base model (backend then rejects "No adapter to export") and a stale
"quantized" flag disabled every method for an unrelated, exportable model. Add
effectiveIsAdapter / effectiveIsQuantized (false outside checkpoint mode) and use
them in the method-reset effect and the MethodPicker disabled state.
- Hide the GGUF LoRA option on a macOS/MLX host (the backend rejects GGUF LoRA on
MLX), so users no longer pick it, wait through the load, and always fail. Disable
the "GGUF adapter" button on a Mac host and never send loraGguf there.
Backend (core/export/export.py):
- Pass the HF token into the GGUF LoRA conversion (save_pretrained_gguf), so a
gated/private base model's config fetch in convert_lora_to_gguf.py is
authenticated; without it the load can succeed but the conversion fails.
- Guard the save_pretrained_gguf capability check with getattr so an older Unsloth
model that lacks the method returns the clean "not supported" message instead of
an AttributeError that surfaces as a generic 500.
* Studio export: address 2nd Codex review (CI index, empty merged, test import)
- studio-export-capability-ci.yml: add --extra-index-url https://pypi.org/simple to
the torch install so torch's transitive deps still resolve; --index-url alone
replaces PyPI with only the CPU wheel index, which does not serve all of them.
- export-page handleStart: reject an empty merged selection (mirrors canExport), so
clicking the panel's Start button with every precision pill deselected no longer
submits mergedSelections: [] and launches an unintended default 16-bit export.
- test_export_imatrix_compressed: the torchao-registry test now reads unsloth/save.py
as text (like the other ast/string checks) instead of `import unsloth.save`, which
raised ModuleNotFoundError in the CPU studio-backend suite that has no unsloth
installed.
* Studio export: make comments succinct across the export changes
* Studio export: use load token for local GGUF LoRA export of gated bases
* Studio export: harden portable torchao path and gate multi-format Hub push
torchao (_unsloth_save_torchao):
- merge to an isolated temp staging dir so a co-selected 16-bit output at save_directory is not deleted
- narrow VLM detection to vision_config / ForVisionText2Text so T5/BART/Whisper are not misrouted
- forward trust_remote_code (from auto_map) to the reload so custom-code models export
Export UI:
- hide portable torchao formats on macOS/MLX (backend rejects quantized export there)
- restrict a Hub merged export to a single format (each writes to the repo root)
* Studio export: torchao tokenizer remote-code + XPU offload, scale GGUF timeout
torchao (_unsloth_save_torchao):
- honor auto_map in the staged tokenizer/processor configs (not just model.config) when
deriving trust_remote_code, so custom-code tokenizers reload after the merge
- offload single-device XPU models to CPU (and empty the XPU cache) before the reload, matching
the CUDA path, so an Intel GPU that fits the model once does not OOM on the second copy
Export orchestrator:
- scale the GGUF wait timeout by the number of requested quants so a multi-quant list export of a
large model does not time out at a flat 3600s
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio export: show portable torchao formats only on non-NVIDIA (CPU) hosts
Portable torchao FP8/INT8 is the fallback for hosts without the NVIDIA compressed-tensors path.
On an NVIDIA GPU the compressed-tensors FP8/FP4/INT formats are the intended path (llm-compressor
auto-installs), so hide the portable duplicates there; keep them on CPU / non-NVIDIA hosts and
continue hiding them on macOS/MLX.
* Studio export: report all output folders and the exported formats
- Multi-format merged export now collects every sibling output directory (one per selected
precision) instead of only the last; the success banner lists them all.
- Show the selected precision formats in the run summary (a Formats row, like GGUF Quantizations),
so the panel says what is being exported rather than just 'Merged Model'.
- Persist the selected formats in the run summary and seed them on mount, so navigating away and
back (or toggling the export method) restores the selection instead of resetting to 16-bit.
* Studio export: list all output formats, add GGUF LoRA target, default Q8_0, auto-select newest checkpoint
- Progress/summary panel now shows a Formats row with the selected merged
formats, and the success banner lists every output folder a multi-format
merged run creates (one line per format) instead of only the last one.
- Merged format selection is seeded from the active run, so navigating away
and back (or switching method cards) no longer resets it to 16-bit.
- GGUF / Llama.cpp now offers an Export target toggle (Full model or LoRA
adapter) for adapter checkpoints, reusing the LoRA GGUF export path.
- Removed the Auto GGUF LoRA output type and defaulted to Q8_0 in the UI,
the request model, and the backend defaults; the outtype list is now
Q8_0/F16/BF16/F32. Core save.py still accepts auto for external callers.
- When a finetune has no checkpoint selected, auto-select the newest one.
* Studio torchao export: robust reload class + optional VLM import
Two fixes to the portable torchao FP8/INT8 export reload, from review of the
narrowed VLM detection:
- Encoder-decoder seq2seq checkpoints (T5/BART/Whisper) are not causal LMs.
With the narrowed is_vlm test they now correctly skip the image-text class,
but fell through to AutoModelForCausalLM and failed to reload after the merge.
Reload them with their own architecture class from the config instead.
- AutoModelForImageTextToText was imported unconditionally at the top of the
torchao path, so on Transformers builds without that class the import aborted
every torchao export (even text-only). Import it lazily only for a VLM, with
the AutoModelForVision2Seq fallback used elsewhere in Unsloth.
* Studio: enable FP8/FP4 compressed export for newer-transformers models
The shipped llm-compressor 0.10.x pins transformers<=4.57.6, so FP8/FP4 export failed
for models needing a transformers 5.x sidecar (Qwen3.5, Gemma-4, Qwen3-Next): the
quantization subprocess crashed importing the removed TORCH_INIT_FUNCTIONS.
Run the quantization against a dedicated llm-compressor-main "shadow": a --target
package dir (transformers 5.10.2 + llm-compressor main + compressed-tensors) layered
over the existing torch. It installs --no-deps so torch is never touched (works on any
Studio torch build), is provisioned lazily and fingerprint-cached, and can be turned
off with UNSLOTH_DISABLE_LLMCOMPRESSOR_MAIN.
- transformers_version.py: provision + validate .venv_llmcompressor.
- export.py: route all compressed exports through the shadow when available; else keep
the workspace 0.10.x path and fail fast past its transformers ceiling.
- save.py: launch _compressed_quantize.py with a clean PYTHONPATH = shadow.
- _compressed_quantize.py: skip linear_attn / vision tower / MTP modules (matches the
RedHatAI and NVIDIA reference quants, and is required by the grouped schemes).
Verified all four schemes (fp8, w8a8, w4a16, mxfp4) on Qwen3.5-9B and Llama-3.2-1B, and
fp8 on Gemma-4, end to end through Studio.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF LoRA export tests
* Fix export CI expectations
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* replaced connect with start
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix
* Studio: build the coding-agent command from the selected server
The API keys panel showed a hardcoded `unsloth start claude`. `unsloth start`
defaults to 127.0.0.1:8888 and only mints a key for a loopback server, so a
non-default port or a tunnel/remote base would target the wrong server or fail
to mint. Build the command from the panel base/key (and emit a key for
non-loopback), matching the other snippets in the panel.
* CLI: keep `unsloth connect` as a hidden alias for `unsloth start`
Avoids breaking existing scripts and docs that still call `unsloth connect`.
* Tests: stub _unstarted_cleanup in same-task disconnect test
The test builds _SameTaskStreamingResponse via __new__, so set the attribute
that __call__ now reads.
* Match coding-agent command loopback check to the CLI 127.0.0.0/8 rule (#6613)
* Keep unsloth_cli.commands.connect importable as a deprecated shim (#6613)
* Format the new coding-agents panel strings and import per biome (#6613)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop the unsloth connect alias and shim; unsloth start is the only command (#6613)
* Route unsloth connect to unsloth start as a hidden backward-compatible alias (#6613)
* Forward unsloth run model-load flags to unsloth start (gguf-variant, context-length, load-in-4bit, tensor-parallel) (#6613)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Session-scope coding agent config in unsloth start
Configure each agent for the current session instead of writing the Studio endpoint, key, and default model into the user's own config. Codex, OpenCode, OpenClaw, and Hermes get a private config relocated through their config-path env vars (CODEX_HOME, OPENCODE_CONFIG overlay, OPENCLAW_CONFIG_PATH plus OPENCLAW_STATE_DIR, HERMES_HOME). Claude Code suppresses the attribution header for the session via the CLAUDE_CODE_ATTRIBUTION_HEADER env var plus a --settings overlay, with no ~/.claude write. --launch uses an ephemeral temp dir removed after the agent exits; --no-launch uses a stable Unsloth-owned dir and prints the matching export lines.
* Read relocated agent session config in Local Agent Guides CI
The contract crosscheck and the openclaw/hermes patch helpers now read each agent's config from the relocated path printed by unsloth start --no-launch (CODEX_HOME, OPENCODE_CONFIG, OPENCLAW_CONFIG_PATH, HERMES_HOME) instead of fixed home paths. The Claude attribution A/B toggles the header for the session only (shipped-config HIT vs vanilla MISS) instead of editing ~/.claude/settings.json.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip the POSIX-only --no-launch parser test on Windows
test_no_launch_output_is_parseable mirrors the #6547 bash CI parser, which greps export/unset lines and only runs on Linux/macOS runners. On Windows --no-launch prints PowerShell ($env: / Remove-Item), so the export-line assertion does not apply there. Cross-OS staging CI surfaced this.
* Size Claude Code's auto-compact window to the loaded model's context
Claude Code auto-compacts against its native (~600k token) window, so against a smaller local model it overflows the server's context (silent truncation) long before it compacts. Set CLAUDE_CODE_AUTO_COMPACT_WINDOW to the loaded model's real context length (the value codex/openclaw already get via model_context_window / contextWindow). Omitted when the model reports no context length.
* Pin OpenCode/Hermes context window and set 90% compaction across agents
Feed every agent the server-determined sequence length (the value /v1/models reports from runtime_context_length) and a ~90% compaction threshold. OpenCode: a custom-provider model with no limit defaults to context 0, which silently disables auto-compaction, so set limit.context/output and scale the compaction buffer to 10% of the window. Hermes: pin model.context_length (it otherwise falls back to a 256k default when the server's /v1/models omits the field) and set compression.threshold 0.9. Claude: add CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=90 alongside the window. Codex (model_context_window) and OpenClaw (contextWindow) already carried the window and auto-manage off it.
* Add `unsloth start pi` recipe
Pi was the only agent without a built-in recipe, so the agent-guides CI
hand-wrote ~/.pi/agent/models.json. Add a first-class `pi` command mirroring
the others:
- write_pi_config writes the session-scoped OpenAI-compatible provider config
(key in the config, like openclaw/opencode).
- pi() launches `pi --provider unsloth --model <id>` (Pi defaults to the google
provider, so the provider/model are pinned on the command line) with HOME
relocated for the session. Pi has no config-dir env var and resolves ~/.pi off
$HOME, so HOME-scoping keeps the user's ~/.pi untouched.
Migrate the agent-guides CI off the hand-written config onto the
`unsloth start pi --no-launch` path (connection + file-edit), with a crosscheck
for the provider api, so the documented recipe is exercised.
* Harden unsloth start for Windows and WSL agent launches
Address the Codex review on PR 6613:
- write_pi_config now pins the loaded contextWindow and a sane maxTokens so Pi
compacts instead of overflowing a small Studio context (it otherwise assumes
its 128000 default), matching the other agents.
- pi() sets USERPROFILE (and HOMEDRIVE/HOMEPATH when present) alongside HOME on
native Windows, where Node resolves ~/.pi via USERPROFILE rather than HOME, so
the session no longer reads or writes the user's real ~/.pi.
- The WSLENV bridge flags path-valued vars with /p so a Windows npm shim under
/mnt receives translated paths, while scalar vars (the numeric context window)
pass through untranslated. WSLENV is deduped on the bare name.
- _print_env prints the launch command with PowerShell-safe quoting so the inline
--settings JSON survives copy-paste on native Windows --no-launch.
Add tests for the WSLENV path flagging, PowerShell quoting, the Pi context
window, and the Pi USERPROFILE relocation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Set CLAUDE_CODE_NO_FLICKER for the Claude session
A local server streams in bursts, so Claude Code's full-screen TUI redraw
flickers between tokens. Disable it for the session via CLAUDE_CODE_NO_FLICKER,
alongside the other CLAUDE_CODE_* session env knobs.
* Add a normalized --yolo flag routed to each agent's auto-approve mode
It is easy to forget which agent spells "run tools without prompting" which way,
so `unsloth start` now accepts all three spellings as one option (--yolo,
--dangerously-skip-permissions, --dangerously-bypass-approvals-and-sandbox) and
routes to the agent's own mechanism:
- claude: --dangerously-skip-permissions
- codex: --dangerously-bypass-approvals-and-sandbox
- hermes: --yolo
- pi: --approve (Pi's only approval gate is project trust)
- opencode: a permission allow block in opencode.json (no CLI flag exists)
- openclaw: tools.exec security=full / ask=off / host=gateway (no CLI flag exists)
Because the option is parsed by `unsloth start`, the "wrong" spelling for an
agent still routes correctly instead of leaking through to the agent and erroring.
IS_SANDBOX is deliberately left unset for Claude so its root/sandbox safety gate
still applies. Adds routing, cross-routing, and per-config tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix review findings: IPv6 loopback command, pi USERPROFILE under WSL, yolo guard
From a 10-reviewer pass over the PR:
- studio/frontend agent-command.ts: normalize bracketed IPv6 hosts. URL.hostname
returns "[::1]" for http://[::1]:8888, which never matched the "::1" loopback
checks, so the copied command embedded the placeholder API key for a local IPv6
server instead of the bare auto-minting command. Now [::1] is treated as loopback
like the CLI's is_loopback_url, so the command matches the CLI contract.
- pi(): also relocate USERPROFILE (and HOMEDRIVE/HOMEPATH) when running under WSL
against a /mnt Windows shim, not just on native Windows. Windows Node resolves
~/.pi via USERPROFILE, and the WSLENV bridge translates the path, so pi no longer
falls back to the user's real ~/.pi in that case.
- _yolo_command_flags: use .get so a config-based agent (or a typo) yields no flag
instead of a latent KeyError.
Adds tests for the WSL pi USERPROFILE relocation, the yolo unmapped-agent guard,
and that opencode/openclaw --yolo stays config-only (no argv flag).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix round-2 review findings: WSLENV /p upgrade, agent help text
- _merge_wslenv now upgrades a user's pre-existing unflagged WSLENV entry (e.g. a
bare HOME or USERPROFILE) to the path-translated form (HOME/p) instead of leaving
it as-is, so a Windows agent shim under WSL receives the translated session path
rather than the raw Linux path.
- Generalize the `unsloth start` registration help to list all six agents (was only
"Claude Code, Codex").
Adds a test for the WSLENV unflagged-entry upgrade.
* Fix round-3 review findings: complete openclaw --yolo, refresh stale copy
- openclaw --yolo now also writes the host approvals file (exec-approvals.json with
defaults security=full / ask=off / askFallback=full) alongside the tools.exec
config. OpenClaw gates tool execution on both layers (the stricter wins), so the
config alone could still leave it prompting or denying. Mirrors `openclaw
exec-policy preset yolo`. ask=off means nothing is ever prompted, so the runtime
socket block is unnecessary.
- Studio API panel copy: clarify that a local server auto-mints the key while a
remote one embeds it in the command, and add pi to the swap hint.
- Local Agent Guides CI: drop the stale "pi has no start.py recipe" note now that
all six agents are driven via `unsloth start <agent> --no-launch`.
Adds the openclaw approvals-file assertions and a no-yolo openclaw test.
* start: parse claude --version with a regex so a format change does not drop optimization flags
* start: offer to install a missing agent (prompt then run its install command)
* start: auto-start a Studio server for --model when none is running, and stop it on exit
* inference: surface an actionable message when llama-server cannot compile a tool grammar
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix review findings: kill the auto-started server tree on Windows; apply the tool-grammar message to the OpenAI passthrough too
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: split --model org/repo:variant so a running session is not evicted
`unsloth start <agent> --model org/repo:QUANT` failed against an already-running
Studio server and, worse, killed whatever model another session had loaded.
/v1/models lists a loaded GGUF under its bare repo id (e.g. unsloth/Qwen3-1.7B-GGUF),
so _resolve_model never matched the `:QUANT`-suffixed request. It then POSTed
/api/inference/load with model_path=org/repo:QUANT, which (a) Hugging Face rejects
("Repo id must use alphanumeric chars, '-', '_' or '.'") and (b) evicts the model the
other session was using, so a second 'unsloth start' in a new tmux/terminal tore down
the first. Re-running the command then attached to the now-empty server, which is why
it 'worked the second time'.
Mirror the org/repo:QUANT -> org/repo + --gguf-variant QUANT shorthand that
'unsloth run' and llama.cpp already accept, splitting it in _connect before we match or
serve. Matching now resolves against the loaded bare repo id (no spurious reload, no
eviction), and any real load uses a valid repo id plus gguf_variant. An explicit
--gguf-variant still wins; local paths and Windows drive letters pass through untouched.
The auto-serve path likewise spawns 'unsloth run --model org/repo --gguf-variant QUANT'.
* start: harden auth-key handling, codex teardown, and CI transcript redaction
Three review findings:
1. CI could leak a live key. agent-guides-drive.sh printed the raw
'unsloth start --no-launch' transcript (which carries export UNSLOTH_API_KEY /
ANTHROPIC_AUTH_TOKEN lines) to the Actions log on both the failure path and the
success path before redact() ran. Add cat_redacted() and use it for those two
prints, so the key is scrubbed on the way to the log while the on-disk file stays
intact for the env parsing that follows.
2. Outages masqueraded as bad keys. _key_accepted caught a broad Exception and
returned False, so a 5xx or timeout while checking a cached key looked like a
rejection: it discarded a good key and minted extra ones (local) or reported 'no
saved key' (remote). Only treat HTTP 401/403 as a rejection; let other errors
propagate so a real outage surfaces.
3. Codex preflight could leave the auto-started server up. _require_gguf_for_codex
runs after _connect may have auto-started Studio but before _run installs its
teardown finally, so a preflight rejection (e.g. a transformers-backend model) left
the server holding the port/GPU until the atexit backstop. Tear it down explicitly
at the point of failure.
Tests: a 5xx on a saved key surfaces without minting; a non-GGUF codex preflight
tears down the auto-served server.
* start: fix IPv6/portless studio URLs, Pi config-dir isolation, and Pi install recipe
Four review findings:
1. Pi ignored the session config when PI_CODING_AGENT_DIR was already set. Pi's
getAgentDir() reads process.env.PI_CODING_AGENT_DIR before falling back to
$HOME/.pi/agent, so a value inherited from the user's shell sent Pi to their real
config and skipped our provider/key (the HOME relocation alone was not enough). Pin
PI_CODING_AGENT_DIR at the session's .pi/agent dir; it is path-valued so the WSL
bridge translates it automatically.
2. Pi install hint dropped Pi's documented --ignore-scripts. Pi's README installs with
'npm install -g --ignore-scripts @earendil-works/pi-coding-agent' and notes it needs
no install scripts, so accepting the prompt now follows that safe recipe.
3. Auto-start ignored a portless UNSLOTH_STUDIO_URL. unsloth run binds to
'parsed.port or 8888', so http://127.0.0.1 launched the child on 8888 but the health
poll (and the returned base) still used port 80, stalling until the startup timeout.
Normalize the base to host:8888 (IPv6-safe) before starting and polling.
4. API-panel command mistook IPv6 loopback for the bare default. The bare 'unsloth
start' only probes 127.0.0.1:8888 on the IPv4 stack, so http://[::1]:8888 must carry
an explicit UNSLOTH_STUDIO_URL. Drop ::1 from the bare-default host set while keeping
it a loopback host (URL emitted, no key needed).
Tests: PI_CODING_AGENT_DIR is set to the session dir; _effective_base normalizes
portless/IPv6 bases; a portless UNSLOTH_STUDIO_URL auto-serves on :8888.
* start: apply fresh-review findings across CLI, CI, and the API-panel command
From a fresh multi-reviewer pass over the merged head plus the latest Codex bot review:
1. Load knobs now always consult the server. _resolve_model matched on model id alone,
so --gguf-variant / --context-length / --no-load-in-4bit / --tensor-parallel were
silently ignored whenever the id was already loaded (asking for UD-Q4_K_XL kept a
Q8_0 serving). With any explicit knob the CLI defers to /api/inference/load, whose
already-loaded dedup answers without reloading when variant and settings match, so a
second session running the same command still attaches without evicting the first.
2. OpenCode --yolo and the session model pin now ride in OPENCODE_CONFIG_CONTENT. A
project's own opencode.json outranks OPENCODE_CONFIG, so a repo config could silently
override the session model and the --yolo permission block; OPENCODE_CONFIG_CONTENT
outranks project config. The API key stays in the private file, never in printed env.
3. The --no-launch recipe's last line is a self-contained one-liner (inline VAR=value
assignments before the command, conflicting vars blanked). People copy just the last
line, and a bare codex/claude there ran against the user's real ~/.codex or Anthropic
credentials with zero isolation, e.g. inheriting a pre-existing damaged ~/.codex
state DB and blaming the recipe. The CI drive script scrubs the key from the one
'invoking:' echo this adds.
4. The auto-serve log is 0600 and the parent handle is closed. It sat world-readable in
the shared tempdir under a predictable name while carrying the minted sk-unsloth-
key from the unsloth run banner.
5. _key_accepted fails with a clean message on outages. Non-auth errors (5xx, network,
timeout) surfaced as a raw traceback; 401/403 still mean a rejected key.
6. _effective_base strips URL paths, and https loopback targets never auto-serve.
http://127.0.0.1:8888/studio polled /studio/api/health (404) and https://127.0.0.1
polled the wrong scheme, both spinning until the 15-minute startup timeout.
7. API-panel command: only literal 127.0.0.1:8888 earns the bare command. localhost can
resolve to ::1, which the bare CLI never probes, so it keeps UNSLOTH_STUDIO_URL.
8. CI artifact sweep covers redacted-configs/ and agent-workdir/, not just logs/.
Tests: 125 CLI tests pass (new coverage for each fix), 156 backend tests pass, ruff
clean. Adds an unsloth connect alias regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: hand Pi a clean screen at launch
Pi paints inline from wherever the cursor sits: its first render assumes a
clean screen instead of clearing or entering the alternate screen itself
(current Pi never emits a clear at startup). Launched under unsloth start,
that left the session starting mid-scroll beneath the connection output.
Clear the screen (click.clear, cross-platform, no-op without a TTY) right
before the Studio banner so Pi opens exactly one line down on a clean
viewport. Launch path only: --no-launch recipes and piped output are never
wiped, and alternate-screen agents are left alone.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: auto-override hermes' 64K context floor for small model windows
Hermes refuses to initialize when the served model's context window is
under 64,000 tokens, and a second copy of the same check rejects the
compression model mid-session. write_hermes_config previously pinned the
real window, so any small local model (e.g. 40,960) failed at startup
with manual config.yaml instructions.
For windows below the floor the recipe now claims 65,536 in
model.context_length, scales compression.threshold so compaction still
fires at 90% of the real window, and sets
auxiliary.compression.context_length to cover the mid-session check.
Windows at or above the floor keep the exact previous behavior.
* ci: install pi with --ignore-scripts, matching the start.py hint
The pi cell predates the pi recipe in start.py and still installed the
package with lifecycle scripts enabled, so CI stopped exercising the
exact command users are prompted to run. npm_retry now passes extra
flags through, the pi branch mirrors the install hint verbatim, and the
stale no-recipe comment is refreshed.
* ci: fail loudly when a relocation var is missing from connect output
The empty-string guards ran after appending /config.toml or /config.yaml,
so they could never fire: crosscheck_contract silently skipped its
contract checks and patch_hermes_tools died on the root path with a bare
traceback. Check the raw variable first and guide_fail with the real
cause.
* staging: 6613 round 6 (https elision, no-launch home reuse, auto-start key fallback)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* Add --with-llama-cpp-dir flag to install.ps1 and install.sh
Users can now pass --with-llama-cpp-dir /path/to/llama.cpp to the
installer to skip downloading or building llama.cpp and use a local
directory instead. A junction (Windows) or symlink (Linux/macOS) is
created at the canonical install location, bypassing both the prebuilt
download (Phase 3) and source build (Phase 4) steps in setup.ps1/setup.sh.
The path is passed via UNSLOTH_LOCAL_LLAMA_CPP_DIR env var which
setup.ps1 and setup.sh read directly.
Ported from the idea in unslothai/unsloth#4384, reimplemented against
current Studio architecture.
* test: add static wiring test for --with-llama-cpp-dir flag
Cross-checks install.sh, install.ps1, studio/setup.sh and studio/setup.ps1
so the flag's contract (parse -> UNSLOTH_LOCAL_LLAMA_CPP_DIR env var -> link
local dir, skip prebuilt download and source build) can't silently regress.
Wired into studio-backend-ci.yml alongside the other tests/sh installer tests.
* Address review feedback on --with-llama-cpp-dir flag
- setup.ps1: delete an existing junction/symlink via DirectoryInfo.Delete()
instead of a recursive remove, which can traverse the link and wipe the
user's real llama.cpp directory on PowerShell 5.1.
- setup.ps1: short-circuit the build chain when a local dir is linked so CMake
never runs inside the user's checkout when it lacks a Windows-layout binary.
- install.sh / setup.sh: resolve paths with CDPATH= cd -P so a set CDPATH
cannot corrupt the resolved path.
- install.sh: seed _WITH_LLAMA_CPP_DIR from UNSLOTH_LOCAL_LLAMA_CPP_DIR so an
exported env var (piped-install style) is honored instead of being clobbered.
- setup.sh: create the root llama-quantize shim when linking a local source
build so GGUF export's check_llama_cpp() still finds it.
- setup.sh / setup.ps1: drop a stale link before the custom-home ownership
assert so re-runs with the flag stay idempotent.
- test: pin the new linked-dir build short-circuit.
* Harden --with-llama-cpp-dir against Codex/Gemini review findings
- install.sh: error when --with-llama-cpp-dir is the final arg with no path,
matching the existing --package/--python post-loop guards (was a silent
fallback to the normal prebuilt/source install).
- studio/setup.sh: canonicalize LLAMA_CPP_DIR before the self-link no-op
compare. _RESOLVED_LOCAL is fully resolved while LLAMA_CPP_DIR was textual,
so a symlinked $HOME made the guard miss and the rm -rf could wipe the
user's real llama.cpp tree.
- studio/setup.sh: make the llama-quantize shim non-fatal; it writes through
the link into the user's tree, which may be read-only (shared/CI cache),
and under set -e a failed ln aborted an otherwise-good reuse.
- studio/setup.ps1: detect a broken junction via Get-Item -Force instead of
Test-Path so a dangling link from a prior run is removed and mklink can
relink to a new valid directory.
- studio/setup.ps1: use Copy-Item -LiteralPath so a source path containing
[ ] isn't treated as a wildcard in the junction copy fallback.
- tests: update the wiring assertions for the LiteralPath copy and the
canonicalized compare.
* Validate/reuse local llama.cpp tree and guard the in-use case
Addresses the second Codex pass on the --with-llama-cpp-dir flag:
- Validate the linked tree before disabling installs (setup.sh + setup.ps1):
reusing a local dir skips BOTH the prebuilt download and the source build,
so the dir must already contain a runnable llama-server (build/bin on
Linux/macOS, build\bin\Release\llama-server.exe on Windows). Bail out with a
clear message instead of linking an unbuilt/wrong-platform checkout and
leaving Studio with no usable binary.
- Treat a canonical-path target as already linked when it holds a build
(setup.sh + setup.ps1): point the flag at ~/.unsloth/llama.cpp itself and an
existing build is reused (skip prebuilt + source) rather than clobbered by
the staged prebuilt installer (which uses os.replace()/replace). An empty
canonical dir still falls through to the normal in-place install.
- Abort when an in-use llama.cpp can't be removed on Windows (setup.ps1):
Remove-Item -ErrorAction SilentlyContinue can silently leave a locked tree
in place; detect that and stop with the same active-process message + exit 3
the prebuilt path uses, instead of junctioning over a half-present dir.
Left as follow-up (already tracked by the PR author as a non-blocker): the
in-app "Update llama.cpp" updater does not yet recognize a local-link install
as externally managed; that fix belongs in studio/backend/utils/llama_cpp_update.py.
* Accept all backend llama-server layouts in --with-llama-cpp-dir validation
The linked-tree validation only accepted build/bin[/Release]/llama-server, but
LlamaCppBackend._layout_candidates() resolves a root-level llama-server first,
then build/bin, then build/bin/Release on Windows. A `make` build or a flat
release extract (binary at the dir root) was therefore rejected with a hard
installer failure even though Studio would have run it.
Validate the same candidate set the backend uses in both setup scripts, and add
wiring-test assertions so the check can't silently narrow again.
* Treat --with-llama-cpp-dir local links as externally managed
A --with-llama-cpp-dir install junctions/symlinks the canonical llama.cpp dir to
the user's own checkout, but two backend paths still treated it as a Studio-owned
tree:
- The in-app updater (llama_cpp_update) offered and could apply an official
prebuilt over the link, writing through it into the user's checkout (or
failing) and silently dropping the link the flag created.
- Orphan cleanup (LlamaCppBackend._kill_orphaned_servers) resolved the linked
root into its kill allowlist, so a llama-server the user launched from the same
checkout was classified as ours and killed on startup.
Detect the canonical dir being a symlink/junction (reparse point) and treat the
install as unmanaged: get_update_status reports unsupported, start_update refuses
with reason "local_link", and the linked root is left out of the orphan
allowlist. Adds behavioral tests (link vs plain dir, updater refusal, and the
spared-vs-killed orphan control).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add behavioral shell test for --with-llama-cpp-dir linking
The existing tests/sh/test_with_llama_cpp_dir_flag.sh is a static grep of the
scripts. This adds a behavioral test that extracts the real link block from
studio/setup.sh (by content anchors, with a self-validating extraction) and runs
it against hermetic fake dirs, asserting the outcomes that matter:
- an external CMake build links and arms neither the prebuilt download nor the
source build
- a flat / make tree (root-level llama-server, no build/bin) is accepted too
- an unbuilt tree is rejected with a non-zero exit and no link left behind
- relinking over a stale link preserves the target's contents (no data loss)
- pointing at the canonical path is a no-op reuse, not a self-referential link
Symlink-identity checks run only where real symlinks exist (skipped on Windows
git-bash copy-mode); the link/skip/no-data-loss checks run everywhere. Wired into
studio-backend-ci.yml next to the static test.
* Install psutil in backend CI so orphan-cleanup tests run
The new orphan-cleanup tests import psutil for the process scan, but the Backend
CI deps step installed studio.txt plus a fixed extras list that omits it, so the
two tests failed with ModuleNotFoundError. Add psutil to both backend pytest dep
steps (kept in shared shape), and guard the import with pytest.importorskip so a
minimal env without psutil skips these tests instead of erroring.
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add FP8/FP4 compressed export to save_pretrained_merged
Adds compressed-tensors export (for vLLM) to save_pretrained_merged /
push_to_hub_merged via llm-compressor, alongside the existing lora /
merged_16bit / merged_4bit / gguf / torchao paths:
model.save_pretrained_merged("model", tokenizer, save_method="fp8")
Supported save_method values: fp8 (FP8_DYNAMIC), mxfp4, nvfp4 (W4A4) and
mxfp8. The LoRA is merged to 16bit at save_directory, then a quantized
checkpoint is written to save_directory + "-<fmt>". nvfp4 needs a small
calibration set (defaults to ultrachat, overridable via calibration_dataset).
Notes:
- llm-compressor is installed lazily on first use, pinning the current torch
and transformers via a constraints file so they are not upgraded (a plain
install pulls transformers>=5 and breaks Unsloth).
- Quantization runs in a separate process (unsloth/_compressed_quantize.py,
launched by file path) so Unsloth's transformers attention patches do not
interfere with the forward llm-compressor runs during calibration, mirroring
how GGUF export shells out to llama.cpp.
- mxfp8 needs a newer llm-compressor (transformers>=5); it is recognised and
raises a clear error until that stack is available.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: main-process guard, calibration subsampling, tokenizer + dtype handling
- Route the 16bit merge through unsloth_generic_save for both LoRA and full
finetuned models, so non-PEFT models are written in 16bit consistently
instead of saving the original (possibly quantized) weights directly.
- Honor is_main_process: only the main process quantizes and writes the
compressed output, so distributed ranks do not race on the same dirs.
- Subsample an in-memory calibration Dataset before save_to_disk so large
training sets are not fully copied to a temp dir.
- Tolerate a missing tokenizer in the converter (data-free exports); still
require one for calibration based schemes.
- Open config.json via a context manager in both files.
- Drop the redundant nvfp4 entry from the unsupported-name check (fp4 covers it).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add direct LoRA to GGUF export and harden FP8/FP4 compressed export
- Run llm-compressor install and scheme check before the 16bit merge so
unsupported schemes (e.g. mxfp8) fail fast without writing a checkpoint
- Only the main process installs, merges, quantizes and uploads; isolate
hub pushes to a temp dir and clean all temp dirs in a finally
- Forward standard save kwargs (state_dict, max_shard_size, ...) to the merge
- Fall back to the first dataset split for Hub calibration ids
- Export LoRA adapters to GGUF via convert_lora_to_gguf.py: modernize
save_pretrained_ggml/push_to_hub_ggml and add save_method="lora" to
save_pretrained_gguf/push_to_hub_gguf; resolve base from the adapter config
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix LoRA GGUF shell-injection test and compressed export trailing-slash path
- Update tests/saving/test_save_shell_injection.py for the new delegation: the
LoRA to GGUF conversion now lives in _unsloth_save_lora_gguf, so assert it
passes argv as a list with no shell=True and that the legacy ggml wrappers
delegate to it instead of calling subprocess.Popen directly
- Normalize the local save_directory before building the "<dir>-<fmt>" sibling
so a trailing slash no longer nests the compressed output inside the 16bit dir
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Polish FP8/FP4 and LoRA GGUF export after review
- Warn (not silently downgrade) when an explicit quantization_method is not a
valid LoRA GGUF outtype; default stays f16
- Correct the inference hardware note: MXFP8 is 8-bit (cc >= 8.9), only FP4
needs Blackwell for full activation quantization
- Document that a local fp8/fp4 save keeps the 16bit merge at save_directory
and writes the quantized checkpoint to save_directory + "-<fmt>"
* Use sequential calibration pipeline and validate Hub access early
- nvfp4 calibration no longer forces the memory-hungry "basic" pipeline. The
quantization runs in a clean subprocess, so llm-compressor's default
sequential pipeline (layer-by-layer onloading) works and lets large models
that do not fit at once still calibrate; fall back to "basic" only if tracing
fails
- For push_to_hub compressed exports, create/validate the repo up front so a bad
token or denied repo fails before the merge and quantization instead of after
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden compressed export: explicit sequential pipeline, base-tokenizer calibration, GPU memory
- nvfp4 calibration now passes pipeline="sequential" explicitly (layer-by-layer
onloading) instead of relying on the inferred default, with a "basic" fallback
- Calibration datasets with a messages column no longer require a chat template:
base / non-chat tokenizers fall back to concatenating message contents
- Free the in-memory model's CUDA memory before the quantize subprocess loads its
own copy from disk (best-effort, single-device non-quantized only; restored
afterward), so a single GPU need not hold two copies at once
- Create the calibration temp dir in the system temp location instead of next to
the save directory, avoiding stray dirs in the workspace
* Free the failed calibration model before the basic-pipeline retry
In the sequential -> basic NVFP4 fallback, release the partially-processed model
and clear the CUDA cache before loading a fresh copy, so the retry does not
transiently hold two model copies on the GPU.
* Harden calibration data handling and compressed-export edge cases
- Calibration messages without a chat template now handle multimodal (list)
content, None content, and null message rows instead of crashing on join
- Raise a clear error when the calibration dataset is empty after subsampling
- Reset llm-compressor's global session before freeing the model in the
sequential -> basic NVFP4 fallback, so the old model is actually released
- LoRA GGUF export accepts a single-element list quantization_method
- Attach datasets metadata to the pushed repo on compressed hub exports
- Warn (instead of silently) if the model cannot be restored to its device
- Raise a clear error if the LoRA base model id cannot be determined
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle DatasetDict calibration, MoE routers, and MTP models in compressed export
- Reduce an in-memory DatasetDict calibration set to a single split before row
subsampling, so save_to_disk does not copy every split to the temp dir
- For MoE models, keep the router/gate unquantized and pass
moe_calibrate_all_experts so every expert is calibrated
- Warn when a model carries MTP / speculative-decoding tensors that the
compressed export does not include
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Support many more compressed-tensors schemes and address review
- Expand save_method to cover the full set of compressed-tensors preset schemes:
FP8 (dynamic/static/block), INT8, W8A8, W8A16, W4A16(+asym), W4A8, W4AFP8,
MXFP4(+A16), NVFP4(+A16), plus the gated MXFP8; calibration is used only for
the static-activation schemes (FP8 static, NVFP4)
- Broaden the near-miss save_method error to cover int/w-prefixed names
- MoE: also keep the Qwen shared-expert gate unquantized
- Strip non-model-input columns from already-tokenized calibration data so the
collator does not choke on a leftover messages column
- Forward the Hub token to the LoRA converter and the quantize subprocess so
gated/private base models and calibration datasets work without a global login
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Collapse compressed-tensors export help line so ruff-format converges
The print line in print_quantization_methods needed two ruff-format passes to
reach a fixpoint (merge implicit string concat, then collapse the single-arg
print). pre-commit.ci applies one pass per run, so it kept reformatting. Land
the converged single-line form directly.
* Add CPU-only regression tests for the export API
Cover all export paths without a GPU, for slow CPU-only CI:
- pure-function checks of the compressed-tensors scheme registry and save_method
normalization (aliases, calibration flags, near-miss errors)
- AST checks that every merged saver dispatches compressed export, the GGUF savers
expose the lora branch, torchao routes PTQ/QAT, the public methods stay attached,
and the export subprocesses remain shell-safe (argv list, sys.executable, no shell)
- monkeypatched dispatch checks that fp8/nvfp4/merged_16bit, the LoRA-GGUF outtype
resolution, and torchao PTQ/QAT reach the right helper with the right arguments
* Run the CPU-only export tests in consolidated CI
tests/saving is --ignored by the Repo tests (CPU) job, so the new GPU-free export
tests are added by path to consolidated-tests-ci.yml (collection sanity + Bucket-A run),
alongside the existing CPU saving tests, so they actually execute on CPU CI.
* Add GPU GGUF export + llama-cli inference smoke test
tests/saving/test_gguf_export_and_inference.py: skipif no CUDA. Trains a tiny
phrase-imprinting LoRA, exports a full-model q8_0 GGUF (merge -> convert_hf_to_gguf
-> llama-quantize), asserts a valid GGUF (magic + size), and - when a llama-cli
binary is available - runs one bounded generation (byte cap + watchdog kill) and
asserts the trained phrase round-trips through HF -> GGUF -> quantize -> inference.
The llama-cli step skips gracefully since the export only builds llama-quantize.
* Fix variant mismatch in compressed (FP8/FP4) export
save_pretrained_merged(..., save_method=fp8/nvfp4, variant=...) forwarded
the variant into the intermediate 16bit merge, so Transformers wrote
variant-named shards (model.<variant>.safetensors). The converter
subprocess then reloaded that directory with the default weight filenames,
so the compressed export failed after doing the merge.
Pop the variant out of the intermediate merge (internal staging that the
subprocess reloads with default names) and forward it via --variant so it
is applied to the final compressed checkpoint instead. Add a CPU AST guard
for the contract.
* Harden export paths from review
- install_llm_compressor: fall back to uv pip when this interpreter has no
pip seeded (uv-created/relocatable venvs), instead of failing with
No module named pip.
- LoRA GGUF export: if convert_lora_to_gguf.py is missing (a prebuilt or
reused CWD llama.cpp install carries binaries but not the converter
script), force a dedicated source checkout that ships it.
- push_to_hub_gguf(save_method=lora): return on non-main ranks, matching the
local save_pretrained_gguf lora branch, so only rank 0 converts/uploads.
- compressed export VLM detection: require a vision_config or a
ForVisionText2Text architecture; a bare *ForConditionalGeneration also
matches text seq2seq models (T5/BART/Whisper) and is no longer treated as
a VLM on its own.
- GGUF GPU smoke test: drop SFTConfig(max_length=1024), which raises under
newer TRL padding-free training; length enforcement is not needed here.
* Add imatrix option to GGUF export, enabling IQ low-bit quants
save_pretrained_gguf / push_to_hub_gguf gain imatrix_file:
None -> no imatrix (unchanged)
'/path' -> pass to llama-quantize --imatrix (a *.gguf_file is renamed to *.gguf)
True -> download the upstream unsloth/<base>-GGUF imatrix (imatrix_unsloth.dat or
.gguf_file), raising a clear error if none exists
An importance matrix unlocks the IQ low-bit quants (iq2_xxs, iq4_xs, ...), which were hard
disabled before. They are gated: requesting one without an imatrix raises a clear error.
- _resolve_imatrix_file resolves path/True (PEFT base first, normalized via get_model_name,
derives unsloth/<base>-GGUF, copies out of the HF cache before renaming *.gguf_file).
- IMATRIX_QUANTS registry replaces the old commented-out IQ entries; save_to_gguf accepts a
resolved imatrix and threads it into the quantize calls.
- The --imatrix flag is emitted by unsloth_zoo's quantize_gguf (companion change). save.py
fails fast with an upgrade hint if the installed unsloth_zoo lacks the imatrix kwarg.
Tests: tests/saving/test_imatrix_export.py (CPU: resolution, repo derivation, IQ gate,
--imatrix wiring) wired into CI; tests/saving/test_gguf_export_and_inference.py extended with
GPU iq2_xxs/iq4_xs export + inference. Verified end to end on Llama-3.2-1B: imatrix
auto-downloaded, iq2_xxs/iq4_xs exported and run via llama.cpp.
Note: requires the companion unsloth_zoo quantize_gguf imatrix change.
* Address imatrix/compressed review feedback: unsloth org GGUF repo, fail-fast, calibration split
- imatrix auto-resolve (imatrix_file=True): derive the upstream repo as unsloth/<base>-GGUF
instead of <org>/<base>-GGUF, so official bases (e.g. meta-llama/Llama-3.1-8B-Instruct) find
the matching Unsloth GGUF imatrix repo rather than failing on a nonexistent meta-llama/...-GGUF.
- Resolve/validate the imatrix before the 16-bit merge in save_pretrained_gguf, so a bad path or
an unavailable upstream imatrix fails fast instead of after a long, multi-GB merge.
- Compressed calibration: when a Hub dataset has no "train" split, resolve the first split name
and slice it, instead of materializing the whole dataset just to take num_samples rows. Keeps
the original materialize-then-subselect path as a last resort.
Tests: add unsloth/<base>-GGUF mapping for an official base id, and create the imatrix file in the
quantize_gguf flag test (quantize_gguf now validates the imatrix exists).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: stop handing CI/user secrets to downloaded llama.cpp binaries
The macOS prebuilt path installs llama.cpp from the unslothai/llama.cpp
fork's latest (unpinned, mutable) release and then executes the
downloaded llama-server / llama-quantize binaries during install-time
validation. binary_env() built that child environment from a full
os.environ.copy(), so a compromised or tampered prebuilt would inherit
every secret in the process: HF_TOKEN and the workflow GitHub tokens in
CI, and HF / cloud credentials for end users running install.sh /
setup.sh.
We publish prebuilts daily, so pinning a release tag is not workable.
Instead, neutralise the impact: these binaries have no reason to read any
token, so strip secret-bearing variables (exact names plus
TOKEN/SECRET/PASSWORD/CREDENTIAL/PRIVATE_KEY/API_KEY markers) before
handing the env to a downloaded binary. The installer's own GitHub and
Hugging Face API calls read os.environ directly, so authentication and
release-API rate limiting are unaffected; PATH, LD_LIBRARY_PATH,
DYLD_LIBRARY_PATH and CUDA/ROCm vars are preserved. One change covers the
install-time validation path for all six macOS workflows and end users.
Follow-up (separate, sequenced): publish build-provenance attestations
from the fork's prebuilt workflows and verify them in CI, so a forged
release is rejected rather than merely starved of secrets.
* Strip KUBECONFIG, SSH_AUTH_SOCK, and PASSPHRASE-marked vars from binary env
Extend the deny-list per PR review: KUBECONFIG and SSH_AUTH_SOCK are
credential pointers/capabilities a downloaded binary never needs, and a
PASSPHRASE marker catches SSH_PASSPHRASE / GPG_PASSPHRASE. Tests updated.
* Studio: also scrub proxy/index env vars and URL-embedded credentials before running prebuilt binaries
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope mlx-ci secrets to the install + download commands for PR #6696
Drop the ambient step-level env block and pass GH/GITHUB/HF tokens only
on the installer and GGUF-download commands, so the directly invoked
llama-quantize / llama-server smoke runs see no secrets. The installer
still reads tokens from os.environ for the releases API and probe fetch.
* Trim verbose comments around the secret-env scrubber for PR #6696
Comment-only: condense the block comments added across this PR. Logic
unchanged (comment_tools.py check confirms code-only signature equal).
* Redirect HOME / cache pointers to an empty dir for prebuilt binaries (PR #6696)
Address Codex P2: stripping token env vars still let a tampered binary
read on-disk token stores (~/.cache/huggingface/token, ~/.aws/credentials,
~/.config/gh) through $HOME and the cache/config pointers. Point HOME plus
the HF / XDG / Windows home pointers at a single empty throwaway dir for
the downloaded-binary env. Defense in depth: a binary resolving the real
home via getpwuid is out of scope and needs OS sandboxing.
* Close residual credential-probe gaps for PR #6696
Address the latest Codex review:
- Strip token-only URL userinfo too (scheme://ghp_token@host), not just
the user:pass form.
- Redirect HOMEDRIVE/HOMEPATH alongside USERPROFILE so a Windows binary
cannot reconstruct the real profile from %HOMEDRIVE%%HOMEPATH%.
- Drop explicit credential-file pointers (NETRC, PIP_CONFIG_FILE,
DOCKER_CONFIG, GIT_CONFIG_GLOBAL) that live outside HOME.
- Probe ldd with a secret-free env: linux_runtime_dirs ran ldd on the
untrusted prebuilt with the inherited os.environ, and ldd may execute
the binary, so it could observe HF_TOKEN/GITHUB_TOKEN during the probe.
Factored the shared scrub into secret_free_environ().
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Separate token-bearing install from binary smoke; drop CI command files (PR #6696)
Address the two P1s in the latest review:
- mlx-ci: GitHub bakes secrets into the run-script text, so inline token
assignments in a step that later runs the prebuilt let a tampered binary
read them from the script. Split into a token-bearing install + download
step that never launches a binary, and a secret-free smoke step that runs
llama-quantize / llama-server.
- secret_free_environ now drops the GitHub Actions command files
(GITHUB_ENV, GITHUB_PATH, GITHUB_OUTPUT, GITHUB_STEP_SUMMARY, BASH_ENV) and
the smoke step unsets them, so a tampered prebuilt cannot inject PATH/env
into the later token-bearing MLX steps.
* Run the prebuilt smoke last, after all token-bearing steps (PR #6696)
Address the P1 workspace-poisoning vector: even with no secrets in its env,
a tampered prebuilt could edit the checkout or installed modules, and the
later HF_TOKEN MLX steps would then execute that poisoned code on push
builds. Move the prebuilt install + smoke to the end of the job so the
untrusted binary runs after every token-bearing step, leaving nothing for it
to corrupt. The MLX GGUF reload uses a source-built llama-cli, not this
prebuilt, so nothing depends on the earlier position.
* Trim comments around the secret-env scrubber and prebuilt CI steps (PR #6696)
Comment-only: condense the security-rationale block comments and merge the
duplicated prebuilt-step description in mlx-ci. Logic unchanged
(comment_tools.py check confirms the code-only signature is equal; install
suite still passes).
* Authenticate the GGUF export release-API lookup with the read-only GITHUB_TOKEN (PR #6696)
* Rename env scrubber off the secret-named identifier CodeQL flags as a clear-text sink (PR #6696)
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Two intermittent Studio CI failures, both runner-environment flakes unrelated
to test logic:
Windows 'Studio install + inference without Visual Studio': the 'Hide Visual
Studio + CMake' step renames C:\Program Files\Microsoft Visual Studio to
simulate a host with no build tools. A background handle on a Program Files
directory (Defender scan or an MSBuild node) makes Rename-Item intermittently
fail with 'Access is denied', and $ErrorActionPreference = Stop turns that into
a hard job failure. Wrap the VS and cmake renames in both Hide steps in a short
Rename-WithRetry (6 tries, 3s apart) to ride out the transient lock.
macOS 'Chat UI Tests': the re-login goto to /login can be interrupted by the
SPA auth guard redirecting to the same /login URL, which Playwright reports as
'Navigation to .../login is interrupted by another navigation to .../login'.
The goto already tolerated ERR_ABORTED; broaden it to also tolerate the same-URL
interrupt (the password-field wait right after confirms we landed on /login),
and add the same signature to the two Playwright flake-retry harnesses as a
safety net for any other navigation.
Validated: playwright_chat_ui.py parses + byte-compiles, both workflow YAMLs
parse, bash -n on the retry harnesses, PowerShell AST parse on all pwsh steps,
and a functional check of Rename-WithRetry (succeeds, and rethrows after
exhausting retries).
* Verify linuxdeploy AppImage digest before use in desktop release
The desktop release workflow downloaded linuxdeploy-x86_64.AppImage from a
GitHub release and ran chmod +x with no integrity check. Pinning the
versioned release path is reproducibility, not integrity: a release asset
can be replaced (or its delivery path compromised) after upload. The next
step builds the AppImage with the Tauri signing private key and a
contents:write GITHUB_TOKEN in scope, so a substituted linuxdeploy that
ran during packaging could exfiltrate signing material or tamper with
published release artifacts.
Pin the immutable SHA-256 of the asset and verify it with sha256sum -c
before chmod +x, so a mismatch fails the job closed before the binary is
ever executable. Extend the existing in-workflow guard to require both the
pinned digest and the verification step, so a future edit cannot silently
drop the check.
* Scope linuxdeploy guard to real step content, not its own text
The self-check searched every workflow line, so the digest assertion was
satisfied by the guard's own expectedLinuxdeployDigest line and the
verification assertion by a comment. Deleting the LINUXDEPLOY_SHA256 env
pin or the actual sha256sum -c command would still have passed.
Match the digest against the LINUXDEPLOY_SHA256 env line specifically and
require sha256sum -c on a non-comment line, so dropping either the pin or
the verification now fails the guard.
* Scope linuxdeploy guard to the Pin step block and check ordering
The previous predicate still scanned the whole workflow, so the literal
sha256sum -c in the guard's own code satisfied the verification check; a
deleted or post-chmod verification command would still pass.
Extract the 'Pin linuxdeploy for AppImage' step block and assert within it:
the LINUXDEPLOY_SHA256 env pins the expected digest, a non-comment line
runs sha256sum -c, and that verification precedes chmod +x.
* Withhold HF_TOKEN from pull_request runs of CI workflows
The pull_request-triggered CI workflows check out and execute PR-controlled
code (install.sh, .github/scripts/**, tests/**) with secrets.HF_TOKEN in the
step environment. For a same-repo PR, GitHub provides repository secrets to the
run, so a malicious or compromised branch could modify a checked-out script to
read and exfiltrate HF_TOKEN, including by writing it into the uploaded logs/
artifact. HF_TOKEN is an external Hugging Face credential of unknown scope, so
this is the high-value exposure.
Gate every HF_TOKEN reference in these workflows with
`github.event_name != 'pull_request' && secrets.HF_TOKEN || ''`, so the real
token flows only on the trusted schedule/push/workflow_dispatch runs and PR runs
see an empty string. All model repos used by these jobs are public
(unsloth/*-GGUF), so anonymous download still works on PRs; install_llama_prebuilt.py
only sends HF auth to Hugging Face hosts and tolerates an absent token.
GITHUB_TOKEN (passed as GH_TOKEN) is intentionally left in place: it is the
auto-provisioned, job-scoped, contents:read token that expires with the job and
gives a same-repo PR author nothing they do not already have, and
install_llama_prebuilt.py needs it to authenticate the GitHub releases API or
the prebuilt llama.cpp download hits the anonymous rate-limit bucket and 403s.
* Trim the HF_TOKEN gating comments to one line per site
Comment/whitespace-only: collapse the per-step rationale to a single line and
shorten the local-agent-guides header note. No workflow logic changes (verified
each file's parsed YAML is identical before/after).
* studio/setup.sh: guard empty CUDA arch detection in the source build
PR #5826 hardened setup.sh for fresh CUDA toolkits, but the source build
still set -DCMAKE_CUDA_ARCHITECTURES only when nvidia-smi reported a
compute capability. When that query returns nothing the build proceeded
with no explicit arch list, so llama.cpp built PTX only. On a driver older
than the toolkit that binary fails at runtime with "the provided PTX was
compiled with an unsupported toolchain" - the build succeeds, so neither
the build-time check nor the CPU fallback caught it (issue #5854).
Resolve the arch list before committing to a CUDA build. A new pure helper
_resolve_cuda_archs parses and de-duplicates the nvidia-smi compute_cap
output and honors an explicit UNSLOTH_LLAMA_CUDA_ARCHS override. When the
result is empty, build CPU llama.cpp instead of a PTX-only binary, with a
clear message pointing at the override - so the user still ends up with a
working llama-server. The override also lets advanced users force a native
build on hosts where nvidia-smi cannot report compute_cap.
No behavior change when an arch is detected: -DGGML_CUDA=ON plus the arch,
CUDA flags and NVCC_PREPEND_FLAGS are assembled exactly as before.
Adds tests/sh/test_resolve_cuda_archs.sh (single/multi/dedup/empty/garbage/
whitespace/override cases), wired into tests/run_all.sh and the
studio-backend-ci.yml shell-test loop.
* studio/setup.sh: resolve nvidia-smi via /usr/bin fallback for arch detection
Addresses review feedback on the empty-CUDA-arch guard: _setup_has_usable_nvidia_gpu
classifies a host as NVIDIA-usable using nvidia-smi on PATH OR /usr/bin/nvidia-smi,
but the new arch detection probed only `command -v nvidia-smi`. On a GPU host where
nvidia-smi is off PATH (reachable only at /usr/bin), arch detection returned empty
and the new empty-arch branch dropped the build to CPU, losing CUDA. Mirror the same
PATH-then-/usr/bin resolution so those hosts still get a native CUDA build.
Also scope _resolve_cuda_archs locals with `local` (no behavior change; it already
runs under command substitution).
* tests: update compute_cap-probe assertion for $_smi_bin resolution
The nvidia-smi /usr/bin fallback parameterized the binary in the compute_cap
probe (_setup_run_smi "$_smi_bin" ...), so the literal-string assertion in
test_compute_cap_probe_timeout_wrapped no longer matched. Assert the probe is
preceded by _setup_run_smi (timeout-wrapped) instead, scanning all occurrences
so the comment mention is ignored. Same intent, binary-agnostic.
* tests: ruff-format the compute_cap probe assertion (pre-commit)
Collapse the backslash-continued assert onto one line and normalize slice
spacing so the ruff-format pre-commit hook (0.6.9) is satisfied. Formatting
only; no behavior change.
* Tighten code comments (no logic change)
* studio(windows): build CPU when CUDA arch is undetectable (#5854)
The Windows source build added -DGGML_CUDA=ON unconditionally but only set
-DCMAKE_CUDA_ARCHITECTURES when $CudaArch was detected. With no detectable
compute capability that produced a PTX-only binary, the same hole the Linux
fix closed. Build CPU llama.cpp in that case, and honor UNSLOTH_LLAMA_CUDA_ARCHS
to force a CUDA build, matching setup.sh. Detected-arch builds are unchanged.
* test: anchor NVCC_PREPEND_FLAGS scope check on the final CPU branch
The undetectable-arch CPU fallback adds an earlier -DGGML_CUDA=OFF, so the
ordering check now anchors on -DGGML_CUDA=ON and the last -DGGML_CUDA=OFF
instead of the first.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Three independent upstream CI fixes that currently fail on every open PR:
verify_import_hoist.py: TARGET-CHANGED only flags a genuine swap (a BEFORE
target no longer reachable in AFTER). A pure superset growth such as adding
import urllib.error next to import urllib.request binds the same top-level
package and loses nothing, so it is no longer a blocker (transformers_version.py).
test_vision_cache.py: run each test from a fresh empty cwd. is_vision_model
calls is_local_path first, and a relative model id that happens to exist on
disk short-circuits before the mocked detection runs; the CI cwd and HF cache
can contain dirs colliding with the synthetic ids, causing 'called 0 times'.
Production code is correct; only the test needed cwd isolation.
consolidated-tests-ci.yml: the llama.cpp smoke probes the first of
llama-cli / llama-mtmd-cli / llama-server that exists instead of hard-requiring
llama-cli, which upstream no longer always builds. llama-cli stays first so it
is preferred when present. Adds Windows .exe + build/bin/Release handling.
Boot `unsloth run --disable-tools` against a small GGUF and drive each
supported coding agent (claude, codex, hermes, openclaw, opencode, pi)
through its documented `unsloth connect <agent> --no-launch` recipe, so
the connect flow in unsloth_cli/commands/connect.py stays exercised end
to end and regressions surface as a failing check.
Per-agent matrix, three jobs:
- connection: assert a non-empty, error-free reply to a trivial prompt
- file-edit: a two-turn create-and-run hello.py test (dispatch/schedule
only, skipped on pull_request)
- prompt-cache: verify llama.cpp prefix-cache reuse across requests
The GitHub-hosted runners are CPU-only, so each request is trimmed to
the smallest prompt that still drives the recipe: claude with --tools to
drop unused tool schemas (--allowedTools only gates permission, it does
not shrink the prompt), hermes with an empty platform_toolsets.cli, and
openclaw with a minimal agent definition. hermes and openclaw run a
multi-turn tool loop in file-edit that a CPU runner cannot finish in
time, so those two cells are best-effort; their endpoint wiring is still
hard-gated by the connection job.
A preflight step HTTP-checks each agent's API dialect before install so a
server-side contract regression is reported separately from agent or
guide drift.
* Windows installer: fix DiskPart UAC, drive-root cache, spurious rename warning, CPU-base messaging
amd-smi gate (DiskPart UAC mid-install): the AMD torch wheel ships hipInfo.exe
inside the venv, and the bitsandbytes fix prepends that venv Scripts dir to PATH.
shutil.which("hipinfo") then found it and flipped _amd_smi_allowed() to True, so
the post-install AMD probe fell through to `amd-smi list` (the venv hipInfo failed
to report gcnArchName, which is why the arch came from the GPU-name table) and
amd-smi elevated, popping the DiskPart UAC. Fix: a hipinfo resolved inside the
active venv (sys.prefix) is the torch-wheel binary, not a HIP SDK, and must not
open the gate. Mirrored in install_python_stack.py, install_llama_prebuilt.py, and
backend utils/hardware/amd.py (the runtime VRAM poller had the same latent prompt).
TORCHINDUCTOR_CACHE_DIR: move from C:\tc to <StudioHome>\TORCHINDUCTOR_CACHE_DIR so
the inductor/Triton cache lives under the user's Studio home, not the system drive
root. Long paths are already enabled above so deep inductor paths still fit.
unsloth.exe rename: skip the rename (and its "pip may fail with WinError 32"
warning) when SKIP_STUDIO_BASE=1. In the install.ps1 flow base packages are not
reinstalled, so unsloth.exe is never rewritten; the self-rename only failed because
setup runs via unsloth.exe (the running launcher holds its own file). The
'studio update' flow still attempts it.
CPU PyTorch messaging: clarify that the CPU base is temporary and setup replaces it
with GPU ROCm wheels, and print an explicit "GPU ROCm PyTorch installed" line after
the AMD wheels land, so the log makes clear the final install is GPU-accelerated.
Adds two regression tests covering the venv-internal vs external hipInfo gate.
Verified end-to-end on a Strix Halo box (Radeon 8060S / gfx1151): install.ps1
--local from this branch completed exit 0 with no DiskPart prompt, no rename
warning, the cache under the Studio home, and "GPU ROCm PyTorch installed
(gfx1151)"; Studio then booted and detected "ROCm (HIP 7.13.99004) -- AMD Radeon
8060S Graphics".
* Windows installer: drop the unreliable unsloth.exe rename and its WinError 32 warning
setup.ps1 used to rename the running unsloth.exe out of the way before the
base-package upgrade so pip could replace it. That rename never actually
worked: setup runs *via* unsloth.exe, so renaming our own running
uv-trampoline launcher failed with a sharing violation (WinError 32) and only
printed a scary 'could not rename unsloth.exe; pip may fail with WinError 32'
warning on every Windows install and update.
It also was not needed. pip tolerates a running/locked console-script .exe: it
moves the old one aside and writes the new one. The base upgrade routes through
pip on Windows, so the upgrade succeeds (or, in the install.ps1 flow with
SKIP_STUDIO_BASE=1, the base is not touched at all) and unsloth.exe is left
intact either way.
Removing the rename block and its failed-install restore block removes the
false warning for all Windows devices in both the install and update flows.
* Windows installer: gate venv-internal hipInfo.exe in PowerShell amd-smi probe; harden venv path checks
Follow-up to PR #6296.
- install.ps1 and setup.ps1: ignore the AMD torch wheel hipInfo.exe that lives
inside the Studio venv when probing for a HIP SDK, so amd-smi no longer reopens
the DiskPart UAC during install/update. Mirrors _path_inside_venv in the Python
installers, which already do this.
- amd.py, install_llama_prebuilt.py, install_python_stack.py: normcase the venv
containment check (Windows paths are case-insensitive) and run the
HIP_PATH/ROCM_PATH candidate through it too.
- setup.ps1: fall back to a short TORCHINDUCTOR cache dir when long paths are
unavailable, and create the dir wildcard-safely.
- tests: isolate sys.prefix in the gate helper, add HIP_PATH/ROCM_PATH cases, and
assert the PowerShell venv exclusion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Windows installer: install ROCm PyTorch directly for a known AMD arch
When the GPU arch is known (name-inferred from the GPU-name table) but ROCm
could not be probe-verified (no HIP SDK, no amd-smi), the bootstrap installed
a CPU PyTorch base that setup.ps1 then force-reinstalled as ROCm. The
repo.amd.com wheels bundle their own runtime (no HIP SDK required), which
setup.ps1 already relies on, so the CPU base was a pure wasted download/install.
- Gate the ROCm index on a known arch, not only on probe-verified ROCm, so a
mapped arch installs ROCm torch directly. Unmapped arches and no-GPU hosts
still get CPU (unchanged).
- Fall back to a CPU base if the ROCm-index install fails, so a transient
repo.amd.com outage does not abort the install (setup.ps1 retries ROCm).
- Correct the stale comment that claimed ROCm wheels need a confirmed HIP SDK.
- Add a regression test for the arch-based gate.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Windows installer: correct the unsloth.exe rename-removal comment
The comment claimed the base upgrade 'routes through pip on Windows' and that
pip 'moves the old unsloth.exe aside, then writes the new one'. That is not what
the code does. install_python_stack tries uv first; on a locked launcher uv
aborts and falls back to pip, but the pip fallback strips --upgrade-package and
base.txt lists only bare unsloth/unsloth-zoo, so pip finds them already
satisfied and no-ops. The running unsloth.exe is left intact at its current
version either way. Reword the comment to describe the real uv-first /
pip-fallback-no-op behavior. No functional change.
* Windows installer: close two gaps in the venv-internal hipinfo exclusion
Review follow-up. The amd-smi/DiskPart gate could still reopen in two cases:
- setup.ps1 ran the HIP probe long before $VenvDir is assigned, so without
VIRTUAL_ENV (the `unsloth studio update` path) $venvRoots was empty and the
venv-internal hipInfo.exe was not recognized. Seed the venv root from
UNSLOTH_SETUP_PYTHON and the default Studio home too (both installers).
- The HIP_PATH/ROCM_PATH candidate was accepted without the venv filter, so an
env var pointing into the venv (AMD wheel) still set $HipSdkInstalled. Run
Test-HipinfoIsVenvInternal on the candidate as well (both installers).
Extend the PS gate test to assert both. Both .ps1 parse clean; install tests
pass (the venv-internal / HIP probe coverage at 359 passed).
* Windows installer: correct the CPU-base message for arches with no ROCm wheels
After gating the ROCm index on a known arch, a mapped arch sets $ROCmIndexUrl
and installs ROCm directly, so it no longer reaches the "temporary CPU base"
branch. That branch is now reached only by a name-inferred arch with no ROCm
wheels (e.g. RDNA2 gfx103X), where setup.ps1 does NOT install ROCm. The old
text ("setup replaces it with GPU ROCm wheels ... the final install IS
GPU-accelerated") was therefore always wrong there. Say plainly that PyTorch
stays on CPU for this GPU.
* Windows installer: seed the venv-internal hipInfo check from a custom Studio home
Test-HipinfoIsVenvInternal seeded the venv root from VIRTUAL_ENV, VenvDir, the
setup python, and the default %USERPROFILE% path only. A standalone
`unsloth studio update` with a custom UNSLOTH_STUDIO_HOME (or STUDIO_HOME alias)
and none of those set would not recognize the venv hipInfo on PATH, reopening the
amd-smi/DiskPart gate. Seed the custom home too, in both installers, and assert
it in the gate test.
* Studio installer: resolve venv aliases and expand ~ in the hipInfo venv filter
Two review points on the amd-smi/DiskPart UAC gate:
1. _path_inside_venv compared os.path.abspath of sys.prefix and the hipInfo
path, which does not resolve symlinks, junctions, or 8.3 short names. A venv
reached through an aliased path then fails the check, so its bundled
hipInfo.exe is mistaken for an external HIP SDK and amd-smi runs (the
DiskPart prompt this fix exists to suppress). Switch to os.path.realpath in
all three copies (amd.py, install_llama_prebuilt.py, install_python_stack.py).
2. setup.ps1's early venv-internal hipInfo probe seeded the venv root from a
custom Studio home (UNSLOTH_STUDIO_HOME / STUDIO_HOME) without expanding a
leading ~, while the canonical resolver does. With a tilde form,
[IO.Path]::GetFullPath kept the literal ~ relative to cwd, so the custom-home
hipInfo escaped the filter and reopened the gate. Expand ~ in the probe the
same way as the resolver.
tests/studio/install/test_pr5940_followups.py: 30 passed (adds a symlink
realpath case and a setup.ps1 tilde-expansion guard).
* Studio installer: mirror the hipInfo venv filter and ROCm wheel pins into install.ps1
Follow-up review on the same install.ps1 paths:
1. install.ps1's venv-internal hipInfo probe (Test-HipinfoIsVenvInternal)
seeded the venv root from a custom Studio home without expanding a leading
~, unlike the canonical resolver and setup.ps1. A tilde form left
[IO.Path]::GetFullPath with the literal ~ (relative to cwd), so the
custom-home hipInfo escaped the filter and reopened the amd-smi/DiskPart
gate. Expand ~ in the probe, matching the setup.ps1 fix.
2. The AMD ROCm path installed torchvision/torchaudio bare while pinning torch
to below 2.12. AMD's per-arch index publishes the companions independently
and may ship torchvision 0.27 (for torch 2.12) before removing 0.26, so a
bare resolve can pick an ABI-incompatible set and fall back to CPU. Add
torchvision/torchaudio floor maps and pass the pinned specs, mirroring
setup.ps1 and install_python_stack.py.
3. The ROCm-to-CPU fallback torch install used Invoke-InstallCommand (no
retry), the only torch step in the file without it. Switch to
Invoke-InstallCommandRetry so the recovery path survives a transient index
failure.
tests/studio/install/test_pr5940_followups.py: 33 passed (parametrized tilde
check over both installers, a torch/companion floor-map parity test, and a
CPU-fallback retry guard).
* Studio installer: scan all PATH hipinfo so the venv copy can't shadow a real HIP SDK
The amd-smi HIP-SDK probe used shutil.which("hipinfo") / Get-Command hipinfo,
which return only the first hit on PATH. The AMD torch wheel ships hipInfo.exe
inside the venv and the bnb fix (plus the Studio backend) prepend the venv
Scripts dir to PATH, so that venv-internal copy lands first. When a real HIP SDK
hipinfo sits later on PATH with HIP_PATH/ROCM_PATH unset, the first-hit probe
stopped at the venv copy, treated it as "not a HIP SDK", and closed the amd-smi
gate -- AMD users in that PATH-only SDK setup lost amd-smi telemetry and could
fall back to CPU. Scan every PATH entry and keep the first hipinfo that is not
venv-internal; only the venv copy is ignored, so the UAC/DiskPart suppression is
unchanged.
Applied to all three Python copies (install_llama_prebuilt.py,
install_python_stack.py, backend/utils/hardware/amd.py) via a new
_external_hipinfo_on_path helper, and both PowerShell callers (install.ps1,
setup.ps1) now use Get-Command hipinfo -All filtered by Test-HipinfoIsVenvInternal.
tests/studio/install/test_pr5940_followups.py: 36 passed (real-PATH scan tests, a
shadow-regression test for the exact venv-first ordering, and a parity check that
every Python copy uses the scanning helper).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio uninstallers: fix leftovers (false "removed", shared icon, llama lock)
Auditing a dual native+WSL uninstall on a real device surfaced three leftovers:
1. uninstall.ps1 removed the data dir (which holds unsloth.ico) before the
shortcuts that reference that icon, so Explorer's icon cache briefly held it
open. Remove-Item -Recurse reported success yet left the locked file, and the
dir was never re-attempted, so it orphaned with a false "removed" log.
_RemovePath now verifies the path is actually gone (retrying transient locks)
and reports honestly, and the data dir is re-swept after the shortcuts go.
2. install.sh writes a shared unsloth.ico to %LOCALAPPDATA%\Unsloth Studio for
the WSL shortcut, but uninstall.sh never removed it, orphaning the icon (and
dir) after a WSL uninstall. uninstall.sh now drops that icon and the dir when
empty, in both the powershell.exe and drvfs-fallback paths.
3. ~/.unsloth/.llama.cpp.install.lock was never removed, so the rmdir of
~/.unsloth failed and the dir lingered. Both uninstallers now remove the lock.
Verified by running both uninstallers on a real dual install: device fully clean
(no install dirs, shortcuts, PATH/registry entries, shared icon, or lock left).
* install.sh: auto-route Strix Halo WSL to an existing Ubuntu 24.04
ROCm-on-WSL is the GPU runtime for Strix Halo and only targets Ubuntu
24.04. When the installer runs in a newer default distro (e.g. 26.04) it
cannot enable the GPU and silently falls back to CPU. If a 24.04 distro
already exists, re-run the install there and stop in the current one so the
GPU path is taken without the user having to know about the distro
requirement.
Runs before venv creation so the wrong distro is left untouched, guards
against re-route loops via UNSLOTH_WSL_REROUTED, leaves a working ROCm
distro alone (librocdxg present), and skips the GGUF-only / opt-out /
non-Strix cases. When no 24.04 distro exists we keep today's behaviour:
continue to CPU and print the `wsl --install Ubuntu-24.04` guidance, never
auto-downloading a distro.
Adds tests/sh/test_strixhalo_wsl_reroute.sh (hermetic: extracts the
function, rewrites its paths to fixtures, mocks wsl.exe) covering the full
decision matrix, wired into tests/run_all.sh.
* uninstall.ps1: keep shared unsloth.ico for a surviving WSL shortcut
A dual native+WSL install shares %LOCALAPPDATA%\Unsloth Studio\unsloth.ico:
install.sh points the WSL shortcut's icon there while the native install owns the
dir. The native uninstaller removed the whole dir unconditionally, so uninstalling
native while keeping WSL left the WSL shortcut with a blank icon. The old code only
avoided this when Explorer happened to hold the icon open, which is unreliable; on a
real dual install the dir was deleted and the WSL shortcut went blank.
_RemoveDataDirKeepingWslIcon now scans the Start Menu + Desktop for a surviving
"Unsloth Studio (WSL ...).lnk" and, if found, removes everything in the data dir
except unsloth.ico (keeping the dir) instead of deleting it; with no WSL shortcut it
removes the dir as before. uninstall.sh still drops the icon and the empty dir when
WSL itself is uninstalled, so every uninstall order ends clean.
Adds tests/studio/test_uninstall_dual_install_icon.ps1 (AST-extracts the helper and
runs it against a temp dir with controlled shortcut dirs) covering the dual,
native-only, empty, and missing-dir cases, wired into the windows-inference smoke
workflow. Verified on a real dual install: native uninstall now keeps unsloth.ico
and the WSL shortcut's icon stays intact.
* installer: condense AMD/ROCm code comments (no behavior change)
Tighten the comments added for the Strix Halo native+WSL installer work so
they are shorter and clearer without losing intent: the venv-internal hipInfo
amd-smi gate, the ROCm torch/companion floor maps, the WSL 24.04 reroute, and
the dual-install uninstall icon handling. Comment-only; code paths unchanged.
107 insertions, 166 deletions across 11 files.
* install.sh: run the Strix Halo WSL reroute before any STUDIO_HOME write
The reroute fired after mkdir -p "$STUDIO_HOME" and the legacy-venv migration,
so rerouting 26.04 -> 24.04 left an empty ~/.unsloth/studio stub in the origin
distro (and ran venv migration in the distro about to be abandoned). Move the
reroute ahead of the venv section so the origin distro is left untouched, matching
the function's own comment. Behavior is identical on every non-reroute path.
* installer: fix ROCm CPU-fallback, hipinfo gate edge cases, uninstall icon, WSL 22.04
- install.ps1: clear $ROCmIndexUrl/$ROCmTorchFloor after the CPU fallback so the
flavor-repair block does not retry the failed ROCm index and abort the install;
pin the ROCm companion specs ($visionSpec/$audioSpec) in the repair path too.
- install.ps1 + setup.ps1: skip a bare drive root in Test-HipinfoIsVenvInternal so a
non-venv UNSLOTH_SETUP_PYTHON does not match the whole drive; iterate
HIP_PATH/HIP_PATH_57/ROCM_PATH and take the first non-venv hipinfo.
- amd.py, install_llama_prebuilt.py, install_python_stack.py: strip surrounding
quotes from PATH entries before probing for hipinfo.
- install.sh: pipefail the WSL reroute curl|sh; do not reroute supported Ubuntu 22.04.
- uninstall.sh: keep the shared unsloth.ico while any Unsloth shortcut (native or
another WSL distro) still references it, in both the powershell and drvfs paths.
- tests: regression coverage for all of the above.
* installer: forward reroute options, guard ROCm bootstrap, harden hipinfo gate
- install.sh: forward the caller's --package/--python/--verbose/--tauri and a custom
UNSLOTH_STUDIO_HOME into the WSL reroute (was a bare default install); bail on
--local; run the reroute BEFORE dependency/uv install so the origin distro is left
untouched; set UNSLOTH_SKIP_ROCM_WSL_SETUP after a failed reroute so the later
ROCm-on-WSL bootstrap does not install into the unsupported origin distro.
- install.ps1 + setup.ps1: Get-Command hipinfo -CommandType Application so only real
executables match (not an alias/function named hipinfo).
- uninstall.ps1: guard $env:APPDATA when building the default shortcut search dirs.
- tests: cover option forwarding, --local bail, the bootstrap guard, and the gate change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer: guard origin ROCm bootstrap on every CPU-only fallback; harden ~ expansion
WSL reroute: the no-wsl.exe, no-24.04-target and --local fallbacks all tell the
user the install continues CPU-only, but only the failed-reroute branch set
UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The later _maybe_bootstrap_rocm_wsl gate keys off
that flag, so the other three branches could still install ROCm into the
unsupported origin distro (e.g. 26.04). Set the skip guard on all of them.
Forward UNSLOTH_ROCM_WSL_AUTO into the reroute so a Tauri/consented GPU bootstrap
carries through to the rerouted 24.04 child instead of dropping to the prompt path.
install.ps1/setup.ps1: guard the venv-probe ~ expansion on a non-empty
$env:USERPROFILE so Join-Path does not throw on a profile-less service account.
Tests: add no-wsl.exe and UNSLOTH_ROCM_WSL_AUTO reroute cases, the USERPROFILE
guard assertion, and route shell-test fixtures through a single trap-cleaned root.
* installer: pin + soften Windows ROCm Python repair, reroute to 22.04, harden gates
install_python_stack.py: the Windows AMD ROCm repair in _ensure_rocm_torch()
installed bare torch/torchvision/torchaudio via the fatal pip_install -- the same
asymmetry already fixed on the PowerShell side. A transient repo.amd.com failure
could abort the whole install even after install.ps1/setup.ps1 fell back to CPU.
Pin companions per-arch (gfx120X/Strix -> the rocm7.2 trio, mirroring the PS floor
maps) and make the retry nonfatal: keep the existing build and let the user re-run
update to retry ROCm, so the chain install.ps1 -> setup.ps1 -> stack stays CPU-safe.
install.sh: reroute now targets an installed Ubuntu 24.04 OR 22.04 (24.04 preferred);
both are AMD-supported for ROCm-on-WSL, matching the leave-alone set, so a box with
only 22.04 reaches the GPU instead of staying CPU-only.
install.ps1/setup.ps1: a bare ~ for UNSLOTH_STUDIO_HOME left an empty Join-Path child
(PS 5.1 throws); fall back to USERPROFILE directly and only join a real remainder.
_path_inside_venv (amd.py + both installers): guard a root-dir sys.prefix so commonpath
can't classify every path on the drive as venv-internal (defensive; venv never at root).
uninstall.sh: guard an empty LOCALAPPDATA in the PS-interop icon cleanup (mirror APPDATA).
Tests: add 22.04-target reroute cases, Windows ROCm pin+nonfatal coverage (text +
behavioral), root-dir guard coverage, and bare-~/LOCALAPPDATA guard assertions.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: match WSL reroute target by exact distro name, not substring
The 24.04/22.04 reroute target was chosen with grep -F (substring), so a custom
distro such as 'Ubuntu-24.04-test' (with no exact Ubuntu-24.04) was picked as the
target; the later 'wsl -d Ubuntu-24.04' then fails and the Strix Halo install stays
CPU-only. Match whole lines (grep -ixF) and reuse the matched name so only a real
Ubuntu-24.04/22.04 is targeted. Adds substring-rejection + exact-vs-custom tests.
* install.sh: keep the WSL reroute target to Ubuntu 24.04 (helper-supported only)
The ROCm-on-WSL bootstrap (scripts/install_rocm_wsl_strixhalo.sh) dies on any
VERSION_ID other than 24.04 and pins the noble repo, so treating 22.04 as
GPU-supported let the parent report a successful reroute while the child fell
back to CPU. Drop 22.04 from the supported set and the reroute target list;
24.04 stays the sole target (keeping the exact whole-line distro match). An
already-working ROCm on any other version is still left alone by the librocdxg
check above.
tests: reroute 22.04 cases updated to the 24.04-only behavior; make the
"no wsl.exe" case hermetic so a real host wsl.exe can't leak in on dev boxes;
stop the tauri exit-order check from mis-flagging the reroute helper's
[ "$TAURI_MODE" = true ] && ... --tauri one-liner.
* installer: tighten comment wording across the Strix Halo install/uninstall paths
Condense the verbose multi-line comment blocks (amd-smi hipinfo gate, ROCm
torch install + CPU fallback, WSL reroute, uninstall icon-keep) into fewer,
clearer lines. Comments and a few docstrings only; no code, logic, or
behavior change. Verified with bash -n, the PowerShell parser, and ast.parse,
and the installer test suite still passes.
* add AGPL-3.0 SPDX headers to the .sh/.ps1 scripts missing them
Every shell and PowerShell script under the Studio/installer surface now
carries the standard SPDX-License-Identifier: AGPL-3.0-only + copyright
header (after the shebang where present): the installer (install.sh,
install.ps1), build.sh, the .github and src-tauri scripts, the installer
test suite, and the moe kernel test. Header-only, line endings preserved;
bash -n, the PowerShell parser, and the installer tests all pass.
* installer: drop the duplicate AGPL header from install.sh and install.ps1
Both already carry an SPDX-License-Identifier: AGPL-3.0-only header below
their usage comment block; the prior header pass added a second one at the
top because it only scanned the first few lines. Remove the duplicate so each
file keeps a single original header.
* installer: force-reinstall CPU fallback torch; propagate Tauri NEED_SUDO from reroute
install.ps1/setup.ps1: when the AMD ROCm wheel install fails and we fall back to a
CPU base, force-reinstall the torch/vision/audio triplet. A failed ROCm install can
leave an unpinned ROCm torch (e.g. 2.10.0+rocm on gfx110X/gfx90a) that still
satisfies the CPU torch>=2.4,<2.11.0 range, so without --force-reinstall uv keeps the
ROCm build and only swaps the companions -- a mismatched venv the flavor-repair block
won't fix. setup.ps1 scopes the forced reinstall to the ROCm-fallback path
() so the genuine CPU-only install stays fast.
install.sh: the Strix Halo WSL reroute treated every nonzero child exit as a reroute
failure and fell back to CPU. In --tauri mode the child uses exit 2 ([TAURI:NEED_SUDO])
to ask the desktop app to elevate for the target distro; capture the child's exit code
and propagate exit 2 in Tauri mode (the child already printed the NEED_SUDO line)
instead of masking it. CLI mode still falls back to CPU on a generic failure.
Tests: reroute Tauri exit-2 propagation (and non-Tauri CPU-fallback) cases;
run_func now preserves the child exit code; force-reinstall assertions for both
PowerShell installers.
Note: codex's _rr_q apostrophe finding is a false positive -- the helper already
emits POSIX-correct 'O'\''Brien' and round-trips under both sh and bash.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.ps1: fix $cpuForce array collapse in the ROCm->CPU torch fallback
An if-expression assignment ($cpuForce = if ($ROCmCpuFallback) { @("--force-reinstall") })
collapses the single-element array to a scalar string, so @cpuForce splatting enumerated
it character-by-character into broken single-letter args (- - f o r c e ...), which made
uv/pip reject the install and aborted the whole Studio setup on the AMD ROCm->CPU fallback
path. Build $cpuForce as a real array assigned outside the if-expression so the splat passes
a single --force-reinstall arg. Genuine CPU-only installs stay fast (empty array, no flag).
Test now asserts the array-build form and rejects the if-expression form.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* uninstall: remove the isolated Node.js runtime (~/.unsloth/node)
The isolated Node.js runtime (install_node_prebuilt.py, added with the managed-Node
change) installs to ~/.unsloth/node in default mode -- a sibling of studio, so deleting
<studio> leaves it behind (~200MB orphaned after uninstall). Both uninstallers already
remove the other default-mode siblings (llama.cpp/.cache/.staging); add node alongside
them. uninstall.ps1 also adds it to the handle-lock sweep so a held node.exe can't block
the delete. Env/custom mode nests node under the custom root, removed with that root.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio/setup.ps1: complete Visual Studio 2026 support for the CUDA llama.cpp build
Builds on #6038 (VS 2026 / v18 detection). Once the generator is detected as
Visual Studio 18 2026, two things still broke the CUDA llama.cpp build:
- the CUDA to VS MSBuild integration copied the CUDA .targets into a hardcoded
VC\v170 (VS 2022) BuildCustomizations folder, so a VS 2026 (v180) toolchain
saw no CUDA toolset and cmake failed with "No CUDA toolset found".
- cmake was installed with no version check, but the "Visual Studio 18 2026"
generator requires CMake 4.2+.
This adds Get-VcBuildCustomizationsDir (derives v160/v170/v180 from the detected
generator, falls back to v170), a CMake 4.2 guard for the VS 2026 generator
(upgrades via winget once, else fails with a clear message), and routes both the
copy target and the failure hint through the derived path.
No behavior change for VS 2022/2019/2017: the folder resolves to v170 and the
guard is skipped. Adds windows-latest Pester unit tests (tests/studio_setup_ps1)
plus a workflow that runs them.
* Address review: make VS 2026 self-contained + gate CMake guard to source build
- Find-VsBuildTools now detects VS 2026: vswhere catalog_productLineVersion 2026
-> "Visual Studio 18 2026", and the filesystem scan covers the "18"/"2026" dirs
(incl. non-standard editions like Preview). Adapted from #6038 by
@LeoBorcherding, so the v180 BuildCustomizations path and the CMake guard are
actually reachable on a VS 2026-only host.
- Move the CMake 4.2 guard out of Phase 1 into the committed-source-build branch.
The preferred prebuilt llama.cpp path never reaches it, so a VS 2026 host on
CMake < 4.2 is no longer blocked from using the prebuilt.
- winget upgrade -> install fallback when the on-PATH cmake is not the Kitware
winget package, and log winget failures instead of swallowing them.
- Add a windows-latest Find-VsBuildTools VS 2026 discovery regression test.
* tests(vs2026): define New-FakeVsTree in BeforeAll so It blocks can see it
The Find-VsBuildTools discovery tests are Windows-only (-Skip on non-Windows),
so they first ran on the windows-latest Pester job, where New-FakeVsTree raised
CommandNotFoundException: it was defined in the Describe body, which Pester 5
executes only during discovery, so the function did not persist into the
run-phase It scope. Move it into a BeforeAll block (which runs in the run phase
and is visible to the It blocks). No production code change.
* Address review: probe cmake generator support, fall back to older VS, fix cmake PATH after winget
The VS 2026 CMake guard previously gated only on the cmake version (>= 4.2)
and hard-failed otherwise. Review on #6473 raised three real gaps:
- A VS-bundled cmake below 4.2 can still drive the VS 2026 generator. Probe
cmake --help (Test-CmakeListsGenerator / Test-CmakeCanDriveGenerator) and
accept it when the generator is advertised, not just on the version floor.
- After winget upgrade/install, an older cmake earlier on PATH kept being
resolved. Add-DefaultCmakeToPath prepends the default install dir so the new
cmake wins before re-probing.
- When cmake cannot drive VS 2026 but an older Visual Studio (2022/2019/2017)
is installed and usable, fall back to it (Get-FallbackVsGenerator) instead of
hard-failing, preserving the pre-VS-2026 build path.
Tests mock the cmake command rather than dropping a shim on PATH: PowerShell
caches its application-path table, so a real cmake on the runner (present on
windows-latest) wins over a PATH shim. A function mock is resolved first and is
cache-proof cross-platform.
* Detect VS installed under the Preview edition dir for older versions
Find-VsBuildTools already scans every subdir for VS 2026, but the older-version
(2017/2019/2022) filesystem fallback and Get-FallbackVsGenerator only checked
BuildTools/Community/Professional/Enterprise. A Preview-channel install lives
under a 'Preview' edition folder, so it was missed when vswhere was also
unavailable. Add 'Preview' to both edition lists and guard each with a Windows
Pester test.
* Add real-VS integration matrix: detect actual VS 2022 and VS 2026 in parallel
The unit tests validate VS detection logic with mocked vswhere and fake install
trees (all five versions). This adds a parallel integration job that runs the
real Find-VsBuildTools / Get-VcBuildCustomizationsDir against the Visual Studio
actually preinstalled on GitHub-hosted runners:
- windows-2022 -> real Visual Studio 2022, expect generator v170
- windows-2025-vs2026 -> real Visual Studio 2026, expect generator v180
It asserts our detection matches the real install, the install path exists, the
derived toolset matches, and that the derived v-number is a real folder on the VS
install. VS 2017/2019/2015 are retired from hosted images, so only 2022 and 2026
can be exercised against a genuine install; the rest stay covered by the mocks.
* Detect VS 2026 via vswhere: it reports productLineVersion '18', not '2026'
Real-VS CI on the windows-2025-vs2026 runner showed vswhere reports
catalog_productLineVersion='18' (the internal major) for Visual Studio 2026, not
the marketing year '2026' that VS <= 2022 report. The vswhere map only had
'2026', so on a real VS 2026 host the vswhere branch returned null and detection
survived only via the filesystem scan (Source='filesystem'); a VS 2026 installed
outside the default Program Files location would not be found at all.
Extract a pure Resolve-VsGeneratorFromLabel that accepts both the year and the
internal-major form ('18'/'17'/'16'/'15' as well as '2026'/'2022'/'2019'/'2017')
and use it for both the vswhere and filesystem branches. Add pure unit tests
(cross-platform) for the mapping, including the '18' -> VS 2026 case.
* ci: dot-source Resolve-VsGeneratorFromLabel in the real-VS integration job
Find-VsBuildTools now calls Resolve-VsGeneratorFromLabel, so the integration
step must extract it too; without it the job failed with the helper not
recognized.
* Defer Visual Studio + CMake to the llama.cpp source build (prebuilt path needs no build tools)
The Windows installer required Visual Studio Build Tools and CMake eagerly in
Phase 1 (winget install + exit 1 if absent), before the llama.cpp prebuilt-vs-
source decision. But the preferred path downloads a prebuilt llama.cpp (no
compiler), the backend only shells out to the prebuilt llama-server.exe, and
PyTorch is pip wheels -- so VS and CMake are only needed for the from-source
build last resort. The eager requirement forced every Windows user to install
multi-GB Visual Studio + CMake they never use, or the installer failed.
Change (mirrors the already-lazy Resolve-CudaToolkit / OpenSSL):
- Phase 1c/1d now only DETECT cmake / VS and log; they never winget-install or
exit. The prebuilt install runs zero build-tool installs and is unblocked on
hosts without build tools.
- New Ensure-BuildToolsForLlamaSourceBuild installs CMake (best effort) + VS
(hard requirement, exit 1 with the existing guidance if it cannot be found),
called only when a source build is actually committed, before
Resolve-CudaToolkit. git stays eager (pip needs it for git+ deps).
Tests:
- Pester: the early probe (Find-VsBuildTools) returns null without exiting when
no VS is present; Ensure-BuildToolsForLlamaSourceBuild no-ops when VS is
already detected.
- New studio-windows-no-vs-smoke.yml: Job A renames Visual Studio + vswhere away
and hides cmake, runs the real install.ps1 --local --no-torch, and asserts the
prebuilt llama.cpp installed (no source-build fallback, no VS/CMake install),
PyTorch CPU imports, the backend is healthy, and a /v1/chat/completions
inference returns a reply -- all with no Visual Studio. Job B confirms the GPU
CUDA prebuilt is available and the resolver runs without VS.
* Fix VS 2026 CUDA source build ordering and fallback VS discovery
Same fix as on the stacked base branch (studio-vs2026-cuda-msbuild):
- Move Resolve-CudaToolkit below the CMake gate/fallback in the source build path. It copies the CUDA MSBuild .targets into the current VS generator's BuildCustomizations folder, so running it before a VS 2026 to older-VS fallback left the .targets under v180 while cmake configured v170 ("No CUDA toolset found"). It now runs after the final generator is selected.
- Get-FallbackVsGenerator now queries vswhere first, matching Find-VsBuildTools, so a VS installed outside the default Program Files roots is found instead of failing with a hard exit.
- Add Pester regression tests: the source build resolves CUDA after the fallback, and the fallback queries vswhere.
* Ensure the Visual C++ Redistributable is present for the prebuilt llama.cpp and PyTorch
The prebuilt llama-server.exe and the PyTorch wheels dynamically link the MSVC runtime (VCRUNTIME140.dll, MSVCP140.dll, VCRUNTIME140_1.dll). The Universal CRT ships with Windows 10+, but the VC++ 2015-2022 redistributable does not, so a clean box can fail to launch llama-server or import torch with a missing VCRUNTIME140.dll.
- Add Test-VCRedistInstalled (System32 vcruntime140_1.dll, with a registry fallback gated on version 14.20+) and Ensure-VCRedist (winget Microsoft.VCRedist.2015+.x64, non-fatal), called as Phase 1b.5 so it runs even on the no-build-tools prebuilt path. It is a no-op when the runtime is already present, which is the common case.
- Add Pester tests for the detection: present via the DLL, present via the registry, absent, and an old 2015-only redist that is too low.
* Add a CI job that validates the VC++ runtime detection on a real Windows runner
Runs on windows-latest and windows-2025-vs2026: asserts Test-VCRedistInstalled reports present on the stock image, removes both detection signals (the System32 DLL via a redirected SystemRoot and the HKLM runtime keys, restorably) to confirm detection fires on a genuinely clean box, then does a literal uninstall/reinstall round trip with the official installer and the Ensure-VCRedist winget path. The runtime is restored before the job ends.
* Dot-source the full logging closure in the VC++ runtime CI job
Ensure-VCRedist calls step/substep, which reach Write-StudioStdoutMirror and Get-StudioAnsi; extract those too so the job does not fail with an unrecognized command. Also note that the runtime is ref-counted by Visual Studio on the hosted image, so the literal package uninstall is a no-op there (the clean-box section already proves detection fires when the runtime is genuinely absent).
* Tighten comments in setup.ps1, the VS2026 tests and workflow
Comment-only: condense the verbose helper/test/CI comments to one or two lines, drop the obvious ones, keep the non-obvious rationale. Verified comment-only by comparing the PowerShell code-token stream before and after (no code tokens changed); Pester suite still green.
* Fold the no-VS and setup.ps1 VS2026 Windows CI into studio-windows-inference-smoke.yml
Move the no-vs-cpu/no-vs-gpu-resolve and pester/vs-integration/vcredist-clean-box jobs into the existing Windows GGUF CI workflow and delete the two standalone files, so a studio change triggers one Windows workflow instead of three. Path filter gains tests/studio_setup_ps1/**; job keys and artifact names stay unique.
* CI: assert a Windows ROCm prebuilt exists in the no-VS resolve job
The no-vs-gpu-resolve job confirmed a Windows CUDA asset but never a ROCm one,
and the resolver step resolves to CPU on hosted runners (no AMD GPU), so the
AMD no-VS guarantee rode only on shared resolver code. Grep the per-gfx
windows-x64-rocm-gfx bundles in the same asset-availability step so a release
that drops the Windows ROCm prebuilts fails loudly.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* Studio: use an isolated Node.js for the frontend build instead of replacing the system Node/npm
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address Node isolation review (no-Node probe crash, PATH refresh, OXC provisioning, venv python, runtime node resolver)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust Node isolation for PR #6533
* Studio Node: don't cache a negative node resolution; accept Node metadata in setup.sh ownership guard
- node_runtime: memoize only a version-adequate executable so a Node installed
by a separate-process 'studio update' is picked up without a backend restart.
- setup.sh: _studio_owned_adoptable also accepts UNSLOTH_NODE_PREBUILT_INFO.json,
matching the setup.ps1 Node ownership guard (custom-home parity).
* Studio setup.ps1: skip OXC npm install gracefully when npm is absent
Mirror setup.sh's `command -v npm` guard so a pip-installed Studio with no
system Node skips the OXC runtime install (validator degrades at runtime) instead
of exit 1 aborting the whole setup. Tighten test_node_probe_guard.ps1's probe
regex so it only matches the two system-version probes, not this new npm guard.
* Wire test_node_probe_guard.ps1 into Windows CI for PR #6533
* Harden isolated Node install and probes for PR #6533
- install_node_prebuilt.py: keep an existing, still-usable isolated Node
when nodejs.org's dist index is unreachable instead of aborting the
update on a transient outage (existing_install_usable + tolerant fetch).
- install_node_prebuilt.py: pin NPM_CONFIG_PREFIX/npm_config_prefix and
drop NODE_PATH in _run_node so any npm -g stays inside the isolated
prefix; Windows npm otherwise writes to %APPDATA%\npm.
- install_node_prebuilt.py: resolve tar hard-link targets against the
archive root (symlink targets stay link-parent relative).
- setup.ps1: wrap the system node/npm probes in try/catch so a present
but broken shim degrades to the bundled Node instead of aborting setup.
- setup.ps1: run the isolated Node install with the handed-off/venv Python
(ReusedSetupPython); the main resolver runs later and bare python may be
a Store stub this early.
- setup.sh: log when the OXC validator runtime is skipped for missing npm,
matching setup.ps1.
- node_runtime.py: move the version-floor comment onto _version_meets_floor.
- Tests for the offline-reuse and broken-shim paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim verbose comments across the Studio Node installer for PR #6533
Comments-only pass: collapse the multi-line section banners to single lines,
drop comments that restate obvious code, and tighten the remaining docstrings
and "why" notes without losing intent. No code changes (verified with an AST
comment-only check on the Python files and a non-comment-diff scan on setup.sh
and setup.ps1). Net 109 fewer lines; the install, decision, and probe-guard
suites stay green.
* Harden Node install from review: validated Python, version floor, legacy home, lock race
For PR #6533, addressing the latest review pass:
- setup.ps1: run the isolated Node install with the validated reused/venv Python.
An incompatible reused interpreter (old venv, conda, stale UNSLOTH_SETUP_PYTHON)
is no longer used; fall back to the resolved python instead.
- setup.ps1: a STUDIO_HOME/UNSLOTH_STUDIO_HOME override equal to the legacy default
now uses the legacy sibling node dir (~/.unsloth/node), matching the runtime
resolver and setup.sh, so OXC can find the Node it installed.
- install_node_prebuilt.py: reject an explicit --node-version below the floor
(^20.19 || >=22.12 || >=23) instead of installing a Node the build cannot use.
- install_node_prebuilt.py: atomically rename a stale install lock before unlinking
so two concurrent runs without filelock cannot both acquire it.
Tests added for the version floor (parametrized + explicit-below-floor rejection).
Full install suite: 937 passed, 1 skipped; setup.ps1 parses; decision tests green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address latest review: armv7l + later-fetch offline reuse for PR #6533
- install_node_prebuilt.py: reject 32-bit ARM (armv7l) up front. Node 24 LTS
ships no linux-armv7l build, so the old path failed late with a confusing
"no sha256"; it now fails fast with a clear unsupported-architecture error.
- install_node_prebuilt.py: extend the offline-reuse fallback to the SHASUMS and
archive fetches. If index.json resolves a newer Node but a later download fails
and a usable isolated Node is already on disk, keep it instead of aborting a
non-force update.
Tests added: armv7l/armhf are unsupported; a SHASUMS failure keeps an existing
usable Node and re-raises when none is present. Full install suite: 941 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add UNSLOTH_STUDIO_HOME node-dir tests (install side + resolver) for PR #6533
* Add regression tests pinning the reuse path read-only and isolating installer writes
Lock in the two invariants behind the isolated-Node design: reusing a good
system Node never mutates the user's Node/npm, and the installer's own npm
calls only ever write inside its install_dir.
- tests/studio/install/test_install_node_prebuilt_logic.py: assert _run_node
redirects NPM_CONFIG_PREFIX/npm_config_prefix into install_dir and drops an
inherited NODE_PATH; assert _ensure_npm_floor scopes the npm self-upgrade to
install_dir (never -g against the system) and is a no-op once npm meets the floor.
- tests/sh/test_system_node_readonly.sh (new, wired into studio-backend-ci.yml):
the setup.sh NODE_SOURCE=system arm runs no global install and sets no
NPM_CONFIG_PREFIX, with a positive control that the bundled arm does.
- tests/studio/test_node_decision.ps1: symmetric structural guard that the prefix
pin and the only global install (bun) live in the bundled branch, not the system arm.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Windows installer: repair a stale CPU PyTorch instead of looping forever
A Windows machine with an NVIDIA CUDA 13 driver (e.g. RTX 6000 Pro on enterprise
drivers) could get permanently stuck at:
Stale venv detected (torch cpu != required cu130).
[ERROR] The existing Studio environment needs repair.
Re-run install.ps1 so it can replace the environment safely with rollback.
Re-running install.ps1 did not help. install.ps1 installs torch with
"torch>=2.4,<2.11.0" --index-url .../cu130 but no --force-reinstall, so when a
torch==X+cpu is already present uv treats it as satisfying the range (PEP 440
ignores the +cpu/+cuXXX local label) and makes no change -- the CPU wheel is
never replaced. setup.ps1 then rejects the venv as cpu != cu130 and exits, but it
cannot create a venv or install torch, so the loop never resolves. The migrated-
venv branch also preserves existing torch and never reinstalls it.
After the install step, detect the installed torch flavor (cuXXX/cpu/rocm) and,
when it does not match the tag implied by the selected index, force-reinstall the
torch/torchvision/torchaudio triplet from the correct index via three
--reinstall-package flags. No-op on a healthy matching venv; skipped for
--no-torch, ROCm (already --force-reinstalls), and CPU-only machines.
Adds two pure helpers (ConvertTo-TorchFlavorTag, Get-ExpectedTorchFlavorTag), a
PowerShell unit test (tests/studio/test_torch_flavor.ps1), and a CI parse gate for
install.ps1 (previously unparsed).
* install.sh: repair a stale CPU PyTorch on Linux too (parity with install.ps1)
install.sh has the same latent bug as the Windows installer: the CUDA torch
install uses "torch>=2.4,<2.11.0" --index-url .../cuXXX with no
--force-reinstall, so an already-present torch==X+cpu satisfies the version
range (PEP 440 ignores the +cpu/+cuXXX local label) and uv leaves it in place.
The migrated-venv branch also preserves existing torch. Unlike Windows there is
no stale-venv check in setup.sh, so on Linux the symptom is silent CPU training
rather than a hard loop -- same root cause.
Mirror the install.ps1 fix: after the install block, detect the installed torch
flavor (_torch_flavor_tag) and, when it does not match the index tag
(_expected_torch_flavor_tag), force-reinstall the torch/torchvision/torchaudio
triplet from the selected index via --reinstall-package. No-op on a healthy
matching venv; skipped for --no-torch, ROCm (its own repair force-reinstalls),
and CPU-only / macOS hosts. Adds tests/sh/test_torch_flavor.sh (run in
studio-backend-ci and run_all.sh).
* Installer: catch CPU-fallback on AMD/WSL too (repair ROCm, warn when unfixable)
Extend the torch-flavor safety net beyond NVIDIA:
- install.sh now auto-repairs a stale CPU torch on standard pytorch.org ROCm
indexes too (the rocm-index install path lacked --force-reinstall, unlike the
Windows ROCm install). Reuses the rocm-adjusted $TORCH_CONSTRAINT + rocm index,
so it pulls the correct ROCm wheels.
- Both installers gain a universal post-install warning: when a GPU build was
expected (cuXXX / rocm, including the repo.amd.com gfx* arch indexes) but torch
is still CPU-only, warn loudly instead of silently training on CPU. This catches
the cases auto-repair cannot safely fix (AMD gfx arch indexes that need
--find-links, a migrated AMD venv on Windows where the ROCm install was skipped).
- Mac / Intel / CPU-only hosts resolve to the cpu index -> expected == installed
-> no-op, no false warning. WSL uses install.sh, so the NVIDIA repair + warning
apply there.
Adds Get-InstalledTorchTag (ps1) and _torch_index_repairable (sh) helpers and
extends both unit tests. gfx*/AMD indexes now map to the 'rocm' expected flavor.
* Installer: tighten torch-flavor comments (no logic change)
Condense the rationale comments added for the stale/CPU PyTorch repair in
install.ps1, install.sh and the two helper unit tests; same intent, fewer
lines. Comment-only: AST parse of install.ps1/setup.ps1 clean, helper unit
tests (15 ps1, 24 sh under bash and dash) and the integration sims
(24 ps1, 28 sh) still pass, banner markers the sims slice on are unchanged.
* Installer: bound torch probe, auto-repair gfx, fix ROCm gate parity
install.ps1: in Get-InstalledTorchTag, call WaitForExit(30000) and drain stdout
and stderr asynchronously instead of reading stdout synchronously first, so a
hung or noisy "import torch" (a wedged CUDA/driver, the exact failure this PR
targets) can no longer block the probe past the timeout.
install.sh and install.ps1: treat the repo.amd.com gfx* indexes as plain
--index-url reinstallable. They are PEP 503 simple indexes uv resolves in full
(torch plus every transitive dep) via --index-url, the same URLs the fresh
ROCm install paths already use, so a stale CPU torch on AMD Strix now auto-repairs
to the correct ROCm build instead of only warning.
install.sh: include */gfx* alongside */rocm* in the bitsandbytes install and
ROCm torch repair gates, so a custom UNSLOTH_AMD_ROCM_MIRROR whose path lacks
/rocm/ still installs the AMD bitsandbytes build and repairs ROCm torch.
tests/sh/test_torch_flavor.sh: gfx indexes now assert repairable, plus a
gfx1151 case and an unknown-mirror not-repairable case.
* install.ps1: guard Get-InstalledTorchTag against an empty python path
Make the early return explicit for an empty $PythonExe instead of relying on
Test-Path -LiteralPath '' returning false, so the probe stays safe under
Set-StrictMode or a future refactor that drops the [string] annotation.
* Studio: drop the VBS launcher to clear the Kaspersky false positive
The Windows shortcut launched Unsloth Studio through wscript.exe ->
launch-studio.vbs, and that VBS used CreateObject("WScript.Shell").Run to
start a hidden -ExecutionPolicy Bypass PowerShell. That wscript + .vbs +
bypass-powershell shape is the canonical trigger for generic VBS-dropper
heuristics (Kaspersky HEUR:Trojan.VBS.Agent.gen). The launcher is benign;
only its shape is the problem.
- install.ps1: stop generating launch-studio.vbs and point the Desktop /
Start Menu .lnk straight at powershell.exe -WindowStyle Hidden running
launch-studio.ps1. The shortcut is saved WindowStyle 7 (minimized) so the
brief console flash is muted. launch-studio.ps1 (health poll, port,
mutex, browser) is byte-for-byte unchanged.
- install.ps1: delete a pre-existing launch-studio.vbs on upgrade, so the
flagged file does not linger on machines that already installed it.
- install.ps1 / install.sh: run the heavier ie4uinit -ClearIconCache plus
StartMenuExperienceHost tile-cache rebuild only on a first install or a
real icon change, instead of on every no-op reinstall. That repeated
clear-cache plus kill cluster is itself a dropper-like behavioral pattern.
- tests: forbid VBS generation and require the legacy-VBS cleanup.
Linux, macOS and WSL install paths are unchanged. WSL already targets
wsl.exe from its .lnk and never used a VBS; its only change is the same
icon-cache gating.
* Studio: add launcher-chain smoke coverage to the Windows UI CI
The shortcut launch path was previously untested: studio-windows-ui-smoke
installed then booted `unsloth studio` directly, so a broken .lnk could ship
silently. After install the job now seeds a legacy launch-studio.vbs, asserts
the upgrade removed it, asserts the .lnk targets hidden powershell.exe (never
wscript.exe), and launches via the shortcut's stored command, waiting for
/api/health to report healthy.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Linux tool calling used Qwen3.5-2B IQ3_XXS. That quant is aggressive
enough that the model intermittently emits a malformed tool call
(doubled </parameter>, stray </tool_call>), and llama-server's
peg-native parser rejects it with a 500, failing the job. Mac and
Windows already run this test at Q4_K_XL; align Linux.
Also copy ~/.unsloth/studio/logs (backend server log + llama-server log)
into the tool-calling and ui-smoke artifacts on stop. Previously only
studio.log + install.log were uploaded, so a /v1/chat/completions or
/api/inference/load 500 had no server-side traceback to diagnose from.
* Package scanners: cut false positives and make the CI gate blocking
scan_packages.py and scan_npm_packages.py red-failed on legitimate
library code, so the security-audit steps were left advisory. Reduce
the false positives at the source and flip both gates to blocking.
scan_packages.py:
- Scan code only: blank comments and bare docstrings/doctests before
matching (line numbers preserved), so prose and >>> examples cannot
trip a finding.
- Drop the platform.system() branch from the anti-analysis regex (under
DOTALL it matched across the whole file, so every cross-platform
library tripped it) and fix the dead /proc/self/status alternative.
- Add a reviewed baseline allowlist (scan_packages_baseline.json) keyed
on (package, basename, check): only non-baselined CRITICAL/HIGH exit
1, and a new kind of finding in a listed file still fails.
- sdist fallback: when --with-deps cannot resolve a shard (a sdist-only
package or a version conflict), drop to per-spec and fetch the raw
sdist from the PyPI JSON API (no pip build, no setup.py), so every
package is still scanned and no shard exits 2.
scan_npm_packages.py:
- Mirror the code-only JS/TS scanning (blank // and /* */ comments,
string/template/regex aware) and the baseline allowlist. The npm
corpus is clean today, so the baseline is empty.
security-audit.yml:
- Flip both scan steps to blocking (SCAN_ENFORCE=1), capturing the
scanner exit via PIPESTATUS so tee does not mask it.
tests/security: add coverage for the strip, baseline and sdist paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review feedback on the package scanners
- Do not blank f-strings during code-only scanning (they evaluate at
import); and when a file uses exec/eval, rescan the original for
payload carriers hidden in a docstring/string so exec(__doc__) style
payloads stay visible.
- sdist fallback: recover transitive deps with their version specifier
(fetch the pinned version, not latest), and recover deps in the
--no-deps branch too so a sdist-only transitive dependency is still
scanned instead of silently skipped.
- Baseline: key by package-relative path, not basename, so a future
same-named file in another directory is not auto-suppressed.
Regenerated the baseline accordingly.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix config.rope_scaling being dropped by the replaced rotary embedding (#2405)
On modern transformers, LlamaModel builds its rotary embedding from config
using unsloth's replacement LlamaRotaryEmbedding class, whose config path
computed vanilla inv_freq and ignored config.rope_scaling entirely. The
llama3/linear/longrope dispatch in patch_llama_rope_scaling rewrites
LlamaAttention.__init__, which no longer constructs rotary embeddings, so it
never fires; the model-level rotary is then copied onto every attention
layer. Result: Llama-3.1/3.2/3.3 ran with unscaled RoPE on the
FastLanguageModel path and collapsed into repetition loops past roughly 29K
tokens (PASS at 28867, FAIL at 31767 in needle retrieval). FastModel was
unaffected because vision.py keeps transformers' own rotary. qwen2, qwen3,
qwen3_moe, mistral and cohere assign the same base class, so any rope-scaled
config of those families was equally exposed.
The fix makes the base class config path compute inv_freq and
attention_scaling via transformers' ROPE_INIT_FUNCTIONS (covers llama3,
linear, dynamic, yarn, longrope), with an inline llama3 fallback reading
factors from config for older transformers, degrading to prior behavior on
any failure. attention_scaling is applied in _set_cos_sin_cache (1.0 default,
exact no-op for unscaled paths) and persists across extend_rope_embedding.
A type(self) guard prevents double-scaling via the legacy scaled subclasses.
Adds tests/utils/test_rope_scaling_drift.py (AST tripwire + behavioral
inv_freq/cos-cache/extension checks, validated to fail 4 of 5 on the unfixed
code) and wires it into the existing consolidated CI HARD GATE step.
Verified on GPU: 48K-token needle retrieval flips FAIL to PASS for
FastLanguageModel in bf16 and 4bit, 20K stays PASS, scaled inv_freq matches
transformers exactly, and the left-padded batch generation guard still gets
exact solo-vs-batched token matches.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: normalize object-style rope_scaling, vectorize llama3 fallback
config.rope_scaling can be a config object rather than a dict on newer
transformers; _rope_scaling_as_dict normalizes it (to_dict/dict/vars
fallbacks) before any .get() access, with a regression test using a
dataclass stand-in. The inline llama3 fallback now uses torch.where instead
of a per-frequency Python loop; verified bit-for-bit equal to transformers
ROPE_INIT_FUNCTIONS for factor 8 (Llama-3.1) and factor 32 (Llama-3.2).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: CPU-safe rope guard tests, normalized config for delegation
The rotary constructor builds per-device CUDA caches, so the behavioral tests
that instantiate it cannot run on GPU-less CI. Restructured into three layers:
the AST tripwire now also asserts the constructor stays wired to
_compute_config_rope_inv_freq; the CPU layer tests that pure helper directly
(llama3 dict, llama3 object, linear object, default type) with no
instantiation; the instantiation and cache tests are gated behind a real CUDA
probe (actual tensor allocation, so import-time CUDA spoofs cannot fool the
gate). Verified: 9 passed with GPU; 5 passed 4 skipped with CUDA hidden; 5
failed 4 skipped on the unfixed code in CPU mode.
Delegation to ROPE_INIT_FUNCTIONS now retries with a shallow config copy
carrying the normalized rope_scaling dict when the original was an object the
installed transformers cannot read; covered by a linear-object test, which has
no inline fallback and passes only through that retry path.
* Tighten comments in rope scaling fix and guard test
Comment and docstring reduction only; verified code-identical with
scripts/comment_tools.py check --strip-docstrings (AST signature match on
both Python files). All guard tests unchanged: 20 passed with GPU, 5 passed
4 skipped with CUDA hidden.
* Apply repo kwarg-spacing format
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Stop HF 429 rate limits from sinking the llama.cpp prebuilt path in Studio CI
The Windows Studio API smoke job failed when anonymous huggingface.co
fetches of the tiny GGUF validation model (stories260K.gguf) hit HTTP 429
on the shared runner IP. The installer correctly refused the unvalidated
prebuilt and fell back to a source build, which the prebuilt assert then
flags. Three layers fix this:
1. Installer: auth_headers sends HF_TOKEN (or HUGGING_FACE_HUB_TOKEN) to
huggingface.co hosts, mirroring the existing GH_TOKEN handling for the
GitHub API rate limit. A redirect handler strips Authorization when a
download is redirected off-host (CDN signed URLs reject foreign auth;
urllib forwards headers on redirect, unlike requests/huggingface_hub).
2. Workflows: the HF_HOME prime steps also prefetch the validation model
so the install's hf_hub_download resolves from the local cache even
when the Hub is rate limiting; cache keys bumped v1 to v2 to repopulate.
This also covers fork PRs, which cannot see secrets.
3. Workflows: every Install Studio / update step that already passes
GH_TOKEN now also passes HF_TOKEN, so both the huggingface_hub path and
the direct URL fallback are authenticated.
Tests: tests/studio/install/test_hf_auth.py covers token-to-host routing,
the cross-host redirect strip, and the download_bytes wiring (offline).
Verified live: authenticated download of the validation model through the
new opener (CDN redirect exercised, pinned sha matches) and an offline
hf_hub_download cache hit against an HF_HOME primed by the new step.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The import-hoist safety step built its changed-file list with a two-dot
diff against the base branch tip on a shallow clone. Once the base
branch moves past a PR's branch point, that diff includes every file
the base changed since, and the verifier compares newer base code
(BEFORE) against the PR's older snapshot (AFTER). This time-reversed
comparison reports the base branch's own refactors as blockers on PRs
that never touched those files, forcing branch updates or admin merges.
Resolve the true merge-base through the compare API, fetch that single
commit by SHA (clone stays shallow), and use it for both the file list
and --before. Verified against the PR 6137 failure: with the base tip
the step flags studio files from a later main commit; with the
merge-base the file list contains only the PR's own files and the
verifier passes.
#5963 folded the manifest resolver into the simple-path resolver,
removed the installer's --simple-policy flag, and moved macOS prebuilts
to the unslothai/llama.cpp fork. The MLX CI prebuilt step still passed
the deleted flag, so argparse exits 2 and the workflow has been red on
every main push since. Point the step at the fork with no extra flags,
exactly like studio/setup.sh on macOS.
* Add regression guard for batched left-padded generation (#1066, #3699)
Three layers of tests plus a path-filtered CI workflow so the left-padding
position_ids / attention-mask bug class cannot silently return:
- tests/utils/test_prepare_inputs_ast_guard.py: import-free AST checks on
_fast_prepare_inputs_for_generation (cumsum-from-mask branch present,
cache_position only as fallback, no mask truncation, model families wired)
- tests/utils/test_prepare_inputs_leftpad.py: CPU behavioral unit test with
synthetic left-padded masks and fake caches; exact expected position_ids
for prefill and cached decode
- tests/utils/test_batched_leftpad_generation_gpu.py: optional GPU e2e,
solo vs batched prefix match, skipped without CUDA
- .github/workflows/batch-inference-guard.yml: ubuntu-latest CPU job running
the two deterministic layers on PRs touching unsloth/models/**
Validated: all pass on main; both CPU layers fail at 6d0f8643~1 (pre #4100)
and at 332eabf3~1 (pre #2216), reproducing the historical bug signatures.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cite staging proof in batch-inference-guard header (staging-2 PRs 170/171)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fold left-padding guard into consolidated Core CI; merge AST + behavioral tests
No new workflow and no new CI job: the guard now runs as one HARD GATE step
inside consolidated-tests-ci.yml, right after the callback signature drift
detector, where the CPU torch stack is already installed. The AST structural
checks and the behavioral unit tests live in a single file
(tests/utils/test_prepare_inputs_leftpad.py); the AST layer stays stdlib-only
with unsloth imported lazily inside the behavioral tests, so import breakage
cannot mask the structural checks.
Revalidated after the merge: 11 assertions pass on main, 8 fail at
6d0f8643~1 (pre #4100).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update staging proof reference for consolidated gate (PRs 170/172)
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Completes the allowScripts rollout: upgrade CI npm to the 11.x line
(node 22 bundles 10.x, which predates the gate) with a loud version
guard so the flag can never silently degrade into a warning, then run
npm ci --strict-allow-scripts. A dependency that introduces install
scripts not covered by the committed policy now fails the job with
npm's approve-scripts/deny-scripts instructions; the pre-commit sync
hook keeps existing pins fresh after bumps.
At UD-IQ2_XXS the temp-0 greedy answer to the capital-of-France probe is
hardware dependent: GitHub ubuntu runners deterministically answer
city=France while other CPUs answer Paris, because the 2-bit argmax
flips with the SIMD kernel path. Seeds do not rescue it: a staging-fork
sweep on the affected runners measured 1/5 Paris at temp 0.7 and 1/5 at
temp 1.0 (different winning seeds on different hardware), so a
retry-across-seeds assert would still flake about a third of runs.
The same sweep on UD-Q4_K_XL answered Paris 13/13 across temp 0, 0.7
and 1.0 with 5 seeds each, including 3x deterministic greedy. Bump the
job's quant (roughly 570 MiB to 1.1 GiB, cache key already includes the
variant) and leave the assertion exactly as it was.
* Studio: npm v12 readiness for install-script gating
npm 12 (July 2026) stops running dependency install scripts unless they
are approved via allowScripts, and npm 11.16 already warns. Studio has
no git or remote URL deps anywhere, so script gating is the only
exposure:
- commit the allowScripts policy that npm approve-scripts writes for
@biomejs/biome and msw, plus a manual fsevents entry: the tooling
cannot match a darwin-only optional dep from Linux, but the strict
check walks the platform independent ideal tree and flags it anyway
- drop the minimum-release-age npmrc alias; npm >=11.16 flags it as an
unknown project config that stops working in npm 12
- approve bun's postinstall in the setup.sh / setup.ps1 bun bootstrap;
under npm 12 defaults npm install -g bun otherwise leaves a broken
stub and setup falls back to the slower npm install path
- fix the stale esbuild comment in studio-frontend-ci.yml: the vite 8
chain ships napi binaries with no install scripts
* Studio: auto-sync allowScripts pins after dependency bumps
The allowScripts entries from #6128 are version pinned, so a biome or
msw bump strands the pin and the approval silently stops matching.
Dependabot cannot maintain the field, so:
- scripts/sync_allow_scripts_pins.py re-pins existing entries from the
versions package-lock.json actually resolves. It never adds or
removes entries, so approving a new script-bearing package stays a
human decision. Bare names and non-exact specs are left alone.
- a pre-commit hook runs it with --fix; pre-commit.ci pushes the fix
commit to PR branches, Dependabot's included, so stale pins heal
without a human in the loop
- a Frontend CI step runs --check plus the offline unit tests as the
backstop when pre-commit.ci is skipped
No dependabot.yml change needed: the /studio/frontend entry already
suppresses version PRs (security only) behind a 7 day cooldown.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make the sync hook robust to lost executable bits
The pre-commit.ci autofix commit dropped the script's exec bit, which
breaks a shebang-style entry. Invoke via python instead and restore
the bit.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
npm 12 (July 2026) stops running dependency install scripts unless they
are approved via allowScripts, and npm 11.16 already warns. Studio has
no git or remote URL deps anywhere, so script gating is the only
exposure:
- commit the allowScripts policy that npm approve-scripts writes for
@biomejs/biome and msw, plus a manual fsevents entry: the tooling
cannot match a darwin-only optional dep from Linux, but the strict
check walks the platform independent ideal tree and flags it anyway
- drop the minimum-release-age npmrc alias; npm >=11.16 flags it as an
unknown project config that stops working in npm 12
- approve bun's postinstall in the setup.sh / setup.ps1 bun bootstrap;
under npm 12 defaults npm install -g bun otherwise leaves a broken
stub and setup falls back to the slower npm install path
- fix the stale esbuild comment in studio-frontend-ci.yml: the vite 8
chain ships napi binaries with no install scripts
* feat: support text-only loading of Gemma 3 27B via FastLanguageModel (skip SiglipVisionModel)
* test: instantiate text-only Gemma3 model and assert no vision tower
Existing tests were AST source-introspection plus a config-resolves-to-
text-config check; none actually instantiated a model from the
text-only config. Add a small integration test that builds a shrunken
Gemma3TextConfig (CPU-cheap), instantiates the matching CausalLM
class, and asserts the resulting model exposes the LM head and has no
vision_tower or multi_modal_projector attribute.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Deduplicate _get_text_only_config into _utils for PR #5816
* Fall back to full model when a VLM has no text-only class for PR #5816
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Preserve quantization_config and clarify warning for text-only loading for PR #5816
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Only take text-only path when the VLM has its own text decoder for PR #5816
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Convert source string-match assertions to AST checks per Gemini review
* Load real VLM text weights on transformers 5.x for text-only mode in PR #5816
transformers >=5 changed Gemma3ForCausalLM base_model_prefix from language_model to model, so a VLM checkpoint's text weights (gemma3: language_model.model.*, gemma3n: model.language_model.*) no longer auto-strip onto the text decoder and were silently initialized random. Add a version-gated key_mapping that remaps them onto the text keys, returning None on transformers <5 where the prefix still strips and a mapping would break the load.
Apply the same family-guarded remap on the load_in_fp8 offline path and for direct FastBaseModel callers, and remap quantization llm_int8_skip_modules off the wrapper prefix after stripping.
Add a regression test that loads real VLM checkpoint weights (the prior tests only instantiated a fresh model so they missed this) and drop the bitsandbytes dependency from the quantization-config test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Separate FP8 text-only cache and hoist the text-only guard for PR #5816
Address review of the text-only changes: (1) _offline_quantize_to_fp8 produced different artifacts for text-only vs full VLM but reused the same <name>-fp8-<mode> cache dir, so one mode could load the other's saved model; decide text-only before the cache name and add a -text-only suffix. (2) FastBaseModel.from_pretrained rewrote the VLM auto class to AutoModelForCausalLM before loading auto_config and before the family check, leaving is_vlm wrong for the fast_inference/vLLM block; hoist the family-guarded text-only decision above those checks and drop the redundant later block. (3) Wire the text-only regression test into the curated CPU pytest job so it runs in CI across the transformers matrix.
* Trim text-only code comments for PR #5816
Shorten and de-duplicate the comments added for the text-only loading work; keep the non-obvious rationale (the transformers >=5 base_model_prefix change) and drop the obvious parts. Comments only, no code changes; AST-based tests still pass on transformers 4.57.6 and 5.4.0.
* Make text-only loading opt-in via a public text_only argument for PR #5816
Rename the internal _force_text_only flag to a public text_only parameter on FastLanguageModel, FastModel and FastBaseModel (and the fp8 helper), defaulting False on all three. Text-only loading is now opt-in (text_only=True) instead of forced on by FastLanguageModel; the family guard and key remap are unchanged. Updated the AST tests for the new parameter and forwarding.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim text-only code comments for clarity
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
transformers-latest ships a new deepseek_ocr2 model whose source-rewriter
compile exceeds the 60s per-model budget on the CI runner, same as the
existing beit/sam/sam_hq entries. Add it to KNOWN_BROKEN_COMPILE Category F
so HF=latest Core stops failing on a new upstream model. The slow compile
path itself remains a follow-up for unsloth_zoo.
Daniel Han