* install: let UNSLOTH_TORCH_INDEX_FAMILY / _URL override CUDA wheel detection
get_torch_index_url (and the studio-update mirror _detect_cuda_torch_index_url)
chose the torch wheel family solely by probing the host GPU, with no override.
In a headless / container / CI build the host driver is visible via the
/proc/driver/nvidia/gpus fallback but nvidia-smi cannot report a CUDA version,
so the function fell back to its cu126 default and installed the wrong wheels
(e.g. a cu128 image got cu126 torch).
Add an explicit override checked before any probing, in both the shell installer
and the Python studio-update path:
- UNSLOTH_TORCH_INDEX_URL full index URL, used verbatim (wins)
- UNSLOTH_TORCH_INDEX_FAMILY family (cpu, cu128, rocm6.4, ...) appended to the
mirror base (UNSLOTH_PYTORCH_MIRROR still honoured)
This matches how the published GPU images select CUDA -- vLLM and SGLang take the
CUDA version from an explicit build ARG rather than detecting it, and the Unsloth
Docker base image already pins the cu128 index directly. Desktop installs are
unchanged: with no override set, detection runs exactly as before.
Adds test_get_torch_index_url.sh cases for the override (family, full URL,
precedence, mirror base, trailing-slash strip, empty-ignored).
* install: make the torch-index override authoritative across ROCm paths
Address review feedback on the override added in this PR so a pinned index is
honoured everywhere, not just in get_torch_index_url:
- Skip the WSL ROCm bootstrap (root privilege + large downloads, probes
/dev/dxg) when UNSLOTH_TORCH_INDEX_URL / _FAMILY is set; it previously ran
before the override was consulted.
- Skip the Radeon/Strix rerouting (which re-probes the GPU and overwrites the
resolved URL with repo.radeon.com / repo.amd.com) when the index is pinned, so
an explicit ROCm override (e.g. UNSLOTH_TORCH_INDEX_FAMILY=rocm6.4) is kept.
- install_python_stack.py: derive _TORCH_BACKEND from the override when
UNSLOTH_TORCH_BACKEND is unset (standalone studio update), so _ensure_rocm_torch
/ _ensure_cuda_torch repair to the requested family instead of re-detecting.
- Strip ALL leading/trailing slashes in the shell override to match the Python
side (avoids 404s on strict pip proxies).
Adds test cases for double-slash and leading/trailing-slash overrides.
* install: honor pinned torch index in CUDA/ROCm repair paths
Follow-up to the override work in this PR: the get_torch_index_url / install.sh
reroute already respect a pinned UNSLOTH_TORCH_INDEX_URL / _FAMILY, but the
Python repair helpers in install_python_stack.py still re-probed the GPU and
could overwrite the pinned family. Make the pin authoritative there too:
- _ensure_cuda_torch: an explicit cu* pin commits to CUDA wheels, so repair a
ROCm-poisoned venv even when no NVIDIA GPU is visible here (headless /
container / CI cross-install), instead of bailing on the GPU-presence gate.
- _ensure_rocm_torch: skip the AMD per-gfx (Strix) reroute when a ROCm index is
pinned, and in the generic reinstall path install from the pinned URL verbatim
rather than re-detecting the host ROCm version. gfx*/rocm7.2 indexes serve
torch 2.11+, so select the 2.11 package specs for a gfx leaf.
- install.sh: raise the torch constraint to 2.11 for */gfx* indexes too, matching
rocm7.2, so a pinned full-URL/family override that returns early keeps a valid
constraint.
Add _explicit_torch_index_url / _explicit_rocm_torch_index_url helpers and tests
covering the no-GPU CUDA pin repair and the explicit gfx index honored verbatim.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: honor torch-index override on the Windows installers too
The pinned-index work landed for install.sh and install_python_stack.py, but the
Windows installers still picked the wheel index from GPU probing. Extend the same
UNSLOTH_TORCH_INDEX_URL / _FAMILY contract so a pinned index wins on every platform:
- install.ps1: Get-TorchIndexUrl returns the pinned URL/family before nvidia-smi
probing; the AMD ROCm reroute is skipped when the index is pinned, so an explicit
cpu/cu* pin on an AMD host is not overwritten.
- studio/setup.ps1: add shared Get-PinnedTorchIndexUrl / Get-TorchIndexLeaf helpers;
the stale-venv check, the install selection and the AMD reroute all honor the pin,
and the CPU/CUDA install pulls from the resolved index URL.
- tests: parity test that all four installers read both override vars and the two
Windows installers gate the AMD reroute on the pinned flag.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: complete pinned-index handling for ROCm/Windows edge cases
Follow-ups to the override work flagged in review:
- install.ps1: a pinned gfx*/rocm>=7.2 index previously skipped the AMD reroute
that sets the torch>=2.11 floor, so the generic install used torch>=2.4,<2.11
and could resolve the known-bad _grouped_mm wheel. Route a pinned ROCm index
through the ROCm install path with the 2.11 floor + companions, and guard the
companion-spec lookup so a skipped reroute block cannot null-deref.
- studio/setup.ps1: the stale-venv check compared the installed flavor (cuXXX/cpu,
with +rocm misread as cpu) against the raw pinned leaf (gfx1151 / rocm6.4), so a
correct pinned ROCm venv was always marked stale. Classify +rocm wheels as the
generic 'rocm' flavor and normalize a pinned rocm*/gfx* leaf to 'rocm' before
comparing (cu* stays specific so cu126-vs-cu128 still rebuilds).
- install_python_stack.py: _ensure_cuda_torch now also reinstalls from a pinned
CUDA index when the venv carries a CPU wheel (headless CPU-venv-to-CUDA
cross-install via 'studio update'), not only when it finds a ROCm build.
- tests: parity assertions already cover all four installers honoring the override.
* install: finish pinned ROCm/CUDA edge cases on Windows + repair path
Follow-ups to the previous round:
- studio/setup.ps1: a pinned gfx*/rocm>=7.2 index now routes through the ROCm
install path with the 2.11 floor + companions (it previously fell through to the
CUDA branch with bare torch/torchvision/torchaudio against the ROCm index). The
CPU/CUDA fallback index is forced to the CPU wheel index when a ROCm index is
active, so a failed pinned-ROCm install does not retry the ROCm mirror.
- studio/setup.ps1: the stale-venv check no longer treats an unrecognized pinned
URL leaf (e.g. a PEP 503 mirror ending in /simple) as a torch flavor tag, which
was marking a correct venv stale; cu*/cpu/rocm/gfx leaves are still compared.
- install.ps1: the post-failure CPU fallback uses an explicit CPU index instead of
, which for a pinned ROCm index was the ROCm mirror itself (so the
'fallback' just retried the failing index and aborted the installer).
- install_python_stack.py: _ensure_cuda_torch now also reinstalls when the venv's
CUDA family differs from a pinned one (installed cu126 vs pinned cu128), not only
CPU->CUDA; the probe reports the installed cuXXX tag for the comparison.
* install: keep the ROCm to CPU fallback install inside the retry-helper window
The pinned-ROCm CPU fallback computes an explicit CPU index, but the comment
explaining why it cannot reuse $TorchIndexUrl pushed the actual
Invoke-InstallCommandRetry / --force-reinstall call more than 600 chars past the
"ROCm PyTorch install failed" message, so test_pr5940_followups's window check
no longer saw the retry helper. Move the CPU-index computation and its comment
above the failure substep so the retrying force-reinstall stays adjacent to the
message. No behavior change: same explicit CPU index, same retry, same
--force-reinstall.
* install: address #6692 review round 5 (ROCm/CPU pin edge cases)
setup.ps1:
- Stale-venv check: treat an AMD/ROCm host (HasROCm or a resolved gfx arch) with
no explicit pin as expecting "rocm", not "cpu", so a healthy +rocm venv is not
flagged stale (which made installer-managed setup exit and direct update rebuild).
- Pinned-ROCm install failure now routes into the force-reinstall CPU branch:
CuTag stays the rocm/gfx leaf on failure, so the condition also checks
ROCmCpuFallback; otherwise the CUDA branch installed from the CPU index without
--force-reinstall and kept the partial ROCm torch.
- Explicit ROCm pin compare no longer collapses gfx*/rocm* to a generic "rocm":
it compares the +rocmX.Y version (and the torch 2.11 line for gfx pins) so
changing the pinned family (e.g. rocm6.4 -> gfx1151) rebuilds and applies it.
install_python_stack.py:
- _ensure_rocm_torch: an explicit ROCm wheel-index pin now bypasses the
NVIDIA-present / no-AMD-GPU / unreadable-ROCm gates (headless/container/CI
cross-install), mirroring the explicit-CUDA-pin bypass in _ensure_cuda_torch.
- Add _ensure_cpu_torch: an explicit CPU pin (FAMILY=cpu or /cpu URL) now has a
repair path that reinstalls CPU torch over an existing CUDA/ROCm build on a
standalone update (which skips install.sh's flavor enforcement).
install.sh:
- Pin torchvision/torchaudio companions alongside torch for the rocm7.2 / per-gfx
index and the Strix reroute (those AMD indexes publish companions independently
and a bare name can resolve a torch-2.12-built wheel, an ABI mismatch).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* torch-index override: classify CUDA pin by leaf; trim blank shell overrides
_ensure_cuda_torch only overrode the NVIDIA-presence gate for *any* pinned index,
so a non-CUDA mirror URL (or a ROCm/CPU pin) on a non-NVIDIA host with ROCm torch
could force a CUDA reinstall over a working ROCm venv. Add
_explicit_cuda_torch_index_url() (leaf cu*), matching the ROCm/CPU helpers, and
gate on it instead.
install.sh::get_torch_index_url treated a whitespace-only UNSLOTH_TORCH_INDEX_URL
/ _FAMILY as authoritative (yielding an invalid index), unlike the Python .strip()
and PowerShell IsNullOrWhiteSpace paths; trim leading/trailing whitespace first.
* install: honor pinned torch index over CVD/GPU gates and fix leaf-based ROCm classification
- install_python_stack.py: an explicit cu* pin now clears the CUDA_VISIBLE_DEVICES
empty/-1 hide gate as well as the NVIDIA-presence gate, so
CVD=-1 UNSLOTH_TORCH_INDEX_FAMILY=cu128 studio update repairs to CUDA wheels
(parity with install.sh's get_torch_index_url override, which skips all GPU
probing). Unpinned CVD=-1 still skips.
- install_python_stack.py: _ensure_cpu_torch installs the bounded _CPU_TORCH_PKG_SPEC
instead of a bare torch/torchvision/torchaudio trio; the /cpu index now also
serves torch 2.11+, which is outside the supported <2.11 range.
- install.sh: the torch>=2.11 constraint case matches the index leaf (rocm7.2|gfx*)
instead of the whole URL, so a mirror base path containing a gfx/rocm7.2 segment
with a cu*/cpu family is not false-matched onto the 2.11 line.
- setup.ps1: the stale-venv check expects rocm torch only for arches the install
path maps to a repo.amd.com wheel index; an unmapped/unreadable arch installs
CPU, so a correct CPU venv is no longer marked stale.
- Tests for each of the above.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: tighten pinned torch-index override edge cases
- install.sh: trim whitespace-only UNSLOTH_TORCH_INDEX_URL/_FAMILY before the
_torch_index_pinned guard, matching get_torch_index_url, so a blank override no
longer skips the WSL bootstrap and Radeon/Strix reroutes while detection still
picks the normal index.
- install.sh / install.ps1 / setup.ps1 / install_python_stack.py: force the torch
2.11 floor only for the gfx families with the <2.11 _grouped_mm bug (gfx120X-all,
gfx1151, gfx1150). A pinned override to gfx110X-all/gfx90a/gfx908 stays on the
default range, matching the automatic AMD path.
- install_python_stack.py _ensure_cuda_torch: treat an untagged CUDA build under a
CUDA pin as a family mismatch (reinstall), and match cuXXX pins narrowly (cu +
digits) so a custom/current mirror leaf no longer forces CUDA over a CPU/ROCm venv.
- install_python_stack.py _ensure_rocm_torch: reinstall when an explicit ROCm pin
names a different ROCm family than the already-installed ROCm torch (the ROCm
analogue of the CUDA cuXXX mismatch repair).
Adds tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: fix second-order edge cases in pinned torch-index ROCm/CUDA handling
Parse the ROCm torch probe positionally so an empty HIP marker is kept:
CPU/CUDA torch no longer reads as HIP, so the ROCm reinstall is not skipped.
Emit one "<marker>|<version>" line (like the CUDA probe) for a robust parse.
Limit the gfx torch 2.11 expectation to the install allowlist
(gfx120X-all/gfx1151/gfx1150). A pinned gfx110X-all/gfx90a/gfx908 index stays
on the default <2.11 specs, so a correct 2.10+rocm wheel is no longer judged a
mismatch and force-reinstalled every update.
Distinguish an AMD per-arch wheel (three-part +rocmA.B.C) from a generic
pytorch.org wheel (two-part +rocmA.B): a gfx per-arch pin over a generic 2.11
wheel now reinstalls the per-arch wheel, while an already-installed per-arch
wheel is not re-flagged (no reinstall loop).
Mirror all of the above in setup.ps1 via new Test-RocmGfx211Leaf /
Test-CudaFamilyLeaf / Get-RocmPinStaleTags helpers, reused by both the
install-spec path and the stale-venv check so they cannot diverge again.
Require a digit after "cu" (^cu[0-9]) in setup.ps1, install.ps1 and install.sh
so a mirror leaf like /custom or /current is not branded CUDA and does not
rebuild the venv every run.
Add tests: CPU/CUDA probe -> has_hip_torch False; gfx110X-all pin + 2.10 wheel
not stale; gfx1151 pin + generic 2.11 wheel stale; gfx1151 pin + per-arch wheel
not stale; /custom and /current not CUDA; plus cross-language allowlist and
cu-digit parity guards, and a PowerShell unit test for the new setup.ps1 helpers.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix ROCm/gfx pin case normalization, ROCm-tag requirement, and CUDA-leaf classification
Normalize torch-index leaves to lowercase before the gfx*/rocm*/cu* allowlist
matches so the canonical gfx120X-all (capital X) gets the torch 2.11 floor in
install.sh (leaf, flavor and repairable helpers). Require an installed +rocm
local tag before a rocmX.Y or non-2.11 gfx pin is judged satisfied in
setup.ps1 Get-RocmPinStaleTags and the Python _rocm_pin_family_mismatch, so an
untagged CPU/CUDA wheel never leaves the pin unapplied. Classify a leaf as CUDA
only via ^cu[0-9]: the Python _TORCH_BACKEND derivation now uses
_is_cuda_family_leaf, and install.sh brands cuda only on cu[0-9]* (unset on an
unknown /current /custom mirror leaf) so the stack probes the GPU instead of
skipping ROCm repair. Add bash, Python and PowerShell tests for capital
gfx120X-all floor, current/custom not-cuda, and untagged-wheel ROCm pins.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: converge torch-index pin detection via a per-venv marker
Introduce a torch-index MARKER that records the exact wheel --index-url used
after each successful torch install, so `unsloth studio update` / repair makes
the "did the pinned index change?" decision by an EXACT string compare rather
than inferring it from the wheel +rocm/+cu version tag. The tag cannot encode
the AMD per-arch gfx family (two 2.11 gfx indexes both install +rocm7.13.0), so
the tag heuristic missed a gfx1151 -> gfx120X-all switch and a custom-URL swap.
Marker path is per-venv (.unsloth-torch-index), one line = the resolved index
URL, written atomically (temp + rename). Path, format and normalization are
shared across all four installers (install.sh, install_python_stack.py,
setup.ps1, install.ps1).
- Reapply gfx pins on a per-arch target change: the marker's exact compare
reinstalls when the pinned index differs, even when both wheels share a tag.
- Honor custom ROCm URL pins during repair: an explicit index whose leaf is not
rocm/gfx/cu/cpu (e.g. simple, current) now reinstalls torch VERBATIM from the
pin when it differs from the marker ("URL wins verbatim").
- Align the KNOWN-2.11 rocm/gfx set to exactly rocm7.2 plus the gfx allowlist
gfx120x-all/gfx1151/gfx1150 in every language; stop treating an unknown newer
rocm (rocm7.3, which does not exist) as the 2.11 line speculatively.
Backward compatible: with no marker (old venvs, torch installed out-of-band) the
existing +rocm/version-tag heuristics still decide, and a matching marker never
reinstall-loops. A cu128 CUDA pin stays a CUDA pin; custom and current leaves are
not CUDA. Adds marker tests (py/sh/ps) plus cross-installer parity checks.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: keep the torch-index marker additive to flavor validation
Three narrow fixes in the marker-based stale-venv detection:
- setup.ps1: a matching marker no longer overwrites the detected installed
flavor. The marker compare is now an additional rebuild trigger, so a stale
wheel (torch swapped to a +cpu build while the marker still records a cuXXX
pin) is still caught by the flavor check instead of being masked as up to date.
- setup.ps1: a supported AMD arch carrying CPU torch is no longer marked stale
and wiped. The downstream AMD Windows ROCm override upgrades CPU torch to ROCm
in place, so wiping first would delete the venv and abort with "Virtual
environment not found". Only a genuinely wrong CUDA wheel still rebuilds.
- install.sh: the Radeon --find-links path records its repo.radeon.com base in
the marker instead of the generic pytorch.org ROCm fallback index, so a later
pin to that generic family correctly reinstalls rather than comparing equal.
Mirrors install.ps1/setup.ps1, which already record the real AMD index.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: honor custom pins and repair pinned venvs in place
Four follow-ups to the torch-index marker work:
- install_python_stack.py: _ensure_cuda_torch/_ensure_rocm_torch now bail when an
explicit custom-index pin names no known torch family, so a verbatim URL override
(a private/simple mirror) is not clobbered by auto-detected CUDA/ROCm wheels
before _ensure_verbatim_torch_index applies it.
- install_python_stack.py: the ROCm marker is additive, not a substitute -- a
matching marker still runs the family/version check so a wheel swapped after the
marker was written is caught. Mirrors setup.ps1.
- setup.ps1: a stale venv under an explicit pin, whose torch still imports, is
repaired in place (force-reinstall torch from the pin in the dependency pass)
instead of wiped. The wipe path only delegates to install.ps1, so on a direct
update it stranded the user at "Virtual environment not found" instead of
applying the new pin. A broken venv or unpinned drift still wipes/delegates.
- install.ps1: when a pinned ROCm install fails over to a CPU base, the marker now
records the CPU index actually used instead of the ROCm pin, so the next managed
setup does not see CPU torch under a ROCm pin and abort as stale.
* setup.ps1: keep the ROCm CPU-fallback force line the pr5940 test guards
5c93ffd4 folded the pin-change force-reinstall into the ROCm CPU-fallback
condition on one line, so the exact literal that test_pr5940_followups.py checks
(if ($ROCmCpuFallback) { $cpuForce = @("--force-reinstall") }) no longer appeared
and the test failed. Split the two conditions into separate if lines: the ROCm
fallback line is restored verbatim and the pin-change force is its own line. Both
still set $cpuForce to the array, so @splat passes one arg.
* install: honor exact CUDA/custom index URL pins in the torch-index marker
Address three Codex review findings on the torch-index marker mechanism:
- install.sh: after the ROCm CPU repair reinstalls torch from the generic
$TORCH_INDEX_URL, record that as the marker source. A Radeon --find-links
install set _TORCH_MARKER_INDEX_URL to its repo.radeon.com base earlier, so
leaving it made the marker misreport Radeon wheels and a later Radeon pin would
compare equal and skip a needed reinstall.
- install_python_stack.py: _ensure_cuda_torch now consults the exact-URL marker
(_marker_pin_mismatch) when the installed +cuXXX tag matches the pinned leaf,
so a same-leaf CUDA mirror change (official cu128 to an internal cu128 mirror)
is reinstalled and re-recorded instead of skipped.
- _normalize_index_url / _normalize_family_leaf (install.sh, setup.ps1,
install_python_stack.py): lowercase only KNOWN wheel-family leaves (rocm/gfx/
cpu/cuXXX) so gfx120X-all still matches gfx120x-all, while a custom
(unknown-family) leaf keeps its case so a verbatim URL pin like /Current does
not compare equal to /current. Tests updated to assert the refined behavior.
* install: fix 3 torch-index marker edge cases (CPU mirror pin, Radeon leaf, migrated venv)
Addresses three review findings on the torch-index override path:
1. CPU index URL change on an already-CPU venv. _ensure_cpu_torch returned
early whenever torch was already a CPU build, so a standalone update that
moved the pin (official /cpu -> a private UNSLOTH_PYTORCH_MIRROR /cpu, same
+cpu tag) never reinstalled. It now consults the exact-URL marker and
reinstalls only when _marker_pin_mismatch reports a different index,
mirroring the CUDA/ROCm same-family handling. A matching marker (or none)
still leaves CPU torch untouched, so there is no reinstall loop.
2. Radeon find-links directory misclassified as a pip ROCm family. A
repo.radeon.com/.../rocm-rel-7.2.1 leaf starts with "rocm" but is a
find-links listing, not a pip --index-url. The old startswith(("rocm",
"gfx")) test routed it into a --index-url reinstall that fails against
find-links. New _is_pip_rocm_family_leaf gates on ^rocm\d / gfx (matching
install.sh's rocm[0-9]* and setup.ps1's ^(rocm[0-9]|gfx)), so a Radeon URL
routes to the verbatim/marker path instead.
3. Migrated venv rewriting its marker to a pin it did not install. install.sh
and install.ps1 write the marker unconditionally, so a migration that
preserves existing torch recorded the newly requested pin and a later
update then found a matching marker and skipped the reinstall the pin
needs (e.g. a per-arch gfx1151 -> gfx120X-all switch, identical +rocm tag).
Both now track _TORCH_INSTALLED_THIS_RUN and write the marker only when
torch was actually installed or repaired this run.
Also add Get-NormalizedFamilyLeaf to the setup.ps1 helper-extraction list in
test_torch_index_marker.ps1 (it was added to setup.ps1 and the shell test in an
earlier round but missed here) and add two unit tests covering findings 1 and 2.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: keep pinned torch repairs on the pinned index
Two fixes for explicit index pins (UNSLOTH_TORCH_INDEX_FAMILY / _URL):
1. install_python_stack.py's repair paths ran uv without clearing the
inherited uv index env vars. uv resolves the default index (--index-url
or --default-index) at the LOWEST priority, so a UV_INDEX or
UV_EXTRA_INDEX_URL mirror in the environment won for any package it
served: a cu128-pinned repair could install torch from the mirror and
then record the cu128 marker it never used. Verified empirically: with
UV_EXTRA_INDEX_URL=.../cu126 exported, uv pip install torch
--index-url .../cu128 resolves torch 2.13.0+cu126. Strip the four uv
index env vars for pinned-index commands only, mirroring the gate
install.sh, install.ps1 and setup.ps1 already have; non-pinned installs
keep the user's mirror.
2. install.ps1 routed any pinned leaf matching rocm* through the ROCm
--default-index path, so a custom find-links leaf like rocm-rel-7.2.1
was treated as a PEP 503 ROCm index and could silently fall back to CPU
torch on resolution failure. Require a digit after rocm, matching
install.sh's rocm[0-9]* and install_python_stack.py's ^rocm\d.
Adds parity + unit tests for both (11 new tests).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: keep pinned repairs off UV_TORCH_BACKEND and narrow setup.ps1's rocm pin match
Round 2 of the pinned-index hardening:
1. _build_uv_cmd converted UV_TORCH_BACKEND into --torch-backend before the
new env isolation could act, and uv's torch backend redirects torch
resolution to its own per-backend index even when --index-url is given
(verified: a cu128-pinned dry run with UV_TORCH_BACKEND=cpu resolves
torch 2.13.0+cpu). Pinned-index commands now never receive the flag and
UV_TORCH_BACKEND joins the stripped env vars, so uv cannot re-read it.
2. setup.ps1's pinned reroute had the same bare rocm* glob install.ps1 had:
a custom find-links leaf like rocm-rel-7.2.1 was routed through the ROCm
--index-url path instead of the verbatim unknown-pin path. Now requires
a digit after rocm, matching install.ps1, install.sh and
_is_pip_rocm_family_leaf.
3. The marker test's case-normalization checks used -eq, which is
case-insensitive in PowerShell, making them vacuous, and the unknown-leaf
expectation was written lowercased while the implementation deliberately
preserves custom-leaf case. Tightened to -ceq with the case-preserving
expected value.
Adds unit + parity tests for 1 and 2 (5 new tests).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: extend the pinned-index guards to every remaining surface
Round 3 of the pinned-index hardening, closing the same holes on the
surfaces the earlier rounds missed:
1. install.sh's pinned-install env scrub now clears UV_TORCH_BACKEND (uv's
torch backend redirects torch resolution to its own per-backend index
even against --default-index), and both PowerShell wrappers clear it in
their pinned-install scrubs, matching install_python_stack.py.
2. setup.ps1's marker stale check still classified any rocm* leaf as a
PyTorch ROCm family while the install selection is digit-gated, so a
custom rocm-current / rocm-rel-7.2.1 pin stale-compared as
not-rocm vs rocm and force-reinstalled on every studio update. The
stale check now uses the same ^rocm\d gate.
3. install_python_stack.py's pinned-command scrub also strips
PIP_EXTRA_INDEX_URL for the pip fallback: pip adds the env extra index
in addition to --index-url, so an inherited mirror could satisfy torch
off the pin while the marker recorded the pinned URL. PIP_INDEX_URL
needs no strip since the explicit --index-url flag overrides it.
Parity + unit tests extended (4 new tests).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: scrub find-links and carry the pinned scrub through pip fallbacks
Round 4 of the pinned-index hardening:
1. UV_FIND_LINKS joins every pinned-install scrub (install.sh, install.ps1,
setup.ps1, install_python_stack.py): uv's --find-links locations can
satisfy torch off the pinned index the same way an extra index does.
2. setup.ps1's Fast-Install restored the scrubbed vars in its finally
BEFORE the pip fallback ran, and never touched the pip env vars at all,
so a failed uv attempt fell back to python -m pip with an inherited
PIP_EXTRA_INDEX_URL / PIP_FIND_LINKS able to win over the pinned
--index-url. The scrub now wraps the whole function (uv attempt + pip
fallback) and includes the pip vars; restore happens after both.
3. install_python_stack.py's scrub also strips PIP_FIND_LINKS for its own
pip fallback, completing the PIP_EXTRA_INDEX_URL fix from round 3.
Parity tests extended (2 new tests).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: digit-gate rocm leaves in marker normalization and ROCm side effects
Round 5 of the pinned-index hardening (three custom-rocm-leaf edge cases):
1. _normalize_family_leaf lowercased every leaf starting with rocm, so a
custom mirror leaf like rocm-Current compared equal to its lowercase form
and a case-only pin change was skipped. URL paths can be case-sensitive.
The rocm prefix is now digit-gated (rocm[0-9]*, matching
_is_pip_rocm_family_leaf) in install.sh, setup.ps1 and
install_python_stack.py, so only true family leaves (rocm7.2) are
lowercased; a custom rocm-* leaf keeps its case.
2. setup.ps1 Test-MarkerPinMismatch compared normalized URLs with -ne, which
is case-insensitive in PowerShell, so a case-only marker change (Simple
vs simple) was treated as matching and the reinstall skipped. Now -cne.
3. install.sh gated the AMD bitsandbytes install and the "repair ROCm torch"
--default-index reinstall on a bare whole-URL rocm glob, so a custom
CPU/CUDA/private index whose leaf merely starts with rocm (rocm-current)
was force-repaired from the wrong ROCm-only path whenever torch.version.hip
was empty. Both now gate on _torch_index_is_rocm_family, computed once from
the digit-gated leaf (rocm[0-9]*/gfx*).
Tests: 4 new parity assertions plus 2 case-sensitivity marker checks.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: apply an explicit custom torch-index pin on the first update
Round 6: an explicitly-set custom (unknown-family) UNSLOTH_TORCH_INDEX_URL
was silently ignored on the first `studio update` of a venv that predates
the marker feature, on both platforms, because the no-marker case was
treated as "do nothing" and the version-tag heuristics cannot judge an
unknown leaf.
1. install_python_stack.py _ensure_verbatim_torch_index now reinstalls
verbatim when the marker is ABSENT (None), not only when it differs, and
short-circuits only when the marker already records this exact pin. It
then writes the marker, so every later update is a no-op. A user who did
not set the override gets pin=None and is untouched, so an out-of-band
torch install is never clobbered.
2. setup.ps1: for an unknown-family pin on a marker-less venv the stale-venv
check now sets PinChangedForceReinstall so the torch block reinstalls in
place from the pin. It deliberately does NOT set shouldRebuild, which
would wipe the venv and strand a direct `studio update`.
3. setup.sh (the Linux `studio update` entry point) skipped
install_python_stack.py entirely when unsloth was already current, so the
marker-driven reinstall (both the verbatim custom pin and the cu/rocm
flavor and family-change repair, e.g. gfx1151 to gfx120X-all) never ran.
It now forces the dependency pass when a torch-index pin env var is set;
the pass is idempotent and no-ops when the marker already matches. This
mirrors setup.ps1's stale-venv pre-check.
Tests: 3 new parity assertions.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test: expect first-update reinstall for a no-marker custom index pin
Follow-up to d671d8fb2: _ensure_verbatim_torch_index now applies an
explicit unknown-family URL pin verbatim on the first update when the
marker is absent (instead of no-op), so the old
test_verbatim_custom_url_no_marker_is_noop assertion was stale. Rewritten
as test_verbatim_custom_url_no_marker_reinstalls_once: asserts the one
verbatim reinstall from the pinned URL, that the marker is written, and
that a second call with the pin still set is idempotent (no reinstall
loop).
* install: gate the pinned update pass on the marker and record a pin baseline
Round 8, two follow-ups to the round-6 first-update pin fix:
1. setup.sh forced the full dependency pass on EVERY `studio update` while a
torch-index pin stayed exported, even after the marker already recorded the
same pin, turning quick updates into the expensive pass every time. It now
probes install_python_stack.py --torch-pin-needs-apply (which reuses the
exact marker normalization) and forces the pass only when the pin is not yet
applied (marker absent or different); an already-applied persistent pin keeps
the fast path. A probe error fails safe toward running the pass. setup.ps1
gets the same probe in its fast path for parity.
2. A known-family full-URL pin on a venv predating the marker (e.g. an installed
cu128 build and UNSLOTH_TORCH_INDEX_URL pointing at a same-family mirror) left
the marker absent forever: the _ensure_* helpers deliberately do not force a
multi-GB reinstall of identical-family wheels on an old venv, so nothing
recorded the pin and every update re-entered the pass. _record_torch_index_pin_baseline
now records the resolved pin as a baseline after the ensure sequence when the
family already matches and no marker exists, so the pin is tracked (a later
genuine change is detected and applied) and the update loop is broken, without
the redundant reinstall.
Tests: 3 new baseline unit tests, 4 new parity assertions, and the CLI probe.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.sh: keep the pin probe's exit 1 from killing the update under set -e
The --torch-pin-needs-apply probe deliberately exits 1 for the common
steady-state answer (pin already recorded, keep the fast path), but it ran
as a bare command under set -euo pipefail, so the whole studio update
aborted before the exit code was even captured. Absorb the status with
|| _PIN_NEEDS_APPLY=$? and pre-seed 0 so all three outcomes route as
documented: 0 runs the pass, 1 keeps the fast path, anything else fails
safe into the pass. Parity test asserts the guard.
* install: strip pin credentials, disable uv config discovery, bound verbatim installs
Four verified fix groups from a 12-reviewer audit of the torch-index
override feature, each reproduced before fixing:
1. Credential persistence: all four marker writers stored the raw pin URL,
so an authenticated pin (https://user:token@mirror/simple) persisted its
credentials in .unsloth-torch-index (mode 0644 under a default POSIX
umask) and install_python_stack.py printed pin URLs verbatim in repair
messages. Userinfo is now stripped before persisting and in every
log/substep that interpolates a pin, via lockstep helpers
(_strip_index_url_credentials in install.sh / install_python_stack.py,
Remove-IndexUrlCredentials in install.ps1 / setup.ps1). The three
normalizers strip too, so an OLD marker that already carries credentials
still compares equal to the same pin: no reinstall loop on upgrade.
Query strings deliberately stay in the marker; two indexes distinguished
only by query must not compare equal.
2. uv configuration discovery beat the explicit pin: with a discovered
uv.toml declaring torch-backend = "cpu" or a [[index]] entry, uv 0.10.12
resolves torch 2.13.0+cpu against an explicit --index-url/.../cu126 pin;
UV_NO_CONFIG=1 restores +cu126 (reproduced both ways). The pinned-install
scrub in all four installers now sets UV_NO_CONFIG=1 and drops
UV_CONFIG_FILE.
3. The verbatim custom-index update path installed a bare, unconstrained
torch trio while fresh installs from the same unknown-leaf pin apply the
supported range; _ensure_verbatim_torch_index now installs the bounded
trio spec, closing the fresh-vs-update asymmetry.
4. Query-bearing pins (.../cu128?token=x) classified by raw leaf split and
force-reinstalled on every update (the installed cu128 never equals
cu128?token=x). Query/fragment are now stripped before leaf
classification in all four implementations; the marker comparison keeps
the query per (1).
Rejected after verification (no change): the pin-baseline record cannot
produce a wrong later decision (every pin change still mismatches and
reinstalls from the new pin); the venv temp-file symlink scenarios require
an attacker who already owns the environment; pathological inputs like
" / cu128 / " have no realistic caller and fail loudly.
Parity, stack, rocm-support, marker (sh + ps1), pin-stale, index-url and
flavor suites all pass (455 python + full shell/ps1 batteries).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: harden custom-pin repair against clobber, broken torch, and pip config
Four follow-ups to the pinned-index audit fixes:
1. setup.ps1 routed an unknown-leaf custom pin through the CUDA branch with
a bare torch trio while install.ps1 (fresh) and the Python verbatim path
bound the supported range; the pinned unknown-leaf route now applies the
same torch>=2.4,<2.11.0 bound. Known cu* leaves and unpinned runs are
unchanged.
2. The final torch safety pass could not repair a clobbered unknown-family
pin: intermediate dependency steps can pull torch from PyPI (the pass
exists for exactly that reason), but the verbatim helper short-circuited
on marker==pin and no flavor tag exists to probe. The helper now keeps a
per-run snapshot of the installed trio (taken after a verbatim reinstall
or on the first matching-marker pass) and reinstalls from the pin when
the final pass sees the trio drifted. Probe failure skips the
comparison; a reinstall refreshes the snapshot, so no loop.
3. _record_torch_index_pin_baseline could freeze a known-family pin as
applied on a venv whose torch is missing or broken (every family helper
returns without reinstalling when its probe fails), making
--torch-pin-needs-apply report done forever. The baseline now probes the
installed flavor and records only on a match: a cuXXX pin requires the
matching +cuXXX tag, cpu requires a cpu build, rocm/gfx requires hip;
probe failure records nothing.
4. The pinned pip fallback stripped PIP_* env vars but user/site pip config
files still applied (a configured global.extra-index-url can satisfy
torch off the pin). PIP_CONFIG_FILE is now pointed at the null device
for pinned commands (pip loads no config files then), in
_install_env_for_cmd and setup.ps1's Fast-Install pinned scrub.
install.sh / install.ps1 have no pip fallback (uv-only), verified.
Tests: 7 new rocm_support tests (snapshot reset fixture), 1 stack test,
2 parity tests. Full battery green (464 python, sh and ps1 suites).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: complete the pin-repair coverage across the fast path and platforms
Three cross-platform follow-ups to the round-2 pin-repair fixes:
1. The --torch-pin-needs-apply probe only compared marker==pin, so a torch
trio clobbered to the wrong family (a cpu wheel replacing cu128 via a
later pip install) with a still-matching marker reported "already
applied" and the _ensure_{cuda,rocm,cpu} repair never ran on the Linux
fast path. The probe is now a testable _torch_pin_needs_apply() that also
checks the installed flavor against a known-family pin (via a shared
_torch_flavor_matches_pin() helper, so the baseline and the probe cannot
drift). An unknown-family pin has no flavor to validate and a failed
probe cannot prove drift, so both keep the fast path.
2. macOS ARM (real CPU/MPS torch, not NO_TORCH) never applied an unknown-
family custom pin on update: both the verbatim path and the baseline
returned on IS_MACOS while fresh install.sh honors the pin, so the marker
was never written and setup.sh forced the dependency pass on every update
forever. The guards are now IS_MAC_INTEL (Intel mac is already NO_TORCH),
and the final pass applies the pin on macOS ARM.
3. The round-2 final verbatim repair sat in the step-13 sequence guarded
not IS_WINDOWS, so on Windows a dependency step that clobbered torch after
the pin was applied was masked by the matching marker (setup.ps1 does not
re-validate the main venv's torch after calling this script -- verified).
Step 13 now runs the verbatim snapshot-drift repair on Windows and macOS
ARM too; the Linux-oriented cuda/rocm/cpu family helpers stay Linux-only.
Tests: 13 new rocm_support cases (flavor drift, macOS ARM, Windows repair),
parity updates. Full battery green (475 python, sh and ps1 suites).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: strip query tokens from the marker and tighten the pin-drift probe
Four follow-ups to the round-3 pin-repair fixes:
1. The credential stripper feeding the torch-index marker and the logged repair
messages dropped only user:pass@ userinfo, so a private feed that carries its
auth token in the query string (.../simple?token=SECRET) persisted the token
in the world-readable marker (mode 0644 under a default umask) and printed it
in substep output. All four strippers (install.sh, install.ps1,
studio/setup.ps1, install_python_stack.py) now drop the query and fragment
before building the sanitized URL. A query is not part of a PEP 503 index's
identity, so this also stops a rotated token from spuriously mismatching the
marker and forcing a needless reinstall.
2. The --torch-pin-needs-apply fast-path probe accepted an untagged CUDA build
(no +cuXXX local tag) under a specific cuXXX pin, but _ensure_cuda_torch
reinstalls exactly that build to enforce the pin. The probe was more lenient
than the repair, so the repair pass was skipped on the fast path.
_torch_flavor_matches_pin now reports a mismatch for an untagged build under a
cuXXX pin, forcing the pass.
3. The probe's ROCm branch accepted any HIP build for a rocm/gfx pin, while
_ensure_rocm_torch decides a reinstall with the per-arch
_rocm_pin_family_mismatch predicate (a generic +rocm7.2 wheel under a per-arch
gfx pin, or a wrong ROCm version, is a mismatch). The probe now reuses that
predicate, so it is as strict as the repair. This needs the installed torch
version, so _probe_torch_flavor now returns (marker, cutag, version) and
_torch_flavor_matches_pin takes the pin URL (extracting the leaf internally).
4. On Windows a known-family cu*/cpu pin is applied to the main venv by setup.ps1
before install_python_stack.py runs; a later dependency step can clobber it,
and the GPU-aware _ensure_{cuda,cpu}_torch self-skip on Windows while the
verbatim helper handles only unknown-family pins, so nothing repaired the
clobber (setup.ps1 does not re-validate the main venv's torch afterward,
verified). New _ensure_pinned_known_family_torch reinstalls a drifted cu*/cpu
pin in the step-13 Windows/macOS-ARM branch; rocm/gfx per-arch specs stay owned
by setup.ps1, unknown-family by the verbatim helper.
A speculative ROCm 2.11 floor was also raised but is unreachable: the rocm7.2
index publishes no 2.x wheel below 2.11.0, and an unknown newer rocm is not
floored speculatively.
Tests: query/fragment strip cases in the sh + ps1 marker suites and the Python
strip/marker tests; the tri-state helper and the probe/baseline harnesses moved
to the (marker, cutag, version) flavor with matching versions; new probe cases
(untagged CUDA, generic-rocm-under-gfx) and 8 _ensure_pinned_known_family_torch
tests; a four-way query-strip parity assertion. Full battery green (1150 python,
sh 26/26 marker, ps1 marker/flavor/pin-stale).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: reinstall markerless gfx pins and cap custom-index updates at torch 2.11
Two follow-ups from the pin-marker audit:
1. A markerless venv with a gfx per-arch 2.11 pin trusted the wheel version
tag, which is byte-identical (+rocm7.13.0) across gfx120X-all / gfx1151 /
gfx1150. A pre-marker install holding one gfx arch's wheel that is now
pinned to a DIFFERENT gfx index was therefore never switched:
_rocm_pin_family_mismatch returns no-mismatch for any three-part +rocm
2.11 wheel, and _ensure_rocm_torch's absent-marker branch fell through to
that heuristic. _ensure_rocm_torch now forces a one-time reinstall when the
marker is absent AND the pin leaf is a 2.11 gfx per-arch index; the reinstall
writes the marker, so the next update compares exactly and does not loop
(the correctly-pinned no-reinstall guarantee then comes from the exact marker
compare, not the ambiguous tag). Non-gfx-2.11 pins (rocmX.Y, non-2.11 gfx)
stay on the tag heuristic -- their tags are distinguishable.
2. The verbatim custom-index update path used _CUDA_TORCH_PKG_SPEC (torch
<2.12.0) while a FRESH install of the same unknown leaf caps torch at
<2.11.0 (install.sh's default TORCH_CONSTRAINT, and setup.ps1's custom-pin
branch), so a private /simple mirror publishing torch 2.11 could upgrade a
`studio update` to a state the fresh installer never produces. Added
_CUSTOM_INDEX_TORCH_PKG_SPEC (torch>=2.4,<2.11.0), used only by the verbatim
path; companions stay pinned for the same exclusive --index-url ABI reason
as _CUDA_TORCH_PKG_SPEC (a bare name could pull a torch-2.12-built
torchvision). _CUDA_TORCH_PKG_SPEC is unchanged (known-family cu/cpu repair
correctly tracks install.sh's widened cu ceiling).
Tests: 2 new markerless-gfx cases (one-time reinstall + marker write + no-loop
second run, and the rocmX.Y absent-marker no-op), the pre-existing markerless
gfx no-reinstall test flipped to assert the one-time reinstall (it had encoded
the old tag-trusting behavior), and the custom-index bound assertions. 488
passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: a matching marker must not mask a broken, clobbered, or misclassified torch
Four round-6 follow-ups, all closing cases where a matching torch-index
marker wrongly vouched for a torch that is not actually the pinned one:
1. _is_cuda_family_leaf matched cu+digits by PREFIX (^cu[0-9]), so a custom
mirror leaf like cu128-private classified as CUDA family; the flavor check
then compared the installed cu128 tag to the whole leaf cu128-private and
forced a reinstall on EVERY update (never converging). The cu family is
now matched EXACTLY (re.fullmatch cu[0-9]+), so a cu-suffixed custom leaf
routes through the verbatim/unknown path with a stable marker. Mirrored in
install.sh (_normalize_family_leaf: strip cu, require an all-digit
remainder) and setup.ps1 / install.ps1 (^cu[0-9]+$).
2. _torch_pin_needs_apply returned False on a failed torch probe (missing or
unimportable) under a matching marker, so setup.sh kept the fast path and
a broken torch was never repaired. A failed probe now forces the pass: the
marker cannot vouch for a torch that does not import, forcing is idempotent,
and once torch imports again the probe succeeds and the forcing stops
(self-resolving). Reverses the round-4 conservative choice for this case.
3. _ensure_verbatim_torch_index snapshotted the installed trio on the first
pass with a matching marker and treated an unimportable torch (snapshot
None) as "no drift, skip", so a torch clobbered to a broken state before
the run was masked. A None snapshot now reapplies the pin. A torch
clobbered to a WORKING-but-wrong build under an unknown-family pin remains
undetectable from metadata (no flavor tag; reinstalling every update would
be the loop this avoids) and is documented as a known limitation.
4. The step-13 Windows final repair reran only the verbatim (unknown-family)
and known-family cu*/cpu paths, so a clobbered explicit rocm/gfx pin (the
wheel setup.ps1 installed from AMD's per-arch index) was left in place. The
branch now also runs _ensure_rocm_torch on Windows for an explicit rocm/gfx
pin; it has a Windows path and no-ops when torch already links HIP, so it
only reinstalls a genuinely clobbered ROCm venv (loop-safe).
Tests: the round-4 failed-probe-trusts-marker test flipped to force the pass;
new cases for the cu-suffix no-loop, the broken-torch verbatim reinstall, and
the Windows rocm final-repair structure; item-2 exact-cu parity assertions.
490 passed. sh/ps1 marker + flavor + pin-stale suites all green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: repair Windows ROCm pins from the pinned URL and honor NO_TORCH
Four round-7 review items, two of them regressions in the round-6 work:
1. _torch_pin_needs_apply ignored UNSLOTH_NO_TORCH. With a torch-index env
var set and no marker, the failed-probe branch forced the dependency pass
on every `studio update`, and the pass (which also honors NO_TORCH) never
installs torch or writes a marker, so nothing could ever stop the forcing.
It now returns False immediately under NO_TORCH: the pin only matters once
torch is actually installed.
2. The step-13 Windows final repair (round-6) restored a clobbered explicit
rocm/gfx pin by calling _ensure_rocm_torch, whose Windows path reinstalls
from the arch AUTO-DETECTED via hipinfo, not from the pin. A user pinning a
different gfx family or a private mirror was restored from the wrong source
(and the wrong marker written), and a headless box was skipped entirely
(the arch probe returns nothing). The repair now goes through
_ensure_pinned_known_family_torch, which reinstalls from the PINNED url with
the same per-arch floor setup.ps1 uses (2.11-line gfx leaves) or a bare trio
(older arches, rocmN mirrors). It is gated on IS_WINDOWS since macOS ARM has
no ROCm, and the existing flavor check keeps it loop-safe (a matching HIP
wheel is left alone).
3. _ensure_verbatim_torch_index's broken-torch check (round-6) used
"_installed_trio_snapshot() is None", but that helper reports a REMOVED torch
as "torch==absent" (a non-None tuple) and a broken import as the stale
on-disk version, so a missing or unimportable torch under a matching marker
was read as "no drift" and skipped. The matching-marker path now confirms
torch health with an import probe (_probe_torch_flavor): a torch that does
not import reapplies the pin, while a healthy torch keeps the snapshot-based
intra-run drift detection.
4. A unit test for _ensure_cpu_torch did not pin NO_TORCH False like its
siblings, so a suite run with UNSLOTH_NO_TORCH=1 in the environment made the
guard return early and the reinstall assertions fail spuriously.
Tests: the round-6 broken-torch verbatim test re-encodes the non-None
"torch==absent" snapshot case (the exact state the old "is None" check missed);
new Windows-ROCm pinned-repair cases (reinstall from the pin, per-arch floor vs
bare spec, matching-wheel no-op, off-Windows no-op); a NO_TORCH fast-path probe
case; the parity test now asserts the Windows final branch does not auto-detect
the ROCm index and that the helper reinstalls from the explicit pin. 494 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: floor the rocm7.2 index in the Windows pin repair; isolate marker tests
Three round-8 review items, two of them downstream of the round-7 changes:
1. _ensure_pinned_known_family_torch gave a rocm<d> index leaf a bare
torch/torchvision/torchaudio trio while flooring only gfx* leaves, so a
Windows venv clobbered under an explicit rocm7.2 pin could reinstall an
unbounded or ABI-mismatched trio from that exclusive --index-url. It now
mirrors the spec the initial ROCm paths pin: the rocm7.2 floor for 2.11-line
gfx leaves and rocm<d> leaves that serve torch 2.11, the <2.11 default for
older rocm versions, and a bare trio only for older gfx per-arch leaves
(which publish no floor), matching _ROCM_TORCH_PKG_SPECS / _ensure_rocm_torch.
2. test_verbatim_custom_url_no_marker_reinstalls_once called
_ensure_verbatim_torch_index twice; the second call now hits the
matching-marker health probe, and with pip_install mocked torch never becomes
importable, so in a no-torch environment _probe_torch_flavor returned None and
forced another reinstall, failing the idempotence assertion. The test now pins
a healthy flavor so the idempotence check is about the marker, not ambient
torch.
3. The TestEnsureRocmTorchMarker fixture patched os.environ per test but not
_TORCH_BACKEND, which install_python_stack.py computes once at import from
UNSLOTH_TORCH_BACKEND. A runner starting with a cuda/cpu backend made
_ensure_rocm_torch early-return and skip the mocked repair these tests
exercise. The fixture now neutralizes _TORCH_BACKEND so the marker tests are
independent of the caller's installer-pin environment.
Tests: the Windows floor-spec test now asserts a rocm7.2 mirror pin uses the
rocm7.2 floor (not bare), plus a new rocm7.1 case that must fall back to the
<2.11 default; the marker suite passes under a hostile
UNSLOTH_TORCH_BACKEND=cuda / UNSLOTH_TORCH_INDEX_URL env. 495 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: apply same-flavor pin repoints, keep ROCm fallback nonfatal, bound custom companions
Four round-9 review items, two of them regressions in the round-7 pin helper:
1. _ensure_pinned_known_family_torch returned as satisfied whenever the installed
flavor matched the pin, so a same-flavor SOURCE change (one /cpu or /cu128
mirror to another, or a gfx1151 -> gfx120x-all per-arch switch, both carrying
the same wheel tag) was never applied, while _torch_pin_needs_apply kept forcing
the pass on the marker mismatch forever. It now also reinstalls when the marker
records a DIFFERENT index of the same flavor, rewriting the marker so the next
update matches (no loop), exactly as the Linux _ensure_{cuda,cpu}_torch helpers
do. An absent marker on an already-matching venv is still left to the baseline
recorder (no forced reinstall of a correct pre-marker venv).
2. That helper reinstalled a Windows ROCm pin with the FATAL pip_install, so when
setup.ps1 had taken its CPU fallback (the pinned AMD index unavailable), the
final repair re-hit the same missing index and aborted the whole install. The
ROCm reinstall is now nonfatal (pip_install_try): on failure it leaves the CPU
base in place and writes no ROCm marker, so the install completes -- matching
_ensure_rocm_torch's Windows path. cu*/cpu pins stay fatal (authoritative source).
3. install.sh left torchvision/torchaudio bare for a pinned custom/unknown-leaf
index (a private /simple mirror), unlike the Python update path's
_CUSTOM_INDEX_TORCH_PKG_SPEC, so a mirror also exposing newer companion wheels
could resolve a torch-2.12-built torchvision against the capped <2.11 torch. It
now bounds the companions (torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0)
for a custom leaf, gated on an empty _expected_torch_flavor_tag so known families
keep their curated bare/floored companions.
4. install.sh's _expected_torch_flavor_tag matched cu[0-9]* by prefix, so a custom
leaf like cu128-private classified as the cu128 family and force-reinstalled a
correct +cu128 wheel on every run. It now requires exact cu+digits (routing the
suffixed leaf to the custom path), matching the Python re.fullmatch(cu[0-9]+) and
PowerShell, and feeding item 3's custom-leaf detection.
Tests: new cases for the same-flavor marker-change reinstall, the nonfatal ROCm
fallback (no marker on failure), the rocm7.2/older-rocm floor selection now split
across the nonfatal path, cu-suffixed custom leaves in test_torch_flavor.sh, and the
custom-leaf companion bounds in test_torch_constraint.sh. 497 python + 143 shell
assertions pass; the marker suite still passes under a hostile
UNSLOTH_TORCH_BACKEND=cuda env.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: bound custom-pin companions on the Windows setup path; isolate pin-probe tests
Two round-10 review items:
1. setup.ps1's custom/unknown-leaf pin branch capped only torch ($cudaTorchSpec)
and still asked the exclusive index for bare torchvision/torchaudio, so a
private mirror that also serves newer companion wheels could install a
torch<2.11 wheel alongside a torchvision>=0.26 / torchaudio>=2.11 built for a
newer torch ABI, after which the marker records the pin as applied. It now
bounds the whole trio (torch>=2.4,<2.11.0 / torchvision>=0.19,<0.26.0 /
torchaudio>=2.4,<2.11.0) for a pinned non-cu-family leaf, matching install.sh,
install.ps1's fresh pinned install, and install_python_stack.py's
_CUSTOM_INDEX_TORCH_PKG_SPEC. This completes the companion-bounds fix across all
three installers; known cu* leaves keep bare specs (the family index bounds them).
2. The _torch_pin_needs_apply probe tests did not pin NO_TORCH False, so a test
process launched with UNSLOTH_NO_TORCH=1 short-circuited the probe (the round-7
guard) and returned False for cases that expect the pass to run. The _needs_apply
helper now patches NO_TORCH (default False) around the call, and the dedicated
no-torch case passes no_torch=True explicitly.
Tests: the cross-platform parity test now asserts setup.ps1 bounds the full trio
(not just torch) for a custom leaf; the pin-probe suite passes under a hostile
UNSLOTH_NO_TORCH=1 environment. setup.ps1 parses clean; 497 python + shell suites
green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: bound custom rocm-* pins, redact diag tokens, snapshot custom pins before base update
Three round-11 review items, all reproduced before fixing:
1. install.sh's custom-index companion bounds gated on _expected_torch_flavor_tag
returning empty, but that helper returned "rocm" for ANY rocm* leaf, so a custom
mirror whose leaf starts with rocm but is not a pip family (a private rocm-current
mirror, a Radeon find-links rocm-rel-7.2.1) escaped the bounds and installed bare
torchvision/torchaudio. It now digit-gates rocm to rocm[0-9]* (matching the Python
_is_pip_rocm_family_leaf ^rocm\d), so those custom leaves return "" and the <2.11
companion caps apply; real rocm7.2 / gfx per-arch indexes still classify as rocm.
2. _tauri_torch_index_family classified by the raw last path segment, so a pinned URL
carrying auth in the query (.../rocm7.2?token=SECRET) had the token echoed verbatim
into the emitted [TAURI:DIAG] line. It now strips query/fragment before classifying
(mirroring the marker/log credential stripping), so no token reaches the diagnostic
output; as a side effect .../cu128?token=x now classifies as cu128 instead of auto.
3. On studio update, the core package step (a newer unsloth can require a torch the
custom pin does not satisfy, pulling a default PyPI trio) runs BEFORE the step-2b
verbatim check, which then recorded the already-clobbered trio as the baseline for a
matching marker and left the pin unapplied. A new _capture_verbatim_baseline() records
the pre-clobber trio before the core step, so the verbatim pass detects the drift and
reapplies the pin. Captures only for a matching custom pin with importable torch; a
mismatched/absent marker or broken torch is left to _ensure_verbatim_torch_index.
Tests: _expected_torch_flavor_tag rocm-current / rocm-rel cases; _tauri_torch_index_family
token/fragment redaction with a no-leak regression guard; _capture_verbatim_baseline
record/skip cases plus an end-to-end clobber-detection scenario; a structural guard that
the capture runs before the core step. 501 python + shell suites pass; install.sh bash -n
clean, shellcheck unchanged from base.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: match rocm family leaves exactly, enforce the rocm7.2 torch line, repair a broken pinned torch
A pinned index is a pip ROCm --index-url family only when its leaf is an exact
rocm<digits> / rocm<digits>.<digits> (rocm7.2) or a gfx* per-arch leaf. The prior
^rocm[0-9] prefix match also caught suffixed private-mirror leaves (rocm7.2-private,
rocm7-current), routing them through the ROCm/companion-family path instead of the
verbatim pin: the companion bounds were skipped and, on a pre-marker venv with a
compatible +rocm wheel, the pin was never applied. Match the family exactly through one
shared helper at every site:
- install_python_stack.py: _is_pip_rocm_family_leaf (re.fullmatch), plus the two other
loose gates it feeds (_normalize_family_leaf, _torch_flavor_matches_pin).
- install.sh: a new _is_pip_rocm_family_leaf routes _expected_torch_flavor_tag,
_torch_index_repairable, _normalize_family_leaf and the ROCm side-effect gate.
- setup.ps1: a new Test-PipRocmFamilyLeaf routes Get-NormalizedFamilyLeaf and both
pinned reroutes; install.ps1 anchors its reroute regex.
_rocm_pin_family_mismatch (and its setup.ps1 mirror Get-RocmPinStaleTags) compared only
the ROCm version, so a +rocm7.2 wheel whose torch release drifted off the 2.11 line
(2.12/2.13 from an out-of-band upgrade or a custom rocm7.2 mirror) satisfied the family
check while violating _ROCM_TORCH_PKG_SPECS['rocm7.2'] (torch>=2.11,<2.12). Flag it stale
so the repair reinstalls to floor; >=2.11 alone is not enough, so the release is compared
exactly against the 2.11 line for a KNOWN-2.11 rocm pin.
_ensure_pinned_known_family_torch returned on a failed import probe, but
_torch_pin_needs_apply forces the dependency pass on that same failed probe: a broken
torch under a known-family pin was left in place and the pass was forced on every update.
Treat an unimportable torch as drift and reinstall the pinned trio (the spec and marker
derive from the pinned leaf, not the absent flavor); once it lands the probe succeeds and
the fast path returns.
Tests: exact-match cases across test_torch_flavor.sh, test_rocm_support.py,
test_cross_platform_parity.py and the two .ps1 helper suites; the rocm7.2 release-line
and broken-probe-reinstall cases; extraction lists updated for the new helpers.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: anchor the PS pinned-ROCm floor gate and bound install.ps1 custom-pin companions
Round 12 made every family CLASSIFIER exact, but the Windows install-flow floor gate reads
$_pinRocm211 directly from the raw pinned leaf with an unanchored -match '^rocm(\d+)\.(\d+)'
BEFORE any exact classification runs. A suffixed custom leaf (rocm7.2-private) matches that
rocm7.2 prefix, so it takes the 2.11-floor branch and is force-routed through the ROCm
install path before the exact-match elseif can send it to the verbatim install. Anchor the
match ($) in both install.ps1 and setup.ps1 so only an exact rocmX.Y leaf is floored; a
suffixed or newer-suffix leaf falls through to the verbatim path. The Python floor
selection is already exact (dict lookups gated on _is_pip_rocm_family_leaf), so only the two
PS scripts needed this.
install.ps1's custom (non-cu-family) pinned-torch install bounded torch>=2.4,<2.11.0 but
left torchvision/torchaudio bare, so a private mirror serving newer companions could pull a
wheel built for a newer torch ABI while the marker records the pin as applied. Bound both
companions (torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0) when the leaf is not a
cu<digits> family index (a cu index bounds its own resolution), matching setup.ps1's
Test-CudaFamilyLeaf gate and _CUSTOM_INDEX_TORCH_PKG_SPEC.
Tests: parity guards for the anchored floor gate in both PS scripts and for install.ps1's
bounded custom-pin companions.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: tighten comments in the torch-index-override paths
Collapse the verbose comment and docstring blocks added across the installer
scripts and their tests to fewer, clearer lines without changing behaviour.
Remove a duplicated CUDA-spec comment block. Comments/docstrings only; no code
changes (AST-verified).
* install: repair a broken pinned torch on Linux, strip trailing slash in tauri family, count the final step
_ensure_cuda_torch / _ensure_cpu_torch returned on a failed import probe (torch present but
unimportable). With an explicit CUDA/CPU pin, _torch_pin_needs_apply forces the dependency
pass on that same failed probe, and the base package update does not force-reinstall an
already-installed torch distribution, so the broken torch was left in place and the pass
reran every update without repairing it. Treat a failed probe under a pin as drift and
reinstall from the pinned index (the reinstall rewrites the marker and the next probe
imports, so no loop). This is the Linux counterpart of the known-family repair fix.
_tauri_torch_index_family stripped the query/fragment before classifying but not a trailing
slash, so a token-authenticated pin like .../cu128/?token=x collapsed to .../cu128/ and fell
through the exact-suffix */cu128 and */cpu arms to "auto". Strip a trailing slash too,
mirroring _torch_index_url_leaf.
The Windows / macOS-ARM final torch-repair step (_ensure_pinned_known_family_torch) runs a
progress step that base_total never counted (the final-step increment was gated to Linux),
so _STEP ran one past _TOTAL on those platforms. Add the missing increment.
Tests: broken-probe reinstall for the CUDA (family and URL pins) and CPU paths; trailing
slash / slash+token cases for _tauri_torch_index_family; a full-flow progress-count guard
asserting _STEP == _TOTAL on Windows and Linux.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: tighten comments in the torch-index-override paths
* install: harden the torch-index pin across all four installers
Redact index-URL credentials from captured install logs before they print on
failure. uv/pip failure text embeds the failing --index-url verbatim, so a
user:token@ or ?token= secret could leak into the console. Add a shared
redaction pass (_redact_install_output / Redact-InstallOutput) wired into the
error-output dump in install.sh, install.ps1, setup.ps1 and
install_python_stack.py. Verbose mode still streams live uncaptured output, so
it is intentionally left unredacted (developer opt-in).
Trim trailing slashes on the PATH only for a verbatim UNSLOTH_TORCH_INDEX_URL
override, preserving a ?query/#fragment token. A whole-URL rstrip corrupted a
base64 token ending in "/", and a single-slash strip left .../cu128//
classifying as an empty leaf. Add _trim_index_path_slashes /
Trim-IndexPathSlashes and route the override through it; strip ALL trailing
slashes in the backend-branding leaf classifier so a double slash still yields
the real leaf.
Reject a trailing-dot ROCm leaf (rocm7.) in the bash family validator so it
matches Python re.fullmatch(rocm\d+(?:\.\d+)?) and the PowerShell regex: both the
major and the minor must be non-empty digits, so rocm7. is a custom verbatim pin,
not a pip ROCm family.
Scrub PIP_NO_INDEX and PIP_INDEX_URL for a pinned install in the two installers
that have a plain-pip fallback (install_python_stack.py, setup.ps1):
PIP_NO_INDEX=1 makes the fallback ignore every index including the pinned
--index-url, and PIP_INDEX_URL replaces it. install.sh and install.ps1 install
via uv --default-index (which ignores pip config/env), so they are unaffected.
Add unit tests (bash, Python, PowerShell) and cross-platform parity tests
covering credential redaction, path-only slash trimming, the rocm7. validator,
the double-slash leaf, and the PIP_NO_INDEX/PIP_INDEX_URL scrub.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: redact captured torch-install output and warn on a failed pinned ROCm repair
Close a redaction gap the earlier pass missed: setup.ps1's direct
`Fast-Install ... | Out-String` branches (ROCm from $ROCmIndexUrl, CPU/CUDA from
$TorchInstallIndexUrl, plus the Triton and T5 sub-venv installs) printed the
captured $output verbatim on failure, bypassing Redact-InstallOutput. A private
index carrying userinfo or a ?token= in the pin could leak into Windows Studio
setup logs. Route every `Write-Host $output` through Redact-InstallOutput.
Warn on a failed pinned Windows ROCm reinstall in
_ensure_pinned_known_family_torch: the branch printed "reinstalling from it" then
called pip_install_try, but had no else, so a failure continued silently and left
the user believing the pin was applied while the old CPU/wrong torch survived.
Mirror the auto-ROCm Windows path and warn, telling the user to retry.
* install: redact captured output on the pip fallback and optional-install failure paths
The uv install path already redacted its captured output, but pip_install's pip
fallback runs through run(), which printed result.stdout verbatim on failure, and
_print_optional_install_failure did the same. A pinned --index-url carrying
userinfo or a ?token= could still leak there when uv is unavailable or the pip
fallback also fails. Route both through _redact_install_output. The verbose
pip_install_try path stays raw (developer opt-in), matching the other installers.
* install: split the survive-updates marker subsystem into a follow-up
The torch-index override PR grew a persisted per-venv marker plus repair
machinery (stale-pin detection, verbatim re-apply, update-time reinstall
triggers) that roughly doubled it. That subsystem is orthogonal to the core
feature and is being reworked in a follow-up (versioned/hashed marker,
full-URL pin baseline), so it moves there wholesale instead of shipping
twice.
What this PR still does: UNSLOTH_TORCH_INDEX_URL / UNSLOTH_TORCH_INDEX_FAMILY
pick the torch wheel index at install time in all four installers, with the
exact rocm/gfx/cpu/cu leaf classification, the torch 2.11 floor for the
per-arch AMD indexes, bounded companions for custom leaves, credential
redaction of captured installer output, path-only slash trimming, and the
uv/pip index env scrubs. Flavor-based repair keeps honoring the pin: a wrong
family under an explicit pin still reinstalls from the pinned URL, and
setup.ps1 repairs a pinned stale venv in place instead of wiping it.
What moves to the follow-up: the .unsloth-torch-index marker file and its
writers/readers/normalizers, exact-URL pin-change detection on update
(same-tag gfx switches, custom-mirror repoints), the verbatim trio snapshot
and clobber re-apply, the pin-baseline recorder, and the
--torch-pin-needs-apply fast-path probe in setup.sh / setup.ps1. Their tests
(the marker sh/ps1 suites, the stale-pin suite, and the marker classes in the
rocm/cuda/parity suites) move with them; the removed code is preserved on a
local archive branch to seed that PR.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: re-apply a ROCm pin over an existing HIP wheel via the version tag
The subsystem split left an explicit ROCm/gfx pin unenforced on `studio
update` whenever the venv already imported ANY ROCm torch: the pinned
reinstall lived inside the `elif not has_hip_torch` branch, so a rocm6.4 to
rocm7.2 switch, a gfx1151 pin over a generic +rocm7.2 wheel, or a broken
2.12+rocm7.2 drift never re-applied the pin.
Restore the markerless half of that detection: _rocm_pin_family_mismatch
compares the pinned leaf against the installed wheel tag (exact rocmX.Y
compare, the 2.11 gfx per-arch allowlist, the untagged-wheel rule), the HIP
probe emits "<hip_marker>|<version>" again so the installed tag is available,
and _ensure_rocm_torch reinstalls from the pinned URL when the tag mismatches
even though HIP torch is present. setup.ps1 mirrors it: the stale-venv check
routes a pinned rocm/gfx leaf through Get-RocmPinStaleTags instead of
collapsing it to a generic "rocm" flavor, and the existing pinned in-place
repair (no wipe) applies the change.
What still waits for the follow-up marker PR, by design: pin changes the
wheel tag cannot see -- a per-arch switch between two 2.11 gfx indexes
(identical +rocm7.13.0 tag), a custom-mirror URL repoint under the same
family leaf, and unknown-family verbatim pins. Those need the persisted
index record.
Tests restored with the code: the _rocm_pin_family_mismatch table, the five
update-path cases (older-rocm reinstall, gfx-over-pre-2.11 reinstall,
matching-pin no-reinstall, non-2.11 gfx no-reinstall, gfx-over-generic-2.11
reinstall), the "|" probe-format guards, and the AST-extracted
Get-RocmPinStaleTags suite for setup.ps1.
* install: compare major-only rocm pins, redact URL fragments, bound pinned CPU trio
Three review fixes on the restored pin-repair path.
The family classifier accepts a major-only rocm<d> leaf (rocm7), but the
mismatch comparators only parsed rocmX.Y, so a rocm7 pin fell through to the
2.11-line fallback and INVERTED both verdicts: an installed +rocm6.4 wheel
compared as satisfied (pin never re-applied) while a matching +rocm7.2 wheel
compared as stale (reinstall loop). Major-only pins now compare on the major
alone in _rocm_pin_family_mismatch and Get-RocmPinStaleTags: rocm6.x under a
rocm7 pin is a mismatch, any rocm7.x satisfies it, an untagged wheel never
does, and a bare +rocm tag with an unreadable version is accepted (matching
the existing lenient unreadable fallback).
The output redactors scrubbed userinfo and ?query= values but not #fragments,
so a pin like https://mirror/whl/cu128#token=secret leaked the secret in
captured uv/pip failure text -- inconsistent with the URL handling itself,
which already treats fragments as sensitive. All four redactors gain a
URL-anchored fragment rule (anchored so a bare "# comment" line in tool
output is never touched).
setup.ps1's CPU branch installed a bare torch/torchvision/torchaudio trio;
fine for the unpinned host default, but a PINNED cpu index routes through the
same branch and the /cpu index serves newer torch, so a fresh pinned CPU
install could land an unsupported trio that _ensure_cpu_torch then keeps
(it accepts any CPU build). Under a pin the branch now installs the bounded
trio mirroring _CPU_TORCH_PKG_SPEC (torch>=2.4,<2.12.0 and matching
companions); the unpinned path is unchanged.
Tests: major-only rows in the Python mismatch table and the AST-extracted
setup.ps1 suite; fragment + query-plus-fragment + bare-hash-comment cases in
all four redactor suites; a parity check that the pinned CPU trio bounds
exist, are gated on the pin, and mirror the Python repair spec.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: tighten comments in the torch index override paths
* tests: track the moved pass-through inheritance in the gguf order check
Main moved the llama_extra_args pass-through inheritance out of the
GGUF branch into _resolve_inherited_extra_args, which runs before it,
so the source-order assertion's "if request.llama_extra_args is None"
anchor no longer exists inside the branch and the check failed after
the main merge. The test now asserts the same property in the current
shape: inheritance before the GGUF branch (a carried --no-mmproj still
shapes the hub guard's companion requirement), and marker, hub guard,
unload in order within the branch. Full file passes (32 tests).
* tests: anchor the inheritance order check on the call, not the definition
source.index("_resolve_inherited_extra_args(") matched the function
definition, which always precedes the endpoint, so the ordering
assertion was vacuously true. Anchoring on "= _resolve_inherited_
extra_args(" pins the first call site inside the load endpoint (line
4505), which is the statement whose position relative to the GGUF
branch the test is meant to guard. 32 tests pass.
* tests: align the gguf order test with main
Main fixed the stale ordering assertion in PR 7252; adopting its
version verbatim removes this file from the branch diff entirely and
avoids a conflict on the next main merge. 32 tests pass.
* install: bound the companion constraints to torch's window everywhere
A full platform x vendor validation matrix over this branch surfaced a
real trio mismatch on the cpu/mac paths: torch is capped <2.11 (installs
2.10.0+cpu) but the bare torchaudio companion resolves 2.11.0+cpu,
because torchaudio 2.11 dropped its exact torch pin. Reproduced in a
sandboxed end to end cpu install. torchvision still exact-pins torch and
self-corrected.
The default companion constraints are now bounded to torch's window
(<0.26 / <2.11) and widen together with the cu* torch window (<0.27 /
<2.12), so every leaf resolves a paired trio. Verified with uv dry-runs
on the cpu, cu130, and rocm6.4 leaves (2.10.0/0.25.0/2.10.0,
2.11.0/0.26.0/2.11.0, 2.9.1/0.24.1/2.9.1) and a rerun of the sandboxed
cpu install, which now lands torch 2.10.0+cpu with torchaudio
2.10.0+cpu.
The Strix WSL reroute now also forwards UNSLOTH_TORCH_INDEX_URL and
UNSLOTH_TORCH_INDEX_FAMILY into the rerouted 24.04 distro; dropping
them silently reverted the child install to auto-detection, defeating
the pin this branch introduces.
test_torch_constraint.sh updated: the bounded companions must appear at
the defaults and the custom-leaf block, no bare companion may remain,
and the cu* widen must carry the companions with it.
* install: harden the override path against reroute drift and credential leaks
Review sweep focused on default-path idempotency found no defects on the
unset path; these fixes cover the override path and failure reporting.
install.sh:
- The early WSL Strix Halo distro reroute now honors an explicit index
pin (UNSLOTH_TORCH_INDEX_URL / _FAMILY): the pin is used in the current
distro instead of probing the GPU and re-entering another distribution,
matching the contract of the later Radeon and Strix guards. Whitespace
only values do not gate, in parity with get_torch_index_url.
- Verbose mode now streams installer output through the credential
redactor; it previously bypassed the redaction the quiet path applies.
The exit code survives the pipe via an rc file since the script runs
under plain sh with no pipefail.
- The kept-release fallback warning now strips credentials from the
index URL before printing it.
install.ps1:
- Bounded torchvision and torchaudio next to every capped torch install
(custom pin, ROCm CPU fallback, CUDA flavor repair). torchaudio 2.11
dropped its exact torch pin from the wheel metadata, so a bare
companion beside torch<2.11 can resolve a mismatched 2.11.0 build,
cu family indexes included. Mirrors the install.sh companion bounds.
studio/install_python_stack.py:
- The verbose failure path now redacts index URLs in pip and uv output
before printing, matching every other output site in the file.
All sh, ps1 and python installer test suites pass (the host-defaults
suite has a known pre-existing failure unrelated to this change).
* install: redact verbose Windows installer output and repair the parity tests
Follow-ups to the override-hardening commit, from review:
- install.ps1 Invoke-InstallCommand and setup.ps1 Invoke-SetupCommand now
pipe verbose output through Redact-InstallOutput per record, and the
three verbose Fast-Install torch call sites (ROCm, CPU, CUDA) do the
same: uv and pip echo the pinned index URL, credentials included, in
their errors, and verbose mode previously bypassed the redaction the
quiet paths apply. ForEach-Object and Out-Host leave $LASTEXITCODE
untouched, verified with a native command exiting 7 behind the pipe.
- test_cross_platform_parity.py: the install.ps1 companion-bounds
assertion now matches the implemented behavior (bounds on every index,
no cu-family exemption, since torchaudio 2.11 dropped its exact torch
pin) instead of requiring the removed $_pinCuLeaf gate.
- test_rocm_support.py: the WSL reroute guard test slices the whole
function body to its closing brace instead of a fixed 1200-character
window, which the new pin-gate preamble had outgrown.
428 tests pass across the parity, install stack and rocm support suites;
the sh and ps1 installer suites pass unchanged.
* install: tighten comments in the torch-index and ROCm/CUDA repair paths
* install: digit-gate the gfx family leaf and honor ROCm pins in the Windows repair
Two review follow-ups on the override path:
- The pip ROCm family predicate accepted ANY gfx-prefixed leaf, so a
custom verbatim pin like /gfx-private classified as a ROCm family and
enabled the ROCm-only side effects (AMD bitsandbytes, ROCm torch
repair) on a mirror that may serve CPU/CUDA wheels. gfx now requires a
following digit (gfx90a, gfx1151, gfx120X-all), consistently in
install.sh, install_python_stack.py, install.ps1 (family gate and
expected-flavor classifier) and setup.ps1, matching the strictness the
rocm side already had (rocm7.2-private stays verbatim). The broader
backend BRANDING globs are unchanged on purpose: radeon repo leaves
(rocm-rel-X.Y) must still brand the rocm backend without being
force-repaired as a family.
- The Windows branch of the ROCm torch repair always installed from the
public per-arch index, ignoring an explicit ROCm-family pin: after a
pinned setup.ps1 install failed to a CPU base, the repair retried
repo.amd.com instead of the pinned index. The branch now resolves
_explicit_rocm_torch_index_url() first, uses it as the install index
when set, and mirrors the Linux pin contract by skipping the NVIDIA
and gfx-detection gates a pin is documented to override.
Source-assertion tests updated to the tightened predicate and the new
repair label. 1165 tests pass across the parity, install stack and
studio install suites; the sh and ps1 suites pass; both PowerShell
installers parse clean.
* Remove scratch archives accidentally committed with the comment pass
The temp/ archive copies of installer and test files were working
scratch, not PR content, and inflated the diff by about nine thousand
lines.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix text-only VLM CPT packing truncation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle streaming vision datasets in packing
* Harden multimodal packing detection
* Preserve safe packing boundaries
* Scope stream packing checks to VLMs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Narrow VLM packing detection
* Align packing mode and eval safety
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add qwen3_5/qwen3_next to PADDING_FREE_BLOCKLIST to avoid packed-sequence contamination
* Detect hybrid linear-attention models structurally instead of by name for packing guard
* Add experimental varlen packing for hybrid linear-attention models
Feed seq_idx to the causal conv and cu_seqlens to the gated-delta scan so
sample packing / padding-free reset state at sequence boundaries for hybrid
linear-attention models (Qwen3.5, Qwen3-Next). Gated behind
UNSLOTH_EXPERIMENTAL_HYBRID_PACKING and fail-closed: when the flag is off or
the accelerated kernels (causal_conv1d + fla) are unavailable, the guard keeps
these models on the padded path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden hybrid linear-attention varlen packing shim
Make patch_hybrid_linear_attention_varlen robust across transformers 4.57.6
through 5.x and TRL 0.22.2 through 1.x, following the import_fixes.py style:
- Read UNSLOTH_EXPERIMENTAL_HYBRID_PACKING at call time so the flag takes effect
when set after importing unsloth.
- Idempotent: repeat calls on a patched model return True without re-validating
the wrappers or double-wrapping; signatures are checked on captured originals.
- Prefer the authoritative packed_seq_lengths (via get_packed_info_from_kwargs)
over position_ids resets, handling pad_to_multiple_of trailing tokens.
- Suppress injection for cached forwards (use_cache / past_key_values) so
generation and eval are left on the untouched decode path.
- Validate every gated-delta module before mutating any (transactional).
- Bind position_ids / use_cache from both positional and keyword args.
- Verify dispatch at runtime (Unsloth wraps each module forward, so the mixer
source is not statically inspectable) and warn once if the shim is never hit.
- Emit one deduped diagnostic on each fail-closed path.
Add CPU unit tests covering the hybrid guard detection, the boundary builders,
and the shim (fail-closed, active, idempotent, cached no-op, runtime handshake).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Abort hybrid packing when the varlen shim is not fully dispatched
The runtime handshake used a single per-module hit flag written by both the conv
and scan wrappers, so a partial dispatch (only one kernel routed through
self.<kernel>) passed the any() check and trained on contaminated data, and a
missing dispatch only logged a warning. Track conv and scan dispatch separately,
require both on every gated-delta module on the first packed forward, and raise
before loss/backward when either is missing (the batch is already flattened, so
there is no padded recovery at that point). Also skip an empty packed_seq_lengths
before it reaches max(), and document the position_ids fallback's left-pad
assumption.
Add tests for no-dispatch and partial (conv-only / scan-only) abort, the
packed_seq_lengths preference over a competing position_ids, MRoPE 3D position
ids, and the pad_to_multiple_of trailing-segment path through the metadata builder.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Import the hybrid packing patch from its submodule to satisfy the import-hoist lint
* Fail closed for hybrid packing on encoder-decoder, chunked-loss, and string-name models
The varlen shim only helps decoder-only hybrid models that run their mixer
through self.<kernel> on a live nn.Module forward. Three cases slipped past
the guard:
- Encoder-decoder configs (is_encoder_decoder) reached the packing path even
though flattening a cross-attention batch is unsound. Block them explicitly.
- TRL's chunked_nll loss (the 1.x default) calls the backbone directly and
bypasses model.forward, so the per-instance forward wrapper that refreshes
the varlen stash never runs. Detect that path and keep the model padded.
- A string model_name reaches the trainer before the module exists, so the
instance shim has nothing to patch. Resolve the config up front and keep
string hybrids on the padded path.
Adds encoder-decoder / decoder-only / chunked-loss / string-model tests.
* Harden the SFT source-injection replacements and forward auth args for string models
The wrapped-packing injection rewrote the sourced unsloth_zoo sft_prepare_dataset
with str.replace anchored on the exact 'All Unsloth Zoo code licensed under
LGPLv3' comment. str.replace never raises on a missing anchor, so a supported
newer unsloth_zoo (the dependency is only lower-bounded) that moved that header
would silently drop the setup while the truncation and pack_dataset edits still
referenced _unsloth_wrapped_packing / _inspect, raising NameError on every SFT
dataset preparation.
- Install the setup at the sft_prepare_dataset signature via re.subn (a structural
anchor that always exists) and raise if even that is missing.
- Route the remaining edits through a _require_replace helper that fails loudly on a
missing required anchor (or warns once for an optional one), formalizing the
verify-then-replace idiom the DPO patchers in this file already use.
- Reuse the guarded _unsloth_pack_has_strategy at the pack_dataset call instead of
re-calling inspect.signature(pack_dataset) unguarded, so a non-introspectable
pack_dataset cannot crash there after the setup already handled it.
- _resolve_string_model_config now forwards token / use_auth_token / cache_dir /
code_revision, so a private hybrid resolves its config instead of falling through
as non-hybrid and enabling packing without the varlen shim.
Adds regression tests for the drift-resistant injection, the helper, and the
string-model auth forwarding.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor top-level SFTConfig.trust_remote_code when resolving a string model
TRL merges the top-level args.trust_remote_code into the load via
model_init_kwargs.setdefault("trust_remote_code", args.trust_remote_code) before
create_model_from_path, so a remote-code hybrid is commonly set with
SFTConfig(trust_remote_code=True) rather than inside model_init_kwargs. The config
probe only read model_init_kwargs, so AutoConfig could fail for such a model, leave
model_config None, and let the guard treat it as non-hybrid, enabling packing
without the varlen shim. Mirror TRL's setdefault (model_init_kwargs wins).
* Tighten hybrid-packing comments for concision
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: alkinun <alkinunl@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherl <61019402+Etherll@users.noreply.github.com>
* test(studio): add e2e test for cpu-fallback overriding vulkan
* feat(studio): add UNSLOTH_LLAMA_CPP_BACKEND env var
* feat(studio): add UNSLOTH_LLAMA_CPP_BACKEND env var
* Preserve UNSLOTH_LLAMA_CPP_BACKEND=cpu across llama.cpp updates for PR #7228
The in-app updater rebuilt the installer command without --cpu-fallback and
only re-asserted Vulkan, so accepting a llama.cpp update after forcing CPU on
an Intel iGPU host re-ran host detection and routed back to the crashing Vulkan
bundle (#7213). Record install_kind in the prebuilt marker and re-assert
--cpu-fallback on update when the installed bundle is CPU.
Also make setup.sh's UNSLOTH_LLAMA_CPP_BACKEND check case-insensitive to match
setup.ps1, and add tests for the updater CPU preservation and the setup.sh flag
plumbing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim and validate UNSLOTH_LLAMA_CPP_BACKEND, warn on unknown values for PR #7228
Trim surrounding whitespace and lowercase the value in both setup.sh and
setup.ps1, so values like ' cpu ' or 'CPU' still force the CPU-only prebuilt.
An unrecognized value (e.g. 'gpu') now prints a warning instead of silently
falling back to auto. Extend test_setup_llama_cpp_backend.py to cover both
scripts, including trimmed, empty and unknown values.
* Preserve arm64 CPU installs on update and honor CPU override in Windows prune for PR #7228
The update-path CPU preservation only matched install_kind ending in -cpu, so
arm64 CPU bundles (linux-arm64, windows-arm64) were re-routed to a GPU or source
build on update. Match the full set of CPU-only kinds instead.
Persisting install_kind also activated the previously inert Windows
mismatch-prune in setup.ps1: on a GPU host with UNSLOTH_LLAMA_CPP_BACKEND=cpu it
saw the windows-cpu marker as mismatched and deleted it every rerun. Normalize
the override once and make CPU expected so a deliberate CPU install is kept.
Extend the tests to cover both.
* Document legacy llama.cpp markers keep heal-to-GPU on update for PR #7228
Legacy prebuilt markers written before install_kind was persisted intentionally
do not force --cpu-fallback on update: the in-app updater lets them re-resolve
(heal to a GPU bundle) per the existing behavior from #6097, and only markers
that explicitly record a CPU install_kind are pinned to CPU. Add a comment and a
regression case documenting the boundary.
* Tighten llama.cpp CPU-fallback comments for PR #7228
* Fix Windows install-prune to keep valid Intel/fallback bundles for PR #7228
Persisting install_kind activated the setup.ps1 mismatch-prune, whose
expectedKinds was incomplete: the non-NVIDIA/non-AMD branch omitted
windows-vulkan (the Intel auto-route) and the GPU branches omitted the
windows-cpu/windows-arm64 fallback the installer uses when a GPU prebuilt is
missing. That made every setup rerun delete and re-download a valid Intel Vulkan
(or CPU-fallback) install. List all kinds the installer can produce per host so
only a bundle the host cannot run is pruned. Cover the full matrix in tests.
* Persist force_cpu marker flag so only forced CPU installs re-assert on update for PR #7228
* Add --force-cpu for deliberate CPU installs and warn on macOS for PR #7228
* Record force_cpu when reusing a matching CPU bundle for PR #7228
* Accept force_cpu keyword in installer test validator fakes for PR #7228
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Fix text-only VLM CPT packing truncation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle streaming vision datasets in packing
* Harden multimodal packing detection
* Preserve safe packing boundaries
* Scope stream packing checks to VLMs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Narrow VLM packing detection
* Align packing mode and eval safety
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add qwen3_5/qwen3_next to PADDING_FREE_BLOCKLIST to avoid packed-sequence contamination
* Detect hybrid linear-attention models structurally instead of by name for packing guard
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Install wrapped-packing setup at the signature, not the Zoo license comment
The _unsloth_wrapped_packing / _inspect setup block was injected by matching the
exact 'All Unsloth Zoo code licensed under LGPLv3' comment line in the sourced
sft_prepare_dataset. The unsloth_zoo dependency is only lower-bounded, so a newer
Zoo that moves or drops that header made the setup a silent no-op while the
truncation and pack_dataset rewrites still emitted references to those names,
raising NameError on every SFT dataset preparation.
Anchor the setup on the function signature instead (a structural location that
always exists) and fail loudly if it cannot be found, so the helper variables are
always defined before they are referenced across Zoo versions.
Adds a regression test that patches in a Zoo source without the license header.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherl <61019402+Etherll@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* install: preserve the previous torch release across every flavor and vendor
A re-run of curl | sh over an existing install was supposed to keep the
user's validated torch release, but the pin required the old build's
local flavor tag to match the freshly chosen index leaf. That gate was
wrong in practice: a PyPI-sourced torch reports a BARE version (on Linux
the PyPI wheel IS a CUDA build), which classified as cpu and never
matched a cu leaf, so a healthy 2.10 on a cu130 host was silently moved
to 2.11 (reproduced end to end); the same happened for any flavor drift
such as cu128 to cu130 after a driver upgrade, and AMD ROCm leaves were
excluded from preservation entirely.
The rule is now release-based and flavor-agnostic: the probed previous
release is pinned whenever it sits inside the final constraint window,
and the pin installs from the freshly chosen index, so the flavor always
follows the machine (NVIDIA cu*, AMD rocm/gfx, Intel/CPU, mac) while the
release follows the user. The pin is evaluated AFTER every index and
constraint decision including the Strix reroute, so raised floors
(rocm7.2 / Strix gfx need torch 2.11 for the _grouped_mm fix) correctly
reject an older release and win. UNSLOTH_TORCH_UPGRADE=1 still opts out,
out-of-window releases are never kept, and probe noise never becomes a
pin.
The kept-release install with its range fallback (for indexes that do
not carry the exact release) is factored into
_install_torch_default_index and used by every --default-index torch
path: the default NVIDIA/CPU/mac path and all three ROCm-index
fallbacks, which previously bypassed the fallback. The Radeon-repo
direct-wheel path keeps its curated per-arch wheel set (those wheels are
already exact-pinned per rocm release).
Platform coverage: install.sh serves Linux, WSL (including the WoA
fallback), and macOS for all vendors; native Windows install.ps1 still
caps at <2.11.0 everywhere, so the silent 2.10-to-2.11 move cannot occur
there (2.11 alignment is a separate follow-up).
Verified: 35-check unit suite rewritten to the new spec (any-flavor
keep, floor rejection, noise, window edges, opt-out, wiring including
pin-after-reroute and helper coverage); end-to-end matrix against
sandboxed UNSLOTH_STUDIO_HOME installs on a cu130 host covering PyPI
bare, cu128 drift, cu130 same-flavor, out-of-window 2.3, the upgrade
opt-out, the hidden-GPU cpu leaf, and a fresh-install control.
* install: honor the kept torch release on the Radeon direct-wheel path
The Radeon repo path installs an explicit wheel trio selected by
_pick_radeon_wheel, bypassing --default-index, so the kept-release pin
only took effect when the listing failed and the install fell back to
the ROCm index. On a re-run over an in-window Radeon install the trio
search started at the newest common minor and silently moved the user
forward (2.9 to 2.10 whenever the repo offered both).
The trio search now starts at the kept release's minor when
_PREV_TORCH_PIN is set and the listing still offers a torch wheel for
that minor. Radeon wheels are patch-curated per rocm release, so the
minor is the unit of preservation there; the raised rocm7.2 / Strix
floors still win because the pin is window-checked against the final
constraint before this point, and gaps keep the existing downward
search / ROCm-index fallback.
Verified with a simulated listing carrying both a 2.9 and a 2.10 trio:
no pin selects the 2.10 trio, a kept 2.9 release selects the matched
2.9 / 0.24 / 2.9 trio, and an unavailable minor degrades to the newest
trio. Added a structural wiring check to test_previous_torch_pin.sh
(now 36 checks).
* install: tighten comments in the torch preservation paths
* install: exact kept release on the Radeon path, pin fallback in ROCm repairs
The minor-level clamp on the Radeon direct-wheel path still allowed
patch drift (a kept 2.10.0 could become 2.10.1 when the listing carried
both) and the downward gap search could settle below the kept minor,
both breaking the exact preservation guarantee the other vendor paths
honor. The kept release now gets an exact-first trio attempt before the
newest-trio search: pick the kept patch (else the newest patch of the
kept minor, for listings that pruned the exact patch) together with the
paired torchvision/torchaudio wheels for that minor. Any gap warns and
falls back to the unchanged newest-trio search, mirroring
_install_torch_default_index, so a rerun installs either the kept
release or the same set a fresh install would choose, never something
in between.
The two ROCm torch repair sites (torch overwritten by dependency
resolution, on the migrated and fresh paths) installed TORCH_CONSTRAINT
directly, so a pinned release missing from the generic ROCm index would
abort the rerun instead of falling back. Both now route through
_install_torch_default_index, which passes extra uv args through
(--force-reinstall) and clears the pin once the fallback fires so later
paths stay consistent.
Verified against synthetic listings: both patches listed keeps exactly
2.10.0; a kept minor missing vision/audio warns and yields the newest
complete trio rather than a silent undercut; a pruned patch stays on
the kept minor; no pin keeps the existing newest-trio behavior. Unit
suite now 39 checks, all passing.
* install: never pin nightly/dev/source torch builds on a rerun
A survey of published torch version strings (PyPI bare, +cpu, +cu116
through +cu132, +rocmX.Y and +rocmX.Y.Z, +xpu, nightly .devYYYYMMDD,
source a0+git, rc tags) showed one gap: nightly, dev, rc, and source
builds passed the loose release-shape check, producing a pin such as
torch==2.11.0.dev20250704 that no stable index carries. The range
fallback rescued the install, but it printed "keeping it" and then
burned a doomed resolve first. The base must now be a plain numeric
X.Y[.Z] release, so those builds skip the pin and go straight to the
newest supported release.
Added unit checks for +xpu and three-component +rocm7.2.1 tags (both
already preserved correctly) and for nightly, a0 source, and rc builds
(never pinned). Suite now 44 checks, all passing.
* install: pair kept-release companions, protect the flavor repair, note substitutions
Three fixes from a 12-way review pass over the preservation work:
The kept-release install left torchvision and torchaudio unconstrained
next to the exact torch pin. torchvision exact-pins its torch in wheel
metadata so it always paired correctly, but torchaudio no longer does:
a kept torch 2.9.0 on cu130 resolved torchaudio 2.11.0 (verified with
uv dry-runs). The helper now pairs both companions to the kept minor
(torchvision 0.minor+15, torchaudio 2.minor); if the index lacks the
paired set the existing range fallback fires. Verified resolving
correctly on cu130, cu126, and rocm6.4.
The wrong-flavor repair at the end of the install was the one remaining
default-index torch install outside the helper. It runs under set -e,
so a retained pin absent from the repair index (reachable when the
Radeon direct-wheel path installed the kept release and dependency
resolution later overwrote it) aborted the installer at the last step
instead of falling back. It now routes through the helper with its
reinstall flags passed through.
The Radeon kept-release path installed a same-series build silently
when the listing had pruned the exact patch; it now prints what it is
substituting.
Unit suite extended with wiring checks for all three (46 checks, all
passing).
* fix(chat_templates): bind loop_messages when default_system_message is None
construct_chat_template(default_system_message=None) built a system part that
binds loop_messages only inside the `{% if messages[0]['role'] == 'system' %}`
arm. The `Fix missing loop_messages` step right below then found no
unconditional `{% set loop_messages = messages %}`, concluded loop_messages was
missing, and rewrote `{% for message in loop_messages %}` back to
`{% for message in messages %}` -- undoing the `messages[1:]` skip.
A caller-supplied system message therefore reached the loop and tripped
raise_exception:
Only user and assistant roles are supported!
Add the `{% else %}` arm so loop_messages is always bound, mirroring the
default_system_message is not None branch minus the default text. That also
stops the rewrite from firing, since the unconditional binding is now present.
Renders before / after, same template, same inputs:
default_system_message input before after
None system msg raise_exception 'Be terse.\n### User: Hi\n'
None no system '### User: Hi\n' unchanged
'You are helpful.' system msg 'Be terse.\n### User: Hi\n' unchanged
'You are helpful.' no system 'You are helpful.\n...' unchanged
The rewrite still fires for templates with no {SYSTEM} part, which is what it
was there for -- verified unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Scope loop_messages binding to {SYSTEM} templates for PR #7199
The None branch now only adds the else arm when system_part contains
{SYSTEM}, so a static prefix with no {SYSTEM} placeholder keeps raising on a
caller system message instead of silently dropping it. Strengthen the tests:
assert the default does not leak when a caller system message is present, and
add a regression test for the static prefix case.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio: allow torch 2.11.x on the CUDA install path
The CUDA torch repair path (_ensure_cuda_torch) installs torch/torchvision/
torchaudio from an exclusive --index-url, so _CUDA_TORCH_PKG_SPEC decides
exactly which torch the Studio venv gets. It was capped at torch<2.11.0, so on
a cu128/cu130 host the venv resolved torch 2.10.x even though the CUDA indexes
now publish torch 2.11.0. That left the Studio venv a torch minor behind the
torch 2.11.0 Docker base image, so the CUDA dedup step would relink base libs
under a mismatched torch.
Raise the upper bound to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so
the CUDA install path lands on torch 2.11.x, matching the rocm7.2 spec and the
base image. The torchao selector already maps torch 2.11 -> torchao 0.17.0, and
_ensure_flash_attn degrades gracefully when no prebuilt wheel matches (Blackwell
skips it outright; non-Blackwell prints a warning and continues), so no other
pin needs to move.
Add test_cuda_torch_spec.py to lock the bound (torch 2.11.x in, 2.12.x out) and
assert the CUDA and rocm7.2 upper bounds stay in lockstep.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test: use zip(strict=True) so a spec length mismatch fails loudly
* install.sh: widen the CUDA torch ceiling to <2.12.0 so a fresh install matches the base
Raising _CUDA_TORCH_PKG_SPEC alone was not enough: that spec only feeds
_ensure_cuda_torch(), the ROCm-poisoning repair path that early-returns on a
normal NVIDIA host. A fresh CUDA install (including the studio Docker build,
which runs `bash install.sh --local`) takes its torch from install.sh's
TORCH_CONSTRAINT, which was still capped at torch>=2.4,<2.11.0, so cu12x/cu13x
resolved torch 2.10.x and the venv landed a minor behind the torch 2.11.0 base
image.
Extend the existing `case "$TORCH_INDEX_URL"` block (which already relaxes
rocm7.2) with a `*/cu[0-9]*` branch that widens the ceiling to <2.12.0, keeping
the >=2.4 floor so an older CUDA index (e.g. cu118) that tops out below 2.11
still resolves. The CPU wheel and older ROCm tags stay on <2.11.0 (the glob
does not match /cpu). torchvision/torchaudio are bare on this install line and
resolve their compatible companions via wheel metadata, matching the rocm7.2
pattern.
Add behavioral tests (Python + shell) exercising the case block: cu118/124/126/
128/130 widen to <2.12.0, rocm7.2 stays 2.11.x, and /cpu plus older ROCm keep
the default <2.11.0.
* install.sh: key the CUDA torch widening off the index leaf, not the full URL
The `*/cu[0-9]*` glob matched a `cu<digit>` segment anywhere in TORCH_INDEX_URL,
so a custom UNSLOTH_PYTORCH_MIRROR whose base path contains e.g. cu128 but whose
final leaf is cpu or an older ROCm tag would still widen TORCH_CONSTRAINT to
<2.12.0, contradicting the block's own comment and letting a CPU / older-ROCm
mirror resolve torch 2.11.x. Match on _torch_index_leaf (the final path segment
the backend classification just above already computes) so only a real cu*/
rocm7.2 leaf is affected; cpu and older ROCm keep the default <2.11.0. Update
the Python + shell tests to mirror the leaf-anchored case and add regression
cases for a mirror base that contains cu128 but resolves to a cpu / rocm7.1 leaf.
* install: freeze the torch trio during the with-deps unsloth installs
Released unsloth wheels can pin an older torch than Step 1 installed
(unsloth 2026.7.2 declares torch<2.11.0), so the with-deps resolve from
PyPI silently downgrades the pinned +cuXXX torch trio to PyPI's default
wheel. The flavor guard cannot catch every such swap: PyPI's torch 2.10
default is itself cu128-flavored, so the cuXXX tag comparison still
matches while the version silently drops. Freeze the just-installed trio
with uv --overrides (overrides replace dependency requirements during
resolution), keeping torch 2.11.0+cuXXX in place while unsloth's other
dependencies resolve normally. Verified on the cu128 path: without the
override torch drops 2.11.0+cu128 -> 2.10.0; with it the trio survives
and unsloth 2026.7.2 + unsloth-zoo install cleanly.
* install: fold UV_OVERRIDE env files into the torch-trio overrides file
The CLI --overrides flag is the command-line form of UV_OVERRIDE, so
passing it replaced any overrides file already exported for the process;
macOS arm64 exports UV_OVERRIDE=overrides-darwin-arm64.txt for the same
generic install path and would have lost those pins. Concatenate any
UV_OVERRIDE files into the temp trio file so both keep applying.
* install: extend the torch-trio overrides guard to migrated installs
Four follow-ups to the Step-2 --overrides guard, all empirically verified:
1. The migrated-environment with-deps unsloth install resolved
unsloth>=2026.7.2 (which pins torch<2.11.0) without the overrides file,
so a migrated CUDA venv on torch 2.11 was silently downgraded -- the
exact bug this branch fixes on the fresh path. The overrides build is
now a function (_build_unsloth_torch_overrides, reading the trio
installed at call time) invoked by both with-deps paths; the migrated
no-torch path installs --no-deps and stays unguarded.
2. The overrides temp file is now cleaned by the EXIT trap (same pattern
as _UV_OVERRIDE_TMPDIR, pre-initialized empty so an inherited value can
never reach the trap's rm); previously any Step-2 failure leaked it.
3. Folding UV_OVERRIDE files used cat, which joins the last requirement of
a file lacking a trailing newline onto the next file's first requirement
(reproduced: idna==3.10certifi==2025.1.31 makes uv fail parsing).
4. Inherited torch/torchvision/torchaudio override lines are now filtered
out when folding: uv intersects duplicate overrides rather than
last-wins (verified on uv 0.10.12: direct conflict is unsatisfiable,
transitive conflict silently backtracks), so a conflicting inherited
trio pin would break the resolve the generated exact pins protect.
Both 3 and 4 are handled by a single newline-terminating awk filter
that preserves non-trio overrides (torchmetrics, torchao, ...).
test_unsloth_torch_override.sh extended: migrated-path coverage, trap
assertion, and a functional fold test (14 checks).
* installer: tighten comments
* install: keep the existing torch release when re-running the installer
Re-running `curl -fsSL https://unsloth.ai/install.sh | sh` over an existing
install rebuilds the venv for clean state, which silently moved users to the
newest torch in range (2.10 -> 2.11 once the constraint widened). A torch the
user already validated must survive an unsloth update.
Before the old venv is moved aside for rollback, its torch version is probed
(last stdout line only, so sitecustomize noise cannot corrupt it). After the
index leaf is chosen, _previous_torch_pin turns that version into a
torch==X.Y.Z pin, but only when it cannot do harm:
- cu*/cpu leaves only; rocm leaves keep their floors (rocm7.2 must land 2.11
for the Strix _grouped_mm fix) and the Radeon wheel-matching path is
untouched.
- The wheel's flavor tag must match the freshly chosen leaf, so a flavor
change (cpu -> cuda, cu126 -> cu130) still installs the correct new build.
- The base must look like a release, so probe noise never becomes a pin.
- UNSLOTH_TORCH_UPGRADE=1 opts out and restores the old always-newest
behavior; the substep line advertises it.
The supported range is kept in _PREV_FALLBACK_CONSTRAINT: if the exact
release is not resolvable from the chosen index (custom mirrors prune old
wheels), the install warns and falls back to the newest supported release
instead of failing the whole run. The later flavor-mismatch repair reuses
TORCH_CONSTRAINT, so a mid-install clobber is repaired back to the kept
release rather than the newest one.
Verified end to end: a venv seeded with torch 2.10.0+cu130 re-run through the
full installer finishes with torch 2.10.0+cu130 (previously 2.11.0+cu130).
Tests: tests/sh/test_previous_torch_pin.sh covers keep/flavor-change/rocm/
noise/opt-out plus wiring (probe ordering before venv replacement, fallback
present, SKIP_TORCH gate).
* install: constrain kept torch pins to the supported window
Review caught that _previous_torch_pin pinned the previous venv's torch on
flavor match alone, so a release outside the installer's active range (a
2.3.x manual install below the >=2.4 floor, or a 2.12.x manual upgrade above
the ceiling) replaced the bounds computed just above it and a rerun kept a
torch the installer otherwise deliberately excludes.
New _torch_release_in_window checks the probed base against the active
TORCH_CONSTRAINT ("torch>=A.B[,<C.D.F]") at major.minor granularity, which
is exact for the windows this script uses (ceilings are always X.Y.0; a
non-.0 ceiling would only make it conservative). Anything unparseable
answers no, so probe noise or a malformed window fails toward the supported
range instead of becoming a pin. _previous_torch_pin takes the active
constraint as a third argument and refuses out-of-window releases; the
in-window keep behavior is unchanged.
Tests: out-of-window rows (2.3.x floor, 2.12.x ceiling, boundary keeps, cpu
and macOS windows, malformed/empty windows) plus direct
_torch_release_in_window coverage.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* test(version-compat): keep GRPO fake-run logits finite on CPU
The GRPO fake-run test samples completions from a tiny untrained model on
CPU. Such a model can emit non-finite logits, so torch.multinomial inside
generate() intermittently raises "probability tensor contains either inf,
nan or element < 0" -- a nondeterministic sampling failure, not a regression
(the Trainer already fixes the seed, but CPU reduction order is not
bit-reproducible). Add a forward hook that sanitizes the LM head logits to a
finite bounded range before sampling, so the fake run reliably exercises the
whole train loop; the test checks the loop runs, not the numerics.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test(version-compat): drop redundant nan_to_num bounds (clamp handles them)
* test(version-compat): scope GRPO finite-logits guard to the GRPO test
Only test_grpo_trains_on_cpu autoregressively samples completions, so it is
the only canary that can hit the non-finite-logits torch.multinomial crash.
Move the _guard_finite_logits hook out of the shared _load_plain() and into
test_grpo_trains_on_cpu so the SFT and DPO canaries keep asserting against the
model's true, unclamped logits.
---------
Co-authored-by: Daniel Han <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(registry): don't register deepseek models at import time
`_deepseek.py` called `register_deepseek_models(include_original_model=True)`
at module scope, so merely importing `unsloth.registry` registered models
(and reached the hub via `list_models`) as a side effect. None of the other
five families (`_gemma`/`_llama`/`_mistral`/`_phi`/`_qwen`) do this; they only
register when `register_models()` asks them to.
Two consequences:
- Importing the registry populated MODEL_REGISTRY on its own (32 entries,
including 10 `deepseek-ai` original models that no other family leaks) and
did network I/O at import time.
- Because the import-time call set the `_IS_DEEPSEEK_*_REGISTERED` guards with
`include_original_model=True`, the later `register_models()` call (which uses
the default `include_original_model=False`) early-returned, so the
original-model set won permanently.
Remove the stray module-level call. The `if __name__ == "__main__"` block below
still registers with `include_original_model=True` for standalone use, so the
generator script is unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test(registry): make import-side-effect test pass on CPU-only runners
The new test spawned a fresh `python -c "import unsloth.registry"` that did
not inherit tests/conftest.py's GPU-free harness, so on no-accelerator CI
runners the child raised NotImplementedError from unsloth_zoo.device_type
before printing REGISTRY_SIZE. With check=True this surfaced only as an
opaque CalledProcessError, turning the "Repo tests (CPU)" job red even
though the registry fix is correct.
Import this directory's conftest inside the child first so it applies the
same device_type stubs and torch.cuda probe patches. Also use check=False
and include the child stdout/stderr in the assertion message so a future
import regression is legible instead of an opaque non-zero exit.
* test(registry): assert register_models() leaks no upstream originals
Adds a fresh-interpreter test that register_models() registers only
unsloth-org models (deepseek still present via the normal path) and never
leaks the upstream deepseek-ai originals that the import-time guard poisoning
used to leak (129 -> 139). Factors the conftest-harness subprocess runner
into a shared helper reused by both registry import tests.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
* Replace standalone Studio wording with Unsloth
Replace the single word Studio with Unsloth wherever it is used as
shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n
locales, workflow display names, comments and docstrings.
Kept unchanged: the full name Unsloth Studio, third party product
names (LM Studio, Visual Studio, Mac Studio), feature names
(Recipe Studio, Fine-tuning Studio and its translations), and all
identifiers such as env vars, commands, paths and filenames.
* Address review feedback on the Studio wording rename
Use "an" before Unsloth where the rename left the article as "a".
Restore the split brand where Unsloth and Studio render as two halves
of the full product name: the onboarding sidebar subtitle and the
IPv6 localhost warning. Scope two messages to the full name Unsloth
Studio where plain Unsloth was misleading: the AMD README bullet and
the CLI studio setup error.
* fix(dataprep): skip .jsonl lines that are valid JSON but not objects
`_read_file_by_format` json.loads each line and hands the result to
`_extract_text_from_json`, which assumes a dict:
for field in self._TEXT_FIELDS:
if field in data and isinstance(data[field], str):
A JSON line does not have to be an object -- `"context"`, `["text"]` and
`42` are all valid JSON. For those, `field in data` stops being a key
lookup and becomes a substring/membership test, so `data[field]` raises:
"context" -> "text" in "context" is True (substring!)
-> TypeError: string indices must be integers
["text", "foo"] -> TypeError: list indices must be integers
42 -> TypeError: argument of type 'int' is not iterable
The TypeError escapes past `except json.JSONDecodeError: continue`, so the
whole load dies on one odd line.
That except clause is also the tell: a *malformed* line is already skipped
gracefully. A *well-formed* line that happens not to be an object should be
too -- it carries no text either way. This makes the two agree.
Reachable from `unsloth-cli.py:253` (`--dataset foo.jsonl` auto-detect) and
`RawTextDataLoader` is exported from `unsloth/__init__.py`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Slim the non-object jsonl regression test and shorten the guard comment
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Stabilize Studio regression tests
Rebuild on current main. Restore the set-membership sidebar account-block matcher
(#6647, which fixed the same order-sensitive regex, was reverted on main, so the
guard is failing on main again) and keep the watchdog replacement-race fix, whose
blocked-watchdog stub now waits without a timeout so a superseded watchdog stays
alive until cleanup regardless of scheduler load.
* Tighten the blocked-watchdog stub comment
---------
Co-authored-by: Daniel Han <unslothshared@gmail.com>
* fix(tokenizer): check for tokenizer.model after saving it, not before
`fix_sentencepiece_tokenizer` creates its temporary directory, then returns
early unless that directory already contains a tokenizer.model:
if not os.path.exists(temporary_location):
os.makedirs(temporary_location) # fresh, empty
if not os.path.isfile(f"{temporary_location}/tokenizer.model"):
return new_tokenizer # always true
old_tokenizer.save_pretrained(temporary_location) # writes that file
The file only appears on the line after the check, so the guard is always
true and the body never runs. Nothing else writes that path either --
`convert_to_fast_tokenizer` saves into a per-name subdirectory, not
`{temporary_location}/tokenizer.model`.
Both call sites are in `get_chat_template` and are commented "Must fix the
sentence piece tokenizer since there's no tokenizer.model file!" -- the
guard defeats the exact intent the caller states. The effect is silent: the
caller still gets a working `new_tokenizer`, but the sentencepiece piece
rename is skipped, so the mapped token (e.g. the eos token remapped to
`<|im_end|>`) is missing from tokenizer.model and GGUF/llama.cpp exports
carry the old piece.
`check_if_sentencepiece_model` in save.py does the same probe in the right
order -- makedirs, save_pretrained, then isfile. Match it.
Tests are added under tests/saving/ next to the existing sentencepiece
coverage, and to the two Bucket-A lists in consolidated-tests-ci.yml, since
Repo tests (CPU) --ignores tests/saving and these need protobuf.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Clear stale tokenizer.model before the sentencepiece guard
The guard now runs after old_tokenizer.save_pretrained, but the default
temporary_location is a fixed reusable directory. A fast-only tokenizer writes
no tokenizer.model, so a stale file from an earlier sentencepiece call could
pass the guard and patch the wrong model (e.g. mixing models in one process,
like a long-running server). Remove any existing tokenizer.model first, and add
a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Empty the reusable sentencepiece scratch directory each call
The final AutoTokenizer.from_pretrained reloads the whole temporary_location, so
removing only a stale tokenizer.model still let other artifacts from a previous
tokenizer (added_tokens.json, chat template, etc.) leak into the reload when the
default reusable directory is used across models in one process. Recreate the
directory instead, and add a regression test for the leaked-artifact case.
* Clear only top-level scratch files, keep subdirectories
Recreating the whole reusable directory deleted the {name} subtree that
convert_to_fast_tokenizer stores a converted tokenizer's source vocab in, so
old_tokenizer.save_pretrained could not copy tokenizer.model and the guard
returned the tokenizer unpatched for those legacy converted tokenizers. Remove
only stale top-level files (all the final reload reads) and leave subdirectories
intact. Add a regression test for the converted-source subdirectory.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the current tokenizer's own source vocab when clearing
On a repeated get_chat_template(map_eos_token=True) call, the returned tokenizer's
vocab_file points back at the top-level tokenizer.model, and the cleanup deleted
that source before old_tokenizer.save_pretrained could re-emit it, so the guard
returned the tokenizer unpatched. Skip removing the old tokenizer's own source
vocab while still clearing stale files from a different tokenizer, and add a
regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use a per-call temporary directory for the sentencepiece fix
The scratch directory defaulted to a single shared path, so concurrent or repeated
get_chat_template(map_eos_token=True) calls could delete or overwrite each other's
tokenizer.model between save and reload (tripping the piece assertion or reloading
the wrong model), and stale files from an earlier tokenizer could leak into the
reload. Work in a unique per-call subdirectory instead: this isolates every call
without deleting anything the caller owns, and replaces the earlier per-file cleanup.
Tests updated to read the patched model from the reloaded directory and to cover
isolation and source-vocab preservation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Pass only the applied token mappings into the sentencepiece fix
get_chat_template mirrors token remaps into tokenizer.model via
fix_sentencepiece_tokenizer, but two caller paths passed a mapping that did not
match what they wrote to the fast tokenizer JSON, so once the sentencepiece patch
runs the model and JSON disagree:
- the mapped-token path skipped entries whose target already existed but still
passed the full mapping, renaming a piece the JSON never changed;
- the EOS-swap path swapped both tokens in the JSON but passed only one direction,
leaving two stop_word pieces and no old EOS piece.
Pass the applied mapping (and both swap directions) instead. Add regression tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten sentencepiece guard comments
* Add SPDX license identifier to sentencepiece guard test
* Reclaim the per-call sentencepiece scratch directory
The per-call tempfile.mkdtemp fixed the shared-directory race but never cleaned
up, so a long-running process leaked one scratch dir per call. The dir cannot be
deleted eagerly for sentencepiece tokenizers because the returned tokenizer's
vocab_file points into it (a later save_pretrained copies the patched
tokenizer.model from there). Reclaim it correctly instead: remove the dir right
away on the fast-only path (the returned tokenizer never references it), and
attach a weakref.finalize so the sentencepiece dir is removed once its tokenizer
is garbage collected. Add regression tests for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the scratch-dir reclaim comment
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Propagate fp8 block_size before the early return in get_lora_parameters_bias
get_lora_parameters_bias set the fp8 block_size on W/W_quant only after the
disable_adapters/merged early return, so on the merged or disabled path (merged
inference, DPO reference model) a block-fp8 weight lost its real block_size and
downstream fp8 kernels fell back to [128, 128]. The non-bias sibling
get_lora_parameters already sets block_size before its early return; move the
block so both behave the same.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard the fp8 block_size against a missing quant state
A decompressed compressed-tensors layer keeps quant_method == "fp8" while its
weight is back to bf16, so it has no quant state and get_lora_parameters_bias
must still return W_quant None for fast_linear_forward to fall back to a plain
matmul. Only attach block_size when a quant state was actually found.
* Guard the sibling get_lora_parameters fp8 block_size against a missing quant state
Mirror the get_lora_parameters_bias guard so a decompressed compressed-tensors
layer (quant_method fp8, bf16 weight, no quant state) does not raise
AttributeError on the fused-LoRA path. Add a CPU-only regression test.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* refactor(studio): move chat model picker into features/model-picker
Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.
* feat(model-picker): add per-model config persistence layer
Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).
* feat(picker): modular backend for chat-template validate + default fetch
New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).
* feat(model-picker): bind picker on-device list to shared hub inventory
Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.
Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.
* feat(model-picker): per-model config step inside the picker
Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.
* feat(chat): apply/persist per-model config through the load flow
handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.
* refactor(chat): remove per-model load config from the right sidebar
The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).
* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section
The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.
* chore(chat): remove dead per-model-config setters + modelControlsDisabled
After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.
* fix(chat): config-step Load actually loads (ignore Load-on-selection)
Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.
* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow
The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.
* feat(model-picker): default chat template from GGUF + thread variant through config flow
Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.
Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.
* feat(model-picker): read safetensors chat template + hide editor where it has no effect
Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.
Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.
* fix(model-picker): set legacy-migration flag only after the write succeeds
Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* MVP model picker fixes
* MVP picker config fix
* MVP safetensors config
* MVP max seq config
* MVP max seq fix
* Fix static max tokens cap ignoring model context
* Fix picker GGUF scan parity
* fix(studio): harden model picker config loading
Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.
* Fix model picker config flow
* Fix model picker config loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid recursive per-model config migration reads
* Apply the displayed context length when loading a GGUF
* Fix template validation, cached template lookup, and failed load rollback
- Validate chat templates with the loopcontrols extension so templates
that use break or continue tags pass the picker validator, matching the
inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
an arbitrary iterdir order, so an older cached revision no longer prefills
a stale template.
- Capture the runtime per-model config before a load and reapply it when the
load fails, so a failed switch leaves the active model context, KV cache,
template, and speculative settings as they were.
* Make chat template view only for safetensors models
Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.
* Fix model picker config edge cases
- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker
* Keep saved GGUF context above the fallback ceiling
* Show the model config in the run settings sidebar
* Fix model config sidebar reset and context slider
- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value
* Fix model picker config and download regressions
- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel
* Fix model picker config and cached download sorting
- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting
* Fix model picker per-model config edge cases
Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.
Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.
Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.
* Fix GGUF context auto-fit and gated model config token
Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.
Send the HF token as a query param so gated safetensors models resolve their max position embeddings.
Derive model default state during render to drop the set-state-in-effect calls.
* Fix native GGUF context ceiling and guard picker template reads
Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.
* Fix model picker lint boundaries
* Fix model picker review findings
Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.
Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.
Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.
* Preserve GGUF context on active reload
* Fix model picker per-model config regressions
- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely
* Fix stale model auto load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker numeric input sizing and constraints
Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.
* Fix picker CI tests and harden chat template resolution for PR #6647
- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Protect future-schema per-model configs from deletion for PR #6647
savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.
* Protect future-schema per-model configs from quota eviction for PR #6647
The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.
* Fix GGUF context persistence, compare context, and rollback settings for PR #6647
Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.
shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.
use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.
* Preserve native path token when reloading the active model for PR #6647
handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.
* Make picker template validation resilient and accept HF generation tags for PR #6647
Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.
* Honor remembered compare config and parse processor chat_template.json for PR #6647
* Fix failed-load rollback context and processor template map fallback for PR #6647
* Restore speculative decoding config on failed-switch rollback
When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.
* Reset max sequence length when a model has no saved config
applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.
* Surface a message when a variant update cannot start
startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.
* Keep per-model speculative choices out of the global default
A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.
* Seed non-active model settings from the app default max length
The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.
* Prefer sidecar tokenizer chat template over the GGUF copy for variants
_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.
* Keep per-model speculative choices load-local in autoload and compare
The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context
* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it
* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports
The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.
* Studio: cache a null default chat template so the viewer stops re-fetching it
A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.
* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context
A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.
* Studio: prompt to re-select a local model file when its lease expired before reload
A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.
* Fix descender-clipping test to tolerate sidebar layout utilities
The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.
* Harden picker chat-template resolution
Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.
Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.
* Guard per-model config against future-schema and lossy migration
Two forward-compatibility gaps in the versioned per-model config store:
- The load/apply path returned and normalized a stored record without checking
its schema version, so a record written by a newer client was reinterpreted
under the current schema and applied to a live model load, even though save,
delete and eviction all refuse to touch future-schema records. Reject
future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
the entries it had just migrated and set the completion flag unconditionally.
When storage was already full of future-schema records (which are unevictable
by an older client), the migrated entries were the only evictable ones and
could be dropped while migration was still marked complete. Protect the
migrated keys during eviction and only mark migration complete when they
survive, so it retries once space frees up.
* Discard chat-template validation results after the dialog closes
Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.
* Record native lease expiry when loading a picked GGUF from the chip
The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer sidecar template for a directly selected local GGUF file
A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.
* Resolve cached chat template per revision, newest first
The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.
* Preserve autoload transport conflicts and surface background busy downloads
- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
instead of clearing it. Clearing it re-keyed the download surface and its
cleanup cancelled the conflict the toast tells the user to resolve, so the
Hub resume affordance was gone the moment it appeared. Return early on
conflict, mirroring the started branch, so resolving it from the Hub still
auto-loads on completion.
- The background-download branch handled started and conflict but silently
dropped a busy outcome, leaving the user with no feedback when a peer variant
of the same repo was already downloading. Surface the same busy toast the
autoload path uses.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* fix(save): unsloth_push_to_hub_gguf(save_method="lora") raises NameError
unsloth_push_to_hub_gguf reads is_main_process at save.py:3181 but never
declares it. Its twin unsloth_save_pretrained_gguf declares it (2783) and
uses it the same way (2839) -- the LoRA branch was copied between the twins,
the parameter it depends on was not. There is no module-level global, so the
name resolves as a global load and the branch raises NameError 100% of the
time.
save_pretrained_gguf(save_method="lora", push_to_hub=True) raises a
ValueError that tells users to "use .push_to_hub_gguf(save_method='lora')
instead" -- the documented escape hatch is the broken call.
Add is_main_process to the signature, positioned as in the twin, and forward
it to unsloth_save_pretrained_gguf on the merged path so the parameter is not
silently ignored there. Default stays True, so nothing changes for existing
callers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(save): preserve GGUF push compatibility
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* fix(dataprep): smart_chunk_text single-chunk path leaks internal tensor type when eos_token_id is None
RawTextDataLoader.smart_chunk_text()'s single-chunk branch only
converts `tokens` to a plain Python list inside the
`if eos_token_id is not None:` guard. When a tokenizer has no
eos_token_id configured, that conversion is skipped entirely and the
function returns whatever internal tensor-like object came out of
the tokenizer normalization step (e.g. a torch.Tensor) as
"input_ids", instead of a list of ints.
The sibling multi-chunk branch a few lines below does the conversion
unconditionally, before checking eos_token_id -- the two branches of
the same method disagree on output type depending purely on whether
the tokenizer has an EOS token. Downstream, create_causal_dataset()
does `labels = [list(ids) for ids in input_ids]`; list()'ing a
tensor produces a list of 0-d tensor elements rather than plain
ints, inconsistent with every multi-chunk sample and liable to break
type inference in Dataset.from_dict()/downstream collation.
Fix: move the list conversion out of the eos_token_id guard,
matching the multi-chunk branch's existing pattern.
Added test_smart_chunk_text_single_chunk_no_eos_returns_plain_list
to tests/test_raw_text.py, confirmed red against unfixed code
(assertion failure: input_ids was a MockTensor, not a list) and
green after the fix. Full tests/test_raw_text.py (both test
functions) passes. ruff check + the repo's ruff-format-with-kwargs
script: clean.
Note: tests/test_raw_text.py does not appear to be wired into any
.github/workflows/*.yml CI job (a pre-existing repo characteristic,
not something introduced by this change) -- verified locally via
`python3 tests/test_raw_text.py`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherl <61019402+Etherll@users.noreply.github.com>
* Fix Studio toast close-button positioning
* Use UTF-8 for locale regression test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden toast close-button positioning
* Limit language menu height
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Single-pass GGUF export for direct outtypes + parallel multi-quant
save_to_gguf defaulted first_conversion to model_dtype before the block
that picks the optimal base conversion, leaving that block dead since it
landed (#3356). Every default export (fast_quantized -> q8_0) therefore
ran two passes: convert HF -> 16-bit GGUF, then llama-quantize -> q8_0,
writing a 2x-size intermediate that the cleanup step deletes again.
- Route single-output exports whose type convert_hf_to_gguf.py emits
directly (f32/f16/bf16/q8_0) through one conversion pass with no
16-bit intermediate. Measured on Qwen2.5-0.5B-Instruct (8-core CPU):
bytes written 1525 MB -> 531 MB (2.9x less), peak extra disk 994 MB
-> 0, wall time neutral on local NVMe (14.8s vs 15.4s). The dequantized
q8_0 tensors are bit-identical to the two-pass output (max diff 0 over
all 290 tensors, same quant-type table). On disk-capped runtimes
(Kaggle 20 GB, Colab) the removed intermediate is the difference
between an export that fits and one that dies - see the Kaggle error
text this file already carries. imatrix runs keep the two-pass route
since only llama-quantize can apply one; explicit first_conversion is
still honored.
- Run independent llama-quantize passes two at a time when several
quant methods are requested (thread budget split between workers,
outputs byte-identical, order preserved). Measured 1.38x wall-clock
on q4_k_m+q5_k_m+q6_k. Sequential under UNSLOTH_ENABLE_LOGGING=1 to
keep subprocess logs readable; kill switch
UNSLOTH_PARALLEL_GGUF_QUANTS=0. Duplicate methods now quantize once.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard parallel GGUF quant on Kaggle and make multi-quant failures atomic
Each llama-quantize pass loads the whole model into RAM, so running two at once on Kaggle can OOM a host that succeeded sequentially; skip the parallel path there. On a failed multi-quant export, stop launching queued passes and remove orphaned quant outputs so a failure leaves no partial GGUFs behind, keeping the 16-bit base for retry. Also accept 0/false/no/off/empty for UNSLOTH_PARALLEL_GGUF_QUANTS so a well-meant 'false' actually disables parallelism, and add tests/saving/test_gguf_single_pass_export.py to the CI saving bucket so the new tests run.
* Preserve pre-existing outputs for canceled quant passes on failure
The parallel cleanup unlinked every requested output name, so a failed rerun could delete a valid model.<METHOD>.gguf left by an earlier successful export for a method whose pass was canceled and never ran this session. Skip canceled futures and only remove outputs from passes that actually executed.
* Gate parallel GGUF quant on available memory and preserve prior outputs
Skip the two-worker path when RAM cannot hold two full-model quantizations at once (and on Colab as well as Kaggle), so a multi-quant export that fit sequentially no longer OOMs. On failure, remove only outputs this run newly created, tracked against a pre-launch snapshot, so a rerun into an existing _gguf directory never deletes a valid artifact from an earlier export.
---------
Co-authored-by: djs <dschroers2@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
RawTextDataLoader.smart_chunk_text takes chunk_size and stride as its own
arguments, so a direct call with stride >= chunk_size bypasses the
constructor validation. In that case `start_idx += chunk_size - stride` is
non-positive, so start_idx never advances past the first window and the
chunking loop never terminates (hangs).
Re-add the chunk_size/stride guard at the top of smart_chunk_text so direct
callers fail fast with a clear ValueError. The constructor keeps its own
guard for the internal callers (defense in depth). Add a regression test
that calls smart_chunk_text directly with stride == chunk_size and
stride > chunk_size and asserts it raises instead of hanging.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
* Unsloth: appearance palettes, customization options, and control restyle
Adds Standard, Classic, and Minimal color palettes to Appearance settings,
each adapting to light and dark mode. Classic is a neutral enterprise look
that reserves its blue accent for toggles, badges, and focus rings; Minimal
is strictly black, grey, and white.
Adds customization options scoped to the active mode: accent, background,
and foreground colors with an in-app color picker, UI and code fonts with a
searchable dropdown covering bundled, device, and imported fonts, font file
import, UI and code font sizes, contrast, pointer cursors, reduce motion,
font smoothing, and translucent sidebar. Settings persist through the
personalization API with backend validation and sync across devices.
Restyles core controls for a cleaner, flatter look in both modes: bordered
white input fields, fully rounded pills for single-row controls, no drop
shadows, simple straight-line chevrons replacing all rounded arrow icons,
and consistent hover tones in dropdown menus. Popovers now portal into the
open dialog so their lists scroll correctly inside modal dialogs.
Moves Language into General settings and Chat defaults into the Chat tab
above the Canvas section.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Unsloth: appearance follow-ups, font options, and settings search
Neutralizes focus and selection rings across all palettes so highlighted
elements, including typing boxes and the selected palette card, never take
the accent color. The custom accent no longer recolors rings.
Restyles the color controls as filled pills showing the hex value inside,
with text and border contrast picked from the color's luminance. Menus in
popovers now match the app's dropdown menus: rounded-lg corners, tighter
padding, accent hover rows, and a bordered search field. Popovers inside
modal dialogs are modal so their lists scroll with the wheel. Outline
buttons share the same dark fills as dropdown triggers.
Adds heading and chat font options next to the UI and code fonts, each
using the searchable font dropdown and persisting through the
personalization API. Removes the translucent sidebar option end to end.
Adds settings search: a search field at the top of the settings sidebar
that filters setting names across every tab, grouped by tab with icons,
and jumps to the tab on click.
* Unsloth: use the shared accent token for dark hover fills
The settings dialog nav, its close button, the model selector, and the
project switcher hovered with hardcoded blue tinted greys (#3a3d43,
#2d2e32) in dark mode while every menu and sidebar uses --accent. All
hover and active pill fills now use the accent token so dark hovers are
the same everywhere and adapt to the active palette.
* Unsloth: settings search polish and jump to matched setting
Widens the settings dialog to 880px and the sidebar column to 248px so
the search field has more room. The search pill aligns with the left
start of the Settings title, gets more spacing above and below, and its
icon and placeholder sit slightly further left.
Search results now jump to the exact setting: rows and sections expose
their label as a data attribute, and picking a result opens the tab,
scrolls the matched row into view, and flashes it briefly.
* Unsloth: settings search bar spans the full nav pill width
The search field now starts and ends at the same edges as the nav hover
pills instead of being inset to the title text.
* Unsloth: address review findings on motion, sync, and font limits
Reduce motion Off now opts back out of the OS reduced-motion preference
for CSS animations via a force-motion class that the media rules skip,
and forcing reduce motion On keeps the loader exceptions (spinners,
loading dots, progress bars) animating.
When the color scheme follows the system, the resolved mode is now part
of the theme store snapshot, so an OS scheme flip re-renders consumers
and reapplies per-mode custom colors instead of leaving stale inline
variables from the previous mode.
Imported fonts get an aggregate size cap (4.4M characters) on both the
frontend sanitizer and the backend model so the persisted store always
fits browser localStorage quotas, with a clear error toast when an
import would exceed it. Backend validation also tightens imported font
names (rejects CSS delimiter characters) and requires strict base64
font data URLs, matching the frontend patterns.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Unsloth: profile toggle to hide the sloth in the chat greeting
Adds a Show greeting sloth switch to Settings > Profile. The chat welcome
hides the mascot when it is off. The preference persists locally and
through the personalization API, with backend validation and tests, and
the row is reachable from settings search in all four locales.
* Unsloth: control restyle, dropdown scrolling, and palette consistency
Settings sidebar puts search on top with the tab list under a small
Settings label. Combobox popups scroll with the wheel inside dialogs by
falling back to manual list scrolling while a dialog scroll lock is
active, and the local model selector popover became modal for the same
reason. Number inputs swap native spinners for a shared grey stepper
that clamps to min, max, and step. Run settings fields in light mode use
the same white fill and border as the settings dialog. Selection and
focus rings derive from each palette's border color instead of near
black, hover borders soften the same way, the Classic sidebar stays
white like Standard, decorative greens follow the palette accent, and
meaning-carrying marks like the hub verified badge keep the brand green
in every palette.
* Unsloth: palette card selection keyed off the palette attribute
Switching palettes restyles the whole page the moment data-palette lands
on the html element, but the React re-render that moves the selection
classes arrives later, so the ring and check briefly stayed on the
previous card with the new palette's colors. The active ring and check
now key off html[data-palette] in CSS, so they swap in the same style
pass that swaps the tokens. Also adds breathing room around the settings
search bar and under the Settings label, shortens the greeting sloth
description, and renames the avatar section to Or pick a sloth profile
picture in all locales.
* Unsloth: restore neutral rings, drop the palette check, sidebar spacing
Puts the ring tokens back to their fixed per palette values and removes
the hover border darkening, undoing the derived border experiment. The
selected palette card no longer shows a check since the ring already
marks it. The settings sidebar search bar, nav pills, and search results
get a little side padding, and the Settings label lines up with the pill
text.
* Unsloth: indicator restyle, sidebar menu customization, edge fade toggle
- Derive focus and selection rings from the border color so indicators
stay 1px and adapt to every theme and palette
- Suppress mouse focus rings except on pressed controls to remove the
selection flash on the avatar and palette pickers
- Defer settings panel rendering so the active nav pill updates instantly
- Customizable sidebar user menu with drag to reorder and shortcuts to
the settings tabs
- Grey hover for the standard light palette instead of green
- Borderless controls in dark mode with fill based focus states
- Profile picture: no picture option, pencil edit icon, atomic selection
- Font dropdowns: narrower triggers and the resolved default shown as
Inter Variable (Default)
- System prompt border darkens on focus
- New appearance setting to swap edge fades for thin divider lines
- Move the theme bootstrap to an external script to satisfy CSP
* Unsloth: harden theme boot and Firefox scroll container focus
- Guard the theme and palette storage reads separately so a blocked
localStorage (private browsing) still resolves a mode from the OS
preference instead of skipping the boot entirely
- Firefox makes scrollable containers keyboard focusable and drew its
3px UA outline on them; swap it for the app's soft 1px indicator
* Unsloth: make the UI and code font settings reach the font utilities
The theme block declared the sans and mono stacks as literals, so
Tailwind inlined them into every font-sans and font-mono utility at
build time and the runtime overrides from Settings > Appearance never
applied. Reference the :root tokens instead, matching how the color
tokens already work.
* Unsloth: in-dropdown font upload, accent meters and avatar, naming cleanup
- Move font importing into each font dropdown: Upload and Select folder
sit side by side under the list, imported fonts get an inline remove,
and the standalone Import font row is gone
- Uploads reuse fonts the user already has (bundled, imported, or
installed, matched by file name with style suffixes stripped) instead
of embedding a duplicate copy; only new fonts are embedded
- Folder scan lists font files from a picked folder in every dropdown
for the session; picking one imports it through the same path
- Fallback avatar uses the control accent with a readable foreground
instead of the neutral primary that rendered black outside standard
- Monitor bars, progress defaults, sliders, and usage meters use the
control accent; warning and danger tiers stay amber and red
- User facing strings that called the app just Studio now say Unsloth
in all four locales, keeping Unsloth Studio and LM Studio intact
* Unsloth: left align the font upload actions and divide them
Upload and Select folder now read from the left like the list items,
with a short vertical rule between the two.
* Unsloth: keep sliders neutral and the chat greeting on Hellix
- Sliders are controls, not meters, so their fill goes back to the
neutral primary instead of the palette accent
- The base h1 rule reads --font-heading with !important and the chat
thread root resets that variable to the sans stack, which pulled the
greeting off Hellix; restore the stack on the greeting element
* Unsloth: move the None avatar cell last and keep footer actions on one line
- None sits after the sloth pictures instead of leading the grid
- Upload shrinks to its label so Select folder no longer wraps
* Unsloth: size the folder action to its label
Both footer actions now hug their content so the hover pill does not
stretch across the leftover row width.
* Unsloth: separators only between unrelated settings clusters
Rows inside a titled section are related, so the per row divide-y is
gone from SettingsSection. A SettingsGroupDivider marks the two real
boundaries in the theme section (colors to fonts, fonts to contrast)
and the Clear all chats row gets its destructive border back now that
divide-y no longer draws one for it.
* Unsloth: balance the two font upload actions
Both actions share the footer row evenly again; nowrap keeps Select
folder on one line at the narrower width.
* Unsloth: drop the theme section dividers and split the chat menu groups
The colors, fonts, and contrast rows read fine without rules, and the
chat menu gains its one real boundary between the pin toggles and the
disclaimer rows.
* Unsloth: normalize oversized sidebar menus and reject newline font data URLs
Two backend validation fixes in PersonalizationCustomization:
- sidebarMenu refused any list longer than the number of distinct ids
because Field(max_length) is enforced before the dedupe validator runs.
A stale or duplicated payload that would normalize to one entry per id
was rejected outright, defeating the normalizer that exists for exactly
that case. Cap the incoming list at a generous multiple so it reaches
the validator; a pathologically long list is still refused.
- The imported font dataUrl validator used re.match on a pattern ending
in $, which also matches just before a trailing newline, so
"data:font/woff2;base64,AAAA\n" passed even though the frontend JS
pattern rejects it. Use re.fullmatch for parity.
Adds covering tests for both.
* Unsloth: preview fonts in their own typeface and slim the color pills
- Every font dropdown entry, the default item, and the closed trigger
render in the font they name, falling back to the UI stack for
families the browser cannot resolve
- Color swatch pills drop from 36px to 28px so they sit closer to the
row label height
* Unsloth: drop the font row and theme section descriptions
The labels carry the meaning on their own; the mode switching note in
particular read long and confusing.
* Unsloth: let the chat greeting follow the heading font setting
The greeting stays on Hellix by default but adopts a chosen heading
font through a --custom-heading-font variable the applier sets only
while an override exists, so the thread root's sans reset for chat
prose no longer hides the user's pick from the greeting.
* Unsloth: divide the theme section clusters and align the color pill height
Separators return between colors and fonts and between fonts and
contrast, and the color pills share the 32px height of the font
dropdown triggers.
* Unsloth: color pills at half the dropdown width
Fixed w-24 against the w-48 font triggers, with tighter padding so the
hex value still fits.
* Studio: update dep-removal test after next-themes was replaced
The frontend no longer declares next-themes or imports it in src (it was
replaced by the custom theme store and boot script), so the checker now
reports its removal as a safe no-op. The C1 and C8 fixtures in
test_frontend_dep_removal.py still asserted next-themes was a used
dependency, which fails the studio frontend CI dependency-removal safety
check. Update C1 to expect a no-op PASS and drop next-themes from the C8
expected failures so the suite matches the checker's correct output.
* Studio: remove unused ageLabel and exportCollectionJsonl helpers
* Studio: fix blocked-storage theme desync, search jump race, font validation
- theme-store.ts: keep an in-memory currentTheme/currentPalette so a selected
value survives when localStorage is blocked (private browsing). The snapshots
previously re-read empty storage and reverted React state to the default while
the DOM already changed. The matchMedia handler no longer re-reads storage, so
it cannot clobber the in-memory choice; cross-tab storage events still adopt.
- settings-dialog.tsx: the search jump waited a single fixed 60ms for the
deferred tab panel to render, then silently missed under render lag. Retry
across animation frames until the target row exists, then scroll and flash.
- settings.py: apply the font-name character check to the four selected-font
fields (uiFont/headingFont/chatFont/codeFont), and forbid backslash, comma,
slash and control characters so a name cannot escape the quoted CSS
font-family or smuggle extra fallbacks. Adds covering tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix appearance customization edge cases for PR #7077
- Reset all local preferences now also clears palette and appearance customization
- Number input wrapper keeps full width so fields fill their flex/grid cell, and the stepper stays pinned to the field edge
- Number stepper snaps to the min anchored step grid like the native spinner instead of leaving a step-invalid value
- Code font now applies to chat code fences and inline code via a dedicated token
- Reduce motion (on/off) is honored by onboarding/tour confetti and the theme toggle view transition
- Re-importing a font under the same name with new bytes now swaps the FontFace
- Keep local customization when a synced record predates the customization field, and re-push it
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align client font name sanitization with server validation for PR #7077
sanitizeFont now strips the same characters the backend _FONT_NAME_FORBIDDEN
rejects (backslash, slash, comma, backtick) plus control chars, so a locally
chosen font name can no longer pass the client but fail the personalization PUT
and silently stall appearance sync.
* Address follow-up review items for PR #7077
- Number input wrapper carries React Flow interaction classes (nodrag/nopan/nowheel) so clicking the stepper arrows increments instead of dragging the node
- Preserve local palette and greeting-sloth toggle when the synced record predates those fields, and re-push them, mirroring the customization handling (new paletteSaved and greetingSlothSaved response flags)
- Add settings-search scroll targets (data-settings-label) for the Profile title, description, display name, nickname, and avatar shape rows
* Preserve absent personalization fields on PUT for PR #7077
A stale client that omits palette or customization previously had those
defaults materialized by model_dump() and persisted, which flipped
paletteSaved/customizationSaved to true and defeated the legacy detection.
The PUT now dumps only the request's set fields and merges them onto the
stored record, so omitted fields keep whatever was already stored.
* Persist theme and palette via a fixed allow-list for PR #7077
The theme/palette values reach setTheme/setPalette from the authenticated
personalization sync, which made the CodeQL clear-text-storage query treat
writing them to localStorage as storing sensitive data. Store a re-derived
literal from a constant map instead, so a plain UI preference is not tracked
as sensitive; behavior is unchanged.
* Harden imported-font handling for PR #7077
- syncImportedFonts: a rejected FontFace.load() only clears the registry entry
if it still points at that face, so a same-name re-import while the old load
was pending is no longer untracked/leaked.
- Cap imported-font names to the backend length (100) so an over-long name can
no longer pass the client but fail the personalization PUT and stall sync.
- Add a backend test that a stale PUT preserves an existing stored palette and
customization (not just that absent fields stay absent).
* Return the merged personalization record from PUT
The PUT /personalization handler returned the request payload, which
Pydantic had already filled with defaults for any field the client
omitted. A partial or stale write (for example a client sending only
theme) therefore got back a response that contradicted both storage and
the next GET: preserved fields like palette and the custom font showed
their defaults instead of the stored values.
Return model_validate(merged) so the response mirrors what was stored.
The stored record is still the full merged dict, so legacy fields the
model does not know about are preserved as before.
* Fix small UI and keyboard-focus defects in appearance settings
- Settings search now scrolls to the result within its destination tab
instead of a same-named row in the previously rendered deferred tab
(for example "Storage" and "Models folder" appear in both General and
Resources).
- The reduce-motion segmented control honors its own Off/On/System choice
by reading useReducedMotionConfig instead of the OS-only useReducedMotion.
- The color picker saturation/value area is operable by keyboard, so the
role="slider" surface responds to the arrow keys it advertises.
- Profile avatars and palette cards show a visible keyboard focus ring
again.
- Guard the persisted appearance-customization write so a blocked or full
localStorage does not throw out of a store action, matching the theme
store.
- Import the appearance store symbols from the settings feature barrel.
* Tighten appearance fix comments
---------
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: resolve llama.cpp prebuilts via the release-assets CDN to avoid GitHub API rate limits
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: resolve manifest-named prebuilt assets on the download-host fast path
Add tag-pinned CDN URLs for any manifest artifact whose hash is keyed under an
upstream-tag alias in the checksum asset, so the fast path resolves the same
assets the API path does. Cover the resolve body directly (only download_bytes
stubbed) and soften the doc's validation-equivalence wording.
* Studio: pin llama.cpp fast path to the releases/latest redirect tag
Derive the authoritative latest tag from GitHub's /releases/latest redirect
target instead of trusting the checksum asset's self-reported release_tag, so the
existing release_tag cross-check in parse_approved_release_checksums is a real
check again: a stale or mis-tagged checksum asset now falls back to the API. Pin
every fast-path URL to that tag. Fall back to the API on a manifest 404 as well,
since an in-progress release can publish the checksum asset before the manifest,
matching the sha256 404 handling. Document the releases/latest (created_at /
make_latest) versus published_at ordering divergence and why it is an accepted,
mitigated tradeoff.
* Studio: drop the llama.cpp prebuilt-resolution doc
Remove studio/docs/llama-cpp-prebuilt-resolution.md and the docstring pointer to
it; the resolution rationale (the created_at/make_latest vs published_at ordering
nuance) stays inline in _download_host_latest_release_tag.
* Studio: tighten llama.cpp download-host fast-path comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Handle linked instruction files in Bash cleanup
* Limit instruction cleanup to managed dependencies
* Make Bash cleanup test portable
* Run junction cleanup regression on Windows
* Keep instruction cleanup CI focused
* Studio: remove AGENTS.md from install artifacts
* Studio: prune CLAUDE.md from install artifacts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Studio instruction cleanup edge cases
* Trim Studio cleanup comments
* Make Studio cleanup safe on PowerShell 5.1
* Fix Studio cleanup ownership boundaries
* Simplify Windows link detection
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix SyntheticDataKit.chunk_data emitting chunks over max_tokens
The multi-chunk path built boundaries from np.linspace(..., n_chunks), but
pairing boundaries[:-1] with boundaries[1:] turns N points into N-1 ranges,
so it produced one fewer, oversized chunk: every chunk exceeded max_tokens
and a document just over the threshold came back as a single unsplit chunk.
Use n_chunks + 1 points so exactly n_chunks ranges are emitted, each within
max_tokens.
Also base n_chunks on the non-overlapped span: consecutive chunks overlap by
overlap, so covering length needs ceil((length - overlap) / stride) chunks, not
ceil(length / stride). The looser count over-counted by one just past a stride
multiple (a 673-token doc became 3 chunks of ~267 instead of 2 of ~369),
emitting an extra redundant chunk. Coverage and overlap are unchanged and every
chunk still stays within max_tokens.
* Condense chunk_data comments and clarify over-split test for PR #7073
---------
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Restore dropped FP8 weight_scale_inv tensors on load
Some block-scale FP8 checkpoints (for example Qwen3.6-27B-FP8, issue #6200) load
with transformers leaving an mlp.gate_proj as a plain bf16 Linear instead of an
fp8 module. Its raw quantized values are read into the bf16 weight and the
weight_scale_inv is dropped as an unexpected key, so the weight is used un-scaled
and the base model is garbage (perplexity around 2 million).
After load, for every checkpoint weight_scale_inv whose live weight is not fp8,
dequantize the orphaned weight in place using the block scale from the checkpoint
index. Modules that were converted correctly keep an fp8 weight and are skipped,
so healthy checkpoints and single-file checkpoints are a no-op.
Verified on Qwen3.6-27B-FP8: 64 gate_proj scales restored, perplexity 2028902 to
8.9. No-op on Qwen3-8B-FP8 (all scales already live).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden FP8 weight_scale_inv restore from review
- Skip restore when the model has no fp8 weights, so an intentionally
dequantized load (load_in_16bit) is never re-scaled and corrupted.
- Thread revision, subfolder and cache_dir through the index and shard
downloads so scales come from the same snapshot as the weights.
- Cover unsharded single-file model.safetensors checkpoints (no index).
- Handle transposed block-scale layouts and skip on a true grid mismatch
instead of applying a wrong scale.
- Match text-only VLM loads where the language_model prefix was stripped.
- Restore on the FastLanguageModel text path too, not only vision.
- Handle a scalar weight_block_size; per-tensor error handling so one bad
tensor cannot abort the rest or hide a partial mutation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address second review round on FP8 scale restore
- Bound peak memory: dequantize block views in place with the fp32 scale
broadcast instead of materializing a full expanded scale and fp32 copy,
so a near-VRAM-limit load is not pushed into OOM by the repair.
- Restore on the sequence-classification load path too.
- Cover more VLM key remappings (language_model.model.* to
model.language_model.*) when matching modules.
- Skip the restore for variant loads (variant=...) rather than risk
applying default-checkpoint scales to variant weights.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align FP8 scale restore revision with the loaded weights and warn on disk-offloaded layers
In llama.py the CausalLM/SequenceClassification weight loads resolve model_name on its
default branch (revision is not forwarded there), so read the dropped weight_scale_inv
tensors from the same default branch instead of the requested revision, avoiding rescaling
default-branch weights with scales from another revision.
In loader_utils.py a disk-offloaded layer keeps its weight on the meta device until the
offload hook materializes it, so the scale cannot be applied in place. Skip such layers
explicitly and print a warning rather than silently leaving them unscaled.
* Tighten comments in the FP8 scale restore path
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
parse_direct_linux_release_bundle and direct_linux_release_plan are no
longer reached by any live code path. Fork Linux installs resolve through
_fork_manifest_release_plans -> _linux_published_attempts, and the upstream
(ggml-org) path uses direct_upstream_release_plan. The dead parser also
called _resolve_linux_bundle_profile, which no longer exists, so its CUDA
branch would raise NameError if ever executed.
Drop both functions and the obsolete TestDirectLinuxNvidiaCpuGate; its live
equivalent TestLinuxPublishedAttemptsNvidiaCpuGate already covers the
NVIDIA no-silent-CPU behaviour.
* Keep native RoPE scaling when extending context; carry rope_theta for linear
When max_seq_length exceeds a model's native window, the loader overwrote the
model's rope_scaling with linear scaling. For models that already ship a scaled
RoPE (llama3/yarn/longrope) that is far worse for long context, and on
transformers v5 the linear dict omitted rope_theta (v5 keeps it under
rope_parameters), so the rotary base fell back to 10000 and broke past ~8K tokens.
Keep the native scaling and just widen the window; only synthesize linear for
plain-RoPE models, and carry rope_theta so v5 keeps the real base.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Only preserve native llama3 when extending context; keep linear fallback otherwise
The patched attention constructor (patch_llama_rope_scaling) rebuilds only linear,
llama3 and longrope and its longrope branch reads a top-level
original_max_position_embeddings, so preserving yarn or a nested-only longrope config
would raise during construction on transformers <= 4.47.1. Keep only llama3 native;
yarn/longrope/other types fall back to the linear override, still carrying rope_theta.
* Correct long-context extension comment to match llama3-only preservation
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix fast_gemv crash on compressed-tensors FP8 models
Loading a compressed-tensors FP8 checkpoint (for example
unsloth/Llama-3.2-1B-Instruct-FP8-Block) with fast_inference=False and
running a forward crashed with 'Parameter object has no attribute absmax'
inside fast_gemv.
A compressed-tensors CompressedLinear exposes an already dequantized bf16
weight at forward time while keeping a weight_scale Parameter. The quant
state resolution in get_lora_parameters/get_lora_parameters_bias fell back
to that weight_scale, so a bf16 weight was routed into the bitsandbytes
fast_gemv/fast_dequantize path, which expects a bitsandbytes QuantState
with an absmax attribute.
Only fall back to weight_scale_inv/weight_scale when the weight is still
fp8. A decompressed bf16 weight then resolves to no quant state and flows
through the normal bf16 path, which already handles bias and the LoRA
backward. Real fp8 and bitsandbytes 4bit weights are unchanged.
* Skip the fast_gemv dispatch test before importing unsloth when bitsandbytes is absent
* Fix Windows installer torch index override
* Clear inherited uv index env vars for pinned installs in studio/setup.ps1 (#6898)
* Harden setup.ps1 index-var clearing to truly remove vars (#6898)
* Apply UV_DEFAULT_INDEX torch index fix to Linux/Mac install.sh (#6898)
* Neutralize all uv index env vars for pinned torch installs (#6898)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix FastSentenceTransformer Qwen embedding preprocessing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Document Transformer.load embedding modality fix for #6881
* Harden #6881 fix and add forwards/backwards-compatible regression tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fall back to Transformer constructor on legacy sentence-transformers without Hub-capable load
* Mirror legacy sentence-transformers fallback in embedding-parity tripwire test
* Tighten #6881 comments and docstrings
* Skip embedding-parity test on CPU-only runners since FastSentenceTransformer requires CUDA
* Honor the transformer module's saved subfolder when loading
modules.json records a path for the Transformer module (root for
decoder embedders like Qwen3-Embedding, 0_Transformer for the classic
layout). Pooling/Normalize already load from their saved path; thread the
same path into Transformer.load as subfolder so config and tokenizer
resolve like stock ST. stays a no-op, so single-module models are
unchanged.
* Make embedding-parity test bf16-aware
fp16 overflows to NaN on bf16-native embedders such as EmbeddingGemma
(Gemma3), producing a false parity failure. Prefer bf16 when the GPU
supports it so the tripwire can guard the full documented embedding
matrix (Qwen3-Embedding, EmbeddingGemma, BGE-M3, all-MiniLM, GTE-ModernBERT),
not just fp16-safe models.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Retry the Studio UI shutdown re-login on transient goto timeout
The Chat UI Playwright smoke intermittently failed at the pre-shutdown
re-login: page.goto('/login') can hit a 60s TimeoutError on a slow runner
even while the server is healthy, and the surrounding except only tolerated
ERR_ABORTED / interrupted-navigation, so a plain timeout hard-failed the job.
Wrap the re-login goto/wait/fill/submit in the same 3-attempt retry the
change-password step already uses (recover_or_replace_page between tries,
per-attempt fail screenshots, wait_for_health pre-gate). The composer wait
stays outside the loop so a retry never re-navigates after login has set
tokens (which would redirect to /chat via the guest guard); it remains the
authoritative confirmation, so a genuinely broken login still fails.
* Catch transient login-request failures and preserve error listeners on recovery
Wait on the /api/auth/login POST inside the retry (via click_and_wait_for_response)
so a transient 4xx/5xx is retried in-loop instead of surfacing only at the
out-of-loop composer wait, matching the change-password step. When
recover_or_replace_page swaps in a fresh page, re-attach the pageerror/console
listeners so error tracking survives the replacement.
* Stabilize floating monitor drag
* Restore floating monitor exit animation
* Harden Windows Studio smoke checks
* Keep API menu badge removed
* Apply no-build-tools env overrides in-script
The runner does not apply step-level env keys containing parentheses,
so ProgramFiles(x86) kept its real value and Find-VsBuildTools still
detected VS through vswhere. Set the overrides inside each pwsh step
instead; child processes inherit them. The resolver step moves to pwsh
because bash cannot export a variable named ProgramFiles(x86).
* Reset chat UI session without a second browser context
macOS runs Chromium with --single-process, where closing the last
context tears down the whole browser, so the shutdown re-login died
with TargetClosedError on new_page. Clear cookies and swap pages
inside the same context instead, opening the replacement page before
closing the old one.
* Keep the no-build-tools Path filtered across session refreshes
install.ps1's Refresh-SessionPath and setup.ps1's Refresh-Environment
rebuild the session Path from the Machine and User registry scopes, so
the process-level filter could be undone mid-install and re-expose
CMake. Filter those scopes in the Prepare step with normalized dir
matching and restore them in cleanup.
* Drop stale localStorage auth tokens before re-login
Auth tokens live in localStorage, not cookies, and the login guest
guard redirects on their mere presence. Remove them during the session
reset so the /login navigation is deterministic instead of relying on
the tolerated redirect bounce.
Michael Han