* Installer: respect a declined Studio auto-start and keep Ctrl+C shutdown logs ordered
The `curl | sh` Studio auto-start prompt had two issues on Linux/macOS/WSL
(install.sh). install.ps1 already gates on input redirection, so Windows is
unaffected.
1. Typing n, or any closed/EOF /dev/tty, still launched Studio. The read
fallbacks defaulted to "y" (read failure, and the no-tty branch), so any
answer other than a cleanly delivered y/n line auto-started a blocking
foreground server. Default those to "n"; a real Enter still counts as yes
via ${_reply:-y}.
2. On Ctrl+C the shell prompt printed in the middle of Studio's shutdown logs.
The non-interactive installer shell took the default SIGINT action and died
before the child finished its graceful shutdown, so the prompt raced ahead
of "All subprocesses cleaned up". trap '' INT in the installer shell so it
waits for Studio's own graceful shutdown.
* Studio: wait for the uvicorn thread before the terminal returns on Ctrl+C
Builds on #6565 by @Imagineer99. The studio server runs uvicorn in a daemon
thread, so on Ctrl+C the process could return to the shell while that thread
was still writing its shutdown logs, interleaving them with the prompt.
Retain the uvicorn thread and join it (flushing stdout/stderr) before terminal
entrypoints return, from run.py's main shutdown path and the CLI shutdown paths.
Refinements over #6565:
- Bound the join at 5s (_SERVER_SHUTDOWN_JOIN_TIMEOUT, matching the existing
_graceful_shutdown subprocess timeouts) so a stalled uvicorn shutdown cannot
hang the terminal; the timeout warning branch is now reachable.
- Restore SIG_DFL for SIGINT/SIGTERM at the start of the signal handler so a
second Ctrl+C force-quits, and drop the redundant in-handler wait (the
post-loop wait already covers the signal path).
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: keep child Ctrl+C working and restore SIGBREAK
- install.sh: run studio in a subshell that resets INT to default
(trap - INT; exec ...) so the foreground child does not inherit the
installer shell's ignored SIGINT, which would otherwise swallow the
studio process's own Ctrl+C and graceful shutdown.
- run.py: also restore SIGBREAK to SIG_DFL in the signal handler so a
second Ctrl+Break force-quits on Windows, matching SIGINT/SIGTERM.
* install.sh: capture studio exit with || under set -e so the migration hint still prints
* Trim shutdown-fix comments to be terser (comments only, no code change)
* Dedup CLI shutdown-wait into finally blocks (review follow-up)
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Windows installer: fix DiskPart UAC, drive-root cache, spurious rename warning, CPU-base messaging
amd-smi gate (DiskPart UAC mid-install): the AMD torch wheel ships hipInfo.exe
inside the venv, and the bitsandbytes fix prepends that venv Scripts dir to PATH.
shutil.which("hipinfo") then found it and flipped _amd_smi_allowed() to True, so
the post-install AMD probe fell through to `amd-smi list` (the venv hipInfo failed
to report gcnArchName, which is why the arch came from the GPU-name table) and
amd-smi elevated, popping the DiskPart UAC. Fix: a hipinfo resolved inside the
active venv (sys.prefix) is the torch-wheel binary, not a HIP SDK, and must not
open the gate. Mirrored in install_python_stack.py, install_llama_prebuilt.py, and
backend utils/hardware/amd.py (the runtime VRAM poller had the same latent prompt).
TORCHINDUCTOR_CACHE_DIR: move from C:\tc to <StudioHome>\TORCHINDUCTOR_CACHE_DIR so
the inductor/Triton cache lives under the user's Studio home, not the system drive
root. Long paths are already enabled above so deep inductor paths still fit.
unsloth.exe rename: skip the rename (and its "pip may fail with WinError 32"
warning) when SKIP_STUDIO_BASE=1. In the install.ps1 flow base packages are not
reinstalled, so unsloth.exe is never rewritten; the self-rename only failed because
setup runs via unsloth.exe (the running launcher holds its own file). The
'studio update' flow still attempts it.
CPU PyTorch messaging: clarify that the CPU base is temporary and setup replaces it
with GPU ROCm wheels, and print an explicit "GPU ROCm PyTorch installed" line after
the AMD wheels land, so the log makes clear the final install is GPU-accelerated.
Adds two regression tests covering the venv-internal vs external hipInfo gate.
Verified end-to-end on a Strix Halo box (Radeon 8060S / gfx1151): install.ps1
--local from this branch completed exit 0 with no DiskPart prompt, no rename
warning, the cache under the Studio home, and "GPU ROCm PyTorch installed
(gfx1151)"; Studio then booted and detected "ROCm (HIP 7.13.99004) -- AMD Radeon
8060S Graphics".
* Windows installer: drop the unreliable unsloth.exe rename and its WinError 32 warning
setup.ps1 used to rename the running unsloth.exe out of the way before the
base-package upgrade so pip could replace it. That rename never actually
worked: setup runs *via* unsloth.exe, so renaming our own running
uv-trampoline launcher failed with a sharing violation (WinError 32) and only
printed a scary 'could not rename unsloth.exe; pip may fail with WinError 32'
warning on every Windows install and update.
It also was not needed. pip tolerates a running/locked console-script .exe: it
moves the old one aside and writes the new one. The base upgrade routes through
pip on Windows, so the upgrade succeeds (or, in the install.ps1 flow with
SKIP_STUDIO_BASE=1, the base is not touched at all) and unsloth.exe is left
intact either way.
Removing the rename block and its failed-install restore block removes the
false warning for all Windows devices in both the install and update flows.
* Windows installer: gate venv-internal hipInfo.exe in PowerShell amd-smi probe; harden venv path checks
Follow-up to PR #6296.
- install.ps1 and setup.ps1: ignore the AMD torch wheel hipInfo.exe that lives
inside the Studio venv when probing for a HIP SDK, so amd-smi no longer reopens
the DiskPart UAC during install/update. Mirrors _path_inside_venv in the Python
installers, which already do this.
- amd.py, install_llama_prebuilt.py, install_python_stack.py: normcase the venv
containment check (Windows paths are case-insensitive) and run the
HIP_PATH/ROCM_PATH candidate through it too.
- setup.ps1: fall back to a short TORCHINDUCTOR cache dir when long paths are
unavailable, and create the dir wildcard-safely.
- tests: isolate sys.prefix in the gate helper, add HIP_PATH/ROCM_PATH cases, and
assert the PowerShell venv exclusion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Windows installer: install ROCm PyTorch directly for a known AMD arch
When the GPU arch is known (name-inferred from the GPU-name table) but ROCm
could not be probe-verified (no HIP SDK, no amd-smi), the bootstrap installed
a CPU PyTorch base that setup.ps1 then force-reinstalled as ROCm. The
repo.amd.com wheels bundle their own runtime (no HIP SDK required), which
setup.ps1 already relies on, so the CPU base was a pure wasted download/install.
- Gate the ROCm index on a known arch, not only on probe-verified ROCm, so a
mapped arch installs ROCm torch directly. Unmapped arches and no-GPU hosts
still get CPU (unchanged).
- Fall back to a CPU base if the ROCm-index install fails, so a transient
repo.amd.com outage does not abort the install (setup.ps1 retries ROCm).
- Correct the stale comment that claimed ROCm wheels need a confirmed HIP SDK.
- Add a regression test for the arch-based gate.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Windows installer: correct the unsloth.exe rename-removal comment
The comment claimed the base upgrade 'routes through pip on Windows' and that
pip 'moves the old unsloth.exe aside, then writes the new one'. That is not what
the code does. install_python_stack tries uv first; on a locked launcher uv
aborts and falls back to pip, but the pip fallback strips --upgrade-package and
base.txt lists only bare unsloth/unsloth-zoo, so pip finds them already
satisfied and no-ops. The running unsloth.exe is left intact at its current
version either way. Reword the comment to describe the real uv-first /
pip-fallback-no-op behavior. No functional change.
* Windows installer: close two gaps in the venv-internal hipinfo exclusion
Review follow-up. The amd-smi/DiskPart gate could still reopen in two cases:
- setup.ps1 ran the HIP probe long before $VenvDir is assigned, so without
VIRTUAL_ENV (the `unsloth studio update` path) $venvRoots was empty and the
venv-internal hipInfo.exe was not recognized. Seed the venv root from
UNSLOTH_SETUP_PYTHON and the default Studio home too (both installers).
- The HIP_PATH/ROCM_PATH candidate was accepted without the venv filter, so an
env var pointing into the venv (AMD wheel) still set $HipSdkInstalled. Run
Test-HipinfoIsVenvInternal on the candidate as well (both installers).
Extend the PS gate test to assert both. Both .ps1 parse clean; install tests
pass (the venv-internal / HIP probe coverage at 359 passed).
* Windows installer: correct the CPU-base message for arches with no ROCm wheels
After gating the ROCm index on a known arch, a mapped arch sets $ROCmIndexUrl
and installs ROCm directly, so it no longer reaches the "temporary CPU base"
branch. That branch is now reached only by a name-inferred arch with no ROCm
wheels (e.g. RDNA2 gfx103X), where setup.ps1 does NOT install ROCm. The old
text ("setup replaces it with GPU ROCm wheels ... the final install IS
GPU-accelerated") was therefore always wrong there. Say plainly that PyTorch
stays on CPU for this GPU.
* Windows installer: seed the venv-internal hipInfo check from a custom Studio home
Test-HipinfoIsVenvInternal seeded the venv root from VIRTUAL_ENV, VenvDir, the
setup python, and the default %USERPROFILE% path only. A standalone
`unsloth studio update` with a custom UNSLOTH_STUDIO_HOME (or STUDIO_HOME alias)
and none of those set would not recognize the venv hipInfo on PATH, reopening the
amd-smi/DiskPart gate. Seed the custom home too, in both installers, and assert
it in the gate test.
* Studio installer: resolve venv aliases and expand ~ in the hipInfo venv filter
Two review points on the amd-smi/DiskPart UAC gate:
1. _path_inside_venv compared os.path.abspath of sys.prefix and the hipInfo
path, which does not resolve symlinks, junctions, or 8.3 short names. A venv
reached through an aliased path then fails the check, so its bundled
hipInfo.exe is mistaken for an external HIP SDK and amd-smi runs (the
DiskPart prompt this fix exists to suppress). Switch to os.path.realpath in
all three copies (amd.py, install_llama_prebuilt.py, install_python_stack.py).
2. setup.ps1's early venv-internal hipInfo probe seeded the venv root from a
custom Studio home (UNSLOTH_STUDIO_HOME / STUDIO_HOME) without expanding a
leading ~, while the canonical resolver does. With a tilde form,
[IO.Path]::GetFullPath kept the literal ~ relative to cwd, so the custom-home
hipInfo escaped the filter and reopened the gate. Expand ~ in the probe the
same way as the resolver.
tests/studio/install/test_pr5940_followups.py: 30 passed (adds a symlink
realpath case and a setup.ps1 tilde-expansion guard).
* Studio installer: mirror the hipInfo venv filter and ROCm wheel pins into install.ps1
Follow-up review on the same install.ps1 paths:
1. install.ps1's venv-internal hipInfo probe (Test-HipinfoIsVenvInternal)
seeded the venv root from a custom Studio home without expanding a leading
~, unlike the canonical resolver and setup.ps1. A tilde form left
[IO.Path]::GetFullPath with the literal ~ (relative to cwd), so the
custom-home hipInfo escaped the filter and reopened the amd-smi/DiskPart
gate. Expand ~ in the probe, matching the setup.ps1 fix.
2. The AMD ROCm path installed torchvision/torchaudio bare while pinning torch
to below 2.12. AMD's per-arch index publishes the companions independently
and may ship torchvision 0.27 (for torch 2.12) before removing 0.26, so a
bare resolve can pick an ABI-incompatible set and fall back to CPU. Add
torchvision/torchaudio floor maps and pass the pinned specs, mirroring
setup.ps1 and install_python_stack.py.
3. The ROCm-to-CPU fallback torch install used Invoke-InstallCommand (no
retry), the only torch step in the file without it. Switch to
Invoke-InstallCommandRetry so the recovery path survives a transient index
failure.
tests/studio/install/test_pr5940_followups.py: 33 passed (parametrized tilde
check over both installers, a torch/companion floor-map parity test, and a
CPU-fallback retry guard).
* Studio installer: scan all PATH hipinfo so the venv copy can't shadow a real HIP SDK
The amd-smi HIP-SDK probe used shutil.which("hipinfo") / Get-Command hipinfo,
which return only the first hit on PATH. The AMD torch wheel ships hipInfo.exe
inside the venv and the bnb fix (plus the Studio backend) prepend the venv
Scripts dir to PATH, so that venv-internal copy lands first. When a real HIP SDK
hipinfo sits later on PATH with HIP_PATH/ROCM_PATH unset, the first-hit probe
stopped at the venv copy, treated it as "not a HIP SDK", and closed the amd-smi
gate -- AMD users in that PATH-only SDK setup lost amd-smi telemetry and could
fall back to CPU. Scan every PATH entry and keep the first hipinfo that is not
venv-internal; only the venv copy is ignored, so the UAC/DiskPart suppression is
unchanged.
Applied to all three Python copies (install_llama_prebuilt.py,
install_python_stack.py, backend/utils/hardware/amd.py) via a new
_external_hipinfo_on_path helper, and both PowerShell callers (install.ps1,
setup.ps1) now use Get-Command hipinfo -All filtered by Test-HipinfoIsVenvInternal.
tests/studio/install/test_pr5940_followups.py: 36 passed (real-PATH scan tests, a
shadow-regression test for the exact venv-first ordering, and a parity check that
every Python copy uses the scanning helper).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio uninstallers: fix leftovers (false "removed", shared icon, llama lock)
Auditing a dual native+WSL uninstall on a real device surfaced three leftovers:
1. uninstall.ps1 removed the data dir (which holds unsloth.ico) before the
shortcuts that reference that icon, so Explorer's icon cache briefly held it
open. Remove-Item -Recurse reported success yet left the locked file, and the
dir was never re-attempted, so it orphaned with a false "removed" log.
_RemovePath now verifies the path is actually gone (retrying transient locks)
and reports honestly, and the data dir is re-swept after the shortcuts go.
2. install.sh writes a shared unsloth.ico to %LOCALAPPDATA%\Unsloth Studio for
the WSL shortcut, but uninstall.sh never removed it, orphaning the icon (and
dir) after a WSL uninstall. uninstall.sh now drops that icon and the dir when
empty, in both the powershell.exe and drvfs-fallback paths.
3. ~/.unsloth/.llama.cpp.install.lock was never removed, so the rmdir of
~/.unsloth failed and the dir lingered. Both uninstallers now remove the lock.
Verified by running both uninstallers on a real dual install: device fully clean
(no install dirs, shortcuts, PATH/registry entries, shared icon, or lock left).
* install.sh: auto-route Strix Halo WSL to an existing Ubuntu 24.04
ROCm-on-WSL is the GPU runtime for Strix Halo and only targets Ubuntu
24.04. When the installer runs in a newer default distro (e.g. 26.04) it
cannot enable the GPU and silently falls back to CPU. If a 24.04 distro
already exists, re-run the install there and stop in the current one so the
GPU path is taken without the user having to know about the distro
requirement.
Runs before venv creation so the wrong distro is left untouched, guards
against re-route loops via UNSLOTH_WSL_REROUTED, leaves a working ROCm
distro alone (librocdxg present), and skips the GGUF-only / opt-out /
non-Strix cases. When no 24.04 distro exists we keep today's behaviour:
continue to CPU and print the `wsl --install Ubuntu-24.04` guidance, never
auto-downloading a distro.
Adds tests/sh/test_strixhalo_wsl_reroute.sh (hermetic: extracts the
function, rewrites its paths to fixtures, mocks wsl.exe) covering the full
decision matrix, wired into tests/run_all.sh.
* uninstall.ps1: keep shared unsloth.ico for a surviving WSL shortcut
A dual native+WSL install shares %LOCALAPPDATA%\Unsloth Studio\unsloth.ico:
install.sh points the WSL shortcut's icon there while the native install owns the
dir. The native uninstaller removed the whole dir unconditionally, so uninstalling
native while keeping WSL left the WSL shortcut with a blank icon. The old code only
avoided this when Explorer happened to hold the icon open, which is unreliable; on a
real dual install the dir was deleted and the WSL shortcut went blank.
_RemoveDataDirKeepingWslIcon now scans the Start Menu + Desktop for a surviving
"Unsloth Studio (WSL ...).lnk" and, if found, removes everything in the data dir
except unsloth.ico (keeping the dir) instead of deleting it; with no WSL shortcut it
removes the dir as before. uninstall.sh still drops the icon and the empty dir when
WSL itself is uninstalled, so every uninstall order ends clean.
Adds tests/studio/test_uninstall_dual_install_icon.ps1 (AST-extracts the helper and
runs it against a temp dir with controlled shortcut dirs) covering the dual,
native-only, empty, and missing-dir cases, wired into the windows-inference smoke
workflow. Verified on a real dual install: native uninstall now keeps unsloth.ico
and the WSL shortcut's icon stays intact.
* installer: condense AMD/ROCm code comments (no behavior change)
Tighten the comments added for the Strix Halo native+WSL installer work so
they are shorter and clearer without losing intent: the venv-internal hipInfo
amd-smi gate, the ROCm torch/companion floor maps, the WSL 24.04 reroute, and
the dual-install uninstall icon handling. Comment-only; code paths unchanged.
107 insertions, 166 deletions across 11 files.
* install.sh: run the Strix Halo WSL reroute before any STUDIO_HOME write
The reroute fired after mkdir -p "$STUDIO_HOME" and the legacy-venv migration,
so rerouting 26.04 -> 24.04 left an empty ~/.unsloth/studio stub in the origin
distro (and ran venv migration in the distro about to be abandoned). Move the
reroute ahead of the venv section so the origin distro is left untouched, matching
the function's own comment. Behavior is identical on every non-reroute path.
* installer: fix ROCm CPU-fallback, hipinfo gate edge cases, uninstall icon, WSL 22.04
- install.ps1: clear $ROCmIndexUrl/$ROCmTorchFloor after the CPU fallback so the
flavor-repair block does not retry the failed ROCm index and abort the install;
pin the ROCm companion specs ($visionSpec/$audioSpec) in the repair path too.
- install.ps1 + setup.ps1: skip a bare drive root in Test-HipinfoIsVenvInternal so a
non-venv UNSLOTH_SETUP_PYTHON does not match the whole drive; iterate
HIP_PATH/HIP_PATH_57/ROCM_PATH and take the first non-venv hipinfo.
- amd.py, install_llama_prebuilt.py, install_python_stack.py: strip surrounding
quotes from PATH entries before probing for hipinfo.
- install.sh: pipefail the WSL reroute curl|sh; do not reroute supported Ubuntu 22.04.
- uninstall.sh: keep the shared unsloth.ico while any Unsloth shortcut (native or
another WSL distro) still references it, in both the powershell and drvfs paths.
- tests: regression coverage for all of the above.
* installer: forward reroute options, guard ROCm bootstrap, harden hipinfo gate
- install.sh: forward the caller's --package/--python/--verbose/--tauri and a custom
UNSLOTH_STUDIO_HOME into the WSL reroute (was a bare default install); bail on
--local; run the reroute BEFORE dependency/uv install so the origin distro is left
untouched; set UNSLOTH_SKIP_ROCM_WSL_SETUP after a failed reroute so the later
ROCm-on-WSL bootstrap does not install into the unsupported origin distro.
- install.ps1 + setup.ps1: Get-Command hipinfo -CommandType Application so only real
executables match (not an alias/function named hipinfo).
- uninstall.ps1: guard $env:APPDATA when building the default shortcut search dirs.
- tests: cover option forwarding, --local bail, the bootstrap guard, and the gate change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer: guard origin ROCm bootstrap on every CPU-only fallback; harden ~ expansion
WSL reroute: the no-wsl.exe, no-24.04-target and --local fallbacks all tell the
user the install continues CPU-only, but only the failed-reroute branch set
UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The later _maybe_bootstrap_rocm_wsl gate keys off
that flag, so the other three branches could still install ROCm into the
unsupported origin distro (e.g. 26.04). Set the skip guard on all of them.
Forward UNSLOTH_ROCM_WSL_AUTO into the reroute so a Tauri/consented GPU bootstrap
carries through to the rerouted 24.04 child instead of dropping to the prompt path.
install.ps1/setup.ps1: guard the venv-probe ~ expansion on a non-empty
$env:USERPROFILE so Join-Path does not throw on a profile-less service account.
Tests: add no-wsl.exe and UNSLOTH_ROCM_WSL_AUTO reroute cases, the USERPROFILE
guard assertion, and route shell-test fixtures through a single trap-cleaned root.
* installer: pin + soften Windows ROCm Python repair, reroute to 22.04, harden gates
install_python_stack.py: the Windows AMD ROCm repair in _ensure_rocm_torch()
installed bare torch/torchvision/torchaudio via the fatal pip_install -- the same
asymmetry already fixed on the PowerShell side. A transient repo.amd.com failure
could abort the whole install even after install.ps1/setup.ps1 fell back to CPU.
Pin companions per-arch (gfx120X/Strix -> the rocm7.2 trio, mirroring the PS floor
maps) and make the retry nonfatal: keep the existing build and let the user re-run
update to retry ROCm, so the chain install.ps1 -> setup.ps1 -> stack stays CPU-safe.
install.sh: reroute now targets an installed Ubuntu 24.04 OR 22.04 (24.04 preferred);
both are AMD-supported for ROCm-on-WSL, matching the leave-alone set, so a box with
only 22.04 reaches the GPU instead of staying CPU-only.
install.ps1/setup.ps1: a bare ~ for UNSLOTH_STUDIO_HOME left an empty Join-Path child
(PS 5.1 throws); fall back to USERPROFILE directly and only join a real remainder.
_path_inside_venv (amd.py + both installers): guard a root-dir sys.prefix so commonpath
can't classify every path on the drive as venv-internal (defensive; venv never at root).
uninstall.sh: guard an empty LOCALAPPDATA in the PS-interop icon cleanup (mirror APPDATA).
Tests: add 22.04-target reroute cases, Windows ROCm pin+nonfatal coverage (text +
behavioral), root-dir guard coverage, and bare-~/LOCALAPPDATA guard assertions.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: match WSL reroute target by exact distro name, not substring
The 24.04/22.04 reroute target was chosen with grep -F (substring), so a custom
distro such as 'Ubuntu-24.04-test' (with no exact Ubuntu-24.04) was picked as the
target; the later 'wsl -d Ubuntu-24.04' then fails and the Strix Halo install stays
CPU-only. Match whole lines (grep -ixF) and reuse the matched name so only a real
Ubuntu-24.04/22.04 is targeted. Adds substring-rejection + exact-vs-custom tests.
* install.sh: keep the WSL reroute target to Ubuntu 24.04 (helper-supported only)
The ROCm-on-WSL bootstrap (scripts/install_rocm_wsl_strixhalo.sh) dies on any
VERSION_ID other than 24.04 and pins the noble repo, so treating 22.04 as
GPU-supported let the parent report a successful reroute while the child fell
back to CPU. Drop 22.04 from the supported set and the reroute target list;
24.04 stays the sole target (keeping the exact whole-line distro match). An
already-working ROCm on any other version is still left alone by the librocdxg
check above.
tests: reroute 22.04 cases updated to the 24.04-only behavior; make the
"no wsl.exe" case hermetic so a real host wsl.exe can't leak in on dev boxes;
stop the tauri exit-order check from mis-flagging the reroute helper's
[ "$TAURI_MODE" = true ] && ... --tauri one-liner.
* installer: tighten comment wording across the Strix Halo install/uninstall paths
Condense the verbose multi-line comment blocks (amd-smi hipinfo gate, ROCm
torch install + CPU fallback, WSL reroute, uninstall icon-keep) into fewer,
clearer lines. Comments and a few docstrings only; no code, logic, or
behavior change. Verified with bash -n, the PowerShell parser, and ast.parse,
and the installer test suite still passes.
* add AGPL-3.0 SPDX headers to the .sh/.ps1 scripts missing them
Every shell and PowerShell script under the Studio/installer surface now
carries the standard SPDX-License-Identifier: AGPL-3.0-only + copyright
header (after the shebang where present): the installer (install.sh,
install.ps1), build.sh, the .github and src-tauri scripts, the installer
test suite, and the moe kernel test. Header-only, line endings preserved;
bash -n, the PowerShell parser, and the installer tests all pass.
* installer: drop the duplicate AGPL header from install.sh and install.ps1
Both already carry an SPDX-License-Identifier: AGPL-3.0-only header below
their usage comment block; the prior header pass added a second one at the
top because it only scanned the first few lines. Remove the duplicate so each
file keeps a single original header.
* installer: force-reinstall CPU fallback torch; propagate Tauri NEED_SUDO from reroute
install.ps1/setup.ps1: when the AMD ROCm wheel install fails and we fall back to a
CPU base, force-reinstall the torch/vision/audio triplet. A failed ROCm install can
leave an unpinned ROCm torch (e.g. 2.10.0+rocm on gfx110X/gfx90a) that still
satisfies the CPU torch>=2.4,<2.11.0 range, so without --force-reinstall uv keeps the
ROCm build and only swaps the companions -- a mismatched venv the flavor-repair block
won't fix. setup.ps1 scopes the forced reinstall to the ROCm-fallback path
() so the genuine CPU-only install stays fast.
install.sh: the Strix Halo WSL reroute treated every nonzero child exit as a reroute
failure and fell back to CPU. In --tauri mode the child uses exit 2 ([TAURI:NEED_SUDO])
to ask the desktop app to elevate for the target distro; capture the child's exit code
and propagate exit 2 in Tauri mode (the child already printed the NEED_SUDO line)
instead of masking it. CLI mode still falls back to CPU on a generic failure.
Tests: reroute Tauri exit-2 propagation (and non-Tauri CPU-fallback) cases;
run_func now preserves the child exit code; force-reinstall assertions for both
PowerShell installers.
Note: codex's _rr_q apostrophe finding is a false positive -- the helper already
emits POSIX-correct 'O'\''Brien' and round-trips under both sh and bash.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.ps1: fix $cpuForce array collapse in the ROCm->CPU torch fallback
An if-expression assignment ($cpuForce = if ($ROCmCpuFallback) { @("--force-reinstall") })
collapses the single-element array to a scalar string, so @cpuForce splatting enumerated
it character-by-character into broken single-letter args (- - f o r c e ...), which made
uv/pip reject the install and aborted the whole Studio setup on the AMD ROCm->CPU fallback
path. Build $cpuForce as a real array assigned outside the if-expression so the splat passes
a single --force-reinstall arg. Genuine CPU-only installs stay fast (empty array, no flag).
Test now asserts the array-build form and rejects the if-expression form.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* uninstall: remove the isolated Node.js runtime (~/.unsloth/node)
The isolated Node.js runtime (install_node_prebuilt.py, added with the managed-Node
change) installs to ~/.unsloth/node in default mode -- a sibling of studio, so deleting
<studio> leaves it behind (~200MB orphaned after uninstall). Both uninstallers already
remove the other default-mode siblings (llama.cpp/.cache/.staging); add node alongside
them. uninstall.ps1 also adds it to the handle-lock sweep so a held node.exe can't block
the delete. Env/custom mode nests node under the custom root, removed with that root.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: label Apple Silicon as Metal/unified memory instead of CPU-only in installers
* Studio: drop redundant aarch64 check from the macOS GPU label detection
* Windows installer: repair a stale CPU PyTorch instead of looping forever
A Windows machine with an NVIDIA CUDA 13 driver (e.g. RTX 6000 Pro on enterprise
drivers) could get permanently stuck at:
Stale venv detected (torch cpu != required cu130).
[ERROR] The existing Studio environment needs repair.
Re-run install.ps1 so it can replace the environment safely with rollback.
Re-running install.ps1 did not help. install.ps1 installs torch with
"torch>=2.4,<2.11.0" --index-url .../cu130 but no --force-reinstall, so when a
torch==X+cpu is already present uv treats it as satisfying the range (PEP 440
ignores the +cpu/+cuXXX local label) and makes no change -- the CPU wheel is
never replaced. setup.ps1 then rejects the venv as cpu != cu130 and exits, but it
cannot create a venv or install torch, so the loop never resolves. The migrated-
venv branch also preserves existing torch and never reinstalls it.
After the install step, detect the installed torch flavor (cuXXX/cpu/rocm) and,
when it does not match the tag implied by the selected index, force-reinstall the
torch/torchvision/torchaudio triplet from the correct index via three
--reinstall-package flags. No-op on a healthy matching venv; skipped for
--no-torch, ROCm (already --force-reinstalls), and CPU-only machines.
Adds two pure helpers (ConvertTo-TorchFlavorTag, Get-ExpectedTorchFlavorTag), a
PowerShell unit test (tests/studio/test_torch_flavor.ps1), and a CI parse gate for
install.ps1 (previously unparsed).
* install.sh: repair a stale CPU PyTorch on Linux too (parity with install.ps1)
install.sh has the same latent bug as the Windows installer: the CUDA torch
install uses "torch>=2.4,<2.11.0" --index-url .../cuXXX with no
--force-reinstall, so an already-present torch==X+cpu satisfies the version
range (PEP 440 ignores the +cpu/+cuXXX local label) and uv leaves it in place.
The migrated-venv branch also preserves existing torch. Unlike Windows there is
no stale-venv check in setup.sh, so on Linux the symptom is silent CPU training
rather than a hard loop -- same root cause.
Mirror the install.ps1 fix: after the install block, detect the installed torch
flavor (_torch_flavor_tag) and, when it does not match the index tag
(_expected_torch_flavor_tag), force-reinstall the torch/torchvision/torchaudio
triplet from the selected index via --reinstall-package. No-op on a healthy
matching venv; skipped for --no-torch, ROCm (its own repair force-reinstalls),
and CPU-only / macOS hosts. Adds tests/sh/test_torch_flavor.sh (run in
studio-backend-ci and run_all.sh).
* Installer: catch CPU-fallback on AMD/WSL too (repair ROCm, warn when unfixable)
Extend the torch-flavor safety net beyond NVIDIA:
- install.sh now auto-repairs a stale CPU torch on standard pytorch.org ROCm
indexes too (the rocm-index install path lacked --force-reinstall, unlike the
Windows ROCm install). Reuses the rocm-adjusted $TORCH_CONSTRAINT + rocm index,
so it pulls the correct ROCm wheels.
- Both installers gain a universal post-install warning: when a GPU build was
expected (cuXXX / rocm, including the repo.amd.com gfx* arch indexes) but torch
is still CPU-only, warn loudly instead of silently training on CPU. This catches
the cases auto-repair cannot safely fix (AMD gfx arch indexes that need
--find-links, a migrated AMD venv on Windows where the ROCm install was skipped).
- Mac / Intel / CPU-only hosts resolve to the cpu index -> expected == installed
-> no-op, no false warning. WSL uses install.sh, so the NVIDIA repair + warning
apply there.
Adds Get-InstalledTorchTag (ps1) and _torch_index_repairable (sh) helpers and
extends both unit tests. gfx*/AMD indexes now map to the 'rocm' expected flavor.
* Installer: tighten torch-flavor comments (no logic change)
Condense the rationale comments added for the stale/CPU PyTorch repair in
install.ps1, install.sh and the two helper unit tests; same intent, fewer
lines. Comment-only: AST parse of install.ps1/setup.ps1 clean, helper unit
tests (15 ps1, 24 sh under bash and dash) and the integration sims
(24 ps1, 28 sh) still pass, banner markers the sims slice on are unchanged.
* Installer: bound torch probe, auto-repair gfx, fix ROCm gate parity
install.ps1: in Get-InstalledTorchTag, call WaitForExit(30000) and drain stdout
and stderr asynchronously instead of reading stdout synchronously first, so a
hung or noisy "import torch" (a wedged CUDA/driver, the exact failure this PR
targets) can no longer block the probe past the timeout.
install.sh and install.ps1: treat the repo.amd.com gfx* indexes as plain
--index-url reinstallable. They are PEP 503 simple indexes uv resolves in full
(torch plus every transitive dep) via --index-url, the same URLs the fresh
ROCm install paths already use, so a stale CPU torch on AMD Strix now auto-repairs
to the correct ROCm build instead of only warning.
install.sh: include */gfx* alongside */rocm* in the bitsandbytes install and
ROCm torch repair gates, so a custom UNSLOTH_AMD_ROCM_MIRROR whose path lacks
/rocm/ still installs the AMD bitsandbytes build and repairs ROCm torch.
tests/sh/test_torch_flavor.sh: gfx indexes now assert repairable, plus a
gfx1151 case and an unknown-mirror not-repairable case.
* install.ps1: guard Get-InstalledTorchTag against an empty python path
Make the early return explicit for an empty $PythonExe instead of relying on
Test-Path -LiteralPath '' returning false, so the probe stays safe under
Set-StrictMode or a future refactor that drops the [string] annotation.
The base-install lines passed a bare unsloth-zoo spec and relied on the
co-installed unsloth>=2026.6.7 (whose pyproject pins unsloth_zoo>=2026.6.5)
to floor unsloth-zoo transitively. Make the floor explicit so the install
scripts stay in sync with pyproject and the bare spec can never resolve below
the version that ships unsloth_zoo.diffusion_studio (the DiffusionGemma Studio
runner), first released in unsloth-zoo 2026.6.4. Leaves the reinstall/upgrade
flags and the git-main overlay untouched.
* Studio: drop the VBS launcher to clear the Kaspersky false positive
The Windows shortcut launched Unsloth Studio through wscript.exe ->
launch-studio.vbs, and that VBS used CreateObject("WScript.Shell").Run to
start a hidden -ExecutionPolicy Bypass PowerShell. That wscript + .vbs +
bypass-powershell shape is the canonical trigger for generic VBS-dropper
heuristics (Kaspersky HEUR:Trojan.VBS.Agent.gen). The launcher is benign;
only its shape is the problem.
- install.ps1: stop generating launch-studio.vbs and point the Desktop /
Start Menu .lnk straight at powershell.exe -WindowStyle Hidden running
launch-studio.ps1. The shortcut is saved WindowStyle 7 (minimized) so the
brief console flash is muted. launch-studio.ps1 (health poll, port,
mutex, browser) is byte-for-byte unchanged.
- install.ps1: delete a pre-existing launch-studio.vbs on upgrade, so the
flagged file does not linger on machines that already installed it.
- install.ps1 / install.sh: run the heavier ie4uinit -ClearIconCache plus
StartMenuExperienceHost tile-cache rebuild only on a first install or a
real icon change, instead of on every no-op reinstall. That repeated
clear-cache plus kill cluster is itself a dropper-like behavioral pattern.
- tests: forbid VBS generation and require the legacy-VBS cleanup.
Linux, macOS and WSL install paths are unchanged. WSL already targets
wsl.exe from its .lnk and never used a VBS; its only change is the same
icon-cache gating.
* Studio: add launcher-chain smoke coverage to the Windows UI CI
The shortcut launch path was previously untested: studio-windows-ui-smoke
installed then booted `unsloth studio` directly, so a broken .lnk could ship
silently. After install the job now seeds a legacy launch-studio.vbs, asserts
the upgrade removed it, asserts the .lnk targets hidden powershell.exe (never
wscript.exe), and launches via the shortcut's stored command, waiting for
/api/health to report healthy.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Make Studio installer resilient to transient uv download failures
Updating an existing Studio install via install.sh could hard-fail and roll
back when a wheel download (torch, unsloth) hit a transient connection reset:
x Failed to download unsloth==2026.6.6
error decoding response body -> error reading a body from connection
-> connection reset
restoring previous environment after failed install...
Root cause: that error chain is a mid-stream HTTP/2 body read failure. uv did
not retry this class until 0.8.16 (astral-sh/uv#15675, h2 was shadowing the
underlying IO error), but the installer pinned UV_MIN_VERSION=0.7.22, so a stale
uv got zero retries and a single blip aborted the whole update under set -e.
Fix (installer only, backwards compatible, no change on success):
- Raise UV_MIN_VERSION to 0.8.16 so stale uv is upgraded to a version that
retries HTTP/2 streaming body errors.
- Export UV_HTTP_RETRIES=5 and UV_HTTP_TIMEOUT=180 (override-preserving :=).
- Add run_install_cmd_retry (retry-with-backoff around run_install_cmd) and use
it for the network-heavy uv pip install steps (torch, unsloth, unsloth-zoo
from git, ROCm torch repair, no-torch runtime deps). Local editable overlays
and venv creation are left to fail fast.
run_install_cmd_retry preserves the final exit code on permanent failure, so the
existing set -e rollback trap still fires.
* Apply the same transient-download resilience to the Windows installer
install.ps1 is the native-Windows installer and had the identical issue as
install.sh: it pinned $UvMinVersion=0.7.22 (below uv 0.8.16, which is where uv
started retrying HTTP/2 streaming body errors), set no UV_HTTP_* defaults, and
ran each 'uv pip install' once via Invoke-InstallCommand, so a single connection
reset aborted the update and triggered the Exit-InstallFailure rollback.
install.ps1:
- Raise $UvMinVersion to 0.8.16.
- Default $env:UV_HTTP_RETRIES=5 and $env:UV_HTTP_TIMEOUT=180 (preserving overrides).
- Add Invoke-InstallCommandRetry and use it for the network-heavy uv pip install
steps (torch, unsloth, unsloth-zoo from git, ROCm torch, no-torch runtime deps).
Local editable overlays and venv creation stay single-shot.
install.sh:
- Align UNSLOTH_INSTALL_RETRIES sanitization with the PowerShell version: a
non-positive-integer value now falls back to the default of 3 instead of
silently disabling retries (set =1 to disable). Keeps both installers identical.
* Adopt pre-marker Studio llama.cpp and sidecar dirs on update
After the uv retry fix, an update now reaches studio/setup.sh, whose
Studio-owned ownership guard rejects a llama.cpp or sidecar venv created by an
earlier install that predates the .unsloth-studio-owned marker:
ERROR: .../llama.cpp already exists and is not marked as a Studio-owned
llama.cpp install.
The marker and UNSLOTH_PREBUILT_INFO.json were introduced in the same commit,
so a directory from before that point carries neither signal and a legitimate
self-update fails for anyone who installed earlier (reported on issue #6274).
Fold a one-time adoption into _assert_studio_owned_or_absent (setup.sh) and
Assert-StudioOwnedOrAbsent (setup.ps1): when a custom-home directory lacks the
marker, backfill it and proceed only when there is positive evidence it belongs
to an established Studio home -- the directory carries UNSLOTH_PREBUILT_INFO.json,
or STUDIO_HOME already holds Studio's CLI shim or studio.conf from a prior run.
Both installers write the shim and studio.conf only after invoking setup, so a
fresh install into a dirty custom home (the case the guard protects) does not
have them yet and is still rejected. The venv marker is excluded because install
writes it before setup and so cannot tell a prior install from a fresh one.
* Review fixes: restrict llama.cpp adoption to dir-local evidence; restore install.sh +x
Addresses the PR review on the marker-migration change.
P1 - the adoption helper keyed on root-level Studio sentinels ($STUDIO_HOME/bin
/unsloth, share/studio.conf), so once a home was recognized every unmarked child
passed to the guard became adoptable, and an unrelated directory at a
Studio-managed path could be silently marked and overwritten. Base adoption on
evidence inside the directory instead:
- UNSLOTH_PREBUILT_INFO.json, written by the prebuilt llama.cpp installer (the
default path, in place well before the marker), or
- a top-level llama-quantize symlink, written by source builds (a plain
llama.cpp checkout keeps the binary under build/bin, not a root symlink).
A foreign llama.cpp now stays rejected even inside an established Studio home,
and sidecar venvs (no such fingerprint) stay subject to the strict guard; their
marker has been written since the guard was introduced, so a real custom install
already carries it.
P2 - restore the executable bit on install.sh; a stray mode change to 100644
would break ./install.sh --local on Unix.
On Windows the prebuilt metadata is the signal; source builds are git checkouts
indistinguishable from a user clone, so they are left to the strict guard.
* Bound UNSLOTH_INSTALL_RETRIES / _DELAY before numeric use
An oversized all-digit override (e.g. a fat-fingered
"99999999999999999999") passed the digit-only validation and then reached the
numeric comparison: POSIX `[ -ge ]` errored with "Illegal number" mid-loop and
could spin instead of falling back, and PowerShell's `[int]` cast threw an
Int32 overflow under $ErrorActionPreference = "Stop" before any install ran.
Sanitize with a length guard + range check (sh) and [int]::TryParse with bounds
(ps1), so out-of-range or oversized values fall back to the default. Bounds:
1..100 retries, 0..3600s base delay.
* Studio installers: scope llama.cpp adoption to prebuilt metadata; reject leading-zero retry delay
setup.sh: drop the top-level llama-quantize symlink as an ownership-adoption signal, leaving UNSLOTH_PREBUILT_INFO.json as the sole fingerprint. The shared ownership guard runs immediately before a destructive replace / rm -rf, and a bare root llama-quantize symlink is user-creatable (a user can keep their own llama.cpp build with such a convenience symlink at a custom UNSLOTH_STUDIO_HOME), so the old check could adopt and then delete a user directory. This matches the Windows installer, which already keeps markerless source builds strict. Pre-marker prebuilt installs still adopt via the metadata file, so the original update fix is preserved.
install.sh: reject leading-zero values for UNSLOTH_INSTALL_RETRY_DELAY. A value like 08 or 09 passed the range check but then hit the backoff doubling $((_ricr_delay * 2)), where a non-octal leading zero is a fatal arithmetic error mid-retry. The 0?* pattern routes such values to the default; bare 0 stays valid.
* Tighten the comments added in this PR
* Condense the comments in this PR
When installed via 'curl -fsSL https://unsloth.ai/install.sh | sh', the
shell reads the script from the pipe on stdin. The optional 'start now'
step launches 'unsloth studio' in the foreground, and as a server it
inherits and drains the rest of the piped script from stdin. The shell
then hits EOF partway through the trailing if/case and dies with
'Syntax error: end of file unexpected (expecting fi)' (reported on WSL,
where /bin/sh is dash). Redirect the launch's stdin away from the pipe so
the script stays intact. The server does not read stdin and Ctrl+C still
works via the controlling terminal.
* install.sh: persist ROCm-on-WSL drop-in even when rocminfo already works
_maybe_bootstrap_rocm_wsl calls _ensure_rocm_probe_env (which exports a
transient HSA_ENABLE_DXG_DETECTION + adds /opt/rocm/bin to PATH on the
installer process) right before the "rocminfo enumerates gfx1151 -> already
set up, return early" gate. On any reinstall over an existing /opt/rocm --
the common case, since the uninstaller keeps shared ROCm userspace but
removes /etc/profile.d/unsloth-rocm-wsl.sh -- that probe env makes rocminfo
succeed, so the gate returns 0 WITHOUT ever persisting the drop-in. The
transient env dies with the installer, so the next login shell (Studio,
llama-server) sees no GPU: torch cuda_avail=False, rocminfo finds nothing,
the llama.cpp ROCm prebuilt segfaults on a GPU it can't reach.
Factor the drop-in writer into _persist_rocm_wsl_dropin() and call it before
the early return so the persistent env is restored whenever librocdxg is
present. Idempotent (only writes when the drop-in is missing), gated on
librocdxg so it never fires on non-WSL/non-ROCDXG hosts, root-writes or
sudo-tees like before. The fast-path branch now reuses the same helper.
Reproduced on gfx1151 (Radeon 8060S) under dash (the curl|sh shell):
before the fix a reinstall left the drop-in absent and torch cuda_avail
False; after, the drop-in is persisted and a fresh login shell reports
cuda_avail True. Verified under both dash and bash, and idempotent on
re-run.
* Studio WSL: load system HIP before a prebuilt's bundled runtime (gfx1151)
The lemonade / published llama.cpp ROCm prebuilts bundle their own HIP
runtime (libamdhip64) built for bare-metal Linux. In WSL the GPU is reached
through the system ROCm's librocdxg bridge over /dev/dxg, which the bundled
runtime cannot drive -- it segfaults on the first GPU call. So:
- install_llama_prebuilt.py: the prebuilt's llama-quantize/llama-server
validation runs with the bundle dir first on LD_LIBRARY_PATH, segfaults
(empty stderr), and the install silently falls back to a CPU source build
(which on this host can't even build for GPU -- hipcc absent). The Strix
Halo WSL user ends up on CPU despite a working GPU.
- llama_cpp.py: even if a GPU prebuilt were kept, the serve-time launcher
put the bundle dir first too, so it would crash at load.
Fix: on a ROCDXG WSL host (gated on /dev/dxg + "microsoft" /proc/version +
a librocdxg-providing /opt/rocm), prepend the system ROCm lib dir to
LD_LIBRARY_PATH so the WSL-capable libamdhip64 + librocdxg load first, while
the bundle still supplies libggml-hip / librocblas with the gfx1151 kernels.
Set HSA_ENABLE_DXG_DETECTION=1 alongside. Added _wsl_system_rocm_lib_dirs()
to both modules (kept identical so a prebuilt that passed install validation
runs the same way at serve time). Strict no-op on bare-metal Linux, NVIDIA,
macOS, and Windows.
Verified on gfx1151 (Radeon 8060S) in WSL (ROCm 7.2.1 + librocdxg, Adrenalin
ROCDXG): before, the lemonade gfx1151 prebuilt segfaulted and the install
fell back to a broken CPU build; after, install_llama_prebuilt validates and
keeps the GPU prebuilt (source=published, prebuilt_fallback_used=False), and
Studio serves Qwen3-1.7B-GGUF at 53 tok/s with the model resident in GPU
memory (llama-server device_info: ROCm0 = AMD Radeon 8060S).
* tests: cover the WSL ROCDXG drop-in + system-HIP-ordering fixes
- _wsl_system_rocm_lib_dirs: no-op without /dev/dxg, on bare-metal Linux,
and on WSL without librocdxg; returns the system lib dir on a ROCDXG WSL
host.
- binary_env: prepends the system ROCm lib dir ahead of the bundle and sets
HSA_ENABLE_DXG_DETECTION on WSL; unchanged on bare-metal Linux.
- install.sh: _persist_rocm_wsl_dropin exists, is gated on librocdxg, and the
rocminfo-already-works early return calls it before returning.
- llama_cpp.py: the serve-time launcher prepends the WSL rocm dirs before the
bundle dir (mirrors binary_env).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten WSL ROCDXG fix comments (no logic change)
Condense the drop-in / system-HIP-ordering comments and docstrings added in
this PR. Verified comment-only via AST parse + py_compile + sh/bash -n, the
308-test rocm_support suite, and a dash functional re-run of the bootstrap
(drop-in still persisted, env still set).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix/uv-bytecode-timeout
* make sure that win installer upgrades uv for bytecode timeout
* Clarify uv bytecode timeout comment in install.sh and install.ps1
* Read installer scripts as UTF-8 in parity test so it runs on Windows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer freshly installed uv when an older one shadows it on PATH
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
PyPI release unsloth 2026.6.3 is now live. Bump the pinned floor in
install.sh and install.ps1 from unsloth>=2026.6.2 to unsloth>=2026.6.3
so fresh installs resolve to the new wheel.
* Installer: harden GPU detection follow-ups after #6174
Ports the NVIDIA-priority and /proc/driver/nvidia/gpus hardening from #6174
to the remaining pathways and adds recovery for already-poisoned venvs:
- install_python_stack.py: add _ensure_cuda_torch so 'unsloth studio update'
force-reinstalls CUDA torch when the venv carries a ROCm build on an NVIDIA
Linux host (the pre-#6174 poisoning signature). Honors UNSLOTH_TORCH_BACKEND,
UNSLOTH_ROCM_TORCH_INSTALLED, and CUDA_VISIBLE_DEVICES=-1/'' opt-outs; never
touches healthy CUDA, deliberate CPU wheels, macOS, or Windows.
- install_llama_prebuilt.py: detect_host gains the /proc NVIDIA fallback and
skips ROCm probes when NVIDIA is usable; forwarded --rocm-gfx/--has-rocm
overrides still win.
- setup.sh: GPU summary classifies NVIDIA first through a timeout-bounded
probe with the /proc fallback; AMD probes are bounded and gain a KFD
vendor_id 4098 fallback; the llama.cpp source build only selects
GGML_CUDA/GGML_HIP when the matching GPU is actually detected.
- install.sh: bound both nvidia-smi calls with a 10s timeout (no behavior
change when healthy or when the timeout binary is absent); classify the
exported UNSLOTH_TORCH_BACKEND on the final index path segment so custom
mirrors containing 'rocm'/'gfx' in their base path are not mislabeled.
- install.ps1 + setup.ps1: NVIDIA probes now require a real 'GPU N:' row from
nvidia-smi -L under a 10s bound instead of bare exit code 0; later CUDA
version and compute_cap queries are bounded too.
Tests: 3 new test files (50+ tests), suite at 788 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Resolve-CudaToolkit driver probe for extracted-function unit test
tests/studio/test_resolve_cuda_toolkit.ps1 extracts Resolve-CudaToolkit alone
into a child pwsh and stubs nvidia-smi with a .ps1 script. The bounded runner
is not in scope there (and ProcessStartInfo cannot dispatch .ps1 stubs), so
the DriverMaxCuda parse silently returned nothing and the major-mismatch
scenarios failed. Fall back to direct invocation when Invoke-NvidiaSmiBounded
is unavailable; production setup.ps1 always has it defined and keeps the
10s bound.
* Treat CUDA_VISIBLE_DEVICES empty or -1 as hidden in NVIDIA-first guards
The NVIDIA-first guards added in this branch only special-cased
CUDA_VISIBLE_DEVICES=-1 at two setup.sh gates and ignored the empty-string
form entirely, while the Python detector (install_llama_prebuilt.py)
already treats both as hidden. On a mixed AMD+NVIDIA host steered to the
AMD card via CUDA_VISIBLE_DEVICES, the guards suppressed the AMD probes,
so setup.sh fell to a CPU llama.cpp build and install.sh picked CUDA
wheels instead of ROCm.
Move the policy into the helpers so every consumer agrees:
- install.sh: new _cvd_hides_nvidia checked first in _has_usable_nvidia_gpu
- studio/setup.sh: same via _setup_cvd_hides_nvidia; the two ad-hoc
CUDA_VISIBLE_DEVICES=-1 gate conditions are now redundant and removed
- studio/install_python_stack.py: _has_usable_nvidia_gpu returns False
when CUDA_VISIBLE_DEVICES is set to or -1 (whitespace tolerated)
Tests: 5 new sh scenarios (hidden via , -1, padded -1, visible device,
and mixed host with hidden NVIDIA restoring the ROCm route) plus a pytest
class covering all three implementations behaviourally.
Addresses the review comment on the NVIDIA-first setup.sh block.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Retrigger CI after PyPI 503 outage during the previous run
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: prevent ROCm torch from installing on NVIDIA Linux hosts
NVIDIA's open kernel module (driver 560+) registers GPU topology nodes in
the KFD sysfs hierarchy with non-zero gpu_id values. The _has_amd_rocm_gpu
(install.sh) and _has_rocm_gpu (install_python_stack.py) sysfs fallbacks
previously treated any non-zero gpu_id as proof of an AMD GPU, so an
NVIDIA-only host with the open kernel driver was misrouted to the ROCm
install path, replacing the correctly-installed CUDA torch with ROCm wheels.
Fixes:
1. install.sh _has_amd_rocm_gpu sysfs fallback: require vendor_id 4098
(AMD 0x1002) in the KFD node properties file before declaring an AMD
GPU present. NVIDIA KFD nodes carry vendor_id 4318 (0x10DE) and are
now skipped.
2. install_python_stack.py _has_rocm_gpu sysfs fallback: same vendor_id
guard. Also preserves the existing fallback for older kernels that
don't ship a properties file (trusts gpu_id alone there).
3. install.sh now exports UNSLOTH_TORCH_BACKEND ("cuda"/"rocm"/"cpu")
immediately after get_torch_index_url() resolves the wheel family.
install_python_stack.py reads this as _TORCH_BACKEND and short-circuits
_ensure_rocm_torch() entirely on cuda/cpu hosts, providing a second
layer of defense that is independent of subprocess GPU detection.
Tests: 9 new cases in TestHasRocmGpuKfdVendorGuard,
TestEnsureRocmTorch, and TestInstallShStructure cover all three changes.
Full test_rocm_support.py suite: 289 passed, 2 skipped, 0 failed.
Closes#6172
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: show actual torch backend in progress step labels
The 'ROCm torch check' and 'ROCm torch (final)' step labels were
hardcoded regardless of whether the installer was targeting CUDA, ROCm,
or CPU. On NVIDIA hosts they showed 'ROCm' even though no ROCm wheels
were being installed, which was misleading.
Add _torch_step_label(suffix) which reads UNSLOTH_TORCH_BACKEND (set by
install.sh) and formats the label as e.g. 'torch check (cuda)' or
'torch final (rocm)'. Falls back to live GPU detection for standalone
studio update runs that bypass install.sh.
* fix: make KFD sysfs vendor check conservative -- skip if no properties file
The previous implementation fell through to `return True` when the KFD
node's properties file was missing (OSError), intending to support older
kernels. But NVIDIA open driver KFD nodes can also lack a properties file
on some kernel versions, so the fallback still produced a false positive.
Change the `except OSError: pass` to `continue` so any node without a
readable properties file is skipped rather than trusted. KFD properties
files exist on every kernel version that actually exposes /sys/class/kfd,
so this does not regress real AMD GPU detection -- if the directory exists
at all, properties files will be present for genuine GPU nodes.
* fix: bulletproof NVIDIA vs AMD GPU detection
Four changes that together ensure ROCm torch can never be installed on an
NVIDIA host regardless of which detection path fires:
1. _has_rocm_gpu() (Python): NVIDIA guard at the top -- returns False
immediately when _has_usable_nvidia_gpu() is True, blocking rocminfo,
amd-smi, and KFD sysfs from producing a false positive even when ROCm
tools are co-installed alongside the NVIDIA driver.
2. _has_amd_rocm_gpu() (install.sh): same NVIDIA guard -- calls
_has_usable_nvidia_gpu first and returns 1 if it succeeds.
3. _has_usable_nvidia_gpu() (Python): adds /proc/driver/nvidia/gpus/
sysfs fallback. The NVIDIA driver populates this directory on Linux
regardless of nvidia-smi state, so a subprocess PATH gap, timeout, or
driver initialisation race can no longer silence NVIDIA detection.
4. _has_usable_nvidia_gpu() (install.sh): same /proc/driver/nvidia/gpus
fallback, tried after nvidia-smi -L rather than instead of it.
Together: NVIDIA wins at every decision point. If nvidia-smi works, it
confirms NVIDIA. If it fails, /proc/driver/nvidia confirms NVIDIA. If
somehow both fail, _has_rocm_gpu still checks NVIDIA first before any AMD
path runs.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: two KFD/proc-only corner cases from Codex review
1. KFD awk state not reset per node file (Ryzen+NVIDIA false positive):
The awk glob processes all topology node properties files in one pass.
Without FNR==1 reset, a Ryzen+NVIDIA host where an AMD CPU-agent node
sets amd=1 (vendor_id 4098, gpu_id 0) can combine with a later NVIDIA
node setting gpu=1 (gpu_id > 0), triggering found=1 before vendor_id
4318 is seen. Added FNR==1{ gpu=0; amd=0 } to reset per file.
2. proc-only NVIDIA not reaching CUDA wheel selection:
_has_usable_nvidia_gpu returning true via /proc/driver/nvidia fallback
left _smi empty, so get_torch_index_url entered the AMD/CPU branch and
selected CPU wheels despite NVIDIA being confirmed. Introduced
_nvidia_detected flag (separate from _smi) so the AMD branch is skipped
whenever NVIDIA is confirmed by any path, while _cuda_ver reads from
_smi when available (with the existing cu126 fallback when _smi is absent).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix Windows installer winget msstore certificate failure
`winget install` was invoked without `--source winget`, so winget also
queried the msstore source. When msstore fails certificate pinning
(error 0x8a15005e, "The server certificate did not match any of the
expected values") winget aborts and demands `--source`, so the Python
(and uv) install fails even though the package exists in the winget
source.
- Pass `--source winget` to all winget install calls (Python x2, uv).
Both packages live in the winget source, so this is strictly correct
and skips the failing msstore round-trip entirely.
- Add a python.org fallback (Install-PythonFromPythonOrg) that downloads
the official installer and runs it silently per-user (no admin/UAC)
when winget is unavailable or fails for any reason. Mirrors the
existing uv -> astral.sh fallback so Python installs without manual
steps. Resolves the latest 3.13.x from python.org with a pinned
fallback, and selects the amd64/arm64/x86 installer per architecture.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Pin remaining setup.ps1 winget calls to --source winget
Two winget invocations in studio/setup.ps1 still queried all sources and
could hit the same msstore certificate-pinning failure (0x8a15005e) that
broke the Python install in install.ps1:
- `winget show Nvidia.CUDA --versions` (CUDA Toolkit version probe)
- `winget install ... ShiningLight.OpenSSL.Dev` (OpenSSL dev for llama-server)
Every other winget call in this file already passes `--source winget`
(Git, CMake, VS Build Tools, CUDA install, Node.js, and setup.ps1's own
Python 3.12 install), so these two were stragglers. Both packages live in
the winget source; pinning it makes setup robust to an unhealthy msstore
source, matching the rest of the file.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stop amd-smi GPU probe from popping a DiskPart UAC prompt
On Windows, AMD GPU detection in install.ps1 and studio/setup.ps1 runs
`amd-smi list` / `static --asic` / `version`. amd-smi (shipped in
System32 by the Adrenalin driver) auto-elevates to read GPU/APU memory
details, surfacing a confusing DiskPart UAC prompt mid-install. The
Studio backend already documents and circuit-breaks on this in
studio/backend/utils/hardware/amd.py, but the installers did not.
Add an Invoke-AmdSmiNoElevate helper (both scripts) that runs amd-smi via
Start-Process under __COMPAT_LAYER=RunAsInvoker so it cannot auto-elevate
(no prompt), with a 30s timeout (matching amd.py) so a flaky amd-smi
cannot stall the install for minutes. On failure/timeout the existing WMI
name -> gfx fallback still resolves the arch, so detection is unchanged on
working hosts.
Verified on a Strix Halo (Radeon 8060S / gfx1151) box: the prompt is gone
and the probe is bounded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add experimental ROCm-on-WSL setup helper for Strix Halo (gfx1151)
install.sh already routes gfx1151 (Radeon 8060S / Strix Halo) to the
repo.amd.com/rocm/whl/gfx1151 wheels once a ROCm runtime is present, but
it does not install AMD's driver/ROCm stack -- a large, admin-gated
prerequisite. scripts/install_rocm_wsl_strixhalo.sh automates the Linux
side on a dedicated Ubuntu 24.04 WSL2 distro: ROCm 7.2 (wsl usecase), the
rocr4wsl HSA runtime, a librocdxg build, env setup, and a PyTorch gfx1151
GPU smoke test. A hard preflight refuses to run until the Adrenalin
>=26.3.1 driver is actually present, so it cannot half-install.
Procedure adapted from AMD's ROCm-on-WSL docs and community gfx1151 notes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Detect AMD GPUs by name so native Windows gets a GPU llama.cpp
The gfx-arch inference from the WMI GPU name was gated behind $HasROCm,
which the hipinfo/amd-smi probe leaves false on the common Windows case
(Adrenalin driver only, no HIP SDK -- and amd-smi often cannot read the
arch without elevation). So an AMD GPU was detected by name but never
mapped to a gfx target, --rocm-gfx was not forwarded, and studio setup
fell back to a CPU llama.cpp build.
Un-gate the inference (install.ps1 + studio/setup.ps1) so it runs whenever
an AMD GPU name is available. The inferred gfx is forwarded as --rocm-gfx,
which makes install_llama_prebuilt.py download the matching lemonade-sdk
ROCm prebuilt (e.g. llama-bNNNN-windows-rocm-gfx1151-x64.zip) -- a
GPU-accelerated llama.cpp that bundles its own ROCm runtime, so it runs
with just the Adrenalin driver. PyTorch's ROCm wheels still require a
confirmed HIP SDK ($HasROCm), so this only affects llama.cpp / inference
and never pulls broken ROCm torch.
Also broaden the name->arch table to every family lemonade ships Windows
assets for: gfx120X (RDNA 4), gfx110X (RDNA 3), gfx1151/gfx1150
(RDNA 3.5), and gfx103X (RDNA 2). Unknown names still fall back to CPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Suppress amd-smi DiskPart UAC prompt in the Python install/runtime paths
The earlier PowerShell guard covered install.ps1 / setup.ps1, but the
Python installer (install_llama_prebuilt.py detect_host,
install_python_stack.py ROCm probes) and the Studio backend monitor
(amd.py) also shell out to amd-smi on Windows, where it auto-elevates and
pops the same DiskPart UAC prompt mid-install / at runtime.
Inject __COMPAT_LAYER=RunAsInvoker into the amd-smi subprocess env on
Windows so it runs un-elevated (no prompt). Callers already tolerate an
empty/failed result and fall back to WMI / name detection (installer) or
the existing circuit breaker (amd.py). Gated to Windows so Linux/macOS
amd-smi behaviour is unchanged.
- install_llama_prebuilt.py: handled centrally in run_capture (covers
detect_host's `amd-smi list` and the version probe).
- install_python_stack.py: new _amd_smi_env() helper on its 3 raw
subprocess.run amd-smi calls.
- amd.py: merge RunAsInvoker into the existing child env.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Tighten AMD GPU name->arch patterns to avoid mismatches
The W9[0-9]{3} and RX 90[0-9]{2} patterns added for RDNA 4 were
speculative and over-broad: W9xxx would also match old GCN FirePro
W9100/W9000 cards (wrong gfx1201 -> a lemonade gfx120X download that
fails validation), and RX 90[0-9]{2} was redundant with the explicit
9070/9060 entries. Drop both; keep only confirmed RDNA 4 SKUs. Unmatched
AMD names still fall back cleanly to CPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fetch the llama.cpp validation model via huggingface_hub
The prebuilt validation downloads a tiny GGUF test model from huggingface
via bare urllib. On Windows / proxy setups where the server sends an
incomplete TLS chain, urllib cannot complete the Amazon CA chain (it does
no AIA intermediate fetching) and fails with CERTIFICATE_VERIFY_FAILED, so
a perfectly good GPU prebuilt is rejected and the installer falls back to a
CPU source build.
Route the validation-model download through huggingface_hub
(hf_hub_download) -- the same mechanism Studio uses for model downloads,
which completes the chain where urllib cannot -- keeping the direct URL as
a fallback. This lets the lemonade ROCm prebuilt validate and install on
cert-restricted machines (verified: hf_hub_download succeeds where urllib
returns CERTIFICATE_VERIFY_FAILED).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Guard the remaining raw amd-smi version probe via run_capture
A ROCm-version detector in install_llama_prebuilt.py called amd-smi version through a raw subprocess.run that bypassed run_capture's Windows RunAsInvoker guard, so it still triggered the DiskPart UAC prompt during setup. Route it through run_capture like the other amd-smi calls.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Forward --rocm-gfx even when the ROCm runtime is unconfirmed
setup.ps1 forwarded --rocm-gfx (and picked the windows-hip llama.cpp
prebuilt) only inside `if ($HasROCm)`. On Adrenalin-only hosts (amd-smi
present but no HIP SDK, so $HasROCm stays false) the gfx arch was
name-inferred but never forwarded, so install_llama_prebuilt.py saw
has_rocm=False and installed the CPU build -- even though the lemonade
gfx1151 GPU prebuilt runs fine there (it bundles its own ROCm runtime;
verified: llama-cli --list-devices -> ROCm0: AMD Radeon 8060S, 69 GB).
Forward --rocm-gfx whenever a gfx arch is known (it is authoritative and
implies ROCm in install_llama_prebuilt.py), and treat a known gfx arch as
windows-hip in the existing-install mismatch check. --has-rocm stays gated
on the confirmed-runtime signal.
Verified on Radeon 8060S / gfx1151: the installer now selects, validates,
and installs llama-b1286-windows-rocm-gfx1151-x64.zip (ROCm DLLs present)
instead of the CPU build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Install AMD ROCm PyTorch on name-inferred gfx hosts (enables Train/Export)
setup.ps1 picked the AMD ROCm PyTorch wheels only inside `if ($HasROCm ...)`.
On Adrenalin-only hosts (amd-smi present but no HIP SDK, so $HasROCm is
false) the gfx arch was name-inferred but the ROCm-wheel branch never ran,
so the host got torch+cpu. With CPU torch, torch.cuda.is_available() is
False, so the Studio backend sets CHAT_ONLY=True and hides Train/Export.
Un-gate the ROCm PyTorch index resolution on a known gfx arch (mirrors the
llama.cpp --rocm-gfx fix). AMD's per-arch Windows wheels
(repo.amd.com/rocm/whl/<gfx>) bundle the ROCm runtime, so they work without
a HIP SDK; a failed install still falls back to CPU.
Verified on Radeon 8060S / gfx1151: torch 2.11.0+rocm7.13.0 installs and
torch.cuda.is_available() -> True, device "AMD Radeon(TM) 8060S Graphics",
GPU matmul OK -> CHAT_ONLY=False -> Train/Export enabled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Force amd-smi un-elevated process-wide in the Python installers
Guarding individual amd-smi call sites kept missing some (install_python_stack.py's probe loop and its Windows GPU re-check), so the DiskPart UAC prompt kept reappearing. Set __COMPAT_LAYER=RunAsInvoker process-wide at the top of install_python_stack.py and install_llama_prebuilt.py on Windows so every amd-smi subprocess (current and future) runs un-elevated with no per-call guard. Safe: these scripts only spawn amd-smi/rocminfo/hipinfo probes and pip/uv. setup.ps1 keeps per-call guards because it also spawns winget installers that need elevation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix Invoke-AmdSmiNoElevate exit code on PS 5.1 + RX 7700S arch match
Start-Process -PassThru leaves the returned process object's .ExitCode
$null after WaitForExit on Windows PowerShell 5.1, so the helper set
$LASTEXITCODE to $null and every caller's `if ($LASTEXITCODE -eq 0 ...)`
was always false -- the amd-smi GPU / gfx-token / ROCm-version detection
branch was effectively dead (masked only because the un-gated WMI
name->gfx inference still ran). Reproduced on PS 5.1.26100.
Rewrite the helper to use [System.Diagnostics.Process]::Start with a
ProcessStartInfo (UseShellExecute=false), whose .ExitCode is reliable,
with async stream reads (ReadToEndAsync) to avoid a pipe-buffer deadlock
and WaitForExit(timeout) to bound a flaky amd-smi. __COMPAT_LAYER=
RunAsInvoker (inherited via the process env) still suppresses the
auto-elevation / DiskPart prompt. Also drops the temp files and the
empty-ArgumentList edge case. Verified: exit code propagates
(7 -> $LASTEXITCODE=7), output captured, env restored.
Also fix the gfx1100 name pattern `RX 7700(?! S)` -> `RX 7700(?!S)` so the
spaceless retail name "RX 7700S" is correctly excluded (it belongs to the
gfx1102 row). Both found by PR review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address PR review follow-ups (install.sh table, update path, tests, WSL)
From the multi-agent PR review:
- install.sh: sync the AMD name->arch table with install.ps1 / setup.ps1
(the bash table had drifted to the old narrow patterns). Adds RDNA 2
(gfx103X), workstation PRO W SKUs, and more Strix Halo/Point names, and
orders gfx1102 before gfx1100 so the spaceless retail name "RX 7700S"
resolves correctly (bash case has no negative lookahead). AMD-ROCm-only:
the name inference stays gated behind _has_amd_rocm_gpu(), so NVIDIA /
CPU / macOS are unaffected.
- setup.ps1: the "dependencies up to date" fast path skipped the torch
reinstall, so an existing user who had CPU torch (installed before
ROCm-wheel support) stayed stuck in CHAT_ONLY. Now, when an AMD gfx arch
is known AND the installed torch is CPU-only, don't skip -- force the
dependency pass so the ROCm wheels install.
- scripts/install_rocm_wsl_strixhalo.sh: resolve the real /opt/rocm dir
instead of hardcoding ROCM_VER for LD_LIBRARY_PATH / the librocdxg
symlink (breaks if amdgpu-install lays ROCm under a patch-version dir);
add a LIBROCDXG_REF pin knob and a "verified against" freshness header.
- tests/studio/install/test_pr5940_followups.py: cover _hf_resolve_url_parts,
_fetch_validation_model_bytes (hf path + urllib fallback), run_capture's
Windows-only amd-smi RunAsInvoker injection, and install.ps1 vs setup.ps1
name-table parity (catches future drift). 14 tests, all passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix DiskPart UAC prompt: skip amd-smi on Windows without a HIP SDK
On Windows, amd-smi re-initialises the ROCm runtime on every invocation
(even `amd-smi version`) and, on hosts without a working HIP runtime
(consumer APUs/dGPUs with only the Adrenalin driver), elevates a child
process at runtime -- popping a UAC/DiskPart prompt. amd-smi's own
manifest is asInvoker, so __COMPAT_LAYER=RunAsInvoker cannot suppress
that runtime elevation (verified: even `amd-smi version` hangs and
times out with RunAsInvoker set).
Replace the ineffective RunAsInvoker-only approach with a real gate:
only spawn amd-smi on Windows when a HIP SDK is detectable (hipinfo
present, so amd-smi runs un-elevated) or the user opts in with
UNSLOTH_ENABLE_AMD_SMI=1. The gfx arch is already resolved from WMI
name inference (forwarded via --rocm-gfx), so ROCm wheel + lemonade
llama.cpp selection is unaffected. Linux/macOS amd-smi never elevates
and is untouched (no regression). RunAsInvoker is kept as harmless
belt-and-suspenders for tools that DO use manifest elevation.
Applied consistently across:
- studio/backend/utils/hardware/amd.py (runtime GPU polling)
- install.ps1, studio/setup.ps1 (install-time detection)
- studio/install_llama_prebuilt.py (prebuilt arch probe + version)
- studio/install_python_stack.py (ROCm version + arch probe)
Verified live on AMD Radeon 8060S (gfx1151), native Windows: fresh
install detects the GPU, installs ROCm torch (torch.cuda.is_available()
True), launches Studio with no DiskPart prompt, and inference, tool
calling, web search, LoRA finetuning, and GGUF export all run on the GPU.
Tests: add 6 _amd_smi_allowed() gating tests + PowerShell-installer gate
assertions; update the three amd-smi monitoring tests to opt in (they
mock amd-smi as available). Full suite: 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: helpful WSL message when the GPU isn't exposed to ROCm
In WSL, an AMD GPU's ROCm-on-WSL runtime is only available with a recent
Adrenalin driver AND a distro AMD supports (currently Ubuntu 24.04). When
neither is in place, GPU detection (rocminfo/_has_amd_rocm_gpu) finds
nothing and we silently fall back to CPU.
Add an actionable hint in the CPU-fallback path, shown only on WSL and
only AFTER detection has already failed -- so it is forward-compatible:
the moment a driver/distro DOES expose the GPU (e.g. if AMD later adds
Ubuntu 26.04 support), detection succeeds and the hint never fires. The
message:
- notes a GPU is plumbed in (/dev/dxg) but no ROCm runtime is exposed,
- lists the two prerequisites (Adrenalin driver + Ubuntu 24.04),
- if the distro is not 24.04, says AMD may not support it yet,
- tells the user to `wsl --install Ubuntu-24.04` and re-run,
- links AMD's ROCm-on-WSL guide + the experimental Strix Halo helper.
Verified live: on Ubuntu-24.04 the hint shows (version-warning omitted)
and the CPU install completes; on Ubuntu-26.04 the extra "this distro may
not be supported" line appears and points to 24.04.
Also fix the experimental scripts/install_rocm_wsl_strixhalo.sh: AMD's
repo.radeon.com/amdgpu-install/ is indexed by unified installer version
(30.30, 31.30, ...), NOT ROCm version, so the hard-coded
amdgpu-install/7.2.0/ path 404'd. Scan the installer dirs newest-first
for a noble .deb matching the target ROCm major.minor (ROCm 7.2 ->
30.30.x/amdgpu-install_7.2.x), falling back to the newest available.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* WSL: fix shortcut collision + pin ROCm-on-WSL driver reqs from AMD docs
Two WSL-related fixes informed by AMD's official ROCm-on-WSL docs and
field reports for Strix Halo / Ryzen AI Max+ (Radeon 8060S, gfx1151):
1. Shortcut collision (real bug). install.sh's WSL branch wrote
"Unsloth Studio.lnk" to the SAME Desktop / Start Menu folder as the
native-Windows installer (install.ps1 New-StudioShortcuts). Running
install.sh in WSL therefore silently retargeted the native shortcut at
the WSL launcher (wt.exe -> wsl.exe), so the desktop/start-menu icon
stopped launching native GPU Studio. Now the WSL shortcut uses a
DISTINCT name -- "Unsloth Studio (WSL - <distro>).lnk" -- and fetches
the Unsloth .ico to %LOCALAPPDATA%\Unsloth Studio so it shows the
proper icon. Native and WSL shortcuts now coexist.
2. Precise ROCm-on-WSL prerequisites. Research (AMD radeon-ryzen WSL
compatibility matrix, gianni.rosagallina.com Feb-2026 guide,
ROCm/ROCm#4952/#5509/#6022) confirms WSL GPU on Strix Halo requires
AMD Adrenalin Edition >= 26.1.1 (26.2.2+ is the first production
ROCDXG/WSL release) + ROCm 7.2.1 + Ubuntu 24.04; an older driver does
not inject the ROCm/DXG runtime into /usr/lib/wsl/lib, so rocminfo sees
only the CPU. install.sh's WSL hint and the experimental
install_rocm_wsl_strixhalo.sh header/preflight now state the exact
driver version (was a guessed ">=26.3.1"), bump ROCM_VER to 7.2.1, link
AMD's radeon-ryzen docs, and document the known librocdxg caveat that
usable VRAM is currently capped at the .wslconfig memory setting.
bash -n clean; install test suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: hint when the AMD driver is too old for ROCm-on-WSL
Adds a detect-and-guide hook for the optional WSL-GPU path. An AMD GPU on
native Windows can also be used inside WSL2, but only with AMD Adrenalin
Edition >= 26.2.2 (the first production ROCDXG/WSL release). Native Windows
GPU works with any recent driver, so this is purely about enabling the WSL
path.
We intentionally do NOT auto-install the driver: AMD referrer-gates driver
downloads (scripted curl/Invoke-WebRequest are blocked) and does not publish
Adrenalin via winget, so no installer can reliably fetch it -- and silently
swapping a live display driver is risky. Instead we point the user at AMD's
official download page (one click), after which the existing WSL detection
lights up automatically.
- install.ps1: new Show-AmdWslDriverHint -- when an AMD GPU is present and the
installed driver predates the 26.2.2 release (DriverDate < 2026-02-01),
print a concise tip with the AMD download URL. Handles DriverDate as either
a CIM DateTime or a WMI string. Suppress with UNSLOTH_SKIP_AMD_DRIVER_HINT=1.
- install.sh (WSL hint): add the direct Adrenalin 26.2.2 download URL and note
that AMD downloads are referrer-gated (open in a browser).
Verified: hint fires on a Sept-2025 driver, auto-suppresses on >= 2026-02-01;
install.ps1 parses; install.sh bash -n clean; suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.ps1: refresh shell icon cache after creating the shortcut
After writing the Desktop / Start Menu .lnk, nudge Explorer to refresh
its icon (ie4uinit.exe -show). Without this, a stale icon cache can show
a blank shortcut icon until the next explorer restart -- most visible
when a shortcut of the same name was rewritten (e.g. a native install
followed by a WSL install, which previously shared the name; now they use
distinct names, but the cache nudge makes the icon appear immediately
regardless). Best-effort and wrapped in try/catch so it never fails the
install. The bundled unsloth.ico itself is valid (verified it renders).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* setup.ps1: don't silently CPU-build llama.cpp on an AMD GPU
For AMD, GPU acceleration comes from the lemonade ROCm prebuilt (it bundles
the ROCm runtime, no HIP SDK needed) and is the preferred/default path. The
source-build fallback is CPU-only -- a HIP/ROCm *source* build would need the
full HIP SDK + ROCm clang toolchain, which the prebuilt exists to avoid.
Previously, if an AMD-GPU host ever fell through to the source build (e.g. the
prebuilt could not be downloaded), it printed "building llama.cpp (CPU-only,
no NVIDIA GPU detected)" and quietly produced a CPU binary -- masking the lost
GPU acceleration. Now that case emits a loud [WARN] explaining the GPU prebuilt
is the AMD path and how to restore it (re-run / check network / set
UNSLOTH_LLAMA_RELEASE_TAG), so AMD never silently degrades to CPU.
No behavior change on the happy path: AMD still gets the GPU prebuilt (verified
on gfx1151: ggml-hip.dll bundled, ~80% GPU compute during inference). NVIDIA
(CUDA source build) and CPU-only hosts are unchanged.
setup.ps1 parses; install suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: remove shared llama.cpp build, kill lock-holders, match WSL shortcut
Three gaps found by running a real uninstall on a native-Windows + WSL host;
all fixes are scoped to Unsloth-owned paths and no-op on the other pathways
(env/custom-root, NVIDIA/AMD/CPU, Mac) so nothing else regresses.
uninstall.ps1:
- Remove the default-mode SHARED llama.cpp build + cache. setup.ps1 installs
them at ~/.unsloth/llama.cpp and ~/.unsloth/.cache -- SIBLINGS of studio,
not under it -- so deleting <studio> left hundreds of MB behind. Now removed
explicitly, then ~/.unsloth is dropped ONLY if empty (never nukes unrelated
content). No-op in env/custom mode (llama.cpp nests under the custom root,
removed already) and when absent. UNSLOTH_LLAMA_CPP_PATH (user-owned) is kept.
- New _StopProcessesLockingRoots: _StopStudioProcesses only matched the venv
unsloth/python/studio exe, so it missed (a) llama-server.exe under llama.cpp
and (b) an orphaned multiprocessing python fork that ran from the SYSTEM
python but loaded a venv DLL (bitsandbytes) -- on Windows an open DLL handle
blocks the directory delete, leaving a half-removed install. The new helper
kills any process whose image path OR loaded module is under a target root
(module scan scoped to python/unsloth/llama-server names; vendor-agnostic).
- _RemovePath now retries (transient post-kill handle release).
uninstall.sh:
- Remove the default-mode ~/.unsloth/llama.cpp + ~/.unsloth/.cache; rmdir
~/.unsloth only if empty.
- WSL Windows-side shortcut cleanup now matches by TARGET (any
"Unsloth Studio*.lnk" whose target launches wsl.exe), covering both the
legacy "Unsloth Studio.lnk" and the new "Unsloth Studio (WSL - <distro>).lnk"
-- and never removes a native-Windows shortcut (which launches wscript.exe).
uninstall.ps1 parses; uninstall.sh passes sh -n and bash -n.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.ps1: invalidate Win11 Start Menu tile cache after creating shortcut
The Start Menu shortcut kept showing a blank/generic icon even after the
Explorer icon-cache rebuild, because Windows 11's StartMenuExperienceHost
keeps its OWN pre-rendered tile-icon cache
(%LOCALAPPDATA%\Packages\Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy\
TempState\TileCache_*.bin + StartUnifiedTileModelCache.dat), separate from
Explorer's iconcache_*.db. ie4uinit and an explorer.exe restart do not touch
it, and they don't recycle the host -- so a rewritten same-name shortcut keeps
showing the first-rendered (often the generic wscript ">") tile until the host
restarts on its own.
Fix: after creating the shortcut, drop only the Start Menu RENDER caches
(TileCache_* + StartUnifiedTileModelCache.dat) and stop StartMenuExperienceHost
(Windows auto-relaunches it), so the tile re-resolves the real icon via the
shell image factory. start2.bin (the user's pinned layout) is deliberately
preserved. Guarded by Test-Path (Windows 10 has no such host -> skipped) and
wrapped in try/catch so it can never fail the install. Windows-only
(install.ps1); no effect on Linux/macOS/Studio.
Verified live: rendering the shortcut via IShellItemImageFactory::GetImage (the
API StartMenuExperienceHost uses) returns the Unsloth sloth icon, color-matched,
after this invalidation -- previously it returned the generic script tile.
install.ps1 parses; install suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ROCm-on-WSL for AMD Strix Halo (gfx1151): auto-setup + runtime enablement
Make Unsloth Studio set up ROCm-on-WSL automatically for AMD Strix Halo
(Radeon 8060S / gfx1151) and use the GPU at runtime, validated end-to-end
on a Ryzen AI Max+ PRO 395 (ROCm 7.2.1 + librocdxg + Adrenalin Apr-2026):
rocminfo enumerates gfx1151, torch.cuda True, ~85.8 GB UMA pool.
Every change is a strict no-op for all other configs (NVIDIA/CUDA,
discrete + native-Linux AMD ROCm, macOS/MLX, Windows, CPU-only, non-Strix
WSL) and can never abort the installer.
- scripts/install_rocm_wsl_strixhalo.sh: rewrite to the validated recipe.
Fixes that would have broken a working box: drop the /usr/lib/wsl/lib
preflight (a working ROCDXG host has only d3d12/dxcore there); remove the
obsolete rocr4wsl step (gone from the 7.2.1 repo; would hard-fail and also
rips out the standard hsa-rocr ROCDXG needs); dynamic librocdxg soname
(was hardcoded 1.1.0; build is 1.2.0); direct apt-repo install; Windows
SDK auto-discovery; persist env to /etc/profile.d + ~/.bashrc; idempotent.
- install.sh: _maybe_bootstrap_rocm_wsl auto-offers/runs the helper when it
detects a Strix Halo APU in WSL (/dev/dxg) with no ROCm runtime, then
loads the env so detection routes to the gfx1151 wheels. Fast-path when
already configured. Fix an inaccurate WSL hint line.
- studio/backend/main.py + worker.py: set HSA_ENABLE_DXG_DETECTION=1
in-process before torch (gated on /dev/dxg AND librocdxg.so), so the
worker uses the GPU even when launched outside a login shell. Mirrors the
existing BNB_ROCM_VERSION injection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: clean up ROCm-on-WSL artifacts + Start Menu tile cache
- uninstall.sh: remove the ROCm-on-WSL helper artifacts -- the librocdxg
build clone (~/.unsloth/librocdxg, which otherwise blocks the empty-dir
rmdir of ~/.unsloth), the throwaway smoke-test venv, the persisted env
(/etc/profile.d/unsloth-rocm-wsl.sh) and the ~/.bashrc block. The system
ROCm userspace is a shared prereq like CUDA and is kept by default;
UNSLOTH_UNINSTALL_ROCM=1 removes it too. No-ops on macOS / non-Strix Linux.
- uninstall.ps1: invalidate the Win11 Start Menu tile cache after removing
the shortcut so its tile disappears promptly (mirrors install.ps1),
preserving start2.bin.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: accurate AMD ROCm messaging (HIP SDK optional, not required)
The Windows installer printed "HIP SDK not found - GPU-accelerated training
unavailable" / "ROCm wheels require the HIP SDK" whenever the HIP SDK was
absent. That is misleading: for a detected AMD GPU arch (gfx1151 etc.),
setup.ps1 installs AMD's bundled-runtime ROCm PyTorch wheels (repo.amd.com)
which ship their own ROCm runtime and do NOT need the HIP SDK -- verified
end-to-end (torch 2.11.0+rocm7.13.0, cuda True, QLoRA training on GPU) on a
Radeon 8060S with no HIP SDK installed.
Gate the GPU-detection + rocm-step messages on a detected gfx arch: when one
is known, state that GPU PyTorch uses bundled-runtime wheels and the HIP SDK
is optional; only when the arch is unknown fall back to the HIP-SDK hint.
Behavior (torch routing) is unchanged; this is messaging only. No-op for
NVIDIA/CUDA, HIP-SDK-present, and CPU paths (they hit earlier branches).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: fix /opt/rocm data-loss + make WSL shortcut create/remove interop-robust
Two fixes from the 3-reviewer regression audit + live testing on a
systemd-enabled WSL distro (interop disabled):
F1 (data-loss, install_rocm_wsl_strixhalo.sh): the /opt/rocm symlink-repair
could force-delete a pre-existing REAL ROCm install. The guard only checked
that /opt/rocm is a real directory, not that it is the stray librocdxg stub.
Now it only touches /opt/rocm when it is NOT a real install (no bin/rocminfo,
bin/hipcc, or .info/version present), and MOVES it aside (rocm.unsloth-stub-bak)
instead of deleting it, so a wrong guess can never lose data.
WSL interop robustness (install.sh + uninstall.sh): both relied on
`command -v powershell.exe`, which is true even when WSL interop cannot EXECUTE
it (on systemd distros powershell.exe fails with "Exec format error"). Result:
the WSL shortcut silently failed to create (install) and to remove (uninstall).
- uninstall.sh: test that powershell.exe actually runs; if not, remove the
"Unsloth Studio (WSL...).lnk" files directly via drvfs (/mnt/<drive>), which
works without interop. The name is WSL-install-specific, so a native install's
"Unsloth Studio.lnk" is never touched.
- install.sh: when the shortcut cannot be created, warn with the manual launch
command + how to re-enable interop, instead of failing silently.
No behavior change on the interop-on path. The regression audit otherwise found
no regressions on Linux/Mac/Windows/CPU/NVIDIA install paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.sh: fast-path fully restores ROCm-on-WSL env when the drop-in is gone
Reinstall regression found by uninstall->reinstall testing: after a Studio
uninstall that removed /etc/profile.d/unsloth-rocm-wsl.sh but KEPT the shared
ROCm (the default), a non-login reinstall hit the bootstrap fast-path
(librocdxg present) and its else-branch only set HSA_ENABLE_DXG_DETECTION --
NOT PATH/LD_LIBRARY_PATH. So rocminfo was not on PATH, GPU detection failed,
and the installer fell back to CPU-only PyTorch.
Fix: when librocdxg is present but the env drop-in is missing, restore the
FULL env inline (HSA + TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL + PATH +
LD_LIBRARY_PATH) so rocminfo is found and detection routes to the GPU, and
recreate /etc/profile.d/unsloth-rocm-wsl.sh so future shells and the Studio
worker get it too. No change to the env-present fast-path or any other host.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: clear Explorer icon cache so shortcut icons aren't blank
Root cause of the persistent blank Desktop + Start Menu icons: Explorer caches
each shortcut's icon in iconcache_*.db and does NOT re-read the .ico when a
same-name .lnk is recreated across reinstalls. The .ico and .lnk are correct
(the shell renders them non-blank via IShellItemImageFactory; the .ico has real
image data at 16/32/48/128 px), but the stale cache entry wins. The previous
fix only ran a weak `ie4uinit -show` + the Start Menu tile-cache clear -- it
never invalidated Explorer's icon cache, so the desktop icon stayed blank.
Fix (native install.ps1 New-StudioShortcuts AND the WSL shortcut path in
install.sh):
- ie4uinit -ClearIconCache (thorough; replaces -show as the primary refresh)
- SHChangeNotify(SHCNE_ASSOCCHANGED) to force a live desktop/taskbar refresh
WITHOUT restarting explorer
- keep the Win11 Start Menu tile-cache invalidation (and add it to the WSL
shortcut path too, preserving start2.bin)
Non-disruptive (no explorer restart). install.ps1 parses clean; install.sh
passes bash -n + dash -n; the heredoc-generated WSL PowerShell parses clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: per-item SHChangeNotify(UPDATEITEM) reliably fixes blank icons
The blank Desktop/Start Menu shortcut icons are a stale Explorer PER-ITEM icon
cache: when a same-name .lnk is recreated across reinstalls, Explorer caches the
previously-resolved (often generic "white page") icon for that item and won't
re-extract the .ico on its own. The .ico and the .lnk's IconLocation are correct
(every icon API renders the sloth) -- only Explorer's cached display is stale.
The previous refresh (ie4uinit -ClearIconCache + a GLOBAL SHCNE_ASSOCCHANGED
broadcast) does NOT recover a stale item -- confirmed by reproduction. The
reliable, NON-disruptive fix (no explorer restart) is a PER-ITEM
SHChangeNotify(SHCNE_UPDATEITEM, SHCNF_PATHW, <lnk path>) for each created
shortcut, which forces Explorer to re-read that exact item's icon.
Verified end-to-end: deliberately staled a shortcut to the generic icon, ran the
installer's exact new refresh code, and the sloth icon recovered with NO explorer
restart (confirmed by capturing the live desktop via PrintWindow).
Applied to both native install.ps1 (New-StudioShortcuts) and the WSL shortcut
path in install.sh. Still clears the on-disk icon cache (ie4uinit) and the Win11
Start Menu tile cache (preserving start2.bin).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: remove leftover llama.cpp .staging root so ~/.unsloth is cleaned
The llama.cpp atomic-install staging root (install_llama_prebuilt.py
INSTALL_STAGING_ROOT_NAME=.staging) is a sibling of the llama.cpp install
dir (~/.unsloth/.staging in default mode). It is normally pruned after a
successful activate, but an interrupted or retained build can leave a
<name>.staging-XXXX tree behind. The uninstallers removed llama.cpp and
.cache but not .staging, so the final empty-dir cleanup of ~/.unsloth failed
and the directory lingered. Reproduced on WSL (Ubuntu-24.04) where an empty
llama.cpp.staging-XXXX dir kept ~/.unsloth alive after uninstall.
Remove ~/.unsloth/.staging in both uninstall.sh and uninstall.ps1. No-op in
env/custom mode (staging nests under the custom root removed already) and
when absent. Cross-platform fix (the staging logic is platform-agnostic).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: WSL-absent hint + fix here-string lint false positive
install.ps1: in the AMD WSL-ROCm driver hint, detect when wsl.exe is absent
and add a one-line "wsl --install -d Ubuntu-24.04" pointer so a Strix Halo
user with no WSL yet gets an actionable next step (the hint previously assumed
an Ubuntu-24.04 distro already existed). Best-effort, informational only.
test_rocm_support.py: test_no_here_strings did a crude substring check that
false-positived on the conda-style block marker
printf '# <<< Unsloth ROCm-on-WSL (gfx1151) <<<' -- a string literal written
into the /etc/profile.d drop-in, also used as a sed delimiter pair by
uninstall.sh, not a here-string. Strip quoted spans before the check so the
lint still catches a real here-string operator but ignores quoted literals.
install.sh remains POSIX-clean (sh -n / dash -n / bash -n all pass).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer: address PR review comments (gfx1150 mapping, amd-smi opt-out, WSL bootstrap, SDK path, make)
Apply the valid bot review findings on #5940; reject the ones that don't hold.
Fixed:
- AMD name->gfx table (setup.ps1 + install.ps1): Radeon 890M and Ryzen AI 9 HX
370/375 are Strix POINT (gfx1150), not Strix Halo (gfx1151). Move 890M / HX 37x
/ AI 9 HX to the gfx1150 row and drop the bogus HX 38x pattern (no such Strix
Halo SKU). Matches the runtime classifier in worker.py (890M/880M -> gfx1150;
8060S/8050S -> gfx1151). Prevents Strix Point hosts from getting the wrong ROCm
prebuilt/wheels.
- amd-smi opt-out (setup.ps1 + install.ps1): an explicit UNSLOTH_ENABLE_AMD_SMI=
0/false/no/off now wins over the HIP-SDK heuristic, so a host with a HIP SDK
binary but a broken runtime no longer gets the DiskPart/UAC prompt the opt-out
exists to avoid.
- amd-smi warning probes (install_python_stack.py): _has_rocm_gpu and
_detect_amd_gfx_codes now gate amd-smi behind _amd_smi_allowed() (and pass
_amd_smi_env()), closing the last unguarded amd-smi spawn on Windows.
- WSL ROCm bootstrap (install.sh): the "already-usable ROCm?" early return now
requires rocminfo to enumerate the real gfx1151 agent instead of the generic
_has_amd_rocm_gpu (whose broad gfx[1-9][0-9] match accepts a fallback
"gfx11-generic" ISA), so a Strix Halo box missing the ROCDXG bridge is no longer
skipped. The shared helper is untouched (no gfx90a regression).
- install_rocm_wsl_strixhalo.sh:
* Quote-safe Windows SDK discovery: the old for-in-$(ls -d "...Program Files
(x86)/...") word-split on the space and never matched; use find + read loop.
* Add `make` to apt prereqs (cmake only recommends it; minimal images lacked it
and the librocdxg `make -j` build failed).
* Verification requires gfx1151 exactly (not gfx1[0-9]) so a generic ISA or an
unrelated RDNA GPU can't pass while the real GPU is absent.
Reviewed but NOT changed:
- "Forward inferred ROCm arch without HasROCm" (setup.ps1): already correct --
--rocm-gfx is forwarded under `if ($script:ROCmGfxArch)`, not `if ($HasROCm)`.
- "Route inferred arch into install.ps1 torch path": not a bug -- install.ps1
installs CPU torch as a base by design and setup.ps1 swaps in the ROCm wheel for
the inferred arch (gate `($HasROCm -or $ROCmGfxArch) -and cpu`); verified live
the native install ends on torch 2.11.0+rocm7.13.0.
- "$p null guard after Start-Process" (install.ps1/setup.ps1): redundant -- the
amd-smi runner uses [Process]::Start wrapped in try/catch, so a null process
already returns "" with LASTEXITCODE=1 (no uncaught exception).
- "ls -> find for /usr/lib/wsl/lib" (gemini): stale -- that heuristic was removed;
only a comment about it remains.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer(rocm-wsl): auto-install the Windows 11 SDK via winget (fewer manual steps)
librocdxg's build needs the Windows SDK 'shared' headers on the Windows host.
Previously the helper just die()d with "install the Windows 11 SDK and re-run" if
they were missing -- a manual prerequisite that broke the otherwise-seamless
`curl ... install.sh | sh` one-liner on Strix Halo.
Now, when the headers aren't found, the helper installs the Windows 11 SDK on the
Windows host from inside WSL via winget (powershell.exe interop), then
re-discovers them. The SDK installer elevates -> ONE UAC prompt on the Windows
desktop; the headers appear under /mnt/c immediately (drvfs is live, no reboot).
The user already consented to the ROCm-on-WSL setup, so no extra prompt is added
beyond the OS UAC gate.
- New _find_win_sdk (space-safe find of the newest installed SDK 'shared' dir)
and _install_windows_sdk_via_winget helpers.
- winget IDs tried newest-stable first: Microsoft.WindowsSDK.10.0.26100, then
.22621. The presence of the headers (re-check) is the source of truth, not
winget's exit code. </dev/null so winget never consumes a piped `curl|sh` stdin.
- Best-effort + non-fatal: interop-off / no-winget / declined-UAC all fall
through to the existing clear manual-install die(). Opt out with
UNSLOTH_SKIP_WIN_SDK_INSTALL=1.
Removes the last avoidable manual step from the WSL Strix Halo path; only the AMD
Adrenalin driver (AMD referrer-gates the download) remains manual. Verified
_find_win_sdk resolves the spaced "Program Files (x86)" path; bash -n clean; all
winget flags validated against `winget install --help`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer(amd): gate install-time amd-smi probe to fix DiskPart UAC prompt
install_python_stack.py's Windows "AMD GPU detected but ROCm torch missing"
warning probe ran `amd-smi list` whenever amd-smi was on PATH -- and amd-smi
ships in C:\Windows\System32 with the AMD Adrenalin driver -- without the
_amd_smi_allowed() gate that every other amd-smi call site in the file uses.
On Adrenalin-only hosts (no HIP SDK) amd-smi elevates a child at runtime and
pops a UAC/DiskPart prompt that __COMPAT_LAYER=RunAsInvoker cannot suppress
(amd-smi's manifest is asInvoker). The probe also ran before the
ROCm-torch-installed check, so it fired on every Windows AMD install.
Gate it behind _amd_smi_allowed() and pass _amd_smi_env(), matching
_has_rocm_gpu()/_detect_amd_gfx_codes(). When skipped, the only loss is the
best-effort "AMD GPU detected" note on HIP-SDK-less hosts.
Adds a per-function AST regression test asserting every function in
install_python_stack.py that names the amd-smi command and spawns a subprocess
also references _amd_smi_allowed() (flags the pre-fix code; passes after).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* studio(cli): fix `unsloth studio stop` crashing on Windows
`stop` used the POSIX `os.kill(pid, 0)` liveness probe, but on Windows
CPython raises OSError (WinError 87, "The parameter is incorrect") for
*every* pid -- alive or dead. `stop` only catches ProcessLookupError /
PermissionError, so the OSError propagated and the command crashed with
a traceback before ever reaching its (correct) `taskkill /F` path.
Add a cross-platform `_pid_alive(pid)` helper (tasklist on Windows,
signal-0 elsewhere) and use it for both the pre-check and the post-kill
wait loop. The actual kill path is unchanged.
Verified on Windows (Python 3.13): os.kill(pid,0) raises WinError 87 for
both a live and a dead pid; `_pid_alive` returns True/False correctly and
the full stop() flow (alive -> taskkill -> dead -> "stopped") passes
end-to-end against a throwaway process.
Adds tests/studio/test_cli_studio_stop_windows.py (AST guard against a
bare os.kill(pid,0) liveness probe + mock-only _pid_alive behaviour for
the win32 tasklist branch and the POSIX signal-0 branch).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer(amd): fix install.sh name->arch table misrouting Strix Point to gfx1151
The bash name->arch inference table in install.sh placed Strix Point
identifiers (Radeon 890M, "Ryzen AI 9 HX 370/375", "AI 9 HX") in the
gfx1151 (Strix Halo) row, diverging from the install.ps1 / setup.ps1
PowerShell tables which correctly map them to gfx1150. It also carried a
stray "HX 38" token absent from the PowerShell source-of-truth.
Align install.sh with the PowerShell tables:
gfx1151 row: 8060S|8050S|8040S|Strix Halo|Ryzen AI Max|AI Max
gfx1150 row: 890M|880M|860M|840M|Strix Point|Krackan|HX 37|AI 9 HX|...
Impact is low (the bash table only feeds the display label _gpu_disp_gfx
and the "set UNSLOTH_ROCM_GFX_ARCH=..." hint; wheel selection is driven
by the detected ROCm version, not this name string) but a Strix Point
user would otherwise see/copy the wrong gfx arch.
Add a parity test (test_install_sh_name_arch_agrees_with_ps_for_strix_and_non_amd)
that parses install.sh's case table and asserts Strix Halo->gfx1151,
Strix Point->gfx1150, RX 7700S->gfx1102, and NVIDIA/Intel->no match,
cross-checking against install.ps1 (the previous parity test only
compared install.ps1 <-> setup.ps1, missing install.sh).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.ps1: keep prebuilt-llama ownership guard within the test's block window
The AMD additions to the prebuilt-llama.cpp block (the windows-hip vs
windows-cpu existing-install kind validation) pushed the
install_llama_prebuilt.py invocation to ~1999 chars after the
"installing prebuilt llama.cpp bundle (preferred path)" anchor, right at
the edge of the 2000-char window that
test_setup_ps1_prebuilt_llama_cpp_has_ownership_guard slices -- so the
helper string was truncated and the test failed with "substring not
found" (CI: Repo tests (CPU)).
The ownership-guard invariant (Assert-StudioOwnedOrAbsent precedes the
install_llama_prebuilt.py call) was already satisfied; only the proximity
to the anchor regressed. Move the "installing prebuilt..." substep to
immediately before the install (after the existing-install pre-cleanup),
which also reads better (validate/clean existing -> then "installing"),
shrinking anchor->helper from 1999 to 413 chars. Behaviour is unchanged
(console message ordering only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.sh: auto-run Strix Halo ROCm-on-WSL setup by default
`curl -fsSL https://unsloth.ai/install.sh | sh` should make a Strix Halo
(gfx1151) GPU usable inside WSL with no extra commands. Previously the
ROCm-on-WSL bootstrap was opt-in: it required UNSLOTH_ROCM_WSL_AUTO=1 or an
interactive [Y/n] at a TTY, and silently skipped under a pipe (no /dev/tty),
so the piped one-liner never set the GPU up automatically.
Flip it to auto-by-default for the single narrow case the existing guards
allow (WSL + Strix Halo + /dev/dxg + no usable ROCm yet) -- exactly the GPU
setup the user ran the installer for. Opt out with
UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The Tauri desktop app keeps its own consent UI
(only auto-runs when it passes UNSLOTH_ROCM_WSL_AUTO=1). All hardware/OS
guards are unchanged, so non-Strix / non-WSL / NVIDIA / native-Linux / macOS /
CPU paths are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* PR comments: condense to be succinct (comments/docstrings only)
Shorten the verbose explanatory comments and docstrings this PR added across
the installer, scripts, backend shims, CLI, and tests -- tighter, fewer lines,
while preserving every non-obvious "why" (os.kill WinError 87, amd-smi
RunAsInvoker/UAC, /dev/dxg + librocdxg gating, the ROCm-on-WSL bootstrap guard
chain, ownership guards, etc.). No executable code, string literals, messages,
or behavior changed.
Verified comments-only: docstring-normalized AST equality (Python, 9 files),
non-comment token equality (PowerShell, 3 files), comment-stripped diff +
sh -n / bash -n (shell, 3 files). Behavior re-confirmed: get_torch_index_url +
gfx name->arch table 44/44 under dash & bash; rocm_support / pr5940_followups /
cli_studio_stop tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Installer: address PR review (amd-smi opt-out, pipefail, multi-distro, non-root)
Fixes valid findings from the Codex/Gemini PR review:
- install.ps1 / setup.ps1: gate the `amd-smi version` ROCm-version fallback with
$amdSmiAllowed so UNSLOTH_ENABLE_AMD_SMI=0 opt-out is honored (the device
probe was gated but this fallback wasn't), avoiding the DiskPart/UAC prompt.
- install_rocm_wsl_strixhalo.sh: make the post-verification rocminfo summary
best-effort (|| true) so head's early pipe-close under `set -o pipefail` can't
fail the bootstrap after gfx1151 was already enumerated; pin the Windows SDK
`winget install` to --source winget (matches the msstore-cert fix rationale).
- install.ps1: python.org fallback installs the py launcher per-user
(InstallLauncherAllUsers=0, avoids admin), and derives the fallback full
version from the requested minor so a non-default UNSLOTH_PYTHON (e.g. 3.12)
isn't silently replaced with 3.13 when the listing is unreachable.
- install.sh: recreate /etc/profile.d/unsloth-rocm-wsl.sh via `sudo tee` for a
non-root reinstall (a plain redirect failed silently, dropping the ROCm env).
- uninstall.sh: scope WSL Windows-side shortcut removal to the current
WSL_DISTRO_NAME (per-distro name or -d "<distro>" arg) so uninstalling one
distro no longer deletes other distros' launchers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Studio ROCm Windows: fix field-reported issues from Strix Halo testers
Four fixes from PR #5940 field reports (Win11 native, gfx1151):
1. bitsandbytes arch-probe spam: bnb's get_rocm_gpu_arch() runs
hipinfo.exe via subprocess PATH at import; the AMD torch wheel ships
hipInfo.exe in the venv Scripts dir, which is only on PATH for
activated venvs. Every bnb import logged "Could not detect ROCm GPU
architecture: [WinError 2]" ERROR + WARNING (even with the HIP SDK
installed, whose bin dir is not on PATH either). Prepend the Scripts
dir to PATH before bnb imports in main.py, worker.py, and
install_python_stack.py, gated on the file existing (only AMD wheels
ship it). Verified on gfx1151: ROCM_GPU_ARCH now resolves to gfx1151
with zero errors.
2. OOM-guard double-tax on native Windows unified APUs: mem_get_info's
total is the WDDM budget the driver grants HIP (BIOS carve + ~half
of remaining RAM) -- the OS share is already outside it. The 0.80
unified cap on top denied loads that fit (field report: 48.49 GiB
budget -> "38.79 GiB allowed" OOM for a 47.29 GiB load with 48.08
free). Use 1.0 on win32 unified; Linux keeps 0.80, discrete 0.90.
3. "Missing VRAM" confusion: log the WDDM budget vs physical RAM with
the fix (BIOS UMA frame buffer / AMD Software Variable Graphics
Memory) when the grant is under 75% of RAM, so a 48 GiB cap on a
96 GiB box reads as policy, not a Studio bug.
4. llama-server fit-step crash (Qwen3.6-27B-MTP + mmproj, lemonade
gfx1151): --fit defaults to 'on' upstream, so the fit step runs even
when Studio already placed the model via -ngl -1, and aborts in
ggml-cuda.cu on some ROCm hosts. Retry the spawn once with --fit off
when the server crashes during startup and Studio's own VRAM math
had placed the model (never when use_fit or an explicit fit flag was
passed). Also keep the TAIL of crash output in the error log (the
diagnostic line prints last; head-truncation cut exactly that) and
reference the full on-disk log.
Verified live on Radeon 8060S: bnb import clean, Qwen3.5-4B-MTP loads
and generates through the new spawn loop, stub-crash retry appends
--fit off and recovers, fraction probes confirm WDDM overcommit and
sub-1.0-only enforcement on current AMD wheels.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio ROCm Windows: GPU-name fallbacks so nothing depends on amd-smi
amd-smi does not reliably exist on Windows: the HIP SDK never ships a
CLI, inbox Windows Update drivers do not, and only some full Adrenalin
packages drop amd-smi.exe into System32 (field report: fresh Win11 +
Adrenalin + HIP SDK, still no amd-smi anywhere). Make every consumer
work without it:
- install_python_stack._detect_windows_gfx_arch: two new probes after
hipinfo/amd-smi -- (2b) the venv Scripts hipInfo.exe shipped by AMD
torch wheels (drives `studio update` on driver-only hosts), and (4) a
last-resort GPU marketing-name -> gfx table via WMI
(Win32_VideoController), mirroring setup.ps1's $nameArchTable so a
standalone repair resolves the arch with zero AMD tooling installed.
- install_llama_prebuilt._resolve_exe: also probe the venv Scripts dir
so a standalone rerun finds hipInfo.exe without HIP_PATH.
- hardware/amd.py _run_amd_smi: which() guard before spawning --
absence now disables the poller in one step instead of burning the
3-strike circuit breaker on FileNotFoundError; corrected the stale
comment claiming Adrenalin ships amd-smi.
Simulated against the real detection functions on gfx1151: amd-smi
absent, present-but-crashing (exit 1), present-but-hanging (60s sleep
vs 5-10s probe timeouts), and hard opt-out -- all resolve gfx1151, no
exceptions, bounded time. Full adversarial install (broken amd-smi
stub first on PATH + UNSLOTH_ENABLE_AMD_SMI=1, fresh uninstall first):
exit 0, name-table arch inference, lemonade gfx1151 b1292 prebuilt,
torch 2.11.0+rocm7.13.0 cuda_avail=True on the 8060S, Studio boots
healthy and stops cleanly.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: per-attempt llama-server log names + amd-smi test portability
Found by cross-platform simulation of the --fit off retry (Windows +
Linux sandboxes, real load_model with stub servers):
- llama-server log filename now carries the spawn-attempt index. The
retry can respawn within the same epoch second; reusing the name
opened the same file with "w" and truncated the crash log the retry
warning had just pointed the user at (proven with a frozen
time.time: one file, crash evidence gone; with the suffix both
attempts keep their logs). Regression-pinned in
test_llama_cpp_wait_for_health.py.
- test_amd_primary_gpu_with_mock now mocks shutil.which alongside
subprocess.run: the amd-smi absence guard which()-checks before
spawning, so on hosts without a real amd-smi (Linux CI, driver-only
Windows) the subprocess mock was never reached and the test failed.
Surfaced by running the suite in a clean Linux sandbox.
Simulation coverage on both OSes: 67-case platform/edge matrix
(real shipped code blocks under win32/linux/darwin spoofs: OOM-guard
fractions + VGM-hint boundary, bnb PATH-prepend gates, retry
eligibility incl. equals-forms and decoy tokens, GPU-name table
adversarial set, WMI fallback without powershell, monitor absence
semantics), 6-scenario live retry matrix (crash-once/crash-always/
exit-zero/explicit-fit/hang/log-collision) against real llama-server
spawns on Windows and WSL (GPU success legs on the 8060S), and a
3-engine browser matrix (chromium/firefox/webkit) driving the live
backend's health + authed /v1 chat completion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: classify unified-memory via props.is_integrated first
Align the ROCm OOM-guard classifier with PR #5988's UMA gate: consult
hipDeviceProp_t.integrated (props.is_integrated) before the hardcoded
arch set. Strictly additive -- truthy upgrades to unified; 0/absent
falls through to the existing gfx1150/gfx1151 + device-name logic, so
wheels that omit or zero the field cannot downgrade the known APU set.
Extends correct unified-cap treatment to APUs outside that set (e.g.
gfx1103 Phoenix iGPUs) and keeps Studio's two unified-memory consumers
on one driver signal. Verified live on gfx1151 (is_integrated == 1 on
the AMD Windows wheel -> ('gfx1151', True) via the new path).
* AMD detection: probe rocminfo with HSA_ENABLE_DXG_DETECTION and sync setup.sh gfx table
Fleet validation on a Strix Halo WSL2 box showed the system rocminfo
(HSA 1.18, ROCm 7.2.1) only enumerates the GPU over /dev/dxg when
HSA_ENABLE_DXG_DETECTION=1, and that rocminfo can sit at /opt/rocm/bin
off PATH outside login shells. Detection probes that miss either of
these report no GPU on a working ROCDXG host and select the CPU build
even though the lemonade bundle offloads fine (95.7 tok/s measured vs
64.5 CPU on the same laptop). Seed the env (a no-op on bare metal) and
the PATH fallback in install.sh, studio/setup.sh, and the installer's
Linux rocm probe, mirroring what main.py/worker.py already do for the
runtime.
Also sync studio/setup.sh's name->gfx table with install.sh: 890M and
the HX 37/AI 9 HX SKUs are Strix Point (gfx1150, not gfx1151), RX 7700S
must match gfx1102 before the gfx1100 row, and the RDNA2/workstation
rows were missing. New parity test pins the two bash tables together so
they cannot drift again.
* Studio: persist server session logs + native-crash stacks to disk
Field report (Strix Halo, 96 GB UMA carve, WSL and native Windows):
"the studio just terminates without a warning". A native crash in the
GPU runtime kills the process with no Python traceback, and a desktop-
shortcut console closes before anything can be read. The server only
ever logged to the console, so there was nothing to send back.
run_server now tees stdout/stderr to
~/.unsloth/studio/logs/server/server-<ts>-pid<n>.log (console behavior
unchanged; file copy is best-effort), arms faulthandler at the same
file so access violations / SIGSEGV leave a stack trace on disk, and
exports PYTHONFAULTHANDLER=1 so training workers inherit crash dumps
on their captured stderr. Armed before `from main import app` so even
import-time failures leave evidence. Keeps the newest 20 session logs;
opt out with UNSLOTH_STUDIO_NO_FILE_LOG=1. Prints "Session log: <path>"
at startup so users know what to attach.
Verified on this box: a forced real segfault (faulthandler._sigsegv)
leaves the full session output plus "Fatal Python error: Segmentation
fault" and the thread stack in the file while the console shows
nothing; a normal server boot captures the startup banner and serves
health as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* AMD probe: honor a pre-set HSA_ENABLE_DXG_DETECTION value
Match the shell helpers, which use the parameter-default form: a user
who exports HSA_ENABLE_DXG_DETECTION=0 to deliberately hide the GPU
from DXG detection should not have the probe override it.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Fix macOS Apple Silicon installs that resolve torch against x86_64
On Apple Silicon, `uv venv --python 3.13` can reuse a cached x86_64
(Rosetta) CPython, often because uv itself is an x86_64 build. The
resulting venv reports macosx_*_x86_64 to the wheel resolver, but PyTorch
has shipped no macOS x86_64 wheels since 2.2.2, so the torch install fails
with "no wheels with a matching platform tag (macosx_..._x86_64)".
Two changes, both scoped to macOS arm64 and additive (no other install
path is affected):
- Create the venv with an arch-explicit `cpython-X.Y-macos-aarch64-none`
request on Apple Silicon (no --python override), so uv cannot fall back
to a cached x86_64 interpreter.
- Harden the existing x86_64 venv guard: when the venv python cannot be
executed (x86_64 binary on a Mac without Rosetta), the platform.machine()
probe returns empty and the recreate was silently skipped. Fall back to
reading the binary's Mach-O arch via lipo/file so migrated or
pre-existing x86_64 venvs are still recreated as arm64.
* Harden arm64 static-arch fallback: file -L and set -e safety
Address review feedback on the lipo/file fallback:
- uv symlinks the venv's bin/python to the base interpreter; plain `file`
reports the symlink ("symbolic link to ...") and the arch substring never
matches. Use `file -L` to dereference (lipo already follows the link).
- Append `|| true` so the command substitution cannot abort the installer
under set -e on a Mac that has neither lipo nor file.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Update Install Scripts
Add SPDX AGPL-3.0 headers to the installer scripts and let the piped web
installs take their common options from the environment.
- install.sh / install.ps1: read UNSLOTH_NO_TORCH (and UNSLOTH_PYTHON for
install.sh) so a piped install needs no positional flags. Flags and the
pipe forms still work; an explicit flag wins.
- Fix the UNSLOTH_STUDIO_HOME example so the variable sits after the pipe
and reaches sh instead of curl.
- Add SPDX headers to install.sh, install.ps1, the uninstall scripts, and
the MLX install scripts.
- Drop the internal test package names from the studio install comments.
* Mirror UNSLOTH_PYTHON env var to install.ps1
install.ps1 now reads UNSLOTH_PYTHON to pin the Python version, matching
install.sh, and lists all three env vars (UNSLOTH_NO_TORCH, UNSLOTH_PYTHON,
UNSLOTH_STUDIO_HOME) in the header examples. The requested version is
preferred during detection and used as the winget install target; behavior
is unchanged when the variable is unset.
On Apple Silicon, uv can create the venv from a cached x86_64 (Rosetta)
Python, so the venv reports x86_64 to wheel resolvers and the torch install
never resolves (the CPU index ships no macOS wheels). Detect an x86_64 venv
on an arm64 host and recreate it with an arch-explicit arm64 CPython,
re-inspecting before the existing 3.13.8 check so both invariants hold.
Skipped when --python is set; non-macOS, Intel, and healthy arm64 venvs are
unaffected.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Version downpairing for Radeon+ROCm PyTorch wheels
The radeon repo has often published the absolute highest version for torch, torchvision, and torchaudio independently without forming a proper "trio" of versions that matches the usual "law of five," which can result in a mismatched trio that fails the sanity check, forcing an unnecessary fallback to the standard ROCm index.
To prevent this issue, this commit:
- Refactors _pick_radeon_wheel to support optional version prefix filtering.
- Implements a two-pass resolution:
1. Identifies the highest available versions for all packages.
2. Calculates the lowest common denominator minor version.
3. Re-picks wheels to ensure torch/audio match and vision is +15.
- Prevents silent fallbacks when compatible sets exist but are not the absolute latest entries in the listing.
* Apply Codex + Gemini suggestions
From Codex:
When the computed target minor is not actually present for one of the packages, _pick_radeon_wheel returns non-zero and this assignment runs under the script's top-level set -e, so the installer exits immediately instead of reaching the existing fallback path. This can happen with gapped Radeon listings, e.g. latest torch/vision imply minor 11, latest torchaudio implies minor 10, but the listing only has a complete older 2.9 trio or no torch 2.10 wheel; the new downpair code then aborts on this line rather than warning and falling back.
From Gemini:
medium
The local keyword is not part of the POSIX shell standard. While many modern shells like bash, zsh, and dash support it, this script uses #!/bin/sh and explicitly aims for POSIX compliance (as noted in the comments around line 1671) to ensure portability across different environments like minimal Docker images, BSD, or BusyBox.
Since the _extract_version function is called within a command substitution subshell (e.g., _torch_ver=$(_extract_version ...)), the variables defined inside it are already isolated from the parent shell's environment. Therefore, local is redundant here and should be removed to maintain portability and consistency with the rest of the script.
* Better implementation of Codex's suggestion
When the first computed target minor is not present for a package, this now clears the wheel and the final check falls back immediately, even if the listing contains a complete older trio. For example, with latest torch/vision implying minor 11, latest torchaudio implying 10, but no torch 2.10/vision 0.25 wheels and a complete 2.9/0.24/2.9 set, the new || _torch_whl="" avoids the earlier abort but still never searches below minor 10, so Radeon installs are skipped despite a compatible set being available.
* Second attempt to better implement Codex's suggestion
When the latest package set is mismatched but the downpair loop finds a lower minor, this accepts any nonempty torch/vision/audio wheels for that minor without verifying the full public versions. In a listing such as torch 2.10.1 plus torchaudio 2.10.0 and torchvision 0.25.0, the new loop marks the trio compatible and installs it, even though the change is meant to avoid unsupported torch/audio mismatches; compare the versions after repicking before setting _radeon_versions_match=true.
* Fix regression from trying to implement Codex
* Apply suggestion from @gemini-code-assist[bot]
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Clear triton when accepting a downpaired set
From Codex:
When this loop downpairs torch/vision/audio to an older minor, _tri_whl remains the absolute newest triton selected before the loop. In the multi-generation Radeon listings this block is meant to handle, that can skip a newer torch generation but still install its newer triton wheel via the later install triton + PyTorch command, producing a mismatched triton/PyTorch set instead of the lower generation's matching triton. Re-pick or clear triton when accepting the downpaired trio.
* Remove trailing whitespace on blank lines for PR #5353
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* fix(studio): set HIP_VISIBLE_DEVICES in apply_gpu_ids for ROCm training workers
Training workers are spawned via multiprocessing spawn before detect_hardware()
runs, so IS_ROCM is still False. If the user never set HIP_VISIBLE_DEVICES in
their shell, _inherits_rocm_visibility is also False, leaving the worker with
only CUDA_VISIBLE_DEVICES set. On ROCm hosts the HIP runtime honors
HIP_VISIBLE_DEVICES over CUDA_VISIBLE_DEVICES, so the worker saw the full
device list and torch raised "no usable HIP accelerator" on some setups.
Fall back to probing torch.version.hip (a build-time attribute, safe to read
before GPU init) to detect ROCm when neither IS_ROCM nor inherited env vars
are available. Mirrors the existing fix in llama_cpp.py for llama-server
subprocess GPU pinning.
Fixes https://github.com/unslothai/unsloth/issues/5180
* test: tighten apply_gpu_ids ROCm fallback assertions
Replace loose OR chain with exact string matches, split into three
focused tests, and add a guard check for the try/except wrapper.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: detect ROCm unified memory (Strix Halo / AMD iGPU) via torch fallback
amd-smi on iGPUs with shared/unified memory (e.g. Radeon 8060S on Strix
Halo) reports only the dedicated VRAM slice (~512 MB) in its metric output,
so get_visible_gpu_utilization() was returning usable_gb ≈ 0.35 GB instead
of the full GTT pool (~128 GB). torch.cuda.mem_get_info() already surfaces
the correct unified-pool size.
Add _reconcile_rocm_unified_memory(): after amd-smi returns a valid result
on a ROCm device, cross-check each device's vram_total_gb against
torch.cuda.mem_get_info(). When torch reports a larger total, replace the
amd-smi VRAM fields in-place. No-op for discrete AMD GPUs where the two
sources agree.
Fixes: "Falling back to all visible GPUs -- model may not fit" on AMD iGPU
machines even when 100+ GB of unified memory is available.
* Apply unified-memory reconciliation in get_gpu_utilization too
The visible-GPU path was already corrected for AMD iGPUs with unified memory
(Strix Halo / Radeon 8060S), but get_gpu_utilization was still returning the
raw 512 MB amd-smi VRAM slice. Studio's /api/train/hardware endpoint and the
live GPU monitor read from this primary path, so users continued seeing the
wrong total even after auto_select_gpu_ids picked the right device.
Refactor to share the per-device correction:
* _apply_unified_memory_correction(metrics, torch_info) -- the actual
replacement logic, in-place on a single metrics dict.
* _reconcile_rocm_unified_memory(...) -- multi-device,
iterates utilization["devices"] (visible-GPU path).
* _reconcile_primary_rocm_unified_memory(...) -- single flat
metrics dict (primary-GPU path), uses parent_visible_spec to pick the
primary index, falls back to ordinal 0 when no visibility env is set.
get_gpu_utilization now calls the primary reconciler under IS_ROCM, so both
endpoints surface the real unified-memory pool on iGPUs while leaving
discrete AMD GPUs untouched (torch_total <= smi_total -> no replace).
* Use 'is not None' and log debug on torch.version.hip probe failures
Two small follow-ups to the apply_gpu_ids ROCm fallback:
1. Match detect_hardware()'s 'getattr(torch.version, "hip", None) is not None'
form so the entire codebase has one canonical 'this torch was built with
HIP' check. On every shipping torch wheel hip is either None or a non-empty
version string, so the new form agrees with the old bool() form on every
real install.
2. Log the probe failure at debug level instead of swallowing it silently.
The broad 'except Exception' is intentional (we never want apply_gpu_ids
to crash a worker over a probe), but the silent pass made it impossible
to tell whether the fallback was firing or being skipped.
* fix(studio): honour HIP_VISIBLE_DEVICES in _get_parent_visible_gpu_spec before IS_ROCM is set
When a user has HIP_VISIBLE_DEVICES set in their shell (e.g. "1" to select
GPU 1) but detect_hardware() has not yet run in the Studio parent process,
IS_ROCM is still False. _get_parent_visible_gpu_spec() was gated on IS_ROCM
so it fell through to CUDA_VISIBLE_DEVICES (unset), saw all physical GPUs,
and auto-selected index 0. apply_gpu_ids then overwrote HIP_VISIBLE_DEVICES
with "0", making the intended GPU invisible to ROCm torch in the worker,
which triggered the "no usable HIP accelerator" error (issue #5180).
Apply the same _inherits_rocm_visibility pattern already used in
apply_gpu_ids: check for HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES in the
environment regardless of IS_ROCM so the correct GPU index is preserved.
* fix(install): harden AMD ROCm GPU detection for multi-GPU and env-filtered setups
The previous rocminfo awk pattern could miss discrete GPUs on machines
where HIP_VISIBLE_DEVICES/ROCR_VISIBLE_DEVICES is used to mask an
integrated GPU — the env vars filter rocminfo output but may not
propagate into the install script subprocess, causing detection to
fail entirely.
Two changes:
- Tighten rocminfo pattern from /gfx[0-9]/ && !/gfx000/ to
/gfx[1-9][0-9]/ — simpler and correctly excludes the CPU agent
(gfx000) without a negative lookahead
- Add sysfs KFD topology fallback: reads
/sys/class/kfd/kfd/topology/nodes/*/gpu_id which is a kernel-level
view unaffected by HIP_VISIBLE_DEVICES or ROCR_VISIBLE_DEVICES
Fixes detection failure reported in Discord by Chains (gfx1201 + iGPU
machine where env var exclusion of the iGPU caused rocminfo to return
no usable device).
* Fix KFD sysfs awk fallback to read properties file
The fallback added by this PR reads /sys/class/kfd/kfd/topology/nodes/*/gpu_id
files but matches the literal token 'gpu_id' against their content. Those
files contain only a single decimal value (e.g. '0' for CPU agents, '50432'
for GPU agents), so the regex never matches and 'found' stays 0, making the
fallback a no-op on every host. The properties file in the same directory
contains key/value lines like 'gpu_id 50432' which is what the existing awk
pattern expects.
Reproduced with a synthetic sysfs layout: against gpu_id files awk exits 1;
against properties files awk exits 0 when any node reports gpu_id > 0.
* fix(setup.ps1): detect AMD ROCm GPU on Windows, bring to parity with setup.sh
setup.ps1 only checked nvidia-smi and fell straight to "gpu: none" on AMD
machines. setup.sh already probed rocminfo/amd-smi/hipconfig/hipinfo.
Add three-tier detection mirroring install_llama_prebuilt.py's detect_host():
1. hipinfo: gcnArchName in output confirms a real HIP GPU (not just SDK)
2. amd-smi list: "GPU: <digit>" data rows as fallback
3. WMI Win32_VideoController: last resort -- detects AMD GPU even without
HIP SDK, then guides user to install it rather than silently going CPU
Also corrects the "none" message to mention AMD ROCm alongside NVIDIA so
users with AMD hardware understand the requirement.
Fixes: rohit-style install where Strix Halo (Radeon 8060S) showed
"gpu: none" even with the HIP SDK present.
* fix(install.ps1): detect AMD ROCm GPU on Windows, bring to parity with setup.ps1
install.ps1 had the same nvidia-smi-only GPU detection as setup.ps1 before
the setup.ps1 fix. Applies the same three-tier AMD detection:
1. hipinfo: gcnArchName confirms real HIP GPU
2. amd-smi list: GPU data rows as fallback
3. WMI Win32_VideoController: detects AMD GPU without HIP SDK and guides
user to install it
Fixes: install.ps1 showing "gpu: none" while setup.ps1 correctly showed
"AMD GPU detected" on the same machine (reported by rohit, RX 7600 XT).
* fix(install.ps1): suppress 'No NVIDIA GPU detected' when AMD GPU is present
* feat: add Windows AMD ROCm PyTorch wheel installation
install_python_stack.py:
- Add _ROCM_WINDOWS_WHEEL_BASE and _ROCM_WINDOWS_RELEASES constants
pointing to AMD repo.radeon.com (ROCm 7.2 -> torch 2.9.1+rocm7.2.1)
- Extend _ensure_rocm_torch() with a Windows branch: detects ROCm via
_has_rocm_gpu() / _detect_rocm_version(), requires Python 3.12 (cp312
is the only ABI AMD publishes for Windows), installs the direct wheel
URL from repo.radeon.com
install.ps1:
- Capture ROCmVersion during AMD detection via hipconfig --version /
amd-smi version (needed for wheel URL selection)
- After Get-TorchIndexUrl, add an AMD wheel override block: when HasROCm
and Python 3.12 detected, set ROCmTorchWheelUrl to AMD wheel URL
- Expand torch install branch to handle ROCmTorchWheelUrl with
uv pip install --force-reinstall --no-cache-dir
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: also install torchvision and torchaudio from AMD Windows repo
AMD publishes matching torchvision-0.24.1+rocm7.2.1 and
torchaudio-2.9.1+rocm7.2.1 cp312 wheels at the same repo.radeon.com
release folder. Install all three in both install.ps1 and
install_python_stack.py Windows ROCm path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat: add ROCm 7.1.1 Windows wheel mapping
AMD uses a different version string for 7.1.1 wheels:
2.9.0+rocmsdk20251116 (date-tagged) instead of +rocm7.1.1.
Adds the 7.1.1 release folder to both install.ps1 and
install_python_stack.py so users with ROCm 7.1 get ROCm
torch instead of falling back to CPU.
* fix: install rocm_sdk_core and rocm_sdk_libraries_custom alongside torch
The AMD Windows torch wheels declare rocm[libraries]==<ver> as a hard
dependency. Without installing rocm_sdk_core and rocm_sdk_libraries_custom
from the same AMD release folder, uv cannot resolve the dependency and
fails with 'No solution found'. Include all 5 wheels in one install call.
* fix: expand ROCm wheel array to scalars for Invoke-InstallCommand
@array splatting inside a scriptblock only works when the native command
is prefixed with '&'. Invoke-InstallCommand uses '& $Command' to run the
block, so @ROCmAllWheelUrls was not being expanded. Extract to scalar
variables $rw0-$rw4 which are captured correctly by the closure.
* fix: use --no-deps for AMD Windows torch wheel install
uv's resolver looks up rocm[libraries]==0.1.dev0 on PyPI during
dependency resolution before downloading any wheels, and fails because
the package doesn't exist on PyPI. --no-deps skips resolution entirely
and installs all 5 AMD wheels directly. The GPU runtime dependency is
satisfied by the HIP SDK, not a Python package.
* fix: setup.ps1 and install_python_stack.py now install ROCm torch on Windows
setup.ps1 was always setting CuTag='cpu' for non-NVIDIA hosts and installing
cpu-only PyTorch, overwriting the ROCm torch installed by install.ps1.
Adds the same AMD wheel selection logic (ROCm version detection, Python 3.12
check, 5-wheel install with --no-deps) to setup.ps1's torch install block.
install_python_stack.py: remove IS_WINDOWS guard from _ensure_rocm_torch()
call site so the Windows path in _ensure_rocm_torch() is reachable during
'unsloth studio update' as well.
* fix: suppress manual-install warning when ROCm torch already present; fix progress counter
- Gate the 'must be installed manually' warning on torch.version.hip being empty
so it doesn't fire when our ROCm torch install succeeded
- Update _TOTAL counter to include the 3 ROCm steps on Windows now that
_ensure_rocm_torch() is called there (fixes 10/9 display)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat: add rocm step display in setup.ps1; fix warning and progress counter
- Add 'rocm' step after 'cuda' in setup.ps1 showing ROCm version or HIP SDK missing
- Move ROCm version detection up to GPU detection block so it's available early
- Suppress 'must be installed manually' warning when torch.version.hip is set
- Fix _TOTAL counter to include ROCm steps on Windows (fixes 10/9 display)
* fix: detect AMD SDK ROCm torch via __version__ when torch.version.hip is unset
AMD's repo.radeon.com wheels (e.g. 2.9.0+rocmsdk20251116) do not set
torch.version.hip, leaving it None. All three probes that relied solely on
torch.version.hip now also check for 'rocm' in torch.__version__.lower():
- hardware.py detect_hardware(): IS_ROCM was never set, causing the studio
to report 'Hardware detected: CPU' even after AMD wheels were installed
and HIP DLLs were on PATH.
- install_python_stack.py _ensure_rocm_torch(): skip-if-already-installed
probe would always reinstall on subsequent runs.
- install_python_stack.py Windows AMD warning: suppression check always
failed, so the 'must be installed manually' note kept appearing after
a successful AMD wheel install.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* perf: drop --no-cache-dir from AMD ROCm torch wheel installs
uv caches downloaded wheels by default; passing --no-cache-dir forced a
full redownload of the ~2 GB torch wheel on every install run. CUDA installs
never had this flag -- AMD was the only path affected.
* fix: use install-state flag instead of subprocess probe for AMD Windows warning
Replace the subprocess torch probe in the post-install warning block with a
module-level _rocm_windows_torch_installed flag set by _ensure_rocm_torch().
Subprocess re-import of torch is unnecessary and fragile -- the install
function already knows whether it succeeded.
* fix: hoist global declaration to top of _ensure_rocm_torch
Python requires the global statement to appear before any assignment
to the variable within a function. Moving it to the function top fixes
the SyntaxError on line 354.
* fix: pass AMD torch install status via env var to suppress false warning
setup.ps1 now sets UNSLOTH_ROCM_TORCH_INSTALLED=1 after a successful AMD
wheel install. install_python_stack.py reads this at the top of
_ensure_rocm_torch() to skip both the subprocess probe and the warning --
no re-import of torch needed, and the warning message now correctly says
'could not be auto-installed' rather than 'must be installed manually'.
* fix: register ROCm DLL directory before torch import on Windows
Python 3.8+ ignores PATH for extension DLL loading on Windows; amdhip64.dll
and other HIP runtime DLLs must be registered via os.add_dll_directory().
Without this, torch.cuda.is_available() always returns False on AMD ROCm
Windows even when HIP_PATH is correctly set in system environment variables.
Reads HIP_PATH / ROCM_PATH env vars first, then falls back to scanning
common ROCm install roots (C:\Program Files\AMD\ROCm, F:\ROCm, C:\ROCm).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: remove hardcoded non-standard ROCm paths from DLL directory scan
Only use HIP_PATH/ROCM_PATH (set by AMD installer) and the standard
C:\Program Files\AMD\ROCm\<version>\bin location. Custom drive paths
like F:\ROCm are user-specific and should not be hardcoded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: prevent torchao overrides step from overwriting AMD ROCm torch
torchao==0.14.0 in overrides.txt declares torch as a dependency. Without
--no-deps, uv resolves torch from PyPI and installs 2.11.0+cpu on top of
the AMD ROCm wheels (2.9.0+rocmsdk20251116). This was the root cause of
'Hardware detected: CPU' -- the AMD wheels were installed but then
immediately overwritten by the overrides step.
When _rocm_windows_torch_installed is True, add --no-deps to the overrides
pip_install call so torchao is installed without pulling in CPU torch.
* fix: add rocm_sdk namespace tarball to Windows ROCm wheel installs
torch/_rocm_init.py calls `import rocm_sdk` at startup, which requires
the rocm namespace tarball (rocm-*.tar.gz) in addition to the SDK wheel
packages. This tarball was missing from both install.ps1 and setup.ps1,
causing ModuleNotFoundError on first torch import.
- Add rocm-0.1.dev0.tar.gz to ROCm 7.1.1 install (provides rocm_sdk namespace)
- Add rocm-7.2.1.tar.gz + rocm_sdk_devel to ROCm 7.2.1 install
- Install tarball in a dedicated step before main SDK/torch wheels
- Switch to @array splatting in install.ps1 scriptblock for dynamic wheel count
- Remove --no-cache-dir from Python-side ROCm wheel install (prevents ~2GB redownload)
* feat: enable ROCm 7.2 torch install + warn on gfx1151 with ROCm < 7.2
Chigoma333 (AMD Radeon 8060S / gfx1151, Strix Halo) confirmed that ROCm
7.1 segfaults when tensors are moved to GPU, but ROCm 7.2 + torch
2.11.0+rocm7.2 works fully including training.
Changes:
- Uncomment (7,2): "rocm7.2" in _ROCM_TORCH_INDEX (was blocked by <2.11.0)
- Add _ROCM_TORCH_PKG_SPECS dict with per-tag version bounds:
rocm7.2 → torch>=2.11.0,<2.12.0; all older tags → <2.11.0
- Add _detect_amd_gfx_codes() helper that parses rocminfo output
- Warn on gfx1151/gfx1150 (Strix Halo) when ROCm < 7.2 is installed,
pointing users at the known segfault and recommending upgrade
- install.sh get_torch_index_url(): enable rocm7.2 case (previously capped
to rocm7.1), cap unknown future tags to rocm7.2
- install.sh: override TORCH_CONSTRAINT to >=2.11.0,<2.12.0 when rocm7.2
index is selected, so pip can actually resolve torch 2.11.0
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: prefer Python 3.12 for AMD ROCm users when 3.13 is also installed
After GPU detection, if ROCm HIP SDK is found and the selected Python
is not 3.12, run a second pass to locate a 3.12 install via py.exe and
PATH (catches uv-managed installs). Switch $DetectedPython to 3.12 so
the venv is created with a compatible interpreter for the cp312-only AMD
Windows torch wheels.
NVIDIA and Intel GPU paths are unaffected -- the re-detection block only
runs when $HasROCm is true.
Fixes: #5301
* fix: also check uv-managed Python 3.12 for AMD ROCm #5301
* fix: hide amd-smi console popups on Windows, guard torch.distributed.is_initialized for ROCm #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: suppress remaining console popups on Windows, patch torch.distributed.is_initialized for ROCm #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: stub all missing torch.distributed attrs for ROCm Windows wheel #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: inject torch.distributed stub when C backend missing in ROCm Windows wheel #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm/windows): pre-stub torch._C._distributed_c10d + raise amd-smi timeout
Two fixes for Windows ROCm regressions reported by electroglyph on #5301:
1. worker.py — torch.distributed stub now fires unconditionally on Windows
The previous stub only injected sys.modules in the except branch, meaning
it was silently skipped when `import torch.distributed` happened to succeed
(the C backend is lazily resolved). The crash then hit later when
transformers/trl triggered the lazy load. Fix: on win32 we pre-populate
sys.modules['torch._C._distributed_c10d'] AND set the attribute on the
torch._C extension module *before* attempting the import, covering both
the early-ImportError and lazy-load failure modes.
2. amd.py — increase amd-smi timeout from 5 s to 30 s on Windows (10 s Linux)
amd-smi on Windows must cold-init the ROCm runtime on first invocation;
5 s was consistently too short, producing repeated 'Command timed out'
warnings in the server log. 30 s gives enough headroom without blocking
indefinitely on broken installs.
3. install.ps1 — widen Python 3.12 enforcement to ROCmGpuLabel (WMI-only path)
Users whose HIP SDK is not on PATH were detected via WMI but not switched
to Python 3.12 before the install started, causing a second pass. Guard
now fires on (HasROCm -or ROCmGpuLabel).
* fix(rocm): guard c10d stub, fix TorchIndexFamily for 7.1, clean dead code + comments
- worker.py: wrap c10d stub injection in `if _c10d_key not in sys.modules` so
Windows NVIDIA users with a real torch.distributed are never affected
- install.ps1: fix Get-TauriTorchIndexFamily receiving hardcoded "rocm7.2"
even when ROCm 7.1 wheels are installed; now branches on $ROCmVersion
- main.py: remove dead `import ctypes as _ctypes` (ctypes is never called)
- hardware.py, install_python_stack.py, worker.py, install.ps1: shorten
verbose multi-line comment blocks throughout
- tests: update 4 stale assertions that expected rocm7.2 to be absent/capped
* fix(tests): match windows AMD warning assertion to actual source string
* chore: trim verbose comment blocks across all ROCm-related files
* fix: guard reconcile call against None numeric_ids; add torchvision lower bounds
* fix(install.ps1): recreate venv with Python 3.12 after ROCm switch
Venv was created with 3.13 before GPU detection ran; switching
$DetectedPython to 3.12 had no effect since $VenvPython still
pointed to the 3.13 interpreter inside the already-created venv.
* ux: detect AMD GPU before Python selection to avoid double venv creation
- Early hipinfo + WMI probe runs before Find-CompatiblePython so Python
3.12 is selected upfront when AMD is detected; venv is now created
exactly once instead of 3.13 then immediately 3.12.
- Post-venv recreation block replaced with a simple warning for the rare
case where AMD was missed by the early probe.
- setup.ps1: show venv's actual Python version (e.g. 3.12) instead of
the system Python found by the pre-activation search (was showing 3.13).
* fix(rocm/win): auto-stub all _distributed_c10d symbols via PEP-562 __getattr__
The bare ModuleType stub caused ImportError when torch._dynamo was imported
(triggered by trainer.py accessing torch._dynamo.config at load time).
torch._dynamo pulls in torch.distributed.fsdp._flat_param which does:
from torch._C._distributed_c10d import FakeProcessGroup
and potentially other symbols. Adding module __getattr__ auto-creates a
stub class for any missing symbol so all such imports succeed without
enumerating every individual symbol. Applied to both the primary stub
and the fallback stub in the except branch.
* chore: trim c10d stub comment
* fix(rocm/win): auto-stub missing torch.distributed attrs (Store, ProcessGroup, …)
* fix(rocm/win): pre-stub fsdp submodules in sys.modules; fix __getattr__ subpackage clash
* feat(rocm/win): arch-aware wheel selector always picks newest ROCm release
Replace HIP-SDK-version-gated wheel selection with GPU arch-based logic.
Select-ROCmWheelRelease (PS) and _select_windows_rocm_release (Python) map
gcnArchName → minimum ROCm version, then pick the newest available release
that satisfies it (currently always rocm-rel-7.2.1 for any supported GPU).
Wheels bundle their own ROCm runtime so the installed HIP SDK 7.1 does not
prevent using 7.2.1 wheels on gfx1200 (RX 9060 XT) and similar RDNA 4 GPUs.
Also installs the bitsandbytes Windows ROCm continuous-release wheel and sets
BNB_ROCM_VERSION=72 in worker.py before ML imports so bnb loads the
libbitsandbytes_rocm72.dll that ships in that wheel.
* fix(rocm/win): stub class metaclass for ProcessGroup.BackendType; amd-smi circuit breaker
torchao.float8.inference accesses ProcessGroup.BackendType as a class-level
attribute. Plain type() stubs have no __getattr__ on the metaclass so this
raises AttributeError. Introduce _StubClassMeta whose __getattr__ returns
child stub classes, fixing the torchao import chain.
Add an amd-smi circuit breaker in amd.py: after 3 consecutive failures the
module stops spawning the process, eliminating the repeated Windows UAC /
DiskPart elevation prompts caused by polling a non-functional amd-smi.
Also guard BNB_ROCM_VERSION=72 behind a DLL existence check so bitsandbytes
fails with its own detection message rather than a harder "DLL not found" when
the Windows ROCm bnb wheel is not yet installed.
* fix: stub __members__ so torchao float8 enum check doesn't crash on ROCm Windows
torchao.float8.inference accesses ProcessGroup.BackendType.__members__
expecting a Python Enum registry dict. _StubClassMeta.__getattr__ was
blocking all dunder attributes, causing AttributeError. Return {} for
__members__ specifically so the isinstance/iteration checks pass cleanly.
* fix: stub distributed tensor/functional_collectives to prevent missing C++ op crash on ROCm Windows
torch._dynamo.trace_rules eagerly loads torch.distributed.tensor at import
time, which pulls in _functional_collectives.py. That file registers Meta
kernels for _c10d_functional C++ ops, but those ops are only registered
by torch._C._distributed_c10d — a C extension absent from ROCm Windows
wheels. Pre-stubbing the affected modules in sys.modules prevents the real
import chain from running and avoids the "operator does not exist" crash.
* fix: give mod stubs __path__ and pre-stub _tensor to fix 'not a package' import error
_make_mod_stub now sets __path__=[] so Python treats stub modules as
packages. Without it, any import of a submodule raises "is not a package".
Also pre-stub torch.distributed._tensor and its submodules so that
_tensor/__init__.py (which re-exports from torch.distributed.tensor) never
runs and torchao's `from torch.distributed._tensor import DTensor` gets a
harmless stub instead of crashing.
* fix: stub torch.ops._c10d_functional namespace with hashable op sentinels
torchao.dtypes.nf4tensor uses _c10d_functional ops as dict keys at import
time (all_gather_into_tensor.default, wait_tensor.default) and
torch.ops.c10d.scatter_.default. None of these ops are registered on ROCm
Windows because torch._C._distributed_c10d (the C extension) doesn't ship.
Replace the whole _c10d_functional namespace with a custom stub whose ops
return hashable .default objects, so dict-key construction doesn't crash.
Also inject a scatter_ stub into torch.ops.c10d if it's missing.
* fix: stub entire torchao package on ROCm Windows instead of individual ops
torchao is not supported on ROCm Windows and its import chain transitively
requires torch._C._distributed_c10d (absent from the ROCm Windows wheel).
Rather than stub each missing op one by one, stub the whole torchao package
upfront. Unsloth uses bitsandbytes for quantization, not torchao, so this
has no functional impact. transformers gracefully handles an importable-but-
empty torchao by disabling TorchAoHfQuantizer.
* fix: set __spec__ on mod stubs so importlib.util.find_spec doesn't raise
Manually-injected sys.modules entries have __spec__=None by default.
importlib.util.find_spec() raises ValueError when it finds a module in
sys.modules with __spec__=None (transformers.utils.import_utils hits this
when checking if torchao is available). Give every stub a minimal
ModuleSpec(name, loader=None, is_package=True) to satisfy find_spec.
* fix: add meta path finder to auto-stub subpackages of stub modules
`import torchao.prototype` goes through the import machinery, not
__getattr__, so an empty __path__ means ModuleNotFoundError. Rather than
list every submodule explicitly, register a MetaPathFinder that intercepts
any import whose parent is one of our stubs (detected by loader=None in the
parent's ModuleSpec). Real installed packages always have a SourceFileLoader
so they are never intercepted. Also register child stubs in sys.modules
from __getattr__ as a belt-and-suspenders measure.
* fix: use _unsloth_stub sentinel instead of loader=None for stub detection
The import machinery overwrites module.__spec__ with the spec returned by
find_spec (which has loader=_StubSubpackageLoader, not None), so the
loader=None check broke for second-level subpackages. Switch to a custom
_unsloth_stub object identity sentinel set directly on each stub module --
it survives __spec__ being replaced and correctly identifies stubs at any
depth (torchao.prototype.safetensors, etc.).
* refactor(rocm/win): switch to repo.amd.com arch-aware index, remove stubs
AMD recommends repo.amd.com/rocm/whl/{arch}/ as the Windows ROCm wheel
source. These wheels bundle their own ROCm runtime, support all Python
versions (not just cp312), and include the full torch._C extension set
(including _distributed_c10d) that the old repo.radeon.com wheel omitted.
Changes:
- install.ps1: remove Select-ROCmWheelRelease + hardcoded cp312 wheel
URLs; remove Python 3.12 forced-preference logic; install via
--index-url repo.amd.com/rocm/whl/{arch-family}/
- studio/setup.ps1: same -- remove Select-ROCmWheelRelease, switch to
repo.amd.com arch-aware index URL
- studio/install_python_stack.py: replace _ROCM_WINDOWS_RELEASES /
_select_windows_rocm_release with _windows_rocm_index_url() using the
_GFX_TO_AMD_INDEX_ARCH map; drop Python 3.12 restriction
- studio/backend/core/training/worker.py: remove all stub machinery
(_make_mod_stub, _StubSubpackageFinder, _StubSubpackageLoader,
_StubClassMeta, torchao/fsdp/dtensor stubs, _c10d_functional ops
stubs, BNB DLL detection) -- no longer needed with new wheel source
* fix(rocm/win): restore _distributed_c10d + torchao stubs; fix BNB install
repo.amd.com torch wheels also omit torch._C._distributed_c10d on Windows
(RCCL is not shipped on Windows). torch/distributed/__init__.py imports
from it unconditionally at module level, so the stub must land in
sys.modules before any torch.distributed import.
torchao (pulled in by transformers.quantizers) walks
torchao.float8.distributed_utils -> torch.distributed._functional_collectives
-> distributed_c10d at import time. Stubbing torchao up-front short-circuits
that chain.
worker.py:
- Restore _make_mod_stub / _StubSubpackageFinder / _StubSubpackageLoader
- Restore _StubClassMeta for ProcessGroup.BackendType attribute access
- Restore _distributed_c10d stub with __getattr__ (Windows only)
- Restore torchao stubs (5 modules, Windows only)
install_python_stack.py:
- BNB AMD wheel install was inside the early-return branch that fires when
torch is already a ROCm build (installed by install.ps1). Move BNB install
outside that branch so it always runs on Windows ROCm — the PyPI
bitsandbytes has only CUDA DLLs and fails to load on ROCm.
* worker: remove _distributed_c10d stub; stub only torchao
The installed torch/distributed/__init__.py from repo.amd.com
(torch==2.10.0+rocm7.12.0) is now properly guarded with
`if is_available():`, so `import torch.distributed` alone is safe.
The crash only comes via torchao's import chain:
torchao.float8.distributed_utils
→ torch.distributed._functional_collectives (unguarded import)
→ torch.distributed.distributed_c10d
→ torch._C._distributed_c10d ← absent on Windows ROCm
Stubbing torchao short-circuits the chain entirely. No need to stub
_distributed_c10d. Remove _StubClassMeta and the _c10d stub block;
keep only _make_mod_stub + _StubSubpackageFinder + torchao seeds.
* fix: BNB AMD wheel skipped + torch.compile segfault on Windows ROCm
install_python_stack.py: the UNSLOTH_ROCM_TORCH_INSTALLED=1 early-return
path (set by setup.ps1 when it installed torch itself) returned before
ever reaching the AMD BNB prerelease wheel install. The PyPI
bitsandbytes==0.49.x ships only CUDA DLLs, so loading it on ROCm fails
with "libbitsandbytes_rocm72.dll not found". Now installs the AMD
Windows BNB wheel before returning on that path too.
worker.py: torch._grouped_mm crashes on gfx1200 (null HIP kernel pointer,
0xC0000005) when torch.compile's JitDecomp system dispatches it during
the first forward pass. Detect Windows ROCm via torch.version.hip
(already in sys.modules from section 1e) and set TORCHDYNAMO_DISABLE=1
to bypass the broken kernel dispatch.
* fix: BNB AMD wheel install fails uv wheel filename check
The bitsandbytes continuous-release wheel is intentionally mismatched:
filename encodes 1.33.7.preview (= 1.33.7rc0 in PEP 440) but wheel
metadata reports 0.50.0.dev0. uv rejects this by default.
Introduce _install_bnb_windows_rocm() helper that sets
UV_SKIP_WHEEL_FILENAME_CHECK=1 only for this specific install, then
restores the previous env value. Both BNB install call sites (the
UNSLOTH_ROCM_TORCH_INSTALLED early-return path and the normal Windows
ROCm path) now use this helper.
* worker: patch _grouped_mm CUDA dispatch on Windows ROCm (gfx1200 null kernel)
TORCHDYNAMO_DISABLE=1 stopped the compiler frontend but not the autograd
JitDecomp system, which also dispatches _grouped_mm and hits the same
null HIP kernel crash (0xC0000005).
Verified that torch.library.Library("aten","IMPL").impl("_grouped_mm", fn,
"CUDA") successfully overrides the broken HIP kernel with a Python mm
fallback on torch==2.10.0+rocm7.12.0.
Schema: _grouped_mm(Tensor self, Tensor mat2, Tensor? offs=None,
Tensor? bias=None, ScalarType? out_dtype=None) -> Tensor
The fallback handles both the simple case (offs=None → torch.mm) and the
grouped case (offs provided → split self by offsets, multiply each group
against the corresponding slice of mat2, then cat results).
Keep _WINDOWS_ROCM_GROUPED_MM_LIB alive at function scope to prevent the
C++ dispatch registration from being freed by GC.
* worker: fix torchao stub — return stub classes not modules for isinstance()
peft/tuners/lora/torchao.py does:
from torchao.dtypes import AffineQuantizedTensor, LinearActivationQuantizedTensor
isinstance(weight, (AffineQuantizedTensor, LinearActivationQuantizedTensor))
The stub __getattr__ was returning stub modules, which isinstance() rejects
with "arg 2 must be a type, a tuple of types, or a union".
Add _StubTypeMeta metaclass whose __instancecheck__ always returns False,
and _make_stub_type() to create stub classes via it. Change _make_mod_stub
__getattr__ to return stub classes instead of stub modules for leaf
attribute access, so isinstance() gets a valid type and returns False.
_StubSubpackageFinder still handles import-style subpackage creation
(those still need module objects in sys.modules); __getattr__ only fires
for from-import or direct attribute access, which are the isinstance paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests: add coverage for Windows ROCm install paths and worker patches
Add conftest.py to fix pre-existing sys.path issue that prevented
test_rocm_support.py from running at all (install_python_stack.py
imports from backend.utils.wheel_utils which needs studio/ on sys.path).
New test classes cover everything added in this session:
- TestWindowsRocmIndexUrl: arch → AMD pip index URL mapping (gfx120X-all,
gfx1151, gfx1150, gfx110X-all, unknown → None, trailing slash)
- TestDetectWindowsGfxArch: hipinfo output parsing, missing/timeout/bad
returncode/no-gcnArchName paths
- TestInstallBnbWindowsRocm: UV_SKIP_WHEEL_FILENAME_CHECK set+restored,
env restored on exception, no-op when URL missing
- TestRocmTorchInstalledEnvVar: UNSLOTH_ROCM_TORCH_INSTALLED=1 skips
pip_install, calls _install_bnb_windows_rocm, sets flag
- TestWorkerWindowsRocmPatches: _grouped_mm CUDA dispatch override,
offs/grouped variant handling, GC-prevention sentinel,
_StubTypeMeta __instancecheck__, _StubSubpackageFinder registration,
torchao key submodule pre-stubbing, TORCHDYNAMO_DISABLE guard
- TestRocmTorchPkgSpecs: rocm7.2 torch 2.11.x spec, default <2.11 cap,
3-tuple shape, _GFX_TO_AMD_INDEX_ARCH RDNA4/3.5/3 coverage
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests: fix encoding, IS_WINDOWS patching, and wrong assertion
- Add encoding="utf-8" to all read_text() calls (54 occurrences) so
tests pass on Windows where the default codec is cp1252 and source
files contain UTF-8 emoji (e.g. ⚠️ in install_python_stack.py)
- Add @patch.object(stack_mod, "IS_WINDOWS", False) to Linux-path
TestEnsureRocmTorch tests so they reach the Linux code path when run
on a Windows machine instead of short-circuiting into the Windows branch
- Fix test_grouped_mm_patch_guarded_by_windows_and_hip_check: the source
uses getattr(_torch_for_rocm, "version", None) not torch.version, so
check for '"version"' and '"hip"' substrings instead
137 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: pin BNB_ROCM_VERSION=72 for torch==2.11.0+rocm7.13.0 compatibility
AMD's pip index now ships torch==2.11.0+rocm7.13.0 (ROCm 7.13).
bitsandbytes auto-detects HIP 7.13 from torch.version.hip and looks for
libbitsandbytes_rocm713.dll, which the AMD Windows prerelease wheel does
not ship (it only ships rocm72.dll), causing a load error at training start.
Fix:
- worker.py section 1f: set BNB_ROCM_VERSION=72 (via setdefault) before
section 2 ML imports, so bitsandbytes always loads rocm72.dll on Windows ROCm
- install_python_stack.py: set BNB_ROCM_VERSION=72 in _install_bnb_windows_rocm()
for any post-install imports; update comment to document root cause
- tests: 4 new assertions covering the fix (141 passed, 2 skipped)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: detect BNB ROCm DLL suffix dynamically instead of hardcoding '72'
BNB_ROCM_VERSION was pinned to '72' which works today (AMD wheel ships
rocm72.dll) but would break again if AMD ships a future wheel with a
different DLL suffix (e.g. rocm713.dll).
Add _detect_bnb_rocm_dll_ver() to install_python_stack.py: scans the
installed bitsandbytes package dir for libbitsandbytes_rocm{VER}.dll
using importlib.util.find_spec (no BNB import needed) and returns the
suffix. '72' remains the fallback when detection fails.
Apply the same detection inline in worker.py section 1f. Both paths
still respect a pre-set BNB_ROCM_VERSION (caller override wins).
Tests: +8 cases covering detection logic and fallback (147 passed, 2 skipped).
* fix: patch torch.distributed stubs in server process for Windows ROCm
On Windows ROCm, torch.distributed ships without process-group helpers
(is_initialized, is_available, get_rank, get_world_size). The worker
subprocess already patches these in section 1e, but the main server
process calls _determine_attention_impl_for_gpu_estimate() which calls
unsloth's resolve_attention_implementation() → is_initialized(), causing:
"Could not resolve attention implementation for '...':
module 'torch.distributed' has no attribute 'is_initialized'"
Fix: patch the missing attrs onto torch.distributed at the top of
_determine_attention_impl_for_gpu_estimate, matching the same stubs
already applied in worker.py section 1e. No-ops on Linux/CUDA where
torch.distributed is fully populated.
* fix: gate _grouped_mm dispatch patch on HIP < 7.13
AMD fixed the gfx1200 null HIP kernel in ROCm 7.13 (torch 2.11+).
Users on the new wheel now get the real GPU _grouped_mm kernel for
MoE workloads instead of the Python mm fallback.
Changes:
- worker.py: add _hip_ver_at_least() helper; wrap full _grouped_mm
patch in `if not _hip_ver_at_least(7, 13):` with else branch that
logs the skip reason; update section-1f comment to document the fix
- test_rocm_support.py: add 5 tests covering the helper definition,
the (7, 13) gate expression, the else branch, the skip log message,
and the AMD-format version string parsing (.split(".")[:2])
Verified: torch==2.11.0+rocm7.13.0 — 3D batch and grouped (offs)
variants both succeed; null crash only present on rocm7.12 and earlier.
* fix: stub is_torchelastic_launched on torch.distributed for Windows ROCm
resolve_attention_implementation calls is_torchelastic_launched() which
does not exist in the incomplete torch.distributed shipped with the
Windows ROCm wheel, causing a warning on every model config load in the
server process. Add it to the stub table alongside the four helpers
already patched in _determine_attention_impl_for_gpu_estimate.
Also adds two tests: one confirming the new stub and one confirming all
five core distributed helpers are covered.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: explicit warnings on AMD ROCm arch/version fallbacks + Fast-Install arg order
setup.ps1:
- Fix Fast-Install argument order: packages before flags, consistent with
all other Fast-Install calls in the file
(was: Fast-Install --force-reinstall --index-url $url torch ...)
(now: Fast-Install torch torchvision torchaudio --force-reinstall --index-url $url)
- Add explicit [WARN] substep when $HasROCm is true but arch mapping fails:
- GPU arch detected but not in supported wheel list → names the arch and
lists supported families so user knows exactly what to report
- HIP SDK present (amd-smi path) but gcnArchName unreadable → instructs
user to re-install the HIP SDK; previously fell back silently to CPU
install.sh:
- Add [WARN] to stderr before silent CPU fallback when AMD GPU is confirmed
(rocminfo/amd-smi) but ROCm version cannot be read from any source
(amd-smi, /opt/rocm/.info/version, hipconfig, dpkg, rpm)
- Add [WARN] to stderr when ROCm version is too old (< 6.0) with upgrade link
install.ps1 and setup.sh: no changes needed (already handle these paths correctly)
* fix: robust gfx arch detection for Strix Halo / HIP-runtime-only installs
Covers users who have the HIP runtime (amd-smi available) but not the
full HIP SDK (no hipinfo), which is common on Strix Halo iGPU systems.
Without this, $ROCmGfxArch stays null and the installer silently falls
back to CPU-only PyTorch despite a working GPU.
Detection waterfall (setup.ps1 + install.ps1):
1. hipinfo gcnArchName -- full HIP SDK (existing, unchanged)
2. amd-smi list gfx pattern -- newer amd-smi versions embed arch
3. amd-smi static --asic -- ROCm 6+ ASIC details with GFX target
4. UNSLOTH_ROCM_GFX_ARCH env -- manual override escape hatch
5. GPU name → arch table -- best-effort from marketing name:
890M / Strix Halo → gfx1151 (RDNA 3.5 iGPU, Strix Halo)
880M / Strix Point → gfx1150 (RDNA 3.5 iGPU, Strix Point)
780M / Phoenix → gfx1103 (RDNA 3 iGPU)
RX 7900/7800/7700 → gfx1100 (RDNA 3 desktop)
RX 9070 XT / 9080 → gfx1201 (RDNA 4)
RX 9070 / 9060 XT → gfx1200 (RDNA 4)
When arch is inferred from name, a Cyan substep tells the user to set
UNSLOTH_ROCM_GFX_ARCH to skip inference on future installs.
WMI block intentionally does not set $HasROCm (no runtime confirmation).
Tests: 11 new tests in TestStrixHaloGfxArchDetection covering all five
detection levels, WMI safety, and gfx regex in both ps1 files.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: resolve hipinfo/hipconfig via HIP_PATH/ROCM_PATH when not on PATH
AMD HIP SDK sets HIP_PATH on Windows but does not always add the bin
directory to PATH. Get-Command hipinfo therefore silently fails and
detection falls through to WMI, which cannot provide a gfx arch, leaving
the user with a CPU-only PyTorch install and no warning.
Changes:
- setup.ps1 / install.ps1: before falling through to amd-smi, attempt to
locate hipinfo.exe and hipconfig.exe under $env:HIP_PATH\bin (then
$env:ROCM_PATH\bin) when Get-Command returns nothing
- Emit a [WARN] with the resolved path and a one-liner to permanently fix
PATH via SetEnvironmentVariable
- Emit a [WARN] when HIP_PATH/ROCM_PATH is set but the exe is still not
found (incomplete SDK install)
- Emit a [WARN] with the first hipinfo output line when hipinfo runs but
returns a non-zero exit code (e.g. "no ROCm-capable device detected")
- 18 new tests in TestHipSdkEnvPathResolution; total 183 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat: print HIP SDK path and full hipconfig version in terminal on AMD detection
Both install.ps1 and setup.ps1 now emit substeps under the gpu step when
AMD ROCm is detected:
gpu AMD ROCm (gfx1200)
HIP SDK: C:\Program Files\AMD\ROCm\7.1
hipconfig: 7.1.51803-d3a86bd04
Previously only the gpu label (e.g. "AMD ROCm (gfx1200)") was shown with
no indication of where the SDK was found or which exact build was active.
The full hipconfig build string (e.g. 7.1.51803-d3a86bd04 instead of just
7.1) is now stored in ROCmVersionFull and also used in setup.ps1's
'rocm' step label.
9 new tests in TestHipSdkDetectedSubstep; total 192 passed, 2 skipped
* fix: Strix rocm7.1 segfault bypass + Ubuntu 24.04 HIP gcc-install-dir
Issue 1 (install.sh): gfx1151/gfx1150 + ROCm 7.1 causes a segfault in
torch._grouped_mm (moe_utils.py:167). The Radeon repo now ships cp313
wheels for rocm-rel-7.1, so _amd_gpu_radeon=true silently lands on the
broken combo. When Strix Halo/Point is detected and TORCH_INDEX_URL is
rocm7.1, override to rocm7.2 PyTorch index, update TORCH_CONSTRAINT, and
set _amd_gpu_radeon=false to bypass the Radeon repo entirely. Emits a
clear [WARN] explaining the segfault and linking to the ROCm upgrade docs.
Issue 2 (setup.sh): ROCm 7.x ships clang-20 which on Ubuntu 24.04+ picks
/usr/lib/gcc/x86_64-linux-gnu/14/ (runtime dir, no C++ headers), causing
'cstdlib file not found' and a failed llama.cpp HIP build. Iterate gcc
versions 14→11 to find the first install dir that has both runtime and
/usr/include/c++/<ver> headers, then pass --gcc-install-dir to clang via
CMAKE_HIP_FLAGS. Fix confirmed by h34v3nzc0dex (llama.cpp 417/417 clean).
11 new tests across TestStrixRocm71Override and TestSetupShGccInstallDir;
total 203 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: BNB_ROCM_VERSION in server process + torch._C._distributed_c10d stubs
Two errors visible in training logs on Windows ROCm:
1. Server process bitsandbytes crash:
"Configured ROCm binary not found at libbitsandbytes_rocm713.dll"
The installed BNB wheel ships rocm72.dll (not rocm713.dll). The
training worker already sets BNB_ROCM_VERSION=72 via DLL detection
but the server process (main.py) imported bitsandbytes before that
ran. Fix: add the same DLL-scan + BNB_ROCM_VERSION assignment to
main.py inside the existing win32 guard, before any downstream
import can pull in bitsandbytes.
2. torch.distributed import failure:
"No module named 'torch._C._distributed_c10d'; torch._C is not a package"
torch._C is a C extension on Windows ROCm — Python cannot do
submodule imports from it, so torch.distributed fails to import
before our attribute stubs could ever run. Fix: inject empty
ModuleType stubs for _distributed_c10d, _distributed_autograd and
_distributed_rpc into sys.modules inside the win32 guard in
hardware.py BEFORE importing torch.distributed, so the import
succeeds and our attribute stubs take effect.
9 new tests in TestServerStartupRocmFixes; total 212 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(win32): populate distributed c10d stub with dummy symbols
torch.distributed tries to `from torch._C._distributed_c10d import
FakeProcessGroup` (and ProcessGroup, Work, Store, etc.). The previous
empty ModuleType stub caused an AttributeError on those names.
Populate every stub with a _Dummy class for each known symbol so the
import chain completes silently on Windows ROCm where torch._C is a
compiled extension and its _distributed_c10d submodule doesn't exist.
Adds four new tests in TestServerStartupRocmFixes covering FakeProcessGroup,
ProcessGroup, setattr population, and all three _distributed_* siblings.
* fix(win32): distinguish HIP SDK installed vs GPU not ROCm-accessible
Previously, when hipinfo was found but exited non-zero (e.g. "no
ROCm-capable device detected"), both install.ps1 and setup.ps1 fell
through to the WMI-label-only branch and printed "AMD GPU detected --
HIP SDK not found" -- factually wrong since the SDK binary is present.
Add $HipSdkInstalled flag (set true when hipinfo binary is found,
regardless of exit code). When HipSdkInstalled && !HasROCm:
- Show "AMD GPU detected -- not ROCm-accessible (HIP <ver>)" instead
- Explain this is a driver issue, not an SDK issue, with a link
- Still run hipconfig version capture so version shows in output
- CPU-only hint now says "GPU not ROCm-accessible" not "require HIP SDK"
Also applies to setup.ps1 (same detection block, same branches).
Adds TestHipSdkInstalledButDeviceInaccessible (11 tests).
* fix(win32): scope ROCm workarounds to AMD hosts only
Three Codex-flagged issues where Windows ROCm workarounds incorrectly
applied to Windows CUDA (NVIDIA) machines:
main.py (P1): BNB_ROCM_VERSION was set unconditionally on all win32
hosts. On NVIDIA, bitsandbytes sees BNB_ROCM_VERSION and looks for a
ROCm DLL that doesn't exist, breaking bitsandbytes initialisation.
Fix: gate the block on HIP_PATH/ROCM_PATH being present (ROCm hosts only).
worker.py (P2): torchao stubs were seeded for all win32 runs, shadowing
real torchao on Windows CUDA and silently disabling torchao quantization
for NVIDIA users. Fix: gate on HIP_PATH/ROCM_PATH (win32 ROCm only).
install_python_stack.py (P1): _detect_windows_gfx_arch() only checked
shutil.which("hipinfo"), skipping the HIP_PATH/ROCM_PATH fallback that
the PowerShell installers use. On installs where the HIP SDK bin dir is
not on PATH, _ensure_rocm_torch() returned early without installing
ROCm wheels or bitsandbytes. Fix: mirror the env-var fallback.
* fix(linux): route Strix + ROCm 7.1 to AMD arch-specific index
Instead of falling back to pytorch.org/rocm7.2, the Strix override now
routes to repo.amd.com/rocm/whl/gfx1151/ (or gfx1150/) which serves
torch 2.11.0+rocm7.13.0 -- AMD's build containing the actual _grouped_mm
kernel fix, verified on real gfx1151 hardware by h34v3nzc0dex.
This exercises the real GPU kernel path rather than the rocm7.2 workaround.
UNSLOTH_AMD_ROCM_MIRROR can override the base URL for air-gapped installs.
Also teaches _tauri_torch_index_family to recognise AMD arch-specific URLs
(repo.amd.com/rocm/whl/gfx*) and return the rocm7.13 family label so
_tauri_gpu_branch correctly classifies these installs as rocm.
Suggested by h34v3nzc0dex based on hardware-verified probe results.
* fix(studio/rocm): gate ROCm-only side-effects on active torch runtime
Address five edge cases flagged during PR review:
1. studio/backend/main.py: BNB_ROCM_VERSION was set whenever HIP_PATH or
ROCM_PATH was present in the environment. A Windows CUDA user who once
installed the HIP SDK and reverted to a CUDA torch wheel still has those
env vars set, so bitsandbytes would try to load libbitsandbytes_rocm72.dll
against a CUDA torch and crash. Now probe torch.version.hip inside the
env-var guard (worker.py already does this).
2. studio/backend/main.py: os.add_dll_directory returned handles were
discarded. Per CPython docs, the directory leaves the DLL search list when
the handle is garbage collected. Retain handles in module-level
_ROCM_DLL_HANDLES list so they survive process lifetime.
3. studio/install_python_stack.py: _install_bnb_windows_rocm() returned None
regardless of pip_install_try outcome, and the caller flipped
_rocm_windows_torch_installed to True unconditionally. On a failed BNB
install the post-install "manual install may be required" warning was
suppressed and the user was misled. Helper now returns bool; caller gates
on it.
4. studio/install_python_stack.py: _detect_windows_gfx_arch returned the raw
capture group, so mixed-case hipinfo output ("Gfx1151") missed the
lowercase keys in _GFX_TO_AMD_INDEX_ARCH and silently fell back to CPU
torch. Lowercase the token.
5. studio/install_python_stack.py: UNSLOTH_ROCM_TORCH_INSTALLED=1 early-
return trusted the env var even when the venv was wiped between runs.
Subprocess-probe torch importability first; fall through to the full
install path if the probe fails.
Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py
(adds one new test for case 5 fall-through).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): worker.py parity + don't roll back ROCm torch on bnb failure
Addresses findings from a 10x reviewer pass on the prior fix commit:
1. studio/backend/core/training/worker.py (parity with main.py):
- Gate the torchao stub block on torch.version.hip / 'rocm' in
torch.__version__ instead of HIP_PATH / ROCM_PATH env-var presence.
Same root cause as main.py: HIP SDK env vars stick around on CUDA hosts.
- Add module-level Windows ROCm DLL registration block. Worker subprocesses
inherit env vars but not the parent's add_dll_directory handles, so the
first `import torch` in the worker could fail to find amdhip64.dll when
HIP_PATH\bin is not on PATH. Mirrors main.py setup. Handles retained at
module scope via _ROCM_DLL_HANDLES.
- Promote _WINDOWS_ROCM_GROUPED_MM_LIB to module scope with `global` in
run_training_process so the torch.library.Library registration survives
past function return / mid-run garbage collection.
- Harden _torch_has_hip() to also accept 'rocm' in torch.__version__
(AMD SDK / Radeon wheels may not set torch.version.hip).
2. studio/install_python_stack.py:
- Don't roll back ROCm torch when bitsandbytes install fails. The prior
commit gated _rocm_windows_torch_installed on _install_bnb_windows_rocm()
returning True; if torch installed successfully but bnb failed, the flag
stayed False and later install steps could overwrite ROCm torch with the
generic CPU torch wheel. Set the flag after torch install; surface bnb
failure as a separate warning instead.
- _detect_windows_gfx_arch now probes in three tiers: UNSLOTH_ROCM_GFX_ARCH
env-var override (matches the PowerShell installer), then hipinfo (PATH
or HIP_PATH\bin), then amd-smi (`static --asic`, `list`). Without the
amd-smi fallback, runtime-only Radeon installs without hipinfo on PATH
made `studio update` return early and leave the venv on CPU torch.
- Linux torch-already-rocm probe in _ensure_rocm_torch now matches the
Windows probe shape: accepts torch.version.hip OR 'rocm' in
torch.__version__ to cover AMD SDK / Radeon Linux wheels.
3. studio/backend/utils/hardware/hardware.py:
- apply_gpu_ids() final-fallback torch probe accepts 'rocm' in
torch.__version__ in addition to torch.version.hip, matching
detect_hardware(). AMD SDK wheels could otherwise leak through with
CUDA-only visibility masks on a spawned ROCm worker.
Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py
(no test changes needed; the probe shape that prints the hip version (or
'rocm' sentinel) preserves the existing non-empty-string contract).
Not addressed in this commit (deferred or out of scope):
- Tag drift / lemonade checksum (PR 5303 surface, not this PR).
- install.sh rocm7.2.1 URL: small fix, separate.
- install.ps1 / setup.ps1 'Radeon 8060S' marketing-name fallback table.
- Strix Halo + ROCm 7.1 routing asymmetry in Python update path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): robustness pass - rocm tag normalisation, Strix routing parity, hardened detection
Robustness pass on top of 76137b2d. Four targeted fixes:
1. install.sh ROCm-tag routing normalisation.
`rocm7.2.1` would route to https://download.pytorch.org/whl/rocm7.2.1
which does not exist (PyTorch publishes major.minor URLs only). Same
for any future patch-level tag. Normalise every rocm{maj.min}* pattern
to the bare {maj.min} index URL.
2. install.ps1 + studio/setup.ps1 marketing-name fallback.
The gfx1151 row matched 890M / Strix Halo / HX 37x / HX 38x / AI 9 HX
but not the actual retail name 'AMD Radeon 8060S Graphics' shipped by
OEMs (Ryzen AI MAX+ 395). Add '8060S' to the regex.
3. install_python_stack.py Strix + ROCm 7.1 routing parity with install.sh.
The shell installer reroutes Strix Halo / Point + ROCm 7.1 to
repo.amd.com/rocm/whl/{gfx}/ (which serves torch 2.11.0+rocm7.13.0
with the upstream _grouped_mm fix). The Python `studio update` path
only warned and still installed the broken generic rocm7.1 wheel.
Mirror the override: detect gfx1151/gfx1150 on ROCm 7.1, route to
the AMD per-gfx index, honour UNSLOTH_AMD_ROCM_MIRROR override.
4. _detect_windows_gfx_arch amd-smi parsing tightened.
The amd-smi fallback added in the prior commit used a bare
`\bgfx[1-9][0-9a-z]{2,3}\b` match against the lowercased stdout,
which could pick up stray gfx references in warnings / device-name
strings. Anchor on labelled lines first (Target_Graphics_Version,
ASIC, Arch, gfx) and fall back to the bare match only when no
labelled line is present.
Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py;
sim_5301 23 cases pass (6 new sims for the Strix override + amd-smi parsing).
* fix(studio/rocm): multi-GPU selection, Strix sibling handling, defensive cleanups
Round 4 robustness pass based on 5 parallel Opus reviewers of head 21773215.
Seven items from across regression / edge-case / error-paths / architecture
reviews:
1. studio/backend/main.py BNB gate: aligned with the broad ROCm check used
everywhere else in this PR (torch.version.hip OR 'rocm' in __version__).
AMD SDK / Radeon Linux wheels do not always populate torch.version.hip;
without this, main.py would silently skip BNB_ROCM_VERSION while worker.py
set it.
2. studio/install_python_stack.py _install_bnb_windows_rocm: init _ok = False
before the try block. Without this, if pip_install_try itself raises
(e.g. OSError on uv binary missing), the finally block restored env vars
correctly but the subsequent `if not _ok:` raised UnboundLocalError,
masking the original exception.
3. studio/install_python_stack.py _detect_windows_gfx_arch:
- Rewrote to use re.findall (not re.search) on both hipinfo and amd-smi
output, dedup tokens preserving order, and select via new
_pick_visible_index() helper.
- HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES (first comma entry, integer)
now picks the right GPU on multi-AMD-GPU hosts. Out-of-range or non-int
values fall back to the first GPU (matches detect_host behaviour in
install_llama_prebuilt.py).
4. studio/install_python_stack.py Strix override now consults the runtime
target before flipping:
- Previous behaviour intersected gfx_codes with {gfx1151, gfx1150} and
picked the first Strix arch, ignoring whether HIP_VISIBLE_DEVICES
selected a non-Strix sibling (e.g. discrete RX 7900 in a mixed APU+dGPU
box). Could install Strix-specific wheels onto a gfx1100 dGPU.
- Now resolves the runtime gfx via _pick_visible_index() and only
overrides when that runtime target is in the Strix set.
5. studio/backend/main.py + studio/backend/core/training/worker.py: ROCm
version dir scan no longer sorts lexically. Previous sort placed "10.0"
before "7.0" alphabetically, which would mis-prioritise ROCm 10.x bin
dirs once AMD ships them. New _ver_key() splits on "." and sorts
numerically with a string fallback.
6. install.sh Strix override URL: replaced ${var%/} (strips one trailing
slash) with a while-loop that strips all trailing slashes, matching
Python's .rstrip("/"). A user setting UNSLOTH_AMD_ROCM_MIRROR with
"http://corp/whl///" no longer ends up with "http://corp/whl///gfx1151/"
which strict pip proxies (artifactory, sonatype) 404 on.
7. studio/install_python_stack.py: bumped torch import probe timeout from
30s to 90s. PyTorch's lazy .so loading can take 60-90s on cold NFS or
USB-backed venvs. The shorter timeout was producing a false "torch
missing" classification and reinstalling a working ROCm torch.
Tests: 231 passed, 1 skipped. sim_5301 30 cases pass (added 7 new sims for
multi-GPU detection, Strix sibling handling, and _ok-init regression).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): worker BNB/grouped_mm broad gate, install.sh Strix visibility, runtime-only ROCm detection
Round-5 robustness pass based on 20 parallel reviewers of head 96b9e465.
1. studio/backend/core/training/worker.py - BNB version pin / dynamo disable
/ _grouped_mm fallback block was still gated on torch.version.hip alone
despite the torchao stub block above already using the broad check. AMD
SDK / Radeon Windows wheels (torch.__version__ contains "rocm" but
torch.version.hip is None) silently skipped the Windows ROCm runtime
patches. Aligned to the same broad check (8/20 reviewers).
2. studio/backend/core/training/worker.py - _hip_ver_at_least() now also
parses the ROCm version out of torch.__version__ (e.g. "2.11.0+rocm7.13.0")
when torch.version.hip is missing, so the kernel-fix gate is correct for
SDK / Radeon wheels too.
3. studio/backend/core/training/worker.py - _grouped_mm_safe_impl with
offs=None now picks torch.bmm/matmul for 3-D inputs instead of always
calling torch.mm. The real _grouped_mm accepts 3-D batched matmul; the
prior fallback raised "self must be a matrix" on MoE workloads (2/20).
4. studio/backend/main.py - dropped the HIP_PATH / ROCM_PATH env-var gate
from the BNB block; probe torch directly. Runtime-only Radeon / AMD SDK
Windows installs do not set those SDK env vars but still ship ROCm torch
(5/20 reviewers).
5. install.sh - Strix override now collects every gfx token from
rocminfo / amd-smi (in enumeration order), then indexes by
HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES so a mixed Strix iGPU + non-
Strix dGPU host where the user selected the dGPU does NOT get rerouted
to the Strix per-gfx index. Mirrors the Python update path (5/20 reviewers).
6. install.sh - Strix detection chain now also probes `amd-smi static --asic`,
matching the PowerShell installer (1/20). Closes the gap on runtime-only
Strix hosts where `amd-smi list` does not surface a gfx token.
7. studio/install_python_stack.py - _has_rocm_gpu() now has the sysfs KFD
topology fallback (/sys/class/kfd/kfd/topology/nodes/*/gpu_id), matching
install.sh. On minimal package-managed installs without rocminfo /
amd-smi GUI tools, `studio update` can now detect the GPU and repair the
venv instead of returning early (2/20).
8. studio/install_python_stack.py - _detect_amd_gfx_codes() now falls back
to `amd-smi list` and `amd-smi static --asic` when rocminfo is missing
(2/20). Strix routing on runtime-only Radeon hosts now matches what
install.sh has done for a while.
9. studio/install_python_stack.py - Strix override now applies even when
has_hip_torch is True. The whole point of the override is to repair an
existing broken torch.version.hip == "7.1" install; skipping the
reinstall left users on the known _grouped_mm segfaulting stack (3/20).
Tests: 231 passed, 1 skipped. sim_5301 30 cases pass. sim_cross 12 pass.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): code review hardening pass
- main.py: numeric DLL sort (string sort picked rocm72 over rocm713);
add basename() to regex; log warning on detection failure; log info
when BNB_ROCM_VERSION is set (mirrors worker.py)
- worker.py: explicit len-guard in _hip_ver_at_least() with warning
logs instead of silent IndexError/ValueError swallow
- hardware.py: isinstance(result, dict) guard before result.get() in
_smi_query() to prevent AttributeError on non-dict backend returns
- amd.py: round() before int() on parsed GPU IDs; log warning when
truncation occurs (defensive against malformed amd-smi output)
- setup.sh: quote --gcc-install-dir value in CMAKE_HIP_FLAGS so paths
with spaces do not break the CMake argument
- install.ps1, setup.ps1: apply colon-split + ToLower() to hipinfo
gcnArchName match (consistent with each other and with setup.sh)
- install.sh: tighten ROCm tag case patterns to explicit
rocmX.Y|rocmX.Y.* to avoid unintended prefix matches
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/training): GPU OOM guard to prevent system freeze on VRAM exhaustion
On RDNA 4 (gfx1200/gfx1201) and other ROCm GPUs, exhausting VRAM can
cause a HIP driver hang that freezes the entire system rather than
raising a recoverable Python exception.
Two-part fix:
- set_per_process_memory_fraction(0.90) caps the HIP/CUDA allocator at
90% of VRAM so PyTorch raises OutOfMemoryError before hitting the
hardware limit, keeping the driver alive and the system responsive
- top-level exception handler detects OOM errors by type and message
and surfaces a clear actionable message to the UI (reduce
max_seq_length, enable gradient_checkpointing, lower batch size)
instead of the raw CUDA/HIP error string
* fix(studio/rocm): OOM guard ROCm-only + unified memory, multi-GPU arch selection
OOM guard (worker.py):
- Scope to _hw.IS_ROCM only -- NVIDIA CUDA has a graceful OOM path and
does not need the allocator cap
- Detect unified memory by comparing torch VRAM against psutil system RAM;
use 0.80 on unified-memory APUs (gfx1151 Strix Halo) where the GPU pool
is carved from host RAM, 0.90 on discrete cards
Multi-GPU arch selection:
- install.ps1 / setup.ps1: replace -match (first hit only) with
[regex]::Matches() to collect all gcnArchName entries, then index by
HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES
- install_python_stack.py: index into full token list before dedup so
HIP_VISIBLE_DEVICES=2 on [gfx1100, gfx1100, gfx1151] resolves gfx1151
- install.sh: remove awk dedup from gfx token collection for same reason
GCC multiarch (setup.sh):
- Only append -linux-gnu when gcc -print-multiarch does not already return
the full triple, fixing double-suffix on Ubuntu 24.04
* fix(tests): update ROCm version cap expectations from rocm7.1 to rocm7.2
Daniel's normalisation commit updated the cap from rocm7.1 to rocm7.2
since PyTorch now publishes that index and rocm7.2 ships torch 2.11.0.
Test expectations were stale.
* fix(tests): correct MLX smoke test losses_per_step assertion
logging_steps=1 with max_steps=30 produces 30 loss entries, not 7.
The assertion was stale from a previous config.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/worker): detect unified-memory APU by GPU name not VRAM/RAM ratio
The previous heuristic (VRAM > 50 % of system RAM) false-positived on discrete
cards in low-RAM systems — e.g. RX 9060 XT 16 GB on a 16 GB or 24 GB machine
would trip the unified-memory path and log "unified memory host" when it should
say "discrete".
AMD iGPUs (gfx1150/gfx1151 Strix Halo, Strix Point, etc.) expose names with a
digit+M suffix ("AMD Radeon 890M"), while discrete cards use "RX NNNN [XT|XTX]"
naming. Matching that suffix is reliable across all current ROCm-capable AMD
consumer GPUs and does not require psutil.
Also includes the device name in the log line to ease future debugging.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(install/setup.ps1): force array on hipinfo gcnArchName parse to fix single-GPU arch truncation
When [regex]::Matches() finds exactly one match, PowerShell's pipeline
unwraps the result to a scalar string. Indexing a scalar string with [0]
returns the first *character*, so a one-GPU system would parse
gcnArchName "gfx1200" as "g", which is not in the supported arch map
and triggers the CPU-only fallback.
Wrapping with @() forces the result to remain an array regardless of
match count. On a single-GPU machine the arch is now correctly read as
"gfx1200" (or whatever the full name is) so the ROCm wheel index is
selected.
Reproducer: hipinfo exits 0 and outputs exactly one gcnArchName line.
Without @(), $_hipAllArches = "gfx1200" (String); $_hipAllArches[0] = 'g'.
With @(), $_hipAllArches = @("gfx1200") (Object[]); $_hipAllArches[0] = "gfx1200".
* fix(studio/rocm): classify unified-memory APU via VRAM/RAM ratio, not arch list
Replace the gcnArchName allowlist {gfx1150, gfx1151} with a
psutil-based heuristic: unified APUs expose the entire system RAM
as the HIP pool (ratio ≥ 0.90), discrete cards are well below that.
No arch name required — future APUs classify correctly without code changes.
Also removes the stale import re / \d[Mm]\b device-name regex that
5d84704 left behind, and logs vram/sys GiB for easier on-hardware
verification.
Addresses h34v3nzc0dex review: Radeon 8060S (gfx1151, 128 GiB
unified) now correctly gets 0.80 cap instead of 0.90.
* fix(studio/rocm): revert to gcnArchName for unified-memory APU classification
VRAM/RAM ratio >= 0.90 false-positives on machines where discrete VRAM
equals system RAM (e.g. RX 9060 XT 16 GB + 16 GB system RAM → ratio 1.0,
incorrectly classified as unified → wrong 0.80 cap applied).
gcnArchName is the correct signal: naming-independent, stable within a
product family, and already parsed throughout this PR. Unified set is
{gfx1150, gfx1151} (Strix Point + Strix Halo).
* fix(studio/llama-prebuilt): resolve hipinfo via HIP_PATH/ROCM_PATH on Windows
shutil.which("hipinfo") returns None when the HIP SDK bin dir is not on
PATH -- the HIP SDK installer sets HIP_PATH/ROCM_PATH but does not always
add the bin dir to PATH. This caused has_rocm=False in the prebuilt asset
selector, so AMD ROCm machines got the CPU llama.cpp zip instead of the
HIP one, silently running all chat inference on CPU.
Add _resolve_exe() that falls back to %HIP_PATH%\bin and %ROCM_PATH%\bin
when shutil.which() finds nothing, mirroring the same fallback already
present in setup.ps1.
* fix(studio/llama-prebuilt): pass --has-rocm from setup.ps1 to skip re-detection
The Python prebuilt installer re-detects ROCm independently via
shutil.which("hipinfo"), which fails when hipinfo is not on PATH
(HIP SDK sets HIP_PATH but doesn't always add the bin dir to PATH).
This caused has_rocm=False and downloaded the CPU llama.cpp zip even
on confirmed AMD ROCm machines.
setup.ps1 already performs reliable ROCm detection with its own
HIP_PATH/ROCM_PATH fallback. Add --has-rocm flag to
install_llama_prebuilt.py so setup.ps1 can forward its result directly,
and pass it whenever $HasROCm is true. The Python script then overrides
has_rocm=True in the HostInfo without re-probing.
* fix(studio/llama-prebuilt): add HIP asset to simple-policy Windows path
direct_upstream_release_plan (used by --simple-policy, which setup.ps1
always passes) only checked has_usable_nvidia on Windows and fell
straight to CPU for AMD ROCm machines, ignoring has_rocm entirely.
The --has-rocm override had no effect because the simple-policy code
path never reached resolve_asset_choice where has_rocm was checked.
Add an elif branch for has_rocm that tries the upstream HIP asset
(llama-TAG-bin-win-hip-radeon-x64.zip) before falling through to the
CPU fallback, consistent with the non-simple-policy path.
* fix(studio/setup.ps1): auto-remove mismatched llama.cpp install kind
When an existing llama.cpp install is the wrong kind for the current
GPU (e.g. windows-cpu on an AMD ROCm machine that should have
windows-hip), the prebuilt installer skips on tag match and never
upgrades. Read install_kind from UNSLOTH_PREBUILT_INFO.json before
invoking the installer and remove the directory if the kind doesn't
match, forcing a fresh download of the correct variant.
* fix(studio/setup.ps1): show live PyTorch install output in verbose mode for ROCm
The ROCm torch reinstall (setup.ps1 phase) always silently captured
output, so in --verbose mode the torch downgrade mid-install
(2.11.0+rocm → 2.10.0 → 2.11.0+rocm) looked like the final state was
2.10.0. Match the CPU/CUDA blocks which show live uv output when
$script:UnslothVerbose is set.
* fix(rocm/windows): set ROCBLAS_TENSILE_LIBPATH for bundled rocblas.dll
The llama.cpp ROCm prebuilt bundles rocblas.dll next to the binary but
not the Tensile kernel library files it depends on at runtime
(rocblas/library/TensileLibrary*.dat + *.hsaco). The bundled DLL
searches for these files relative to its own location by default, i.e.
<binary_dir>/rocblas/library/, which does not exist in the prebuilt
install tree. This causes a silent crash on the very first GEMM
(prefill) with no output from llama-server, seen by the caller as
WinError 10054 / 10061. Model load and the single-token warmup pass
because they use simpler code paths that do not trigger rocBLAS GEMM.
Fix: set ROCBLAS_TENSILE_LIBPATH in the subprocess env to
<HIP_PATH>/bin/rocblas/library so the bundled DLL finds the kernel
files from the system ROCm installation. Uses setdefault so a user-
supplied env var is never overwritten. No-ops on CUDA and CPU (no
HIP_PATH) and on Linux (win32 branch only).
Reproducer log:
rocBLAS error: Cannot read .../Release/rocblas/library/TensileLibrary.dat
rocBLAS error: Could not initialize Tensile host:
directory_iterator: The system cannot find the path specified.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(install.sh): restore gfx token dedup in Strix multi-GPU awk indexer
536a54df removed the per-source `| awk '!seen[$0]++'` dedup from the
_gfx_all collection step but left the indexer awk as bare NF, so on a
mixed-arch host (e.g. dGPU gfx1100 + Strix iGPU gfx1151) where
rocminfo emits each gfx token twice (Name: field + ISA triple),
HIP_VISIBLE_DEVICES=1 indexed vals[1] = the second gfx1100 occurrence
instead of gfx1151, triggering the Strix routing on the wrong GPU.
Add !seen[$0]++ to the indexer awk so duplicate tokens from the same
GPU collapse to one entry before the HIP_VISIBLE_DEVICES index is
applied -- matching exactly what the Python side does with dict.fromkeys()
in _detect_amd_gfx_codes(). The comment above the block ("skip
duplicates") already documented this as the intended behaviour.
* fix(studio/install): correct _TOTAL progress count on Windows
base_total += 3 fired for all non-macOS platforms including Windows,
but flash-attn (line 1620) and ROCm torch final (line 1705) are both
guarded by 'not IS_WINDOWS and not IS_MACOS', so on Windows with torch
enabled _TOTAL was 13 while only 11 _progress() calls actually execute.
Split into +1 for the ROCm torch check (all non-macOS) and +2 for the
two Linux-only steps, so Windows gets _TOTAL=11 and Linux gets 14.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(install.ps1): enforce torch>=2.11.0 for gfx120X and Strix on Windows
The AMD arch-specific index (repo.amd.com/rocm/whl/gfx120X-all/ and
gfx1151/) publishes torch wheels from 2.7.1 through 2.11.0. Without a
version floor pip can resolve to torch 2.10.0+rocm7.12 on RDNA 4
(gfx120X) or torch 2.10.0+rocm7.1 on Strix (gfx1151/gfx1150), both of
which have a null-pointer crash in torch._C._grouped_mm (TheRock
issues #5284 / #3284). torch 2.11.0+rocm7.13 contains the fix.
Add $ROCmTorchFloor alongside $ROCmIndexUrl: set to torch>=2.11.0 for
the two affected arch families, null for all others. Wire it into the
uv pip install call so the broken wheels are never selected.
* fix(rocm/windows): address Codex nits - deterministic DLL suffix, CUDA llama.cpp kind, HIP_VISIBLE_DEVICES arch indexing
- install_python_stack.py / worker.py: _detect_bnb_rocm_dll_ver() and the
inline worker probe now collect ALL libbitsandbytes_rocm*.dll suffixes and
return max() by numeric value instead of stopping at the first glob hit.
Filesystem glob order is not guaranteed; this ensures '713' always wins
over '72' when both variants are present in the wheel.
- setup.ps1 (expectedKind): add 'windows-cuda' branch so NVIDIA hosts are
not treated as 'windows-cpu'. Previously an existing windows-cuda prebuilt
was always considered a mismatch on non-ROCm machines, forcing an
unnecessary re-download on every update.
- setup.ps1 (amd-smi gfx arch): collect ALL gfx tokens from amd-smi list
output in GPU order and honour HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES
when selecting which arch to use. On mixed-arch AMD systems where the
visible GPU is not the first enumerated one, this prevents installing an
incompatible wheel index. Falls back to index 0 (same as before) when the
visibility var is unset or is a comma-separated list.
- test_rocm_support.py: add test_picks_highest_suffix_when_multiple_dlls to
cover the multi-DLL case that was previously untested.
* fix(rocm): misleading amd-smi log, BNB spec consistency, torch ceiling for AMD index
amd.py: split 'returncode != 0 or not stdout' into two separate branches.
Previously, exit-0 with empty output logged 'amd-smi returned code 0' (which
reads as success, not a warning) and incorrectly incremented the circuit-breaker
counter. Now: non-zero exit logs the code and counts toward the limit as before;
empty stdout on exit 0 logs at DEBUG level and does not penalise the counter
(amd-smi --json always emits at least [] on exit 0, so this branch is rare and
is not a tool failure).
main.py: replace spec.origin / os.path.dirname() with
spec.submodule_search_locations to match install_python_stack.py and worker.py.
For normal wheel installs both approaches reach the same directory, but using
submodule_search_locations is the canonical way and handles editable bitsandbytes
installs correctly. Also use max() by numeric suffix (same as the other two sites)
instead of a sort-then-break loop.
install.ps1: add <2.12.0 ceiling to the torch constraint for gfx120X (RDNA 4)
and gfx1151/gfx1150 (Strix). AMD actively publishes new versions on their
per-arch index; without a ceiling, a future 2.12.0+rocmX.Y wheel would be
pulled in automatically before being validated on these architectures. The
ceiling matches the existing Linux install_python_stack.py constraint for the
same arches. Bump both when 2.12.x is confirmed working.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm): torch floor in setup.ps1, torchvision pin for Strix, rocmsdk in _hip_ver_at_least
setup.ps1: add \ (mirrors install.ps1) and derive \
from it. Previously the AMD index install called 'Fast-Install torch torchvision
torchaudio --force-reinstall --index-url \' with no version
constraint, so pip could resolve torch 2.10.0+rocm7.12 for gfx1151/gfx1200 --
the exact broken wheel the PR is meant to avoid. Now gfx120X and Strix enforce
'torch>=2.11.0,<2.12.0', matching install.ps1 and the Linux constraint.
install_python_stack.py: pin torchvision and torchaudio in _strix_override_pkgs.
The Strix Linux override uses --index-url (exclusive, no PyPI fallback); bare
unversioned 'torchvision' and 'torchaudio' could resolve a build from AMD's
index targeting a different torch major, causing ABI/version mismatches at
runtime. Now pinned to '>=0.26.0,<0.27.0' and '>=2.11.0,<2.12.0' respectively,
matching _ROCM_TORCH_CONSTRAINT['rocm7.2'].
worker.py: extend _hip_ver_at_least to handle AMD SDK wheel version strings.
The fallback regex r'rocm(\d+)\.(\d+)' cannot match '2.9.0+rocmsdk20251116'
(no rocmX.Y component), so the function always returned False on SDK/Radeon
wheels -- installing the Python _grouped_mm workaround on wheels that already
have the working HIP kernel. Added a second check: if the version string
contains '+rocmsdk', assume >= 7.13 (the rocmsdk format post-dates the
gfx120X null-kernel fix) and skip the fallback.
* fix(rocm): warn on OOB HIP_VISIBLE_DEVICES, bail on empty numeric_ids mask
- setup.ps1: when HIP/ROCR_VISIBLE_DEVICES names an index beyond the
detected GPU count, emit a yellow warning and fall back to GPU 0
instead of silently reading allGfxArches[-1] (wrong arch)
- hardware.py _reconcile_primary_rocm_unified_memory: distinguish
numeric_ids=None (no env var, use torch ordinal 0) from numeric_ids=[]
(empty mask / HIP_VISIBLE_DEVICES=-1, no GPU visible); bail out early
in the empty case to avoid querying torch.device(0) incorrectly
* fix(rocm): gate StubSubpackageFinder on win32 ROCm, add gcnArchName fallbacks
- worker.py _StubSubpackageFinder: the meta_path append was running on
every platform on every call to run_training_process; moved it inside
the if _is_win32_rocm: block since stubs are only seeded there and the
finder is a pure accumulation on Linux/Windows CUDA
- worker.py OOM guard: AMD SDK / Radeon wheels may not populate
gcnArchName, causing Strix Halo to be misclassified as discrete and
get the 0.90 cap (12.8 GB OS headroom) instead of 0.80 (25.6 GB);
now tries gcn_arch_name / arch_name / gfx_arch_name variants first,
then falls back to device-name matching (890M -> Strix Halo,
880M -> Strix Point) with a debug log when the fallback fires
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm): pin torchvision/torchaudio in setup.ps1, remove -Unique from arch array
- setup.ps1 ROCm torch install: torchvision and torchaudio were passed
bare alongside pinned torch>=2.11.0,<2.12.0 for gfx1151/gfx1200 arches.
AMD publishes packages independently so a future torchvision 0.27 (for
torch 2.12) on the same arch index would cause pip ResolutionImpossible
or an ABI-incompatible install. Added torchvisionFloorMap and
torchaudioFloorMap mirroring install_python_stack.py's strix override
(torchvision>=0.26.0,<0.27.0, torchaudio>=2.11.0,<2.12.0) and derived
ROCmVisionSpec/ROCmAudioSpec used in all three Fast-Install call sites.
- setup.ps1 amd-smi arch detection: Select-Object -Unique was collapsing
same-arch multi-GPU arrays (e.g. two gfx1151 APUs -> 1-element array)
causing HIP_VISIBLE_DEVICES=1 to trigger a false out-of-range warning
and fall back to GPU 0 even though the correct GPU would have been at
index 1. Removed -Unique; added comment noting the positional-index
assumption and its non-contiguous-GPU limitation.
* fix(rocm): add 8060s/8050s to OOM guard device-name fallback, extract classifier helper
Path 3 of the OOM guard device-name fallback only checked for 890m/880m
(gfx1150 Strix Point SKU names). Strix Halo (gfx1151) ships as Radeon 8060S
(Ryzen AI MAX+ 395) and Radeon 8050S (cut-down SKU) -- neither matches, so
the fallback returned is_unified=False and applied the 0.90 fraction instead
of 0.80, leaving ~12.8 GiB OS headroom on a 128 GiB pool instead of ~25.6 GiB.
Fix: add 8060s and 8050s to the name-match set. Also correct the comment that
mislabelled 890M as a Strix Halo name (it is Strix Point).
Refactor: extract the three-path classifier into _rocm_classify_unified_memory()
so it can be unit-tested directly. Add 31 test cases in test_rocm_oom_guard.py
covering all three paths and the regression case (Radeon 8060S Graphics).
Reported-by: h34v3nzc0dex
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm): pass explicit dtype on bf16-unsupported hardware (RDNA2)
dtype=None lets unsloth auto-detect the model dtype. On RDNA2 (gfx103x,
e.g. RX 6600) is_bfloat16_supported() incorrectly returns True, so unsloth
picks bf16 and the first bf16 kernel dispatch triggers:
LLVM ERROR: Cannot select: intrinsic %llvm.amdgcn.fdot2.bf16.bf16
Replace every dtype=None in load_model() with _auto_dtype which resolves
to None when bf16 is supported (all modern NVIDIA + RDNA3+) and
torch.float16 otherwise. This gives RDNA2 users a working float16
training path without touching NVIDIA behaviour at all.
Fixes: https://github.com/unslothai/unsloth/issues/5337
* fix: reduce log noise for expected non-issues on Windows ROCm
Three log lines fired at warning/error level for conditions that are
completely expected on a Windows HIP SDK-only setup:
amd.py
- amd-smi WinError 2 (FileNotFoundError): downgrade warning -> debug.
amd-smi ships with Adrenalin, not the HIP SDK; absence is normal.
- 'disabling' message: downgrade warning -> info with clearer text
'not available (not installed; expected on HIP SDK-only systems);
GPU VRAM polling disabled'
hardware.py
- torch.distributed.Store missing: downgrade warning -> debug.
The distributed stub added in this PR intentionally omits Store; the
attention-impl fallback to eager is expected and non-actionable.
worker.py
- causal-conv1d: add early Windows exit (info) in both
_ensure_causal_conv1d_fast_path and _causal_conv1d_install hook;
no cp313/win_amd64 wheel exists, so the install always fails.
- FLA: add early Windows exit (info) in
_ensure_flash_linear_attention_unconditional; triton dependency has
no cp313/win_amd64 wheel.
- Defense-in-depth: _install_package_wheel_first non-HIP PyPI failure
logs info+debug on Windows instead of error; FLA failure logs
info+debug on Windows instead of warning.
* [AMD] FIx installation of bitsandbytes when it's from .dev and skip rebuilding llama.cpp if we build it manually.
* fix: use force_pip for Windows ROCm bitsandbytes prebuilt wheel install
uv rejects the bnb continuous-release wheel due to filename/metadata
version mismatch (1.33.7.preview vs 0.50.0.dev0). Switch to force_pip=True
(pip bypass) instead of the UV_SKIP_WHEEL_FILENAME_CHECK env var workaround
-- cleaner and consistent with how the Linux path handles it.
BNB_ROCM_VERSION is still set post-install to the detected DLL suffix so
the worker subprocess loads the correct libbitsandbytes_rocm{VER}.dll even
when torch.version.hip reports a newer HIP version than the wheel ships.
* fix: three small correctness fixes found in PR review
- _install_bnb_windows_rocm: use UV_SKIP_WHEEL_FILENAME_CHECK=1 with
try/finally instead of force_pip=True so the env var is always
restored and the failing CI test passes
- _determine_attention_impl_for_gpu_estimate: gate torch._C distributed
stubs on IS_ROCM so Windows CUDA users keep the real extension
- install.ps1 amd-smi fallback: collect all gfx tokens and index by
HIP_VISIBLE_DEVICES, matching the hipinfo path on multi-GPU hosts
* fix: stub torchao in export subprocess on Windows ROCm
On Windows, the ROCm build of PyTorch ships without the distributed
C extension (torch._C._distributed_c10d). torchao, which is pulled in
transitively by transformers.quantizers at import time, walks into
torch.distributed._functional_collectives -> distributed_c10d and
crashes with:
No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package
This only affected the export subprocess because the training subprocess
already applied an identical torchao stub (introduced separately to fix
the same root cause). The export subprocess had no such guard and died
during 'Importing Unsloth...' before any model loading could happen.
Fix: apply the same _StubSubpackageFinder / torchao stub pattern to the
export subprocess entry point, gated on Windows ROCm detection, before
any import of transformers or unsloth_zoo.
Root cause tracked in ROCm/TheRock#3284 (libuv / torch.distributed
missing on Windows ROCm builds).
Ref: https://github.com/ROCm/TheRock/issues/3284
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh, setup.sh: add GPU arch step logging to match PS1 scripts
Both shell scripts were missing the step "gpu" terminal log block that
install.ps1 and setup.ps1 emit. This adds equivalent output: GPU label
with gfx arch (e.g. "AMD ROCm (gfx1151)"), ROCm root path, hipconfig
version, and marketing name substep. Includes the same gfx arch detection
chain (rocminfo → amd-smi list → amd-smi static --asic), UNSLOTH_ROCM_GFX_ARCH
env override, and name-based arch inference table (Strix Halo/Point, RDNA 3/4)
as the PS1 versions. install.sh also replaces bare echo blocks for the AMD
ROCm and CPU-only cases with formatted substep output.
* Fix BNB_ROCM_VERSION gate, ROCm GPU mask preference, APU unified memory and Release build for PR #5301
- main.py: gate BNB_ROCM_VERSION on the rocm bnb DLL or HIP_PATH/ROCM_PATH instead of importing torch on every Windows host
- hardware.py: prefer HIP/ROCR visible-device masks only on ROCm hosts so a stale mask cannot override CUDA_VISIBLE_DEVICES on NVIDIA
- llama_cpp.py: set GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 only for unified-memory APUs (gfx1150/gfx1151)
- setup.sh: pass -DCMAKE_BUILD_TYPE=Release for the HIP source build
- add test_amd_apu_unified_memory.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: guard recompile_limit + fix AMD VRAM monitor fallback
trainer.py: torch._dynamo.config.recompile_limit does not exist in
some ROCm torch builds (e.g. pytorch.org/whl/rocm6.2 wheels). Guard
the assignment so training doesn't crash on RDNA2/RDNA3.
hardware.py: when amd-smi/nvidia-smi is unavailable or returns no
usable data (HIP SDK-only Windows, Docker, unexpected JSON format),
the existing fallback used torch.cuda.memory_allocated() which is
process-specific and reads near-zero even with a fully loaded model.
Switch to torch.cuda.mem_get_info() via _torch_get_per_device_info()
which reports system-wide VRAM occupancy so the GPU monitor shows
real usage on all AMD systems without requiring amd-smi.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: Windows VRAM monitor via Performance Counter API
When amd-smi/nvidia-smi is unavailable on Windows, query dedicated GPU
VRAM via Windows Performance Counters (same source as Task Manager).
This gives system-wide cross-process usage, fixing the near-zero reading
caused by torch.cuda.mem_get_info only seeing the Studio server process.
Linux fallback path unchanged (mem_get_info is system-wide on ROCm).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: rename to _rocm_windows_perf_counter_vram_gb, scope to IS_ROCM
Function is AMD ROCm specific — amd-smi absent on Windows when only the
HIP SDK is installed. Scoped to IS_ROCM so NVIDIA Windows path is
untouched (nvidia-smi handles that case).
* fix: AMD VRAM monitor — Linux DRM sysfs + Windows perf counter
Linux: read /sys/class/drm/card*/device/mem_info_vram_used|total for
system-wide GPU memory across all processes. No tools required, always
present on Linux AMD systems.
Windows: Windows Performance Counter API (already added).
Both paths are gated on IS_ROCM and only fire when amd-smi is absent.
torch mem_get_info remains as last resort (process-local).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: AMD GPU monitor — utilization, temperature, and power for Windows and Linux fallback paths
- Windows: GPU utilization via \GPU Engine(*engtype_3D*)\Utilization Percentage perf counter
- Windows: temperature and power via ADL (atiadlxx.dll, ships with Adrenalin)
- Linux: GPU utilization via DRM sysfs gpu_busy_percent
- Linux: temperature via hwmon temp1_input (millidegrees C)
- Linux: power via hwmon power1_average / power1_input (microwatts)
All paths are no-op fallbacks (None) when the source is unavailable.
Mirrors what nvidia-smi provides on the CUDA path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: remove ADL ctypes — does not support AMD iGPU (Strix Halo)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Detect CUDA UMD Version from newer nvidia-smi output (#5812)
Newer NVIDIA drivers (e.g. 610.x on Windows) print the driver's max
CUDA capability as "CUDA UMD Version: X.Y" instead of the legacy
"CUDA Version: X.Y" header. The installers and Studio setup scripts
were only matching the legacy spelling, so on a fresh RTX 5090
laptop with a 13.x driver they failed to detect any CUDA version
and fell through to the cu126 wheel default.
Accept both spellings everywhere we parse nvidia-smi output:
- install.ps1: Get-TorchIndexUrl regex now allows " UMD"
- install.sh: two-expression sed (POSIX BRE has no "?"); the two
patterns are mutually exclusive per line, head -1 picks the match
- studio/setup.ps1: Get-PytorchCudaTag and the $DriverMaxCuda
detector both relaxed
- studio/install_llama_prebuilt.py: substring scan replaced with a
regex search using the same pattern
- tests/sh/test_get_torch_index_url.sh: new make_mock_smi_umd helper
plus three UMD cases (13.3 -> cu130, 12.8 -> cu128, 11.8 -> cu118);
all 30 tests pass locally
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
PyPI release unsloth 2026.5.8 is now live. Bumps the pinned floor in
install.sh and install.ps1 from unsloth>=2026.5.7 to unsloth>=2026.5.8
so fresh installs resolve to the new wheel.
* Fix unsloth studio update silently downgrading on macOS arm64
Root cause: studio/install_python_stack.py's "Updating base packages"
step passes `--upgrade-package unsloth -r base.txt -c constraints.txt`
with base.txt's `unsloth` and `unsloth-zoo` entries unpinned. On macOS
arm64 the resolver silently backtracks to an older unsloth (2026.5.2 or
even 2025.7.2) whenever a transitive constraint (the most common one is
bitsandbytes wheel availability: 0.49.0+ ships macosx_14_0_arm64 wheels,
older versions do not) makes the unpinned requirement satisfiable by an
older release. install.sh already maintains an explicit `unsloth>=N.N.N`
floor for the same reason, but the floor was missing from the in-venv
update path.
Reproduced on macos-14 across 2026.3.18 / 2026.4.8 / 2026.5.2 / 2026.5.6
starting states. All four ended on unsloth==2026.5.2 after a clean
`unsloth studio update` invocation (2026.5.6 was a true downgrade,
others were stale or partial advances).
Fix mirrors install.sh: query PyPI at runtime for the current latest
version of unsloth and unsloth-zoo, then pass `unsloth>=<latest>` and
`unsloth-zoo>=<latest>` as extra positional pins alongside the existing
`--upgrade-package` flags. Network failures fall back to the historical
unpinned behaviour so offline installs continue to work. Applied to all
three upgrade branches (standard update, local-repo overlay, no-torch).
Also fix the cosmetic `Hardware detected: MLX -- Apple Silicon (i386)`
banner. platform.processor() reads `uname -p` which returns "i386" on
many universal2-shaped Python builds even on a native arm64 interpreter;
platform.machine() is the reliable source ("arm64" once is_apple_silicon
has gated us).
* Dedup floor-pin call sites + LRU cache PyPI lookup
Three upgrade branches each rebuilt the same conditional `unsloth>=` /
`unsloth-zoo>=` arg list with two PyPI round-trips per branch -- six
round-trips per `unsloth studio update` invocation. Extract a
`_pin_floor_args(*, include_unsloth=True)` helper and wrap
`_resolve_latest_pypi_version` in `functools.lru_cache` so the three
branches share a single PyPI request per package.
Functionally equivalent; pure cleanup on top of the previous commit.
* Warn when PyPI is unreachable so the silent fallback is visible
If `_resolve_latest_pypi_version` returns None for either lookup the
floor args are silently dropped, which restores the pre-fix resolver
behaviour. Print a single cyan `warning` line in `_pin_floor_args` when
that happens so users behind a proxy / captive portal / firewalled
PyPI mirror know the upgrade has degraded -- and can supply network
egress or a `--index-url` mirror and retry.
* Soft floor with unpinned-fallback for hosts where floor is unsatisfiable
Reviewer found that the unconditional unsloth-zoo>=LATEST floor turns
a previously-resolvable macOS 13 arm64 update into a hard resolver
failure: unsloth-zoo 2026.5.4 requires mlx-vlm>=0.4.4 -> mlx>=0.30.0,
and mlx 0.30+ only publishes macosx_14_0_arm64 wheels. The pre-fix
behaviour backtracked to an older unsloth instead of erroring. We
should not turn "stale" into "fail".
Add pip_install_with_floor_fallback: first try the install with the
floor appended; if the resolver cannot satisfy it (subprocess exit
code != 0), retry the install without the floor and print a clear
warning. The fall-through preserves the legacy "succeed-but-stale"
contract on hosts where wheel availability is the bottleneck.
Also extend pip_install_try with a req= kwarg so the floor attempt
can pass `-r base.txt` like pip_install does, and add an
UNSLOTH_NO_PYPI_FLOOR=1 opt-out for air-gapped CI / corporate PyPI
mirrors that intentionally do not expose pypi.org directly.
All three upgrade branches (standard, local-repo, no-torch) now go
through the helper so the fallback behaviour is consistent.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add second fallback level: floor without constraints
macOS arm64 floored attempt with -c constraints.txt fails because the
single-env constraint `transformers==4.57.6` conflicts with the new
unsloth-zoo 2026.5.4 -> mlx-vlm 0.4.4+ -> transformers>=5.1.0 chain.
First fallback level retries the floored install without constraints
(transformers freely resolves to a mlx-vlm-compatible version);
downstream pip_install calls still apply constraints.txt to anything
that doesn't transitively conflict.
If THAT still fails (wheel availability rather than constraint
conflict), drop the floor and fall back unpinned as before.
Verified locally with uv pip compile against aarch64-apple-darwin
python-3.13: strict-constrained floor errors, no-constraint floor
resolves cleanly to unsloth==2026.5.7 + unsloth-zoo==2026.5.4 +
transformers==5.5.0 + mlx-vlm==0.5.0.
* setup.sh/.ps1: also gate fast-path on unsloth-zoo being up to date
The version-check fast-path in setup.sh / setup.ps1 only looked at
unsloth itself. If unsloth was at the PyPI latest but unsloth-zoo was
stale, the gate set _SKIP_PYTHON_DEPS=true and install_python_stack.py
never ran -- so the new floor pin from PR #5767 had no effect for the
exact "unsloth at latest, zoo behind" state several reviewers flagged.
Probe both packages' installed-vs-latest versions and only skip the
deps step when BOTH match. When either is behind, fall through to
install_python_stack.py so the new resolver fix gets a chance to run.
Verified setup.sh with `bash -n`; the setup.ps1 change uses PowerShell
if-expressions for the null-default pattern rather than bash-style
${var:-default} which is not valid PowerShell.
* Skip unsloth-zoo floor too for custom no-torch test packages
Reviewer found the asymmetric guard: the no-torch branch was already
gating the unsloth floor on package_name == "unsloth" (test side
packages may not publish to PyPI), but the unsloth-zoo floor was
still added unconditionally. A custom no-torch update that ships its
own forked zoo metadata could now hit a public PyPI floor that does
not match the fork's published version.
Add a symmetric `include_zoo` parameter to `_pin_floor_args` and
gate both pins on the same `package_name == "unsloth"` check.
* Address review feedback: simpler except clause + private-index note
Gemini flagged TimeoutError in the PyPI fetch exception list. OSError already
covers socket timeouts and the 3.11+ TimeoutError subclass on every supported
Python, so drop the redundant entry and explain what each remaining exception
catches.
Codex flagged that floor lookups against pypi.org could break installs behind
a lagging private mirror. Step 3 of pip_install_with_floor_fallback already
recovers transparently in that case; expand the docstring so the behavior is
discoverable without reading the body.
* extras-no-deps: skip transformers==4.57.6 on macOS arm64
Reviewer flagged that the resolver-selected transformers from the
no-constraints base step on macOS arm64 (transformers 5.x for mlx-vlm
0.4.4+) gets silently downgraded back to 4.57.6 by extras-no-deps.txt
during the very next step, breaking mlx-vlm imports at runtime even
though unsloth itself reports as latest.
Add a PEP 508 platform marker so the pin only applies off macOS arm64.
constraints.txt still enforces 4.57.6 everywhere else; mlx-vlm only
publishes wheels for darwin arm64, so other platforms are unaffected.
* setup.sh/.ps1: gate fast-path zoo probe on _PKG_NAME == unsloth
Reviewer found the asymmetric custom-package regression: the new
zoo-aware fast-path probes public unsloth-zoo unconditionally, but a
custom STUDIO_PACKAGE_NAME side build may ship its own zoo fork via
dependency metadata and not install public unsloth-zoo at all. The
previous behaviour (skip Python deps if the custom package itself is at
its declared latest) is preserved by only running the zoo probe when
the managed package literally IS unsloth.
Matches the include_zoo gate already in _pin_floor_args() at
install_python_stack.py.
* install_python_stack: all-or-nothing floor + uv-to-pip retry
Two reviewer findings on the floor-pin helpers:
1. _pin_floor_args() previously kept a half-floor if one PyPI lookup
succeeded and the other failed. With unsloth at latest but the zoo
lookup down, the resolver could still backtrack zoo while we
required unsloth at latest, defeating the pin. Return [] on any
lookup failure so the unpinned legacy path runs cleanly.
2. pip_install_try() ran ONLY uv when USE_UV was true; a uv-specific
failure short-circuited to False even when pip itself could have
applied the floor. Mirror pip_install()'s uv-to-pip fallback: try
uv, fall through to pip on non-zero exit, and only then give up.
* extras-no-deps: rewrite marker without `not` for PEP 508 parsers
pip's vendored packaging rejects `not (...)` in PEP 508 markers; the
grammar only specifies `and` / `or` between boolean atoms. The staging
macos-14 matrix failed every job at "Installing extras (no-deps)" with
`Expected a marker variable or quoted string`. Apply De Morgan's law
so the marker uses `or` between two `!=` checks, which both pip and
uv parse cleanly. Behaviour identical: skip the 4.57.6 pin only on
darwin arm64; pin everywhere else.
* constraints: skip transformers==4.57.6 pin on macOS arm64 too
Marker-gating the extras-no-deps.txt pin was not sufficient. Every
subsequent pip_install in the update pipeline passes
-c single-env/constraints.txt, and constraints.txt itself pinned
transformers==4.57.6 unconditionally. The latest staging-2 run shows
the base step's no-constraints fallback installed transformers 5.5.0
correctly, but a later constrained step (extras / studio / data-designer
deps) silently downgraded it back to 4.57.6, leaving mlx-vlm 0.5.0
in the venv with an unsatisfied transformers>=5.5.0 requirement.
Apply the same `sys_platform != "darwin" or platform_machine != "arm64"`
marker to the constraints.txt entry so it is inert on darwin arm64.
Other platforms still pin 4.57.6 because mlx-vlm only publishes wheels
for darwin arm64; no other platform is affected.
* constraints: carve out darwin arm64 from every == pin
Marker-gating only transformers was not enough; staging-2 still failed
with the same `transformers==4.57.6 in venv after the update` outcome
because the resolver hit a `huggingface-hub==0.36.2` (and adjacent)
conflict with mlx-vlm's `huggingface-hub>=1.5.0` requirement, then
fell back to a stale stack even after my no-constraints level fired
on the base step.
Apply the same `sys_platform != "darwin" or platform_machine != "arm64"`
marker to every == pin in constraints.txt. Range pins (mcp, fastmcp,
websockets) stay active everywhere because they do not conflict with
the mlx-vlm chain. mlx-vlm only publishes wheels for darwin arm64, so
no other platform is affected.
* install_python_stack: also --upgrade-package transformers and mlx-vlm
Staging-2 showed that even after the constraints.txt carve-out for
darwin arm64, the venv still ended up with the OLD `transformers==4.57.6`
paired with a NEW `mlx-vlm==0.5.0` from unsloth-zoo's transitive
upgrade. The resolver's --upgrade-package flag only freshens the named
packages and their newly-pulled transitive deps; transformers was
already installed at a version that satisfied unsloth-zoo's range
(`>=4.51.3,<=5.5.0` with exclusions), so the resolver did not upgrade
it -- even though mlx-vlm 0.5.0 requires `transformers>=5.5.0`.
Add `--upgrade-package transformers` and `--upgrade-package mlx-vlm`
to all three base-step branches. Both are no-ops when the package is
absent (mlx-vlm only ships wheels on darwin arm64); on darwin arm64
this is what nudges the resolver to upgrade both together so the
final venv is internally consistent. On Linux/Windows, transformers
stays at 4.57.6 because constraints.txt still pins it there and
mlx-vlm never enters the resolution.
* install_python_stack: explicit mlx-vlm + transformers realign on macOS arm64
Even with --upgrade-package hints, uv leaves the venv with the
already-installed transformers (4.57.6 inherited from the OLD venv's
constrained install) when that version still happens to satisfy
unsloth's own metadata range -- but it does not also re-resolve
mlx-vlm's stricter `transformers>=5.5.0` requirement, so the venv
ends up with mlx-vlm 0.5.0 paired with transformers 4.57.6 and
mlx-vlm imports break at runtime.
After the base step, on darwin arm64 only, run an explicit
`pip install --upgrade mlx-vlm transformers` with constrain=False.
This forces both packages through the resolver again as direct
top-level requirements, so transformers is pulled up to whatever
mlx-vlm's metadata requires (5.5.0 today). No effect on any other
platform because mlx-vlm has no wheels off darwin arm64 and the
branch is gated on IS_MAC_ARM.
* requirements: marker-gate every == pin that conflicts with mlx-vlm chain
Staging-2 kept ending up with transformers==4.57.6 even after the
realign step, because studio.txt unconditionally pins
huggingface-hub==0.36.2 (and datasets==4.3.0). Installing studio.txt
with constraints active pulls the resolver back to a huggingface-hub
that only recent transformers (4.x) supports, which silently downgrades
the realigned 5.5.0 to 4.57.6 -- exactly the inconsistency we tried to
prevent.
Also extras-no-deps.txt still pinned trl==0.23.1 unconditionally; the
0.23.1 wheel transitively requires huggingface-hub<1, same coupling.
Marker-gate all three. The carve-out is identical to constraints.txt's:
inactive on darwin arm64 (where the mlx-vlm chain dictates newer
versions), active everywhere else (where Linux/Windows users rely on
the single-env pins). mlx-vlm only publishes wheels for darwin arm64
so no other platform is affected.
* realign: --force-reinstall mlx-vlm + transformers + huggingface_hub
Plain --upgrade does not force uv to re-resolve mlx-vlm's transformers
requirement when the already-installed transformers happens to satisfy
unsloth's own range. Switch to --force-reinstall on the three packages
so the resolver tears them down and brings them back together with
consistent versions. Include huggingface_hub because transformers 5.x
requires hf-hub>=1.5.0 and the resolver would not touch it otherwise.
* realign: pin transformers via mlx-vlm's own metadata spec
`pip install --force-reinstall mlx-vlm transformers` still resolved to
an already-installed transformers 4.57.6 because uv treats it as
satisfying unsloth's transformers range without re-checking mlx-vlm's
stricter requirement. Pull mlx-vlm's actual transformers specifier
from its installed metadata at runtime and pass it as an explicit
version requirement (e.g. `transformers>=5.5.0` for mlx-vlm 0.5.0).
That removes the resolver's wiggle room: it MUST pick a transformers
satisfying mlx-vlm AND unsloth, which on darwin arm64 with the latest
unsloth-zoo means transformers==5.5.0. Falls back to unpinned
`transformers` if metadata read fails, so this never errors.
* realign: uninstall-then-install to bypass uv's incumbent bias
Every flag-based approach failed: --upgrade, --upgrade-package,
--force-reinstall, and even an explicit `transformers>=5.5.0`
requirement all left the venv with transformers==4.57.6 because uv
treats the already-installed version as satisfying unsloth-zoo's
range and refuses to disturb it, even when it does not satisfy
mlx-vlm's stricter requirement.
Replace the realign step with an explicit uninstall of the conflicting
trio (transformers / mlx-vlm / huggingface_hub) followed by a fresh
install. With no transformers in the venv, the resolver MUST pick a
version satisfying every installed package's metadata, which on
darwin arm64 with the latest unsloth-zoo is uniquely 5.5.0.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim verbose comments across PR #5767 changes
* Simplify mac-arm64 fix: install MLX stack with --no-deps
The previous approach (PyPI floor pin + 3-level fallback + macOS arm64
realign step + marker carve-outs on every == pin) was fighting symptoms.
The root cause is that unsloth-zoo declares mlx-vlm>=0.4.4 as a darwin
arm64 dep, and mlx-vlm 0.5.0's metadata pulls in transformers>=5.5.0,
which conflicts with the main venv's transformers==4.57.6 pin and forces
the resolver to backtrack unsloth.
Severing that chain at its source: install mlx + mlx-metal + mlx-lm +
mlx-vlm with --no-deps BEFORE unsloth-zoo. The resolver sees mlx-vlm
already installed (>=0.4.4) and never inspects its transformers metadata.
Per-model transformers version routing is already handled at runtime by
the side-car venvs in utils/transformers_version.py (.venv_t5_530 for
Ministral/GLM/Qwen3 MoE, .venv_t5_550 for Gemma 4).
Net change: -224 / +71 lines across install.sh, install_python_stack.py
and the three requirements files.
Reverted:
- _resolve_latest_pypi_version + _pin_floor_args + pip_install_with_floor_fallback
- macOS arm64 realign step (pip uninstall + reinstall)
- --upgrade-package transformers --upgrade-package mlx-vlm in base steps
- All ; sys_platform != "darwin" or platform_machine != "arm64" markers
in constraints.txt, studio.txt, extras-no-deps.txt
- pip_install_try restored to its pre-PR signature
Added:
- install.sh: Apple Silicon MLX --no-deps install before unsloth (both
fresh and migrated branches)
- install_python_stack.py: same step gated on IS_MAC_ARM and not skip_base
Kept (independent bugs):
- setup.sh / setup.ps1 dual-package zoo version check
- platform.processor() -> platform.machine() hardware-detect fix
* Minimise PR to mac-arm64-specific changes only
Revert setup.sh and setup.ps1 to main -- the dual-package zoo check was
defensive and not strictly needed once mlx-vlm is installed --no-deps
(the resolver-backtrack scenario that produced stale zoo no longer happens).
Tighten remaining comments in install.sh and install_python_stack.py.
Final PR-attributable changes:
install.sh +24/-5 (MLX --no-deps in 2 places)
studio/install_python_stack.py +19 (MLX --no-deps + IS_MAC_ARM)
studio/backend/utils/hardware/hardware.py +6/-6 (processor() -> machine())
studio/backend/requirements/*.txt unchanged
* Revert "Minimise PR to mac-arm64-specific changes only"
This reverts commit 9470daa855.
* Revert "Simplify mac-arm64 fix: install MLX stack with --no-deps"
This reverts commit f8a43b87e8.
* Revert "Trim verbose comments across PR #5767 changes"
This reverts commit c3f293a10f.
* Simplify mac-arm64 fix: --no-deps MLX + METADATA patch
Root cause: unsloth-zoo declares mlx-vlm>=0.4.4 as a darwin-arm64 dep, and
mlx-vlm 0.5.0's published metadata declares transformers>=5.5.0. Every
subsequent resolver run with constraints.txt's transformers==4.57.6 sees
the conflict and backtracks unsloth to escape it (user-reported downgrade).
The aggressive pin doesn't reflect what mlx-vlm actually requires at
top-level import time -- the symbols it loads (AutoProcessor, AutoTokenizer,
ProcessorMixin, BatchFeature) are stable across transformers 4.51+. Model-
specific submodules that genuinely need 5.x APIs are only loaded once the
3-tier transformers dispatcher (utils/transformers_version.py) has activated
the matching .venv_t5_530 / .venv_t5_550 side-car at runtime.
Fix: on Apple Silicon, install the MLX stack with --no-deps then rewrite
mlx-vlm/mlx-lm's installed METADATA to declare transformers>=4.51.3. Now
the resolver sees mlx-vlm 0.5.0 as compatible with the main venv's
transformers==4.57.6 and there's nothing to backtrack.
Reverts the previous heavy machinery:
- _resolve_latest_pypi_version, _pin_floor_args, pip_install_with_floor_fallback
- macOS arm64 realign step (pip uninstall + reinstall)
- --upgrade-package transformers --upgrade-package mlx-vlm in base steps
- All ; sys_platform != "darwin" or platform_machine != "arm64" markers
in constraints.txt / studio.txt / extras-no-deps.txt
- setup.sh / setup.ps1 dual-package zoo check (Windows never had the bug;
with this fix in place stale zoo no longer happens on macOS either)
- pip_install_try restored to pre-PR signature
Kept:
- install.sh: MLX --no-deps install in fresh + migrated branches
- install_python_stack.py: same step gated on IS_MAC_ARM and not skip_base
- _relax_mlx_metadata() helper, called immediately after each MLX install
- studio/backend/utils/hardware/hardware.py: platform.processor() ->
platform.machine() cosmetic fix
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use UV_OVERRIDE to relax mlx-vlm transformers pin
uv supports --overrides / UV_OVERRIDE which globally overrides any package's
stated dependency requirement. mlx-vlm 0.5.0 declares transformers>=5.5.0
and mlx-lm 0.31.3 declares transformers>=5.0.0; neither is true at top-level
import time (their imports use AutoProcessor / AutoTokenizer / ProcessorMixin /
BatchFeature which are stable across transformers 4.51+). Per-model 5.x
routing is handled at runtime via the .venv_t5_530 / .venv_t5_550 side-cars.
Override file (overrides-darwin-arm64.txt) declares transformers>=4.51.3 ;
exported via UV_OVERRIDE env var on Apple Silicon by both install.sh and
install_python_stack.py. uv then resolves mlx-vlm as compatible with the main
venv's transformers==4.57.6 (constraints.txt) and unsloth advances cleanly to
LATEST.
Drops, vs. the previous attempts:
- _resolve_latest_pypi_version + _pin_floor_args + pip_install_with_floor_fallback
(floor-pin machinery -- replaced by single UV_OVERRIDE line)
- macOS arm64 realign step (pip uninstall + reinstall)
- --upgrade-package transformers --upgrade-package mlx-vlm in base steps
- All ; sys_platform != "darwin" or platform_machine != "arm64" markers
- _relax_mlx_metadata() helper + sed METADATA patch (uv reads from index, not
dist-info, so dist-info patches were ineffective)
Kept:
- install.sh / install_python_stack.py: MLX latest install on Apple Silicon
(now without --no-deps, the override lets the resolver pick a consistent set)
- studio/backend/utils/hardware/hardware.py: platform.machine() cosmetic fix
* Trim UV_OVERRIDE comments; bump override floor to 4.57.6
Match the main venv's constraints.txt pin exactly so the override file
reads as the actual installed version rather than mlx-vlm's API floor.
Comments collapsed to one-liners where possible.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: auto-recover when shadowed 'unsloth' on PATH hides the frontend dist
The CLI launcher derives `_PACKAGE_ROOT` from where `unsloth_cli` imports
from, and `studio/backend/run.py` derives its default `frontend_path` from
`Path(__file__).resolve().parent.parent / "frontend" / "dist"`. When
another `unsloth` (a separate venv with `pip install unsloth`, a system
install, an older venv earlier on PATH) wins `which unsloth`, both
resolve into a site-packages tree that ships frontend source files but no
vite-built `dist/`. The backend warned `[WARNING] Frontend not found at
...` and then happily served 200 on every `/api/*` route while returning
`{"detail":"Not Found"}` on `/`. The 404 was silent to users -- the
process was healthy, the log line scrolled by, and the only symptom was a
blank browser tab.
This is a real situation: many devboxes carry a workspace venv with
`unsloth` installed years before the user runs `curl|sh` to install
Studio. The installer-managed binary at `~/.local/bin/unsloth` exists
but loses to the older venv on PATH order.
Three layers of fix, additive:
Layer C -- runtime auto-discovery (unsloth_cli + run.py)
The CLI now resolves `--frontend` explicitly before spawning `run.py`,
probing in order: package-local default, installer venv site-packages
(`$STUDIO_HOME/unsloth_studio/lib/python*/site-packages/...` and the
Windows `Lib/site-packages/...` equivalent), and editable-install source
roots read from `__editable___*_finder.py` MAPPING dicts in the installer
venv. `run.py` does the same probe as a backstop for direct `python
run.py` invocations.
Layer E -- loud structured error
The silent `[WARNING]` is replaced with a `SystemExit` that names every
candidate path tried and lists the four one-line fixes (run the absolute
path, pass `--frontend`, pass `--api-only`, reinstall). Suppressed only
in `--api-only` mode where no UI is served by design.
Layer F -- installer self-check (install.sh + install.ps1)
At the tail of install, both installers compare `command -v unsloth`
(POSIX) / `Get-Command unsloth` (PowerShell) against the just-installed
binary. If a different path wins, a yellow `warning` block names the
shadowing binary and prints the alias / absolute-path / PATH-reorder
fixes. install.sh uses the venv Python for path canonicalization so it
also works on macOS (BSD `readlink` has no `-f`).
Cross-platform notes:
- Glob patterns probe both `lib/python*/site-packages` (POSIX) and
`Lib/site-packages` (Windows).
- Canonical-binary path branches on `sys.platform == "win32"` to pick
`unsloth.exe` over `unsloth`.
- install.sh fixed for macOS; install.ps1 is the Windows analog.
Tests: `studio/backend/tests/test_frontend_resolution.py` covers five
cases via AST-load of the helpers (no uvicorn / FastAPI import needed,
matching `test_host_defaults.py`'s style):
1. Resolver returns None when nothing exists anywhere.
2. Resolver picks the first existing candidate when the default works.
3. Fallback to `$UNSLOTH_STUDIO_HOME` site-packages dist when the default
is missing.
4. Fallback to an editable-install source root via MAPPING parsing.
5. Resolver tolerates a non-existent `$UNSLOTH_STUDIO_HOME`.
All 5 new + 2 existing host-default tests pass.
* Studio: address review feedback on PR 5782 (Windows hardlink, Win path hint, broader tests)
Four parallel platform reviews (Windows, Linux, macOS, general) on the
initial commit surfaced a small batch of correctness items, all addressed
here:
Windows install.ps1 (medium severity, false positive on every install):
The user-facing shim at $StudioHome\bin\unsloth.exe is a hardlink to
$VenvDir\Scripts\unsloth.exe (created at line 1582). Resolve-Path does not
de-duplicate hardlinks, so the previous string compare always saw the two
paths as different and the new "another 'unsloth' wins on PATH" warning
would fire on every fresh Windows install. Switched to content-hash
equality via Get-FileHash, which collapses hardlinks, symlinks, and
identical copies to a single identity. Also restricted the probe to
Get-Command -CommandType Application so PowerShell aliases / functions /
scripts named "unsloth" don't false-trigger.
Windows run.py SystemExit hint (medium severity, defeats the recovery UX):
The structured error printed Path(STUDIO_HOME)/"unsloth_studio"/"bin"/
"unsloth.exe" on every platform, but on Windows the installer places the
shim at $STUDIO_HOME/bin/unsloth.exe (no unsloth_studio segment) and the
venv binary at $STUDIO_HOME/unsloth_studio/Scripts/unsloth.exe. The hint
pointed at a non-existent path on Windows. Branch on sys.platform ==
"win32" to emit the real shim location; Linux / macOS keep the unsloth_
studio/bin/unsloth layout.
MAPPING regex robustness (low):
[^\n]* silently failed if a future setuptools / black reformat wrapped
the MAPPING dict across multiple lines. Tightened to [^}]* + re.DOTALL,
which still rejects nested dicts (setuptools never emits those for
editable installs) but tolerates either single- or multi-line literals.
install.sh broken-venv edge case (low, macOS reviewer):
Previously _canon fell back to echoing the raw input when the venv python
failed, which would make two symlinked-but-identical paths look different
and false-trigger the warning. Now _canon returns empty on failure and
the caller skips the whole comparison if either side is unresolvable.
argparse default + log readability (nits):
run.py's argparse --frontend default now reuses the module-level
_DEFAULT_FRONTEND_PATH constant so it stays in lockstep with run_server's
default. The [OK] log message resolves the chosen path so support output
is always absolute.
Tests grow from 5 to 8 in studio/backend/tests/test_frontend_resolution.
py (10/10 with the existing host-default tests):
- Windows-layout fallback: Lib/site-packages with capital L.
- Multi-line MAPPING dict: locks in the [^}]* + re.DOTALL behaviour.
- SystemExit message contract: every actionable fix string and the
attempted-paths list must appear; pins the user-facing recovery
message so a future refactor doesn't drop a bullet.
End-to-end re-verified on this box: shadowing workspace_22/bin/unsloth
still serves 200 on / through the editable-finder fallback, with the
follow-up resolve-then-log change yielding [OK] Frontend loaded from
/mnt/disks/unslothai/ubuntu/unsloth/studio/frontend/dist.
Out of scope (called out by reviewers but deferred):
- _resolve_frontend_path candidate ordering still tries _PACKAGE_ROOT
first. For the rare case where a shadowing install carries an older
built dist, this serves the stale UI instead of the fresh one. Fix is
non-trivial (the --local workflow intentionally wants _PACKAGE_ROOT to
win when the cloned repo is the source of truth), so leaving it for a
follow-up.
- studio/backend/colab.py still bails out on missing frontend instead of
routing through the new resolver. Pre-existing behaviour, separate PR.
- _resolve_frontend_path is duplicated across run.py and unsloth_cli/
commands/studio.py. Minor maintenance concern; consolidation is
natural in a later refactor.
* Studio: guard ast.literal_eval result with isinstance(dict)
Addresses gemini-code-assist[bot] high-priority inline review on PR 5782
flagging that `mapping.get('studio')` could raise AttributeError if the
MAPPING regex matched a brace-delimited literal that ast.literal_eval
parsed as a non-dict (set, list, None). The regex `\{[^}]*\}` happily
matches `{1, 2, 3}` and literal_eval returns a set; the previous code
then crashed on .get().
Setuptools's editable-install template only emits dict literals so this
is defensive rather than a live bug, but the guard is one line per call
site and prevents a future template change from taking out backend
startup or CLI invocation.
Both call sites (studio/backend/run.py:558 and
unsloth_cli/commands/studio.py:234) now bail out on the finder file when
isinstance(mapping, dict) is False; the resolver keeps probing the
remaining finders, so a malformed entry in one finder cannot poison the
discovery of a good one elsewhere.
Adds test_resolver_does_not_crash_on_non_dict_mapping_literal to
test_frontend_resolution.py, which writes one bad finder (MAPPING is a
set literal) alongside one good finder (MAPPING is a real dict) and
asserts the resolver returns the good finder's dist path. Without the
guard this test crashes with AttributeError; with the guard it passes.
11/11 tests green.
PyPI release unsloth 2026.5.7 is now live. Bumps the pinned floor in
install.sh and install.ps1 from unsloth>=2026.5.6 to unsloth>=2026.5.7
so fresh installs resolve to the new wheel.
Tagged on main as v0.1.416-beta.
* ci: broaden Linux llama.cpp runtime pattern to lib*.so*
#5741 patched the explicit Linux pattern list to add
``libllama-*-impl.so*`` after ggml-org/llama.cpp#23462 (between
b9279 and b9283) split each binary's entry code into a paired
``lib<binary>-impl.so`` shared library. Same class of upstream
repackaging will hit us again whenever a new shared lib is added.
Mirror what macOS already does and replace the per-lib list with a
single ``lib*.so*`` glob. ``copy_globs`` (line 3614) unions
patterns, so the per-variant ``libggml-cuda.so*`` / ``libggml-hip.so*``
entries were never filtering anything; the spec lives in
``runtime_payload_health_groups`` (line 5209) which keeps the
explicit minimum-required list per variant.
Dry-run against b9296-bin-ubuntu-x64.tar.gz: 40 files copied (all
ggml, llama, mtmd, impl variants + the two binaries we ship), 22
skipped (other CLIs, rpc-server, LICENSE). Functionally equal to
the post-#5741 set.
* cleanup: trim #5741 comments on the pydantic split
Comments added in #5741 explained the original bug in full each
time. They are mostly redundant with the commit message and the PR.
Trim them to one short paragraph per site.
No behavior change.
* ci: narrow Windows runtime pattern to llama-server.exe + llama-quantize.exe
Studio only invokes llama-server and llama-quantize. Mac and Linux
already filter to those two binaries; Windows was the odd one out
with ``*.exe`` copying every CLI upstream ships (llama-cli,
llama-bench, llama-mtmd-cli, ...).
Dry-run on b9296 (win cpu-x64, cpu-arm64, cuda-13.1, hip-radeon):
20 unused EXEs skipped per variant, all DLLs (incl. the new
llama-*-impl.dll family) still copied via ``*.dll``.
``existing_install_matches_choice`` already checks llama-server.exe
exists explicitly (line 5297), so the health gate is unchanged.
Bundles three independent CI regressions hitting the maintainer PR
backlog. Each one is verified end-to-end on a staging fork against
real Ubuntu / macOS / Windows GitHub-hosted runners before this
lands.
1. Windows --no-torch install: pydantic + pydantic-core drift to
incompatible versions under `uv pip install --no-deps -r
no-torch-runtime.txt` because pip resolves each independently
from latest. pydantic.VERSION 2.13.4 pins pydantic-core==2.46.4
but pydantic-core 2.47.0 was the freshest published wheel, so
`import pydantic` raised
`SystemError: pydantic-core 2.47.0 is incompatible with the
current pydantic version`. Resolve pydantic WITH deps in a
focused pip call (install.sh, install.ps1,
install_python_stack.py) before the --no-deps no-torch-runtime
pass so pip pins pydantic-core to the version pydantic declares.
pydantic's transitive deps (annotated-types, pydantic-core,
typing-extensions, typing-inspection) are torch-free. Drop the
redundant `Patch Studio venv with full typer / pydantic dep
trees` workaround from the four Windows smoke YAMLs.
Supersedes #5733 + #5734.
2. Linux Studio Update CI: upstream llama.cpp b9261+ split each
binary's entry code into a paired `libllama-<binary>-impl.so`
shared library. `llama-server` and `llama-quantize` NEEDED-link
against `libllama-server-impl.so` / `libllama-quantize-impl.so`
with RUNPATH `$ORIGIN`, so the prebuilt overlay must copy those
alongside the binaries. Without that, ldd reports them missing,
preflight rejects, the installer falls back to source build, and
studio-update-smoke annotates `setup.sh idempotency regressed`.
Add `libllama-*-impl.so*` to the Linux runtime patterns and lock
the pattern in test_rocm_support.TestRuntimePatterns.
3. Mac Studio UI Chat: change-password submit clicked while
disabled. The disable gate only checked new + confirm password
length, but Playwright's first click landed before the
current-password field's React state had committed, so the form
was simultaneously logically-invalid (current_password empty) and
the button was disabled. Tighten the gate to require
`currentPassword.length >= 8` and mirror the same check in the
submit handler so Enter / autofill cannot bypass.
Supersedes #5738.
PyPI release 2026.5.6 is now live; update install.sh and install.ps1 to
pin against the new minimum so fresh installs pick up the latest wheel.
Co-authored-by: Michael Han <michaelhan2050@gmail.com>
* studio: regenerate desktop launcher on `unsloth studio update`
Today `unsloth studio update` only mutates the venv. The macOS .app bundle,
the Linux .desktop file, and the shared launch-studio.sh stub bake their
paths and `studio_install_id` at install time and never refresh. Users who
update an existing Studio install report the Dock / Applications icon still
pointing at the old launcher; only a fresh `curl ... install.sh | sh`
fixes it because that path re-enters install.sh's create_studio_shortcuts.
Wire the same logic into the update path:
- install.sh: add --shortcuts-only. Skips the heavy install steps, resolves
STUDIO_HOME / OS / DATA_DIR through the existing _resolve_studio_destinations
+ platform detection, then calls create_studio_shortcuts and exits.
- unsloth_cli/commands/studio.py: after setup.sh succeeds, call install.sh
with --shortcuts-only. Prefers a local checkout's install.sh (when
STUDIO_LOCAL_REPO is set) or one shipped under _PACKAGE_ROOT, and falls
back to fetching the upstream installer from https://unsloth.ai/install.sh
for PyPI-installed users (the wheel does not ship install.sh).
Net effect: `unsloth studio update` now refreshes the macOS .app stub,
launcher script, studio.conf, and Linux .desktop entry on every update, so
the desktop icon stays in sync with the venv that setup.sh just updated.
Env-override and Tauri modes keep their existing behavior (no persistent
menu shortcuts, but the launch-studio.sh is still regenerated).
Windows is unchanged here; setup.ps1 already handles its own Start Menu /
Desktop .lnk creation on update.
* studio: also regenerate Windows .lnk shortcuts on update
Mirror the macOS fix: install.ps1 gains --shortcuts-only that short-circuits
to New-StudioShortcuts, and unsloth studio update calls it after setup.ps1
the same way it now does on macOS / Linux.
PyPI installs do not ship install.ps1, so the Python helper fetches the
upstream script from https://unsloth.ai/install.ps1 and pipes it into
powershell.exe -Command - with an explicit Install-UnslothStudio call
appended (irm | iex relies on the trailing @args, which is empty when
launched from stdin).
setup.ps1 alone never recreates the Start Menu / Desktop .lnk targets or
the launch-studio.{ps1,vbs} scripts, so without this update users on
Windows hit the same stale-icon regression that triggered the macOS PR.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: rename unsloth.exe to .deleteme before update on Windows
Pip's editable reinstall calls uninstall first, which deletes every RECORD
entry. unsloth.exe is one of them, and Windows refuses to delete a file
whose image is mapped into the running process tree. The first
unsloth studio update after install therefore fails with:
OSError: [WinError 32] The process cannot access the file because it
is being used by another process: ...\Scripts\unsloth.exe
Windows does allow renaming an in-use exe, so move it aside before
_run_setup_script kicks pip. pip then drops a fresh unsloth.exe at the
original path; the *.exe.deleteme left behind is cleaned up at the start
of the next update once the previous shim has exited.
* studio: rename unsloth.exe from setup.ps1 to reliably bypass exe lock
* studio: print python -m workaround when Windows exe lock blocks update
* studio: use python -c hint (unsloth_cli has no __main__)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: reshape --shortcuts-only Tauri guard to pass exit-order test
* shorter comments in update / launcher regen logic
* studio update: env-mode passthrough + non-silent shortcuts-only error
* studio update: address codex/gemini PR review
- Strip install.ps1's `Install-UnslothStudio @args` auto-invoke before
appending an explicit `--shortcuts-only` call so PyPI Windows installs
don't re-run the full installer over stdin.
- subprocess.run(input=wrapper, ...) now uses encoding="utf-8" so box
drawing chars in install.ps1 don't UnicodeEncodeError on CP1252.
- Wrap _run_setup_script in try/except to restore unsloth.exe from
.deleteme if setup fails, and mirror that rollback inside setup.ps1
when install_python_stack.py exits non-zero.
- Capture subprocess return codes in _refresh_desktop_shortcuts and
echo a one-line warning on non-zero so silent stale-shortcut failures
surface.
- Drop --local from the Windows lock-recovery hint so users on PyPI
installs don't accidentally switch into editable-checkout mode.
- Quote $VENV_ABS_BIN/unsloth in the install.sh shortcuts-only error
so paths with spaces print legibly.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio update: harden Windows refresh per multi-reviewer pass
- PowerShell stdin path now writes the wrapper to a UTF-8 BOM tempfile
and runs it via `-File`. `powershell.exe -Command -` decodes stdin
with the OEM code page, which mangles box-drawing chars in the
fetched install.ps1; -File reads the BOM and decodes UTF-8 cleanly.
- _restore_self_exe_lock_windows now treats a zero-byte unsloth.exe as
a partial-write and prefers the .deleteme copy. setup.ps1 mirrors
the same check.
- _release_self_exe_lock_windows uses os.replace for atomic overwrite
so a stale .deleteme from an aborted prior update doesn't break the
rename.
- Lock-recovery hint mentions that --local should be re-added when
the user installed from a repo checkout.
* studio update: respect Tauri context and tidy Windows .deleteme
Tauri's update.rs spawns `unsloth studio update`; without a signal,
the CLI's _refresh_desktop_shortcuts would call install.{sh,ps1}
--shortcuts-only and create duplicate ~/Applications/Unsloth Studio.app
(or .desktop / .lnk) entries that collide with the Tauri bundle.
- update.rs now sets UNSLOTH_TAURI_UPDATE=1 on the spawned child.
- studio.py's update() skips _refresh_desktop_shortcuts when that env
var is set; Tauri owns its own bundle entries.
- After a successful Windows update, drop the .deleteme orphan so
repeated updates don't accumulate stale binaries that could later
be promoted by _restore_self_exe_lock_windows on a cross-version
failure.
- Tempfile for the PyPI-fallback PowerShell path now uses an
unsloth-studio-refresh- prefix so AV/EDR rules and user greps can
identify it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio update: drop obsolete WinError 32 hint, echo Tauri skip
The rename trick in _release_self_exe_lock_windows + setup.ps1's
restore now handle the .exe-lock case in-flow; the printed hint
suggested re-running update via venv python, but that just re-enters
the same update() and hits the same failure if the rename didn't help.
Removing the misleading hint and its helper.
Also surface a one-line typer.echo when refresh is skipped under
UNSLOTH_TAURI_UPDATE so --verbose logs make the branch visible.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
unsloth 2026.5.3 was just published to PyPI. Update install.sh and
install.ps1 so fresh installs pull the new release (5 occurrences each).
Co-authored-by: Daniel Han <info@unsloth.ai>
* studio/install: fix mac desktop shortcut spawning and lifecycle
The macOS .app generated by install.sh ships a shell-shim wrapper that
is unsigned and has no NSAppleEventsUsageDescription in its Info.plist,
so AppleEvents from the bundle are denied by TCC. The launcher's
`osascript ... tell application "Terminal" to do script ...` call
silently fails and the script falls back to the headless nohup branch,
where the user sees no Terminal window at all. Each click of the Desktop
shortcut then leaks an unattached server (no PID file, no cleanup) and
the launcher times out after 60s without ever opening a browser.
Replace the AppleScript spawn with a `.command` file + `open -a Terminal`.
Terminal handles `.command` natively through Launch Services, no
AppleEvents permission required, works with unsigned bundles.
The new design also decouples the studio server from the Terminal:
- Server is started via nohup, detached from any TTY. Warm relaunches
(server still alive) hit the existing fast path: the launcher's
`_find_healthy_port` returns the running port and the browser opens
in ~80ms with no Terminal involvement.
- The `.command` file is a log viewer (`tail -F` of studio.log), not
the server's parent. It also runs a watcher subshell that polls the
server PID and kills `tail` when the server exits. This means
clicking "Stop server" in the UI causes the Terminal window to drop
to no-running-processes state, so the user can close the window
without the "Do you want to terminate running processes" dialog.
- A trap on HUP/INT/TERM/EXIT in the `.command` file sends SIGTERM
(then SIGKILL at +0.5s) to the server PID, so closing the Terminal
window also stops Studio. Best of both worlds: fast warm relaunch
AND "close terminal == quit Studio".
Also:
- Drop POLL_INTERVAL_SEC from 1 to 0.25. With Python studio startup
at ~2s, the 1s poll added up to 1s of slack between server-ready
and browser-open. 0.25s tightens cold-launch latency at no
meaningful CPU cost.
- Refuse to install the `.app` bundle through a symlink. If a prior
install (e.g. a --tauri build) left $HOME/Applications/Unsloth\\ Studio.app
as a symlink, mkdir -p follows it and writes the new bundle contents
through to the target. Detect and rm the symlink before mkdir -p.
Test plan:
- Existing studio-mac-update-smoke.yml CI runs install.sh end-to-end
on macos-14 and asserts /api/health returns healthy.
- Manual: click Desktop shortcut from cold state, Terminal opens with
logs streaming, browser opens at ~2s. Re-click while Studio still
running, browser opens in <200ms, no new Terminal. Click "Stop
server" in the UI, Terminal closes cleanly with no prompt. Close
Terminal via Cmd+W, server stops within 1s.
* studio/install: trim verbose comments in _spawn_terminal
* studio/install: harden trap quoting in generated .command
The trap bodies in the .command file were written with broken
quoting:
trap "rm -f "$PID_FILE" 2>/dev/null" EXIT
Shell parses this as three concatenated tokens ("rm -f " + unquoted
$PID_FILE + " 2>/dev/null") then runs the trap. With paths that
contain spaces, the unquoted expansion word-splits and the rm
either no-ops or removes the wrong path. Default $HOME has no
spaces so the bug is latent, but it should be space-safe.
Switch both trap bodies to single-quoted form so $WATCHER_PID,
$TAIL_PID, and $PID_FILE expand at signal time inside properly
quoted positions. Shellcheck-clean on the generated .command.
* studio/install: exec studio in nohup wrapper so PID is the server
Without the explicit exec, `nohup sh -c "$_cmd"` runs `_cmd` as a
child of the wrapper shell. Whether sh exec-optimizes that single
command is shell-specific (macOS /bin/sh does, dash does, some bash
configurations do not). When the optimization does not fire, `$!`
records the wrapper PID rather than the studio PID, so:
- the watcher in the generated .command monitors the wrapper, not
the actual studio process; closing the Terminal can leave studio
running if the wrapper exits first
- SIGTERM from shutdown_studio goes to the wrapper rather than the
server
Force the replacement with exec so the recorded PID is always the
studio process regardless of shell version.
Flagged by both gemini-code-assist and codex in PR review; verified
correct.
* Fix orphan-on-spawn-failure, graceful kill, and nested symlink for PR #5496
Three issues found while testing the new macOS spawn path:
1. _spawn_terminal returned 0 even when 'open -a Terminal' failed, so
the nohup'd server was left orphaned with no Terminal owner. Wrap
the .command write + chmod + open chain in 'if {...}; then return 0;
fi', and on failure SIGTERM the orphan (with a 3s grace) before
falling through to the generic terminal-spawn fallback.
2. The generated .command sent SIGKILL only 0.5s after SIGTERM, shorter
than studio/backend/run.py's _graceful_shutdown windows (5s inference
+ 5s export). Wait up to 12s for the server to exit on its own.
3. The .app symlink guard only checked the top-level path. If a prior
corrupted install left Unsloth Studio.app/Contents (or its MacOS or
Resources children) as a symlink, mkdir -p still wrote through them.
Check all four bundle paths, and refuse to continue if the bundle
path exists as a regular file.
---------
Co-authored-by: Daniel Han <info@unsloth.ai>
* Add Apple Silicon MLX routing
Rewrite __init__.py: detect MLX on macOS arm64 before any torch imports
Extract original GPU init to _gpu_init.py (unchanged)
MLX path imports FastMLXModel from unsloth_zoo, skips all GPU code
GPU path unchanged: from ._gpu_init import *
* Add Apple Silicon MLX routing
- Rewrite __init__.py: detect MLX on macOS arm64 before any torch imports
- Extract original GPU init to _gpu_init.py (unchanged)
- MLX path imports FastMLXModel from unsloth_zoo, skips all GPU code
- GPU path unchanged: from ._gpu_init import *
* mlx with studio
* mlx with studio
* updating temporary install.sh
* updating temporary install.sh
* adding t_v5 path
* adding t_v5 path
* fixing vision training
* fixing vision training
* adding chat
* adding chat
* minor
* minor
* Adding export and fixing training issues, inference with lora adaptors
* Adding export and fixing training issues, inference with lora adaptors
* fix: MLX worker pass load_in_4bit, override is_vlm based on dataset, streaming for VLM
* fix: MLX worker pass load_in_4bit, override is_vlm based on dataset, streaming for VLM
* Merge mlx-apple-silicon into main
* update install.sh to point to main branch
* update install.sh to point to main branch
* fix: export returns 3 values (success, message, output_path) matching upstream worker
* fix: export returns 3 values (success, message, output_path) matching upstream worker
* fix(mlx): show training-process peak memory in Studio UI, not system-wide
Studio UI was showing ~95 GB during MLX training because get_gpu_utilization
read "In use system memory" from IORegistry's AGXAccelerator — system-wide
GPU memory across all processes (training + backend + browser + Display).
Now the trainer's mx.get_peak_memory value is forwarded through the
progress event and surfaced via /api/train/hardware while training is
active. Falls back to the system-wide reading when training is not running.
* fix(mlx): show training-process peak memory in Studio UI, not system-wide
Studio UI was showing ~95 GB during MLX training because get_gpu_utilization
read "In use system memory" from IORegistry's AGXAccelerator — system-wide
GPU memory across all processes (training + backend + browser + Display).
Now the trainer's mx.get_peak_memory() value is forwarded through the
progress event and surfaced via /api/train/hardware while training is
active. Falls back to the system-wide reading when training is not running.
* fix(mlx): make is_bfloat16_supported detect M1/M2 (no native bf16)
M1 and M2 chips emulate bf16 in software on the GPU, causing 40-70%
slower prefill compared to native fp16. M3+ have native bf16 (macOS
Sonoma+ MPSGraph). Replaces the always-True stub with chip-aware
detection via mx.device_info.
* fix(mlx): make is_bfloat16_supported() detect M1/M2 (no native bf16)
M1 and M2 chips emulate bf16 in software on the GPU, causing 40-70%
slower prefill compared to native fp16. M3+ have native bf16 (macOS
Sonoma+ MPSGraph). Replaces the always-True stub with chip-aware
detection via mx.device_info().
* feat(mlx): wire training_type="Full Finetuning" through MLX worker
Compute use_lora from the UI's training_type before loading the model,
pass full_finetuning=not use_lora to FastMLXModel.from_pretrained, and
let the existing 'if use_lora' branch skip get_peft_model. Matches the
GPU worker's flow.
* feat(mlx): wire training_type="Full Finetuning" through MLX worker
Compute use_lora from the UI's training_type before loading the model,
pass full_finetuning=not use_lora to FastMLXModel.from_pretrained, and
let the existing 'if use_lora' branch skip get_peft_model. Matches the
GPU worker's flow.
* fix(mlx): pass save_method='merged_16bit' from Studio's export page
Previously the MLX path called save_pretrained_merged with no
save_method, which fell through to a no-op that didn't actually fuse
LoRA into the base. Now Studio's "Merged Model" export properly
fuses LoRA + dequantizes any 4-bit base to bf16, matching the GPU
behavior for the same UI option.
* fix(mlx): pass save_method='merged_16bit' from Studio's export page
Previously the MLX path called save_pretrained_merged() with no
save_method, which fell through to a no-op that didn't actually fuse
LoRA into the base. Now Studio's "Merged Model" export properly
fuses LoRA + dequantizes any 4-bit base to bf16, matching the GPU
behavior for the same UI option.
* fix(studio): pass private to MLX push, return 3-tuples consistently
MLX push_to_hub branch now forwards private=private (matches GPU)
Existing 2-tuple early-returns ('repo_id+token required', 'PEFT model
needed') were tripping the route's 3-tuple unpack. Added a None
output_path so the unpack always succeeds.
* fix(studio): pass private to MLX push, return 3-tuples consistently
- MLX push_to_hub branch now forwards private=private (matches GPU)
- Existing 2-tuple early-returns ('repo_id+token required', 'PEFT model
needed') were tripping the route's 3-tuple unpack. Added a None
output_path so the unpack always succeeds.
* studio wirings
* studio wirings
* Merge pull request #5 from Manan17/feat/quant_config
studio wirings
* fix(mlx): wire train_on_completions for VLM via per-template lookup
Mirror the GPU worker: stop excluding VLMs and stop hardcoding
template detection. Look up the model in MODEL_TO_TEMPLATE_MAPPER and
fetch the per-template instruction/response markers from
TEMPLATE_TO_RESPONSES_MAPPER. The frontend already force-disables
train_on_completions for vision+image and audio cases, so backend
just trusts the flag.
* fix(mlx): wire train_on_completions for VLM via per-template lookup
Mirror the GPU worker: stop excluding VLMs and stop hardcoding
template detection. Look up the model in MODEL_TO_TEMPLATE_MAPPER and
fetch the per-template instruction/response markers from
TEMPLATE_TO_RESPONSES_MAPPER. The frontend already force-disables
train_on_completions for vision+image and audio cases, so backend
just trusts the flag.
* wire in lora rslora, init lora weights, random_state
* wire in lora rslora, init lora weights, random_state
* loftq studio error message fix
* loftq studio error message fix
* handle unknown optim and lr scheduler
* handle unknown optim and lr scheduler
* Merge pull request #6 from Manan17/update/peftkwargs
Update/peftkwargs
* feat(mlx): pass finetune_language/attention/mlp/vision flags to FastMLXModel
Studio's four UI checkboxes now actually flow through to MLX get_peft_model
(which was just updated in unsloth-zoo to honor them). Also drops the
incorrect train_projector wiring that tied projector LoRA to the
attn/mlp flags — those are language-side toggles, not projector toggles.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* feat(mlx): pass finetune_language/attention/mlp/vision flags to FastMLXModel
Studio's four UI checkboxes now actually flow through to MLX get_peft_model
(which was just updated in unsloth-zoo to honor them). Also drops the
incorrect train_projector wiring that tied projector LoRA to the
attn/mlp flags — those are language-side toggles, not projector toggles.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* feat(mlx,ux): auto-imply finetune_language_layers when user picks attn/mlp
UI guardrail. The four checkboxes (vision/language/attention/MLP) carry
"scope × module-type" semantics that aren't obvious — picking just
"Attention modules" + "MLP modules" without "Language layers" naturally
reads as "fine-tune attn/mlp" but our backend reads it as "fine-tune
attn/mlp modules in *no* tower" → empty target_modules → zero
trainable params → crash inside value_and_grad.
If user selected attn or mlp module types but no layer scope, default
to language scope. Power users can still explicitly choose
language=False, vision=True if they want vision-only fine-tuning of
attn/mlp.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* feat(mlx,ux): auto-imply finetune_language_layers when user picks attn/mlp
UI guardrail. The four checkboxes (vision/language/attention/MLP) carry
"scope × module-type" semantics that aren't obvious — picking just
"Attention modules" + "MLP modules" without "Language layers" naturally
reads as "fine-tune attn/mlp" but our backend reads it as "fine-tune
attn/mlp modules in *no* tower" → empty target_modules → zero
trainable params → crash inside value_and_grad.
If user selected attn or mlp module types but no layer scope, default
to language scope. Power users can still explicitly choose
language=False, vision=True if they want vision-only fine-tuning of
attn/mlp.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* fix(mlx): wire top_k, repetition_penalty, and VLM top_p through to mlx-lm/mlx-vlm
Inference UI sliders for top_k and repetition_penalty had no effect on
MLX, and VLM top_p was also silently dropped. Plus a latent pre-existing
bug: mlx_vlm.generate_step expects temperature= (long form), but we
were passing temp= which silently fell into **kwargs — every VLM chat
was effectively greedy regardless of the temperature slider.
Text path (_generate_text):
make_sampler now receives top_k in addition to temp/top_p
make_logits_processors built and forwarded when repetition_penalty is
non-trivial (skip when 0.0/1.0 to avoid pointless overhead)
VLM path (_generate_vlm):
Pass top_p, top_k, repetition_penalty as kwargs (mlx_vlm.stream_generate
forwards them to generate_step's sampler/logits_processor builders)
Rename temp= → temperature= so it's actually consumed
Verified end-to-end with a smoke test on Qwen2.5-0.5B-Instruct (text) and
Qwen2.5-VL-3B-Instruct (VLM): each of {greedy, top_p=0.5, top_k=10,
rep_pen=1.5} now produces a distinct output, proving the parameters
reach the sampler.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* fix(mlx): wire top_k, repetition_penalty, and VLM top_p through to mlx-lm/mlx-vlm
Inference UI sliders for top_k and repetition_penalty had no effect on
MLX, and VLM top_p was also silently dropped. Plus a latent pre-existing
bug: mlx_vlm.generate_step expects temperature= (long form), but we
were passing temp= which silently fell into **kwargs — every VLM chat
was effectively greedy regardless of the temperature slider.
Text path (_generate_text):
- make_sampler now receives top_k in addition to temp/top_p
- make_logits_processors built and forwarded when repetition_penalty is
non-trivial (skip when 0.0/1.0 to avoid pointless overhead)
VLM path (_generate_vlm):
- Pass top_p, top_k, repetition_penalty as kwargs (mlx_vlm.stream_generate
forwards them to generate_step's sampler/logits_processor builders)
- Rename temp= → temperature= so it's actually consumed
Verified end-to-end with a smoke test on Qwen2.5-0.5B-Instruct (text) and
Qwen2.5-VL-3B-Instruct (VLM): each of {greedy, top_p=0.5, top_k=10,
rep_pen=1.5} now produces a distinct output, proving the parameters
reach the sampler.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* feat(mlx): map format_type to MLX save_method, reuse local save dir for hub push
export_merged_model: format_type="4-bit (FP4)" → save_method="merged_4bit"
(was hardcoded merged_16bit, ignoring the UI choice).
Both export_merged_model and export_base_model now pass save_directory=
to push_to_hub_merged so it reuses the just-written local folder
instead of re-saving under a relative "username/model" directory.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* feat(mlx): map format_type to MLX save_method, reuse local save dir for hub push
- export_merged_model: format_type="4-bit (FP4)" → save_method="merged_4bit"
(was hardcoded merged_16bit, ignoring the UI choice).
- Both export_merged_model and export_base_model now pass save_directory=
to push_to_hub_merged so it reuses the just-written local folder
instead of re-saving under a relative "username/model" directory.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* restore install
* restore install
* fix(mlx): restore FastVisionModel as a distinct class
unsloth/__init__.py was assigning `FastVisionModel = FastLanguageModel`
right after defining `class FastVisionModel(FastLanguageModel)` with a
`for_training` static method. The alias erased the class binding, so
the documented `FastVisionModel.for_training(model)` call from upstream
Unsloth's VLM notebooks raised `AttributeError` on MLX.
Remove the offending alias. `FastVisionModel` is now a real subclass of
`FastLanguageModel` again — inherits `from_pretrained` /
`get_peft_model` / `for_inference`, exposes `for_training` as a no-op
pass-through (no-op because MLX doesn't have a train/eval mode flag;
the call exists purely for GPU/MLX notebook parity).
Verified end-to-end: Qwen3-VL-2B + LaTeX_OCR LoRA + vision LoRA via
FastVisionModel.from_pretrained → get_peft_model → for_training →
MLXTrainer.train runs 10 steps cleanly (loss 1.10 → 0.12, no NaNs,
peak 5.89 GB).
Studio's path (FastLanguageModel.from_pretrained for any repo,
auto-detect VLM in the loader) is unaffected. Tier-1 review finding #8.
* fix(mlx): restore FastVisionModel as a distinct class
unsloth/__init__.py was assigning `FastVisionModel = FastLanguageModel`
right after defining `class FastVisionModel(FastLanguageModel)` with a
`for_training` static method. The alias erased the class binding, so
the documented `FastVisionModel.for_training(model)` call from upstream
Unsloth's VLM notebooks raised `AttributeError` on MLX.
Remove the offending alias. `FastVisionModel` is now a real subclass of
`FastLanguageModel` again — inherits `from_pretrained` /
`get_peft_model` / `for_inference`, exposes `for_training` as a no-op
pass-through (no-op because MLX doesn't have a train/eval mode flag;
the call exists purely for GPU/MLX notebook parity).
Verified end-to-end: Qwen3-VL-2B + LaTeX_OCR LoRA + vision LoRA via
FastVisionModel.from_pretrained → get_peft_model → for_training →
MLXTrainer.train() runs 10 steps cleanly (loss 1.10 → 0.12, no NaNs,
peak 5.89 GB).
Studio's path (FastLanguageModel.from_pretrained for any repo,
auto-detect VLM in the loader) is unaffected. Tier-1 review finding #8.
* Studio: harden MLX training and export, restore GPU init guards
Studio export
Restore Tuple[bool, str, Optional[str]] contract on export_merged_model,
export_base_model, export_gguf, and export_lora_adapter, populating
output_path on successful local saves so routes/worker/CLI/frontend
details.output_path is non-empty again.
Lift the GPU save_method assignment out of the local-save branch so
Hub-only merged exports (save_directory='', push_to_hub=True) no longer
hit UnboundLocalError on the push branch.
For MLX merged and base hub-only export, stage to a tempfile.TemporaryDirectory
before push_to_hub_merged instead of passing save_directory=''.
Source _IS_MLX from unsloth instead of recomputing the platform check
(single source of truth, also enforces mlx-package availability).
Studio MLX training/inference
Pass token=hf_token into FastMLXModel.from_pretrained for gated/private
models, matching the inference path.
Strip hf_token and wandb_token from wandb.init(config=...) so secrets
do not leak into the W&B run config.
Replace load_from_disk(local_datasets[0]) with the existing
UnslothTrainer._resolve_local_files / _loader_for_files helpers so
uploaded JSON/JSONL/CSV/Parquet files train through the normal datasets
loader (load_from_disk still used for HF save_to_disk directories).
Make the dataset slice helper inclusive at the end and treat 0 as a real
index instead of "unset", matching the GPU and embedding paths.
Add a status_message -> message alias inside _send so the existing parent
pump (training.py) renders MLX status updates instead of blanks.
Forward min_p through generate_chat_response into _generate_text /
_generate_vlm and into make_sampler / vlm_kwargs so the sampling control
is no longer a no-op on MLX.
Wrap unsloth_zoo.mlx_loader / mlx_trainer imports with a clearer
ImportError pointing users at install.sh for Apple Silicon.
Exit the MLX stop-polling thread on EOFError/OSError instead of
busy-looping when the queue/pipe is permanently closed (one-line
why-safe rationale inline).
Studio frontend
ParamsSection subscribes to platform deviceType via the Zustand hook so
the gradient checkpointing dropdown re-renders after the async device
fetch completes.
Studio hardware
get_gpu_utilization MLX branch now reads _read_apple_gpu_stats once and
derives VRAM totals from psutil, removing the second ioreg subprocess
per utilization poll.
Unsloth core
Restore the os.geteuid == 0 guard around the CUDA ldconfig recovery
that was lost when GPU initialization moved into _gpu_init.py, plus the
non-root manual-fix warning branch. Non-root CUDA users no longer shell
out to ldconfig at import time.
Load dataprep/raw_text via importlib so the MLX import path no longer
pulls torch in through dataprep/__init__.py -> synthetic.py.
FastVisionModel.from_pretrained overrides the inherited delegator only
to inject text_only=False; this is an extension, not a duplication, and
is needed so VLM checkpoint loads keep the vision tower.
Wrap the MLX-branch unsloth_zoo import with a clearer ImportError.
* Studio: regression tests for MLX training/export and GPU init ldconfig guard
tests/python/test_gpu_init_ldconfig_guard.py asserts the geteuid root
check still wraps the ldconfig recovery and the non-root branch warns
bnb users; AST + source-text inspection so the test runs without torch.
tests/studio/test_export_output_path_contract.py covers the
Tuple[bool, str, Optional[str]] return contract on every export method,
the output_path assignment after successful local save, the Hub-only
GPU save_method binding fix, the MLX hub-only TemporaryDirectory
staging, and the single-source `_IS_MLX` import from unsloth.
tests/studio/test_mlx_training_worker_behaviors.py covers token
forwarding to FastMLXModel.from_pretrained, wandb config secret
stripping, file-aware local dataset loading, status_message ->
message aliasing, inclusive slice semantics, EOFError/OSError stop
thread exit, and the friendly mlx_loader / mlx_trainer ImportError.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(mlx): cap inference memory + release wired on unload + tame worker pre-pin
Three memory-hardening fixes for Studio's MLX path:
1. Inference applies the same Metal caps as the trainer.
load_model previously only called set_wired_limit(100% of recommended)
with no upper memory_limit, leaving large VLM checkpoints unbounded
during the loader allocation. Add _configure_memory_limits() that sets
memory_limit to 85% of recommended and wired_limit to min(recommended,
memory_limit) — matching MLXTrainer's defaults so behavior is the same
whether the user trains or just runs inference.
2. unload_model releases pinned memory back to the OS — but only when
the cache is empty. Without this, pinned wired bytes stayed allocated
to MLX after the model was gone, starving other apps. The release is
guarded on `not self.models` so unloading one of several cached
models doesn't un-pin weights still in use.
3. Worker pre-cap is conservative instead of aggressive.
The previous pre-pin set_wired_limit(100% of recommended) competed
with MLXTrainer's later more conservative cap. Replace with the same
85%-memory / min(rec, memory) pair that the trainer applies later
(idempotent re-apply). Bounds the model load + LoRA setup window
without over-pinning.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests/studio: regression tests for the _IS_MLX dispatch gate
Two gates drive every MLX-vs-CUDA dispatch decision in Studio:
1. unsloth._IS_MLX in unsloth/__init__.py — evaluated once at import
time, read by Studio worker code to choose the GPU vs MLX trainer
and inference paths. Defined as
Darwin AND arm64 AND find_spec("mlx") is not None.
2. utils.hardware.detect_hardware() — runtime probe with priority
CUDA > XPU > MLX > CPU. The MLX branch is reached only when both
CUDA and XPU are unavailable and the host is Apple Silicon and
mlx is importable.
Neither gate had a direct test. Adds tests/studio/test_is_mlx_dispatch_gate.py
with six tests:
test_is_mlx_gate_uses_three_required_predicates
AST-walks unsloth/__init__.py and asserts the _IS_MLX assignment
is a BoolOp(And) of platform.system()=="Darwin",
platform.machine()=="arm64", and find_spec("mlx") is not None.
Catches accidental rewrites that drop a predicate.
test_is_mlx_gate_true_on_apple_silicon_with_mlx_present
Spoofs platform to Darwin/arm64, injects a fake mlx module so
find_spec returns a real ModuleSpec, re-evaluates the gate
expression. Verifies it flips True under the exact conditions
Studio expects.
test_is_mlx_gate_false_when_mlx_missing
Spoofs Apple Silicon but with mlx absent. Verifies the gate stays
False (so a Mac without mlx installed does not pretend to have
MLX support).
test_is_mlx_gate_false_on_non_apple_silicon
Canary on the actual Linux+CUDA / AMD / Intel test host: the gate
must remain False regardless of whether mlx happens to be
importable. Protects existing GPU users from accidental MLX
hijack when MLX support evolves.
test_detect_hardware_picks_mlx_when_only_apple_silicon_available
Forces torch.cuda and torch.xpu off, spoofs Apple Silicon, injects
fake mlx and mlx.core. detect_hardware() must return DeviceType.MLX.
test_detect_hardware_picks_cuda_on_real_host
Canary: on a real CUDA host detect_hardware() must return
DeviceType.CUDA. Protects against the MLX branch shadowing CUDA
dispatch on NVIDIA / AMD ROCm hosts.
Uses the same monkeypatch.setitem(sys.modules, ...) fake-mlx pattern as
the existing test_mlx_inference_backend.py — no new test infrastructure,
no real mlx install required.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add AGPL-3.0 SPDX header to Studio MLX regression tests
Four Studio MLX test files shipped without an SPDX-License-Identifier:
studio/backend/tests/test_mlx_training_worker_config.py
tests/studio/test_mlx_training_worker_behaviors.py
tests/studio/test_export_output_path_contract.py
tests/studio/test_is_mlx_dispatch_gate.py
They sit in or alongside studio/backend/, which is governed by
studio/LICENSE.AGPL-3.0, and exercise AGPL Studio code. Add the same
"# SPDX-License-Identifier: AGPL-3.0-only" header that's already on
test_mlx_inference_backend.py so the license declaration matches
the code under test rather than defaulting to the repo-root
Apache-2.0.
* Wrap MLX submodule imports with friendly install hint
The _IS_MLX block at the top of unsloth/__init__.py already catches the
missing-package case with a friendly install hint, but the follow-up
"from unsloth_zoo.mlx_trainer import ..." and "from unsloth_zoo.mlx_loader import ..."
lines run unguarded. An Apple Silicon user who has unsloth-zoo installed
but on an older version (e.g. the current PyPI release, before the MLX
modules ship) sees a raw ImportError on the submodule rather than the
hint that points at install.sh.
Wrap the two submodule imports in the same try/except shape so the
friendly install message fires whether the package is missing entirely
or just predates the MLX submodules. No-op once both packages release
together; smooths the transitional window where unsloth/main has merged
but unsloth-zoo on PyPI has not.
---------
Co-authored-by: DoubleMathew <mmathew23@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* install: support STUDIO_HOME / UNSLOTH_STUDIO_HOME for custom install paths
Currently install.sh and install.ps1 hardcode all install paths off
$HOME / $env:USERPROFILE with no env-var fallback. This blocks
workspace-isolated installs (CI sandboxes, per-PR test environments,
multi-tenant boxes) unless the entire HOME / USERPROFILE is faked,
which also relocates ~/.gitconfig, ~/.ssh, and other unrelated state.
Add an opt-in env-var override that does only what is needed.
Resolution priority (highest first):
1. HOME / USERPROFILE explicitly redirected vs the password-database
default. Detected via getent (Linux), dscl (macOS), or
[Environment]::GetFolderPath (Windows). Best-effort: when the
detection mechanism is unavailable the check is skipped and we
fall through to step 2.
2. UNSLOTH_STUDIO_HOME, if set.
3. STUDIO_HOME, if set (alias for convenience; the variable name
already matches the internal var install.sh sets).
4. Default: legacy $HOME/.unsloth/studio (or
$USERPROFILE\.unsloth\studio on Windows). Identical to today's
behavior when no env var is set.
When an env var override fires:
* DATA_DIR is nested inside ($STUDIO_HOME/share, or $StudioHome\share
on Windows) so the runtime launcher and shortcuts find studio.conf
in the same place install-time wrote it.
* The unsloth CLI shim lands at $STUDIO_HOME/bin/unsloth (Unix) or
$StudioHome\bin\unsloth.exe (Windows). On Windows the shim already
lives under $StudioHome; the change only redirects DATA_DIR and
skips the persistent registry PATH update.
* Persistent shell PATH modifications are skipped (no .bashrc /
.zshrc / .profile append on Unix; no Add-ToUserPath on Windows).
Caller is expected to invoke via absolute path or add the bin dir
to PATH explicitly. Avoids polluting the user's profile with a
workspace-scoped path that may be deleted.
The Unix launcher script is the only piece that must read DATA_DIR
at runtime (it sources studio.conf from there). The hardcoded
DATA_DIR inside the LAUNCHER_EOF heredoc is replaced with an
@@DATA_DIR@@ placeholder substituted via sed at install time, using
the same approach the script already uses for other install-time
substitutions.
Default path behavior is unchanged: when no env var is set and HOME
is not redirected, install.sh / install.ps1 produce exactly the same
file layout as today.
Test scenarios verified locally on install.sh:
* Default (no env vars) -> $HOME/.unsloth/studio (legacy)
* HOME=/tmp/x -> /tmp/x/.unsloth/studio
* UNSLOTH_STUDIO_HOME=/tmp/y -> /tmp/y as STUDIO_HOME root
* STUDIO_HOME=/tmp/z (alias) -> /tmp/z as STUDIO_HOME root
* HOME redirect + env var (HOME wins) -> install follows HOME
* Unwritable override -> exits with clear ERROR message
* install: priority change -- env vars now win over HOME redirect
Flip the resolution order so explicit env vars take precedence over
HOME / USERPROFILE redirection.
New priority (highest first):
1. UNSLOTH_STUDIO_HOME, if set.
2. STUDIO_HOME, if set.
3. HOME / USERPROFILE explicitly redirected.
4. Default.
Rationale: the env vars are explicit single-purpose signals (the user
typed UNSLOTH_STUDIO_HOME=... specifically to redirect Studio). HOME
redirection is broader and incidental -- the user may have redirected
HOME for unrelated reasons (workspace tools, container builds) without
wanting Studio to follow it. When both are set, the more specific
signal should win.
When only HOME is redirected (no env var), behavior is unchanged from
the previous commit: install follows $HOME.
* install: address review feedback (sed escape, downstream propagation, edge cases)
Fixes from gemini-code-assist + chatgpt-codex-connector + reviewer.py
20-parallel run on the open PR.
install.sh:
* Escape sed replacement metacharacters before substituting @@DATA_DIR@@.
Two-stage escape: ' -> '\'' for safe single-quote shell embedding,
then \, &, | for sed replacement string + chosen delimiter. Heredoc
switched to single-quoted DATA_DIR='@@DATA_DIR@@' so we only need
single-quote escaping at runtime. Verified end-to-end with paths
containing & and | (the sed delimiter).
* Pass UNSLOTH_STUDIO_HOME into both setup.sh invocations
(--local and PyPI paths) so the downstream install resolves the
same Studio root install.sh picked.
* macOS .app stub: replace hardcoded
exec "$HOME/.local/share/unsloth/launch-studio.sh" with
exec "$_css_data_dir/launch-studio.sh" so the .app launches the
resolved launcher even in env-override mode.
* Use mkdir -p -- and cd -- when validating the env override so
paths starting with - cannot be misread as flags.
install.ps1:
* Drop .Guid from [guid]::NewGuid().Guid: the property does not
exist; the probe filename was always identical and not unique.
Default ToString() on System.Guid produces the canonical UUID
string we want.
* Guard LOCALAPPDATA before Join-Path to avoid aborting the
installer in service / CI contexts where LOCALAPPDATA is unset
(Join-Path under $ErrorActionPreference='Stop' would otherwise
throw). Computed once into $defaultDataDir; both 'profile' and
'default' branches reuse it.
* Set $env:UNSLOTH_STUDIO_HOME for the duration of the
'unsloth studio setup' subprocess so studio/setup.ps1 and
unsloth_cli see the same install root install.ps1 picked.
Restored in a finally block.
studio/setup.sh:
* Honor UNSLOTH_STUDIO_HOME / STUDIO_HOME (alias) when resolving
STUDIO_HOME, VENV_DIR, VENV_T5_*_DIR. Falls back to the legacy
$HOME/.unsloth/studio when no override is set.
studio/setup.ps1:
* Same change in PowerShell: honor $env:UNSLOTH_STUDIO_HOME /
$env:STUDIO_HOME for $StudioHome / $VenvDir resolution.
unsloth_cli/commands/studio.py:
* Replace the module-level constant
STUDIO_HOME = Path.home() / ".unsloth" / "studio"
with a resolver that honors UNSLOTH_STUDIO_HOME / STUDIO_HOME
before falling through to the legacy default. Same precedence
the installers use.
Verified locally: 6 install.sh scenarios still produce correct paths
(default, HOME redirect, env var, alias, both, bad override). New
sed-escape unit tests pass for paths containing & and |. Python
resolver matches priority: UNSLOTH_STUDIO_HOME > STUDIO_HOME > default.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: portable sed (no -i.bak) per gemini review feedback
GNU sed -i.bak vs BSD/macOS sed -i.bak vs BusyBox sed have subtly
different semantics. Use the POSIX-portable redirect-then-mv pattern
instead. Functionally identical, runs everywhere.
* studio: persist UNSLOTH_STUDIO_HOME so fresh shells find custom installs
Without this, a custom-root install (UNSLOTH_STUDIO_HOME=/work/studio
bash install.sh --local) only worked in the same shell that ran the
installer. Closing the terminal and reopening lost the env var, the
PATH was deliberately not persisted, and the Python CLI fell back to
~/.unsloth/studio. Result: 'Studio not set up' or quietly operating on
a stale legacy install.
Three persistence layers, all backwards-compatible (default installs
emit zero changes):
1. Unix studio.conf
install.sh now writes 'export UNSLOTH_STUDIO_HOME=...' next to
UNSLOTH_EXE in studio.conf when in env-override mode. The launcher
sources studio.conf at startup so the exec'd binary gets the var.
Default installs do not write this line; studio.conf stays
byte-identical to before.
2. Windows launch-studio.ps1
install.ps1 prepends '$env:UNSLOTH_STUDIO_HOME = ...' to the
generated launcher when in env-override mode. Default installs
produce the same launcher content as before.
3. Python sys.prefix inference
storage_roots.studio_root() and unsloth_cli/commands/studio.py
now infer the install root from sys.prefix when no env var is
set (Path(sys.prefix).parent for unsloth_studio venvs). Catches
direct invocations of <STUDIO_HOME>/bin/unsloth that bypass the
launcher entirely.
unsloth_cli/commands/studio.py also re-exports the resolved
UNSLOTH_STUDIO_HOME via os.environ.setdefault so child processes
(setup script, backend run.py) inherit it.
Backend storage roots (storage_roots.studio_root, cache_root) now
respect the env var via the shared resolver. run.py PID file,
transformers_version.py T5 venvs, and model_config.py vision-check
venv all switch to studio_root() so custom installs are
self-contained.
studio/setup.ps1: T5 sidecar venvs now resolve under $StudioHome
(was $env:USERPROFILE\.unsloth\studio\.venv_t5_*).
studio/setup.sh + studio/setup.ps1: llama.cpp build dir nests under
$STUDIO_HOME / $StudioHome when env-override is active, otherwise
keeps the legacy ~/.unsloth/llama.cpp.
Verified locally:
* studio.conf write block: env-override mode emits the export line;
default mode does not (byte-identical to today).
* PowerShell heredoc interpolation: correct output for both modes.
* studio_root() resolver: default, UNSLOTH_STUDIO_HOME, STUDIO_HOME
alias, and sys.prefix-based inference all return correct paths.
* cache_root() now derives from studio_root().
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: tilde expansion + macOS .app stub safe-quoting
Two fixes from running a 25-scenario simulation sweep against install.sh
across path edge cases (spaces, apostrophes, ampersands, pipes,
backslashes, dollar signs, Unicode, trailing slash, relative paths).
1. UNSLOTH_STUDIO_HOME=~/foo was landing as literal '~/foo' (env vars
are not subject to tilde expansion). Added a POSIX-portable case
block in install.sh, install.ps1, studio/setup.sh, studio/setup.ps1
that expands a leading ~ or ~/ to $HOME / $env:USERPROFILE.
The prefix-removal pattern is single-quoted ('${var#'~/'}') so the
shell does not tilde-expand the pattern back to $HOME/ before
matching -- a subtle dash/bash gotcha.
2. macOS .app stub used an unquoted heredoc ('<< STUB_EOF'), so any
$VAR / backtick / etc in the path would expand at .app launch time.
Switched to single-quoted heredoc ('<< 'STUB_EOF'') with a
placeholder + sed substitution + single-quoted shell embedding,
matching the @@DATA_DIR@@ pattern already used for launch-studio.sh.
Verified: 25/25 simulation scenarios pass on Linux dash + bash,
including paths with $VAR, &, |, \\, ', spaces, and Unicode. End-to-end
install in env-mode + fresh-shell launcher invocation confirmed: studio
binds to /api/health from a clean env, and sys.prefix-based inference
correctly returns the workspace root.
* install: stop accidentally treating default installs as env-override
Reviewer.py 20-runs cycle 1 found a unanimous P1 regression: a default
'unsloth studio update' relocates llama.cpp from ~/.unsloth/llama.cpp
to ~/.unsloth/studio/llama.cpp, because the CLI was re-exporting
UNSLOTH_STUDIO_HOME unconditionally and install.sh / install.ps1 were
passing it into setup.{sh,ps1} unconditionally. The setup scripts
treated the var's mere presence as "env-override mode" and relocated
the llama.cpp build dir away from the legacy path, breaking the
runtime backend's _find_llama_server_binary lookup on default installs.
Fixes:
* unsloth_cli/commands/studio.py: _resolve_studio_home now returns
(path, is_custom). Re-export only when is_custom -- a real env
override or a sys.prefix inference that resolves to a non-legacy
path. Default installs leave UNSLOTH_STUDIO_HOME unset.
* install.sh: gate UNSLOTH_STUDIO_HOME on $_STUDIO_HOME_REDIRECT == env
before calling setup.sh. Use 'env $VARS bash setup.sh' so the var
is set only for the subprocess, never leaked.
* install.ps1: gate $env:UNSLOTH_STUDIO_HOME on $StudioRedirectMode
-eq 'env' before invoking 'unsloth studio setup'. Restore prior
value in finally block (unset if it wasn't set).
* studio/setup.sh + setup.ps1: decide llama.cpp install root from
the resolved $STUDIO_HOME (not from env-var presence). If the
resolved path equals the legacy default ($HOME/.unsloth/studio),
fall back to ~/.unsloth/llama.cpp. This makes setup robust against
a stale UNSLOTH_STUDIO_HOME inherited from a parent process that
happens to point at the legacy default.
* studio/backend/core/inference/llama_cpp.py:
- _find_llama_server_binary() now searches studio_root() / llama.cpp
AND the legacy ~/.unsloth/llama.cpp (de-duped). Custom-root
installs become discoverable; default installs unaffected.
- kill_orphaned_servers ownership allowlist also includes
studio_root() / llama.cpp so custom-root processes are cleanable.
Verified locally:
* 25/25 sim scenarios still pass (path edge cases unchanged).
* setup.sh unit test: default-mode lands UNSLOTH_HOME at $HOME/.unsloth;
env-mode lands at $STUDIO_HOME.
* Python CLI unit test: default-mode returns is_custom=False and does
NOT setdefault UNSLOTH_STUDIO_HOME; env-mode sets is_custom=True.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: || exit 1 on STUDIO_HOME subshell (dash set -e gap)
Gemini review feedback: in dash, set -e does not trigger on subshell
failures inside variable assignments. If 'cd -- "$_override" && pwd'
fails, STUDIO_HOME stays empty and DATA_DIR collapses to /share. Add
explicit '|| exit 1' on both install.sh:187 and setup.sh:413.
* install.sh: argv-safe setup invocation for paths with spaces
Cycle 2 reviewer.py 20-runs found a unanimous P1: passing the env-var
through 'env $_STUDIO_ENV_FOR_SETUP' word-splits on whitespace, so a
custom root like '/tmp/Unsloth Studio' becomes 'UNSLOTH_STUDIO_HOME=
/tmp/Unsloth' followed by env trying to exec 'Studio'.
Replaced with a tiny helper that prepends the env-var directly to the
argv (no string-form intermediary), so spaces are preserved as a
single argument. Default-mode invocation skips the env-var entirely.
Verified: 'UNSLOTH_STUDIO_HOME=/tmp/test space/studio' now reaches
setup.sh as a single value.
* studio: tighten sys.prefix inference + Tauri env handling + llama.cpp env
Cycle 3 reviewer.py findings (3 P1s converging):
* sys.prefix inference too broad: a developer venv named 'unsloth_studio'
was being treated as a custom Studio root. Narrow with an installer-
sentinel check (presence of share/studio.conf or bin/unsloth shim
inside the parent dir) in both unsloth_cli/commands/studio.py and
studio/backend/utils/paths/storage_roots.py.
* Tauri studio/src-tauri/src/process.rs::find_unsloth_binary() hardcoded
~/.unsloth/studio. Honor UNSLOTH_STUDIO_HOME / STUDIO_HOME (in that
priority order) before falling back to legacy.
* unsloth-zoo's GGUF export binds LLAMA_CPP_DEFAULT_DIR at import time
from UNSLOTH_LLAMA_CPP_PATH. For env-override installs, persist
UNSLOTH_LLAMA_CPP_PATH alongside UNSLOTH_STUDIO_HOME in studio.conf
(Unix), in the generated PowerShell launcher (Windows), and via
os.environ.setdefault in the Python CLI when running on a custom
root, so GGUF export uses the custom-root llama.cpp build instead
of the legacy ~/.unsloth/llama.cpp.
Default behaviour unchanged: no env vars are written to studio.conf
in default mode, no LLAMA_CPP_PATH is set, and the dev-venv inference
falls through to legacy when no installer sentinels are present.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: desktop_auth env-aware + legacy-root llama.cpp consistency
- desktop_auth.rs: honor UNSLOTH_STUDIO_HOME / STUDIO_HOME for the
.desktop_secret path so Tauri desktop login works against custom-root
installs instead of always reading ~/.unsloth/studio/auth/.
- install.sh / install.ps1 / unsloth_cli/commands/studio.py: when an env
override resolves to the legacy default ($HOME/.unsloth/studio), set
UNSLOTH_LLAMA_CPP_PATH to ~/.unsloth/llama.cpp (matching setup.sh /
setup.ps1's legacy-equality branch). Previously the persisted value
pointed at $STUDIO_HOME/llama.cpp, which was a non-existent location
and broke unsloth-zoo's import-time GGUF binding for that edge case.
* studio: tauri studio_root helper + marker-file persistence + ~ expansion
Address cycle-5 reviewer findings:
- Add studio/src-tauri/src/studio_root.rs: shared resolver with
UNSLOTH_STUDIO_HOME / STUDIO_HOME (priority order), tilde expansion
(~, ~/..., ~\...), installer-written marker fallback, then
~/.unsloth/studio. 5 unit tests cover the expansion paths.
- Tauri lookups now go through the shared resolver:
- process.rs::find_unsloth_binary
- desktop_auth.rs::desktop_secret_path
- main.rs::setup_logging (tauri.log under custom root)
- commands.rs::open_logs_dir (opens custom root dir)
- install.rs work_dir uses parent of resolved root (avoids creating
a stray ~/.unsloth on a custom-root install)
- install.sh / install.ps1 (env-mode only): write
~/.unsloth/studio-home marker so the desktop app launched from
Finder/Start Menu (no shell env inheritance) still resolves the
custom root.
- install.sh / install.ps1 non-interactive completion: when
StudioRedirectMode=env, print the absolute custom-root shim path
since the persistent rc/registry PATH update is intentionally
skipped in env-override mode.
- unsloth_cli/commands/studio.py: replace setdefault() with
truthy-check so a blank UNSLOTH_STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH
in the parent env doesn't suppress the inferred custom root.
40/40 cargo test --bins pass.
* studio: validate marker file + write in --tauri mode + propagate to subprocess
Cycle-6 reviewer follow-ups:
- studio_root.rs marker resolver now validates the persisted path before
using it. A stale ~/.unsloth/studio-home pointing at a deleted/moved
workspace is ignored (resolution falls back to the legacy default
rather than hijacking it). Validation accepts share/studio.conf
sentinel or bin/unsloth shim. Trailing newline strip uses
trim_end_matches(['\n','\r']) so paths whose content legitimately has
leading/trailing spaces survive.
- install.sh / install.ps1: marker write moved out of the launcher
generation path so it runs before the Tauri-mode early exit. Both
shell-launcher and Tauri-installed env-mode roots now persist the
marker. Removed the duplicate marker write that was previously inside
install.ps1's $studioHomeExport block.
- studio/src-tauri/src/install.rs: pass UNSLOTH_STUDIO_HOME to the
installer subprocess (when not already in scope) so app-initiated
repair / update flows reach the same root the running app uses.
cargo test --bins -- --test-threads=1: 44/44 pass (4 new tests for
marker validation: sentinel accepted, bin shim accepted, empty dir
rejected, missing path rejected).
* studio: fix Tauri legacy-fallback regression + stale marker cleanup
Cycle-7 reviewer follow-ups (regression I introduced in cycle 6):
- studio_root.rs: add StudioRootSource enum + resolve_studio_root_with_source().
Lets callers distinguish a real custom override (Env / Marker) from the
legacy fallback (Default).
- studio/src-tauri/src/install.rs: only forward UNSLOTH_STUDIO_HOME to the
installer subprocess when the resolution source is Env or Marker. The
Default fallback must NOT be passed -- install.sh / install.ps1 treat
any non-empty UNSLOTH_STUDIO_HOME as env-override mode and would
relocate DATA_DIR to $STUDIO_HOME/share and _LOCAL_BIN to $STUDIO_HOME/bin
(regressing default Tauri repair / update flows from the legacy
~/.local/share/unsloth and ~/.local/bin).
- install.sh / install.ps1: clear stale marker on default / HOME-redirect
installs. A user who first installed with UNSLOTH_STUDIO_HOME=/work/studio
then later reinstalls without env vars no longer has the desktop app
hijacked by ~/.unsloth/studio-home pointing at the old custom root.
- install.sh / install.ps1: when env mode wins over a redirected
HOME / USERPROFILE, write the marker into the OS-reported real profile
home (getent / dscl on Unix; [Environment]::GetFolderPath on Windows)
so a later desktop launch from the user's normal session still finds
it. Falls back to the current HOME / USERPROFILE.
cargo test --bins -- --test-threads=1: 45/45 pass (1 new for the source
enum invariants).
* install: scrub stale marker from real-home on HOME-redirect cleanup
Cycle-8 reviewer follow-up: the previous cleanup branch only removed
\$HOME/.unsloth/studio-home, leaving a stale marker in the real
password-database home after a prior env-mode install. A later default
install with redirected HOME / USERPROFILE would still see the desktop
app resolving the old custom root.
- install.sh: compute the real password-database home (via getent /
dscl) unconditionally, and scrub markers from BOTH \$HOME and the
real-home in the default / HOME-redirect cleanup branch.
- install.ps1: build a profile-candidate list (current USERPROFILE
+ OS-reported real profile) and remove markers from EVERY candidate
in the default / profile-redirect cleanup branch.
bash -n + cleanup smoke verified.
* revert: drop Tauri env-var support + marker file mechanism
Keep this PR scoped to shell installer + Python backend env-var support.
Tauri desktop integration with custom Studio roots is deferred to a
separate, focused PR.
Reverts to pre-PR state:
- studio/src-tauri/src/process.rs (find_unsloth_binary)
- studio/src-tauri/src/desktop_auth.rs (auth_secret_path)
- studio/src-tauri/src/main.rs (setup_logging tauri.log path)
- studio/src-tauri/src/commands.rs (open_logs_dir)
- studio/src-tauri/src/install.rs (work_dir + subprocess env)
- studio/src-tauri/src/studio_root.rs DELETED
Removes from install.sh / install.ps1:
- ~/.unsloth/studio-home marker write/read/cleanup
- HOME-redirect-aware marker location logic
What this PR keeps (the original scope):
- install.sh / install.ps1: UNSLOTH_STUDIO_HOME / STUDIO_HOME env-var
resolver with HOME-redirect detection, tilde expansion, legacy
fallback. Default installs are byte-identical to pre-PR.
- studio/setup.sh / studio/setup.ps1: legacy-equality llama.cpp path.
- studio.conf / launcher persists UNSLOTH_STUDIO_HOME +
UNSLOTH_LLAMA_CPP_PATH for fresh shells (env-mode only).
- unsloth_cli/commands/studio.py: env > sys.prefix sentinel > legacy
resolver, conditional re-export.
- studio/backend/utils/paths/storage_roots.py: same resolver.
- Backend modules use storage_roots (run.py, model_config.py,
transformers_version.py, llama_cpp.py).
cargo test --bins -- --test-threads=1: 34/34 pass (pre-PR baseline).
bash -n install.sh: clean.
* install: cycle-10 fixes (default launcher, --tauri guard, env-mode shortcuts, win PATH)
- install.sh launcher: default and HOME-redirect installs keep the
legacy DATA_DIR=\"\$HOME/.local/share/unsloth\" runtime form so a
later shell with a different \$HOME still resolves DATA_DIR. Only
env-mode bakes the resolved absolute path. Restores byte-identical
default behavior.
- install.sh / install.ps1: fail fast when --tauri is combined with
UNSLOTH_STUDIO_HOME / STUDIO_HOME. The desktop app still resolves
the legacy ~/.unsloth/studio root, so a custom-root --tauri install
would yield a desktop app that cannot find its binary or auth
secret. Print the right alternative.
- install.sh / install.ps1: skip persistent desktop / Start-Menu
shortcuts in env-override mode. Workspace-scoped installs would
otherwise leave launchers pointing at a path the user may delete.
Default and HOME/profile-redirect installs keep the shortcut.
- install.ps1: re-prepend env-override \$ShimDir AFTER
Refresh-SessionPath. Refresh rebuilds PATH as Machine > User >
current \$env:Path, so a previously-installed legacy User PATH
entry would otherwise win precedence over the current-session
env-override shim.
bash -n install.sh, pwsh parser install.ps1 + setup.ps1: clean.
cargo test --bins -- --test-threads=1: 34/34 (Tauri unchanged).
* install: cycle-11 fixes (env-mode launcher writes, --tauri legacy passthrough, run.py llama path)
- install.sh / install.ps1: env-mode no longer skips the entire
create_studio_shortcuts / New-StudioShortcuts function. Move the
early-return INSIDE those functions, just before the persistent
desktop / Start-Menu shortcut creation. The runtime launcher
(launch-studio.sh / launch-studio.ps1), studio.conf with
UNSLOTH_STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH exports, and the icon
ARE always written so env-mode shims can resolve via fresh shells.
- install.sh / install.ps1: --tauri guard passes through when the
override resolves to the legacy default ($HOME/.unsloth/studio /
%USERPROFILE%\.unsloth\studio). The desktop app already uses that
path, so explicit-equality is a supported edge case (matches the
llama.cpp legacy-equality branch).
- studio/backend/run.py: when launched directly (bypassing the
unsloth CLI), set UNSLOTH_STUDIO_HOME and UNSLOTH_LLAMA_CPP_PATH
before the rest of import chain runs so unsloth-zoo's import-time
LLAMA_CPP_DEFAULT_DIR binding picks up the custom-root build. Only
set when STUDIO_ROOT is a real custom override; legacy default
installs leave them unset.
bash -n install.sh, pwsh parser install.ps1: clean.
python ast parse studio/backend/run.py: clean.
cargo test --bins -- --test-threads=1: 34/34 pass (Tauri unchanged).
* install: cycle-12 fixes (--tauri trailing slash + main.py uvicorn env)
- install.sh / install.ps1 --tauri legacy passthrough: strip trailing
separators before comparing the override to the legacy default.
Previously UNSLOTH_STUDIO_HOME=\"\$HOME/.unsloth/studio/\" (with
trailing slash) was rejected even though it resolves to the
supported legacy root.
- studio/backend/main.py: when launched directly via
\`uvicorn main:app\` from a custom-root venv (bypassing both
unsloth_cli and run.py), export UNSLOTH_STUDIO_HOME and
UNSLOTH_LLAMA_CPP_PATH before any unsloth-zoo import so its
import-time LLAMA_CPP_DEFAULT_DIR binding picks up the custom-root
build. Only sets when STUDIO_ROOT is a real custom override.
bash -n install.sh, pwsh parser install.ps1, python ast main.py: clean.
Smoke probe: UNSLOTH_STUDIO_HOME=\$HOME/.unsloth/studio/ install.sh --tauri
no longer exits with the unsupported-custom-root error.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.ps1: skip CWD-relative venv migration in env-override mode
The legacy ~/unsloth_studio venv migration path on Windows reads
%USERPROFILE%\unsloth_studio\Scripts\python.exe (a fixed home-relative
path). Under env-override mode this would Move-Item the user's
pre-existing default-install venv into $StudioHome\unsloth_studio,
breaking the default install and contaminating the workspace root.
Gate the migration on $StudioRedirectMode -ne 'env' so workspace-scoped
installs leave the user's default-install venv untouched.
No Linux equivalent: install.sh migrates from \$STUDIO_HOME/.venv which
is already env-mode-aware (points at the workspace root, not \$HOME).
* install: cycle-14 fixes (Tauri env scrub + setup.ps1 missing-root error)
Tauri does not honor UNSLOTH_STUDIO_HOME / STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH
yet -- the desktop app's Rust paths use the legacy ~/.unsloth/studio root.
If the user's shell has these env vars set, spawned Python subprocesses would
diverge from the Rust paths (custom-root Python <-> legacy-root Rust).
Scrub the three env vars at all Tauri subprocess spawn sites:
- process.rs: backend launch
- desktop_auth.rs: provision-desktop-auth subprocess
- install.rs: install.sh / install.ps1 invoked from the desktop app
(also prevents the --tauri guard from rejecting an inherited override).
setup.ps1: when UNSLOTH_STUDIO_HOME points at a non-existent directory,
'Resolve-Path -LiteralPath' threw a confusing PSObject error under
$ErrorActionPreference = "Stop". Test-Path the override first and emit a
friendly "run install.ps1 to create the install root" message instead.
* install: cycle-15 fixes (preserve UNSLOTH_LLAMA_CPP_PATH + add update.rs scrub)
UNSLOTH_LLAMA_CPP_PATH is a pre-existing custom-llama.cpp-directory override
the Python backend (studio/backend/core/inference/llama_cpp.py) and unsloth-zoo
intentionally support. It is unrelated to the Studio install root. Cycle 14
over-scrubbed it from the Tauri spawn sites, regressing desktop GGUF/llama.cpp
workflows for users who set it in their shell.
- process.rs / desktop_auth.rs / install.rs: stop scrubbing
UNSLOTH_LLAMA_CPP_PATH; only scrub UNSLOTH_STUDIO_HOME and STUDIO_HOME.
- update.rs: missed Tauri spawn site -- add the same UNSLOTH_STUDIO_HOME /
STUDIO_HOME scrub so 'unsloth studio update' from the desktop app updates
the legacy-root install Tauri actually manages.
Verified: cargo test --bins -- --test-threads=1 -> 34/34 pass.
* install.sh: document apostrophe-escape derivation inline
The shell quoting at install.sh:642 / 659 / 679 / 680 / 823 has been
flagged as broken across multiple review cycles, but every end-to-end
verification (DATA_DIR=\"a b's&c|d\$e\" -> generated launcher -> source ->
recovered exact input) passes. The proposed "8 backslash" fix would
double the escape and actually break what currently works.
Strengthen the inline comments to spell out the derivation:
- shell pattern \"s/'/'\\\\''/g\" passes \"s/'/'\\''/g\" to sed (\\\\ -> \\)
- sed replacement '\\'' yields close-quote / escaped-quote / open-quote
- stage 2 (\\, &, |) only needed where the value is then sed-replaced
into a launcher template via s|@@DATA_DIR@@|VALUE|g
studio.conf is written via printf, not sed, so it only needs stage 1.
No behavior change, only inline doc to head off future false positives.
* install/setup .ps1: use -LiteralPath for $StudioHome-derived paths
Pre-PR, $StudioHome was hardcoded to %USERPROFILE%\.unsloth\studio --
no wildcard characters possible. The PR introduces UNSLOTH_STUDIO_HOME /
STUDIO_HOME, so $StudioHome (and every path derived from it: $VenvDir,
$VenvPyExe, $UnslothExe, $UnslothHome, $LlamaCppDir, $VenvT5_*, etc.)
can now contain bracket characters that PowerShell would interpret as
wildcards.
Reproducer (from cycle 17 review 20):
pwsh> Test-Path 'studio[abc]/Scripts/python.exe'
False
pwsh> Test-Path -LiteralPath 'studio[abc]/Scripts/python.exe'
True
Switch the relevant Test-Path / Remove-Item / New-Item / Move-Item calls
in install.ps1 and studio/setup.ps1 to -LiteralPath. Sites where the
path is fixed (the shim under %LOCALAPPDATA%\Microsoft\WindowsApps,
$RepoRoot from -PSCommandPath) keep the wildcard-aware form.
* install/setup .ps1: fix New-Item -LiteralPath regression from cycle 17
Cycle 17 added -LiteralPath to all $StudioHome-derived path operations,
but New-Item has no -LiteralPath parameter (verified pwsh 7.6 syntax:
"New-Item [-Path] <string[]> [-ItemType <string>] ..."). Every directory-
creation site would throw "A parameter cannot be found that matches
parameter name 'LiteralPath'" at runtime, blocking T5 sidecar setup,
llama.cpp parent creation, and StudioHome creation.
Likewise, "Split-Path -LiteralPath $X -Parent" cannot mix LiteralPath
with -Parent (separate parameter sets). The default LiteralPath mode
already returns the parent.
Switch to [System.IO.Directory]::CreateDirectory($X), which natively
takes a literal path, and drop the trailing -Parent on Split-Path.
Verified end-to-end on a bracketed path "/tmp/...[abc]":
- CreateDirectory: created
- Test-Path -LiteralPath: detects
- nested CreateDirectory(Split-Path -LiteralPath ...): works
* install/setup .ps1: extend -LiteralPath sweep to remaining \$StudioHome paths
Cycle 17/18 missed several wildcard-aware operations on user-controlled
\$StudioHome-derived paths. Reviewers identified remaining sites:
install.ps1:
- \$UnslothExePath (Test-Path / Resolve-Path) at the shortcut creator
- \$VenvDir (Get-ChildItem) at the no-torch-runtime resolver
- \$ShimDir (New-Item Directory -- replaced with .NET CreateDirectory)
- \$ShimExe (Test-Path / Remove-Item / re-prepend guards) -- the shim
lives at \$StudioHome\\bin\\unsloth.exe in env-override mode, so it
inherits bracket sensitivity from \$StudioHome.
- \$UnslothExe (Copy-Item fallback) when HardLink fails.
studio/setup.ps1:
- \$LlamaServerBin (Test-Path) at the prebuilt-bundle / source-build
validation gates (3 sites). \$LlamaServerBin lives under \$BuildDir
under \$LlamaCppDir under \$UnslothHome under \$StudioHome.
New-Item HardLink keeps -Path because creating a non-existent target
with brackets succeeds (verified via direct pwsh smoke test).
* install: cycle-20 fixes (more setup.ps1 -LiteralPath + shell-quote launch hints)
setup.ps1: extend -LiteralPath sweep to remaining \$BuildDir-derived paths
that the cycle-19 commit missed:
- \$CmakeCacheFile (Test-Path + Select-String -Path)
- \$buildTmp (10 Test-Path / Remove-Item sites in source-build cleanup)
- \$QuantizeBin (Test-Path)
- \$altBin (Test-Path)
These all live under \$BuildDir -> \$LlamaCppDir -> \$UnslothHome ->
\$StudioHome, which is now user-controlled via UNSLOTH_STUDIO_HOME.
Bracket characters in the override would silently skip rebuild
detection or leave stale build artifacts.
install.sh: shell-quote the launch-instruction substep lines for env-
override mode. UNSLOTH_STUDIO_HOME values containing spaces or
apostrophes (e.g. "/tmp/O'Brien Studio") would print copy-paste-
unsafe commands -- the install succeeded but the printed launch
instructions split at the space. Now wraps with the canonical
'\\''-style escape so the printed lines parse with bash -n.
Verified end-to-end:
- printed shim line: '/tmp/O'\''Brien Studio/bin/unsloth' studio ...
- bash -n on the printed line passes.
* install.ps1: -LiteralPath for macOS-stub-launcher \$appDir-derived paths
The shortcut/launcher generator at install.ps1:418-693 writes the
stub launcher, .vbs, and icon under \$appDir = \$StudioDataDir, which in
env-override mode is \$StudioHome\share. Cycle 17/19/20 missed the
following wildcard-aware ops on these paths:
- Test-Path \$appDir (with New-Item Directory swap to .NET CreateDirectory)
- Set-Content -Path \$launcherVbs (for the WSH .vbs stub)
- Test-Path / Copy-Item \$bundledIcon (bundled icon copy)
- Test-Path / Remove-Item \$iconPath (icon header validation)
In env-override mode \$StudioHome can contain bracket characters;
without -LiteralPath the .vbs write fails outright and the icon
validation can either skip a present icon or fail to delete a
malformed one. (The COM shortcut creation downstream returns early
in env-override mode, so its path values don't need this treatment.)
* install: don't override pre-existing UNSLOTH_LLAMA_CPP_PATH in launchers
Cycle 14/15 established UNSLOTH_LLAMA_CPP_PATH as a pre-existing
custom-llama.cpp-directory override the Python backend and unsloth-zoo
intentionally support, independent of the Studio install root.
The launchers (studio.conf sourced by Unix launch-studio.sh, and the
PowerShell launch-studio.ps1) were unconditionally re-exporting it,
which silently overrides a user's pre-existing value when they invoke
the launcher from a shell where UNSLOTH_LLAMA_CPP_PATH is already set.
Make the assignment conditional in both launchers:
install.sh studio.conf:
if [ -z "\${UNSLOTH_LLAMA_CPP_PATH:-}" ]; then
export UNSLOTH_LLAMA_CPP_PATH='...'
fi
install.ps1 launch-studio.ps1:
if (-not \$env:UNSLOTH_LLAMA_CPP_PATH) {
\$env:UNSLOTH_LLAMA_CPP_PATH = '...'
}
UNSLOTH_STUDIO_HOME stays unconditional: the launcher is bound to a
specific install, so its STUDIO_HOME must always match that install.
* install.sh: harden --tauri legacy resolver against CDPATH and symlinks
Reviewer cycle 23 (inst 19) noted that the bare \`cd -- ... && pwd\` form
in the --tauri legacy comparison can echo a CDPATH-prefixed path when the
user has CDPATH set in their environment, contaminating the resolved
absolute path used in the legacy-equality check.
Switch to \`CDPATH= cd -P -- ... && pwd -P\` so:
- CDPATH= clears the cd-prefix-echo behavior
- -P / pwd -P resolves any symlinks to a canonical path
No behavior change for users without CDPATH set; correctness fix for
users who have it set in their shell.
* install + llama_cpp backend: cycle-24 hardening
Three real findings from cycle 24 reviewers:
1. install.sh:231 + studio/setup.sh:413 -- main \$STUDIO_HOME
resolvers used the same bare \`cd -- ... && pwd\` form that cycle 23
only fixed for the --tauri guard. Switch both to:
\$(CDPATH= cd -P -- "\$override" && pwd -P)
so relative custom-root values don't get CDPATH-prefixed or have
the cd-on-CDPATH stdout newline contaminate the captured value.
2. install.sh --tauri legacy root used logical \$HOME/.unsloth/studio
while the override side was canonicalized via pwd -P. A symlinked
\$HOME (e.g. /home/alice -> /u/alice) made the comparison fail even
when both sides pointed at the same directory. Canonicalize the
legacy side too when the dir exists.
3. studio/backend/core/inference/llama_cpp.py:_find_llama_server_binary
searched \$STUDIO_HOME/llama.cpp first then ~/.unsloth/llama.cpp
in default-mode installs. setup.sh / setup.ps1 only install llama.cpp
under \$STUDIO_HOME/llama.cpp in env-override mode; in default mode
it always lives at ~/.unsloth/llama.cpp. The post-PR search would
pick up a stale partial install at ~/.unsloth/studio/llama.cpp over
the real legacy binary.
Mirror setup's legacy-equality check: when studio_root() resolves
equal to ~/.unsloth/studio, search ONLY the legacy ~/.unsloth/llama.cpp.
Otherwise (env-override custom root), search custom first, legacy
fallback.
* install + setup: canonicalize legacy-equality comparison sites
Cycle 24 made \$STUDIO_HOME canonical via 'CDPATH= cd -P -- ... && pwd -P',
but the legacy-equality comparison sites still used the bare logical
"\$HOME/.unsloth/studio" string. With a symlinked \$HOME (e.g.
/home/alice -> /u/alice), the comparison fails even when both sides
point at the same dir, and llama.cpp ends up under a custom-root path
the Python backend's legacy comparison cannot find.
Reviewer cycle 25 inst 2 reproduced this with HOME=/tmp/link -> /tmp/real
and UNSLOTH_STUDIO_HOME=\$HOME/.unsloth/studio: setup.sh resolves
UNSLOTH_HOME to /tmp/real/.unsloth/studio while the backend search
resolves both physically equal and looks at /tmp/link/.unsloth/llama.cpp.
Canonicalize the legacy side at all four sites:
- install.sh:695 (create_studio_shortcuts llama.cpp path)
- studio/setup.sh:577 (UNSLOTH_HOME selection)
- install.ps1:462 (launcher UNSLOTH_LLAMA_CPP_PATH path)
- studio/setup.ps1:1829 (UnslothHome selection)
Apply CDPATH= cd -P -- ... && pwd -P (Unix) or Resolve-Path -LiteralPath
(Windows) when the legacy dir exists. unsloth_cli/commands/studio.py
already does this via Path.resolve().
* llama_cpp: gate _kill_orphaned_servers studio-root allowlist on env-override
Cycle 24 fixed _find_llama_server_binary to only search
\$STUDIO_HOME/llama.cpp when STUDIO_HOME is a real env override (not
the legacy default), but the symmetric _kill_orphaned_servers
allowlist still appended _sr() / "llama.cpp" unconditionally.
In default mode _sr() resolves to ~/.unsloth/studio, so
~/.unsloth/studio/llama.cpp would be treated as a Studio-owned install
root for the orphan-kill scan even though the default installer does
not own that path. A llama-server process running there from a
different tool or a stale partial install would be killed.
Apply the same legacy-equality check used in _find_llama_server_binary
and the install/setup scripts: only add _sr()/"llama.cpp" to the
allowlist when STUDIO_HOME != legacy default.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.sh + setup.ps1: canonicalize both sides of legacy-equality check
Proactive audit pass found one real asymmetry the cycle-by-cycle
review process had not yet flagged:
- install.sh:704 / install.ps1:469 are gated on env-mode and only
run when STUDIO_HOME has already been canonicalized (cycle 24).
Symmetric.
- studio/setup.sh:577 / studio/setup.ps1:1829 run UNCONDITIONALLY,
including in default mode. In default mode STUDIO_HOME is set to
the bare logical \$HOME/.unsloth/studio (setup.sh:416) or
Join-Path \$env:USERPROFILE ".unsloth\\studio" (setup.ps1:1480).
Cycle 25 canonicalized only the legacy side, creating an
asymmetry under symlinked \$HOME / junctioned %USERPROFILE%.
Result of the asymmetry: a default-mode install on a host with
\$HOME=/tmp/link -> /tmp/real treats the legacy default as a custom
root, putting llama.cpp at \$STUDIO_HOME/llama.cpp instead of
~/.unsloth/llama.cpp -- and the Python backend's _find_llama_server_binary
(which uses .resolve() on both sides) then can't find the install.
Fix: canonicalize STUDIO_HOME on the fly at the comparison site, in
both setup.sh and setup.ps1. Symmetric with the now-canonicalized
legacy side from cycle 25, regardless of which mode set STUDIO_HOME.
The other two comparison sites (install.sh:704, install.ps1:469) are
already symmetric because they only run when STUDIO_HOME comes from
the env-override resolution path that already does pwd -P / Resolve-Path.
unsloth_cli/commands/studio.py + studio/backend/run.py + main.py +
llama_cpp.py already use .resolve() on both sides -- symmetric.
* install.ps1: env-override resolution uses .NET API for literal paths
Gemini code-review (review 4177641398, commit 2ea2c91) caught two
remaining New-Item -Path sites in the env-override resolution block
that the cycle 18 sweep missed:
- Line 123: New-Item -ItemType Directory -Path \$envOverride
- Line 132: New-Item -ItemType File -Path \$probe (writability test)
Both use -Path which interprets square brackets as wildcards. For a
user with UNSLOTH_STUDIO_HOME=C:\\workspaces\\studio[abc], both calls
would fail before the install starts. New-Item also has no
-LiteralPath in PowerShell 5.1.
Replace both with the .NET API:
- [System.IO.Directory]::CreateDirectory(\$envOverride)
- [System.IO.File]::WriteAllText(\$probe, "") -- closes the file
handle before the Remove-Item below.
End-to-end verified with /tmp/test-envoverride-[abc]-* path:
CreateDirectory + WriteAllText + Test-Path -LiteralPath all work.
* comments: condense multiline blocks added by this PR
Across the 27-cycle review process, comments accumulated as multiline
blocks explaining each fix's history (cycle numbers, prior bugs,
reviewer rationale). Compress every block to 1-2 lines that capture
just the WHY, dropping cycle references and history that belongs in
the PR description / commit log instead.
Net: 268 deletions / 124 insertions (-144 lines) of comments only.
Behavior unchanged. Verified: bash -n, pwsh parser, python ast.parse,
cargo check all pass.
* install.ps1: use 'return' over 'exit 1' for Install-UnslothStudio bail-outs
Per Gemini review #4177659001: when users run install.ps1 via
'irm ... | iex', 'exit 1' inside the function terminates the entire
PowerShell process and closes the user's terminal. 'return' bails out
of the function while keeping the shell open, matching existing error
sites at lines 34, 50, 57.
Three sites fixed: --tauri+env-override guard, env-override mkdir/access
failure, and write-probe failure. The 'exit' calls at lines 591/611
are inside a generated launcher here-string (a separate top-level .ps1
that runs as its own process), so they correctly stay as 'exit'.
* install.{sh,ps1}: address Gemini review #4177680451
Three medium fixes:
1. install.sh redirection detection: canonicalize both sides of the
$HOME vs passwd-DB comparison via 'CDPATH= cd -P -- ... && pwd -P'
so a trailing slash on $HOME (or symlink-vs-realpath mismatch with
getent/dscl output) doesn't misfire the redirection branch.
2. install.sh shim symlink: 'ln -sf' into an existing directory creates
the link INSIDE it ($_LOCAL_BIN/unsloth/unsloth instead of the
intended file). Pre-strip a real (non-symlink) directory at
$_LOCAL_BIN/unsloth before linking.
3. install.ps1 ShimExe: add -Recurse to Remove-Item so the launcher
refresh recovers if $ShimExe somehow exists as a directory rather
than a file (would otherwise drop into the catch and skip the
shim update).
* install.ps1: use 'throw' over 'return' for fatal validation failures
Cycle 28 reviewer.py (12/8 RC/APPROVE) caught a regression introduced
by the previous Gemini-review fix (#4177659001 -> commit 393e676b).
'return' inside Install-UnslothStudio kept iex'd terminals alive but
made 'pwsh -File install.ps1' exit with code 0 on fatal validation
failures (--tauri+custom-root rejected, STUDIO_HOME unwritable, etc.),
so CI / wrapper scripts treated failed installs as successful.
'throw' satisfies both constraints:
- pwsh -File install.ps1: exits with code 1 (CI sees failure)
- irm | iex: shows error to user, does NOT close the host terminal
Three sites: --tauri+env-override guard, mkdir/access failure,
write-probe failure. Verified throw -> exit code 1 under pwsh -File.
* install.ps1 launcher: single-quote child -Command path
Cycle 28 P2 finding: the generated launch-studio.ps1 builds the child
PowerShell -Command string with the executable path inside double
quotes, so a custom Studio root containing PowerShell metacharacters
(\$, backtick) re-expands in the child shell. Example:
D:\work\\\$job\studio -> child reparses \$job and runs the wrong path.
Fix: single-quote the path inside the child command and double any
apostrophes (PowerShell's literal-quote-escape form) so paths like
"O'Brien Studio & x|y" or "C:\work\\\$bad\studio" survive verbatim.
* install: harden custom Studio root handling
- install.sh shim refresh: refuse to recursively delete a real directory
at $_LOCAL_BIN/unsloth before creating the symlink. The previous rm -rf
could destroy unrelated user data living at that path.
- install.ps1 shim refresh: drop -Recurse from Remove-Item on $ShimExe and
refuse early when the shim path is a directory; mirrors the install.sh
guard so a directory at $StudioHome\bin\unsloth.exe is not blown away.
- install.ps1 PATH wiring: remove the redundant first $ShimDir prepend in
env-override mode; the post-Refresh-SessionPath prepend is the one that
takes effect, and the duplicate left $ShimDir in $env:Path twice.
- install.ps1 manual launch instructions: single-quote the printed shim
and Activate.ps1 paths so '$' / backtick metacharacters in custom roots
do not reparse when the user copies and pastes the command.
- studio/setup.sh: validate writability of UNSLOTH_STUDIO_HOME with the
same [ -w ] check install.sh already has, so a read-only override fails
with a clear message instead of an obscure uv pip permission error.
- Drop the STUDIO_HOME alias everywhere (storage_roots.py, studio.py,
install.sh, studio/setup.sh, install.ps1, studio/setup.ps1). The name
is too generic and an ambient STUDIO_HOME from unrelated tooling could
silently redirect the install. Only UNSLOTH_STUDIO_HOME is honored.
- unsloth_cli/commands/studio.py: defer UNSLOTH_STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH
re-export from import time into a helper invoked by the studio app
callback. Importing the module no longer mutates os.environ as a side
effect, so test runners and CLI introspection stop leaking those vars
into unrelated subprocesses.
- studio/backend/core/inference/llama_cpp.py: replace set-mutation inside
list comprehension with an explicit dedup loop for readability.
* install: harden custom Studio root edge cases
- install.ps1 shim refresh: move the directory-collision preflight outside
the lock-handling try/catch. The previous throw inside the try block was
swallowed by the surrounding catch and downgraded to a "Continuing with
the existing launcher" warning, leaving the install in a broken state
with no usable shim on disk.
- storage_roots.py / unsloth_cli/commands/studio.py: tighten the bin-shim
sentinel from .exists() to .is_file(). A directory at the candidate
bin/unsloth (or bin/unsloth.exe) path would otherwise false-positive
the venv inference and pick the wrong Studio root.
- storage_roots.py / unsloth_cli/commands/studio.py: wrap the env-var
override Path(...).expanduser().resolve() in try/except (OSError, ValueError),
matching the defensive pattern already used in studio/backend/main.py
and studio/backend/run.py. An invalid override (unresolvable network
drive, bad characters) now falls back to the un-resolved path instead
of crashing at import time.
* install: fail fast on missing custom root, allow brackets in shim path
- install.ps1 shim hardlink: switch the New-Item -ItemType HardLink call
from -Path to -LiteralPath so a custom Studio root containing bracket
characters does not fail under PowerShell's wildcard-aware -Path
parameter. Matches the -LiteralPath usage on every other Test-Path /
Remove-Item / Copy-Item call against the same shim path.
- studio/setup.sh override branch: replace the silent mkdir -p of the
override directory with an existence check that exits 1 with a clear
message. setup.sh runs against an existing install (via 'unsloth
studio update'), so a typo in UNSLOTH_STUDIO_HOME must not materialize
an empty workspace dir. Brings the Unix flow in line with setup.ps1,
which already errors on a missing override root.
* llama_cpp: scope orphan-server kill to the active install root
_kill_orphaned_servers used to unconditionally include the legacy
~/.unsloth/llama.cpp tree in install_roots, even when the running
Studio is in env-override mode and operates out of a custom root.
On a single OS user running both a default-install Studio and a
custom-root Studio concurrently, the custom Studio would kill the
default Studio's llama-server during startup orphan cleanup.
Hoist _is_custom_root out of the import try/catch so the legacy-
append decision sees it (default to False on ImportError so default
mode behaviour is unchanged), and gate the legacy ~/.unsloth/llama.cpp
append on `not _is_custom_root`.
* install: harden custom-root .venv migration and shim hardlink
- install.sh / install.ps1 OLD-layout .venv migration: gate on
default-mode only. Without the guard, pointing UNSLOTH_STUDIO_HOME at a
workspace that already has .venv (e.g. an unrelated Python project)
caused the torch validation to fail and the installer to recursively
remove the user's project venv. Mirrors the existing env-mode skip on
the CWD-relative venv migration immediately below.
- install.ps1 shim hardlink: revert to New-Item -ItemType HardLink -Path.
-LiteralPath is not accepted on the HardLink ItemType in any PowerShell
version, so the previous form always threw and silently fell back to
Copy-Item, breaking hardlink-update propagation. Bracket characters in
$ShimExe are still defended by the directory-collision preflight added
earlier.
- storage_roots.py / unsloth_cli/commands/studio.py: strip whitespace
from the UNSLOTH_STUDIO_HOME env var before the truthy check so a
blank " " override does not become a real path with trailing spaces
(which would silently break every downstream Studio path operation).
* Studio paths: tolerate stat / resolve failures during root inference
- storage_roots._infer_studio_home_from_venv: wrap the share/studio.conf
and bin/shim is_file() sentinel checks in try/except OSError. A
PermissionError on a restricted candidate dir would otherwise propagate
out of studio_root() and crash module import in run.py / main.py /
transformers_version.py / model_config.py at server startup.
- llama_cpp._kill_orphaned_servers: broaden the studio_root() guard from
ImportError-only to (ImportError, OSError, ValueError) so transient
resolve / sentinel failures do not crash the orphan-killer at server
startup. Matches _find_llama_server_binary's existing pattern.
- llama_cpp._find_llama_server_binary: nest the inner resolve() in its
own try/except and fall back to unresolved-path comparison instead of
dropping the custom search root entirely. A transient resolve() error
on the legacy path no longer loses the custom-root llama.cpp lookup.
* Add Studio install-root resilience tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: isolate custom-root installs from default-install state
- llama.cpp discovery in env-override mode no longer falls back to the
legacy ~/.unsloth/llama.cpp tree. The orphan-cleanup path already
excludes that root in custom mode; aligning discovery prevents a
custom-root Studio from launching a sibling install's binary it then
refuses to manage. Users who want a shared build set
UNSLOTH_LLAMA_CPP_PATH explicitly.
- Generated POSIX launcher (install.sh heredoc) namespaces LOCK_DIR with
a hash of DATA_DIR and persists the launched port to
$DATA_DIR/studio.port; in env-override mode the fast-path attaches only
to a port we ourselves wrote, never to a sibling Studio that happens
to be healthy on 8888..8908.
- Generated Windows launcher (install.ps1 heredoc) bakes a per-install
$portFile and SHA-256-suffixed mutex name, mirroring the POSIX side;
Find-HealthyStudioPort uses the port file in env-override mode.
- studio/setup.sh and studio/setup.ps1 require an .unsloth-studio-owned
marker before deleting $STUDIO_HOME/.venv_t5*, $STUDIO_HOME/llama.cpp,
and the sidecar T5 venvs in env-override mode. The marker is dropped
after fresh creation so subsequent runs of 'unsloth studio update'
proceed cleanly. Mirrors the existing .venv guard in install.sh.
- Wrap bare Path.resolve() calls on the legacy STUDIO_HOME constant in
studio/backend/main.py, studio/backend/run.py, and
unsloth_cli/commands/studio.py in the same try/except (OSError,
ValueError) used adjacently, so a restricted parent or recursive
symlink on $HOME does not crash module import / CLI startup.
* Studio: guard env-mode workspace against destructive cleanup
- install.sh and install.ps1 unconditionally rm -rf / Remove-Item the
new-layout $STUDIO_HOME/unsloth_studio when it has a python; in
env-override mode that path is a user-chosen workspace, mirroring
the .venv migration concern the .venv branch already guards. Refuse
to remove an existing $STUDIO_HOME/unsloth_studio that lacks Studio
sentinels (share/studio.conf or bin/unsloth).
- studio/setup.ps1 only checked Test-Path -PathType Container on the
custom root; setup.sh and install.ps1 both also write-probe via
WriteAllText / Remove-Item. Add the matching probe so 'unsloth
studio update' against an ACL-restricted root fails fast with a
clear message instead of erroring later while creating sidecar
venvs.
* Add Studio install/setup workspace-isolation tests
* Studio: tighten installer rationale comments
- install.sh: collapse a 5-line restatement into 3 lines, naming
env-mode behavior up front and the byte-identical pre-override
fallback after.
- install.ps1: correct misleading hardlink comment that claimed the
directory-collision preflight guards against wildcard expansion;
bracket characters in $ShimExe still glob-expand here, with the
Copy-Item -LiteralPath fallback handling them.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Split: keep only 2 file(s)
* Studio: harden env-mode workspace guards across installers and update path
Tightens the UNSLOTH_STUDIO_HOME custom-root protections so destructive
installer paths cannot displace unrelated user data when the override
points at a workspace.
install.sh / install.ps1: env-mode sentinel that gates rm -rf $VENV_DIR /
Remove-Item $VenvDir now requires share/studio.conf or the bin/unsloth(.exe)
shim to be a real file or symlink. Previously a directory at bin/unsloth or
bin\unsloth.exe satisfied the check (-e and bare Test-Path accept any path
type), so a workspace with unrelated content under unsloth_studio plus a
sibling directory at bin/unsloth could be wiped.
studio/setup.ps1: stale-venv rebuild branch now mirrors install.ps1's
env-mode guard before Remove-Item -LiteralPath $VenvDir -Recurse -Force.
Without this, "unsloth studio update" pointed at a custom workspace whose
unsloth_studio venv fails torch validation deletes the venv even when the
root carries no Studio sentinels.
studio/setup.sh / studio/setup.ps1: prebuilt llama.cpp install path now
calls _assert_studio_owned_or_absent / Assert-StudioOwnedOrAbsent before
invoking install_llama_prebuilt.py, and writes the .unsloth-studio-owned
marker on success. install_llama_prebuilt.py uses os.replace() to move
any existing install_dir aside before staging, so an unrelated
$STUDIO_HOME/llama.cpp could otherwise be displaced before the existing
source-build ownership guard ever ran.
* Studio: gate ownership guards on canonical custom-root and add venv marker
Tightens UNSLOTH_STUDIO_HOME ownership semantics so they fire only for a
genuinely custom root, never for an explicit override that resolves to the
legacy default. Adds an in-VENV marker that lets a partial install be
repaired and provides a strong primary sentinel for the deletion guard.
studio/setup.sh + studio/setup.ps1: hoist the canonical $STUDIO_HOME vs
legacy-default comparison so it sits next to the marker definition, derive
_STUDIO_HOME_IS_CUSTOM / $StudioHomeIsCustom once, and gate the
_assert_studio_owned_or_absent / Assert-StudioOwnedOrAbsent helpers and the
prebuilt llama.cpp marker writes on that flag instead of raw env-var
presence. UNSLOTH_STUDIO_HOME=$HOME/.unsloth/studio (legacy override) no
longer trips the guard for pre-PR T5 sidecar venvs or llama.cpp dirs that
predate the .unsloth-studio-owned marker. The duplicate canonical block
inside the llama.cpp section is removed; the new flag is reused.
studio/setup.ps1: Assert-StudioOwnedOrAbsent's marker check now requires
-PathType Leaf so a directory at .unsloth-studio-owned cannot satisfy it.
The in-place git-sync branch in the source-build path now calls
Mark-StudioOwned after a successful sync so a later prebuilt-update path
does not fail Assert-StudioOwnedOrAbsent on the same root.
install.sh + install.ps1: write $VENV_DIR/.unsloth-studio-owned right after
uv venv succeeds and accept it as the primary sentinel in the env-mode
deletion guard. This recovers from a partial install that was previously
unrepairable, and is a stronger sentinel than sibling shim files (the
marker is inside the venv that is about to be wiped, so an unrelated
workspace cannot accidentally satisfy it).
install.sh: drop the standalone -L test on $STUDIO_HOME/bin/unsloth in the
deletion guard. -L returns true for any symlink including symlinks to
directories and broken symlinks; -f already accepts the legitimate
file-targeted symlink shape created by ln -s at install.sh:1864.
* Studio: close residual workspace-isolation gaps for custom roots
Four follow-on hardenings that close the remaining cross-root leaks the
custom-root install plumbing still left open.
studio/setup.ps1 in-place git-sync: when the source-build path finds an
existing $LlamaCppDir/.git, it ran git remote set-url, checkout -B, and
clean -fdx in place before any ownership check. The previous fix marked
the tree as Studio-owned AFTER the sync but did not guard the BEFORE
case, so an unrelated workspace .git could be silently rewritten on the
first source-build under a custom UNSLOTH_STUDIO_HOME. Add the same
Assert-StudioOwnedOrAbsent guard already used by the prebuilt path and
the temp-dir swap path (gated on $StudioHomeIsCustom for parity).
Launcher port-file workspace isolation: the env-mode launchers' fast
path attached to any backend listening on the cached port that returned
a healthy /api/health, even when that backend belonged to a different
install root. studio/backend/main.py /api/health now returns the
resolved studio_root; install.sh _check_health and install.ps1
Test-StudioHealth verify it against UNSLOTH_STUDIO_HOME when set, so a
stale studio.port pointing at a sibling Studio is rejected instead of
opening the wrong UI.
studio/src-tauri preflight + commands: the Tauri desktop app stays on
the legacy root by design. process.rs / install.rs / desktop_auth.rs /
update.rs already strip UNSLOTH_STUDIO_HOME and STUDIO_HOME from their
CLI subprocesses, but preflight.rs run_cli_probe / probe_cli_capability
and commands.rs check_install_status did not, so a desktop launch from
a shell carrying those env vars produced status reflecting a different
root than the desktop manages. Mirror the existing scrub.
install.sh shim install: the previous `rm -f -- $_shim_path; ln -s ...`
pair leaves a window with no shim if interrupted. Use ln -sfn for an
atomic replace; the -n flag prevents descent into a symlink-to-directory
target (the existing directory guard above already rejects a real dir).
* Studio: replace launcher root verify with hex digest baked at install time
The previous launcher identity check returned the absolute resolved Studio
install root from /api/health and matched it against $UNSLOTH_STUDIO_HOME
in the launcher. Three problems that this commit closes:
- POSIX launcher used a raw bash `case` against the JSON-encoded value, so
paths containing characters that JSON escapes (e.g. /tmp/back\slash,
/tmp/O"Brien) caused the launcher to reject its own healthy backend.
- /api/health is unauthenticated and Studio supports `-H 0.0.0.0`, so any
reachable client could read the absolute install path (username, home
dir, workspace name, CI checkout path).
- The verification was gated on $UNSLOTH_STUDIO_HOME being set at runtime,
so a default-mode launcher would attach to a sibling env-mode Studio
listening on the same port instead of starting its own.
The fix replaces the raw path with a SHA-256 hex digest computed at install
time and baked into the generated launcher (mirroring how @@DATA_DIR@@ is
substituted today):
studio/backend/main.py: /api/health now returns `studio_root_id =
sha256(str(_studio_root()))` instead of the raw `studio_root` path.
install.sh: computes `_css_studio_root_id` once from $STUDIO_HOME using
python3, bakes `_EXPECTED_STUDIO_ROOT_ID='@@STUDIO_ROOT_ID@@'` into the
launcher heredoc, and adds `s|@@STUDIO_ROOT_ID@@|...|g` to the existing
sed pipeline for ALL modes (env / home / default). _check_health verifies
the baked id substring-matches the JSON response. Hex-only so no shell or
sed escape corner cases.
install.ps1: same shape on Windows. SHA256 the $StudioHome bytes, lower
hex, bake `$_ExpectedStudioRootId = '...'` into the launcher heredoc.
Test-StudioHealth now compares `$resp.studio_root_id -eq
$_ExpectedStudioRootId` unconditionally (no special-case for env-mode).
Default-mode launchers also bake their expected id, so two coexisting
Studio installs on the same machine can no longer cross-attach.
* Studio: harden launcher root-id and split install-time mode from runtime env
- install.sh launcher: compute studio_root_id with the venv Python (uv-managed
systems may not have system python3) and canonicalize STUDIO_HOME with
cd -P/pwd -P so default and home-redirect modes match the backend's
Path(sys.prefix).resolve() canonicalization. Fail fast instead of silently
baking an empty discriminator.
- install.sh launcher heredoc: gate PORT_FILE / namespaced LOCK_DIR on a baked
install-time mode flag (@@INSTALLED_IS_ENV_MODE@@) instead of the runtime
UNSLOTH_STUDIO_HOME variable so a sourced custom-root studio.conf cannot flip
a default-mode launcher into env-mode behavior with stale state.
- studio/backend/main.py: cache the studio_root_id digest at module load so
/api/health does not recompute hashlib + filesystem probes on every poll.
- studio/backend/core/inference/llama_cpp.py: widen the studio_root() probe
except clause from ImportError to (ImportError, OSError, ValueError) so it
matches the sibling _kill_orphaned_servers handler and tolerates Path.resolve
failures from broken symlinks or odd codecs.
* Studio: align launcher root-id digest with backend canonicalization
- studio/backend/main.py: hash the already-resolved _STUDIO_ROOT_RESOLVED
instead of recomputing str(_studio_root()); the default fallback in
storage_roots returns Path.home()/.unsloth/studio without .resolve(), so
on systems where $HOME is a symlink (NFS / AFS / Docker) the cached
digest now matches install.sh's cd -P/pwd -P canonicalization and the
launcher no longer rejects its own healthy backend.
- install.ps1: canonicalize $StudioHome via Resolve-Path before the SHA256
compute (env-mode already resolves at line 121, only default and profile
branches were raw); a junctioned USERPROFILE now produces the same digest
the backend computes via Path.resolve() for the same install.
- install.sh launcher template: substitute the non-user-controlled
@@STUDIO_ROOT_ID@@ and @@INSTALLED_IS_ENV_MODE@@ placeholders before the
user-controlled @@DATA_DIR@@ pass so a $DATA_DIR that contains the
literal placeholder text cannot be mutated by the second sed.
* Studio: tighten installer rationale comments
* Studio install: extend workspace-guard test coverage
Add behavioral coverage for env-mode workspace guards across install.sh,
install.ps1, studio/setup.sh, studio/setup.ps1, the launcher root-id
discriminator, and the backend's /api/health response. Also refresh the
custom-mode llama.cpp resilience assertion so it matches the implementation
that intentionally excludes the legacy tree from search_roots.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor STUDIO_HOME alias, fix workspace-guard test harness, harden rollback
The PR title and description promise STUDIO_HOME as a priority-2 alias
to UNSLOTH_STUDIO_HOME, but the implementation only read the longer name
in all six resolution sites. Wire the alias through install.sh,
install.ps1, studio/setup.sh, studio/setup.ps1, the Python storage_roots
resolver, and the unsloth_cli studio resolver. UNSLOTH_STUDIO_HOME wins
when both are set (more specific signal beats the generic alias).
Whitespace-only values are now treated as unset to match the Python
resolvers' .strip() semantics, preventing install/runtime layout drift
where the installer would create a literal " " directory while the
backend fell through to the legacy default.
Error messages and the substep status line report the env-var name the
user actually set ("UNSLOTH_STUDIO_HOME=..." vs "STUDIO_HOME=...") so
diagnostics stay accurate under either spelling.
Test harness fix: tests/test_studio_install_workspace_guard.py extracted
the install.sh venv-replacement block, but after the merge that block
delegates to _start_studio_venv_replacement (defined further up in
install.sh, not in the extracted snippet). Five sentinel-positive tests
echoed RESULT=ok but never moved $VENV_DIR. Add a single
_INSTALL_GUARD_STUBS constant that stands in a minimal mv-based stub
plus a no-op substep, and route every inline test script through a new
_build_install_guard_script() helper. All 50 tests now pass (was 45/50).
Rollback hardening: Start-StudioVenvRollback / Restore-StudioVenvRollback
/ Complete-StudioVenvRollback in install.ps1 used plain Test-Path,
Move-Item, Remove-Item against paths derived from $StudioHome. With a
custom UNSLOTH_STUDIO_HOME containing brackets (the very motivation for
the broader -LiteralPath sweep this PR set out to do), rollback would
silently misbehave under wildcard interpretation, turning a recoverable
install error into a destroyed env. Same fix for the --local Tauri
overlay block (Test-Path / Copy-Item / Get-FileHash on $VenvDir-derived
paths).
* Replace studio_root_id path-hash with per-install opaque id
The previous design computed studio_root_id as sha256 of the resolved
$STUDIO_HOME path, both at install time (baked into the launcher) and
at backend startup (returned via /api/health). This worked but had
three weaknesses:
1. Information disclosure on -H 0.0.0.0: anyone reaching /api/health
could confirm a guessed install path (username, workspace name,
etc.) by replaying the same hash.
2. Canonicalization brittleness: launcher (cd -P/pwd -P) and backend
(Path.resolve()) had to produce identical strings, which required
careful symlink/junction handling on every site (cycles 17-27 of
the PR review history were entirely about closing this drift).
3. Stale-launcher attach: an uninstall + reinstall at the same path
produced the same hash, so a launcher from the previous install
would silently attach to the new (incompatible) backend.
Replace the path-hash with a per-install opaque id:
- install.sh and install.ps1 generate 32 bytes from the platform CSPRNG
(/dev/urandom on POSIX with a python3 secrets fallback;
RandomNumberGenerator.Create().GetBytes on Windows) and persist it to
$STUDIO_HOME/share/studio_install_id with mode 0600. Atomic
temp-file-rename so a crash mid-install can't leave a half-written id.
The check 'if [ ! -s "$_css_id_file" ]' / Test-Path makes generation
idempotent across re-runs (so re-running install.sh doesn't invalidate
previously-baked launchers in the same install root).
- studio/backend/main.py replaces hashlib.sha256 with
_read_studio_install_id(), which reads $STUDIO_HOME/share/studio_install_id
once at module load. Validates the content against ^[0-9a-f]{64}$ so
malformed/truncated/uppercase/wrong-length content returns "" and
triggers the launcher's existing "no baked id, accept any healthy
Unsloth backend" fallback path.
- /api/health field name (studio_root_id) and wire format (64 hex chars)
preserved for compatibility with launchers already shipped via earlier
PR iterations.
Tests:
- Drop test_install_sh_root_id_matches_backend_resolved_under_symlinked_home
and test_install_ps1_canonicalizes_studio_home_before_root_id_hash --
the entire reason these existed (cd -P/Resolve-Path/Path.resolve()
digest agreement under symlinks/junctions) is moot when the id comes
from a file rather than from the path.
- Drop test_main_py_studio_root_id_hashes_resolved_root_not_unresolved
(no more hashing).
- Rewrite test_main_py_studio_root_id_caches_at_module_load to assert
the file-read pattern; add test_main_py_read_studio_install_id_validates_hex_and_handles_missing
to pin the exact rejection rules (empty / non-hex / wrong case /
wrong length all -> "").
- Rewrite test_install_sh_create_shortcuts_uses_venv_python_first as
test_install_sh_create_shortcuts_seeds_id_from_csprng_with_python_fallback
with a behavioral subprocess check that re-invocation is idempotent.
- Rename test_check_health_handles_path_with_backslash_via_hash to
test_check_health_handles_arbitrary_id_token (the JSON-escape concern
it pinned is preserved -- ids are hex-only by construction -- but the
test no longer derives the id from a path).
- Add test_install_sh_install_id_survives_symlinked_studio_home as a
regression test pinning that the new design has zero canonicalization
drift across symlinked parents.
- Update test_install_sh_bakes_studio_root_id_into_launcher and
test_install_ps1_bakes_studio_root_id_into_launcher to assert the
CSPRNG seed and the file location.
49/49 tests pass. Behavioral verification: install.sh-style generation
is idempotent across runs, three parallel installs at different roots
get distinct ids, reinstall at the same path produces a new id (so
stale launchers correctly fail to attach to the new backend), and
symlinked-\$HOME no longer causes launcher/backend disagreement.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
Bumps the unsloth>= install floor in install.sh and install.ps1 from
2026.5.1 to 2026.5.2 so fresh curl/iwr installs pull the just-released
PyPI version that ships PR #5296: Studio chat history and image
attachments work again with newer @assistant-ui/react.
Studio bound to 0.0.0.0 by default and the installer silently auto-started
a server at end of install, exposing it on the network without consent and
contradicting the privacy-first / local-only guarantee.
- studio/backend/run.py: run_server() and argparse --host default to 127.0.0.1
- unsloth_cli/commands/studio.py: studio_default() and run() --host default to 127.0.0.1
- install.sh: drop -H 0.0.0.0 from generated launcher template; replace silent
auto-start with a [Y/n] prompt; add cloud/network note to manual hint
- install.ps1: drop -H 0.0.0.0 from PowerShell launcher template; replace
silent auto-start with a Read-Host [Y/n] prompt; add cloud/network note
- studio/setup.sh: drop -H 0.0.0.0 from launch hint; add cloud/network note
- README.md: simplify launch examples to `unsloth studio -p 8888`; note
-H 0.0.0.0 is available for cloud/LAN use
Tests:
- studio/backend/tests/test_host_defaults.py
- tests/studio/test_cli_studio_defaults.py
- tests/sh/test_install_host_defaults.sh
* Fix check for libcurl hearders in install.sh
Checking for `dpkg` has no relation to libcurl headers at all. If the package is installed, then an executable `curl-config` is available in Ubuntu/Debian as it can be seen here:
https://ubuntu.pkgs.org/26.04/ubuntu-main-amd64/libcurl4-openssl-dev_8.18.0-1ubuntu2_amd64.deb.html
This also fix the installation in ArchLinux, provided the needed packages are installed previously as shown in the error message when a package is missing.
* Fix check for libcurl hearders in studio/setup.sh
Use the same check as install.sh.
---------
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
When --local is passed, also overlay unsloth-zoo from the upstream main
branch (--no-deps --reinstall-package) on top of the PyPI install. This
keeps the editable unsloth checkout paired with the latest unreleased
unsloth-zoo, mirroring the existing -e $_REPO_ROOT --no-deps overlay.
Applied to all four --local paths in install.sh (migrated, fresh no-torch,
fresh with-torch, auto-torch fallback) and the three corresponding paths
in install.ps1.
Lee Jackson