Resolve three conflicts, keeping both sides in each case:
- studio/backend/tests/test_llama_cpp_update.py: the _clean_state fixture
keeps this branch's routes.inference stub (so _run_update never blocks on a
real Studio backend singleton) alongside main's _whisper_chain_status stub.
- studio/setup.sh: keep both llama.cpp state flags. This branch's
_LLAMA_CPP_DEFERRED (WSL2 aarch64 background CUDA build is success, not
degraded) and main's _LLAMA_CPP_NO_SPACE are independent and both are read
later in the file.
- unsloth/models/_utils.py: keep the DGX Spark / N1X UMA helpers and their
four patch calls, then main's patch_unified_memory_safetensors_load. The
Spark block stays first so patch_dgx_spark_memory_config still sets
PYTORCH_CUDA_ALLOC_CONF before anything can touch the allocator; the
safetensors patch gates lazily inside its wrapper, so installing it after
does not init CUDA.
Verified every line main added since the merge base is still present in
install.sh, install.ps1, studio/setup.sh and studio/install_llama_prebuilt.py,
and likewise for this branch's own additions.
The staleness gate compared only the toolkit major, so a host with
glibc >= 2.41, a CUDA 13.0/13.1/13.2 toolkit and a cu13-capable driver
kept that toolkit and skipped the CUDA 13.3 provisioning. The build then
hit the rsqrt header clash this script exists to avoid, and GGUF
inference stayed on the CPU server.
The gate now parses the toolkit minor and flags a 13.0-13.2 toolkit when
the detected glibc is 2.41 or newer (getconf first, ldd as a fallback;
an unparseable version keeps the previous major-only behavior). The
driver check still applies, so a host whose driver cannot run cu13 is
never pushed onto a 13.3 install.
Verified over the (toolkit release) x (glibc) x (driver major) matrix
with mocked nvcc output: 13.0/13.1/13.2 on glibc >= 2.41 are stale;
13.3, 13.4 and a two-digit 13.10 are kept; the same toolkits on older
glibc are kept; pre-13 stays stale on any glibc; and nothing is flagged
when the driver reports CUDA 12.x. Version parsing verified for 2.39,
2.41, 2.42, 3.0, empty and garbage inputs.
Main fixed the stale ordering assertion in PR 7252; adopting its
version verbatim removes this file from the branch diff entirely and
avoids a conflict on the next main merge. 32 tests pass.
source.index("_resolve_inherited_extra_args(") matched the function
definition, which always precedes the endpoint, so the ordering
assertion was vacuously true. Anchoring on "= _resolve_inherited_
extra_args(" pins the first call site inside the load endpoint (line
4505), which is the statement whose position relative to the GGUF
branch the test is meant to guard. 32 tests pass.
Main moved the llama_extra_args pass-through inheritance out of the
GGUF branch into _resolve_inherited_extra_args, which runs before it,
so the source-order assertion's "if request.llama_extra_args is None"
anchor no longer exists inside the branch and the check failed after
the main merge. The test now asserts the same property in the current
shape: inheritance before the GGUF branch (a carried --no-mmproj still
shapes the hub guard's companion requirement), and marker, hub guard,
unload in order within the branch. Full file passes (32 tests).
Ninth review round; each item reproduced before fixing.
The inner WSL install ran install.sh without /usr/lib/wsl/lib on PATH,
so its GPU detection (which checked PATH and /usr/bin only) could pick
CPU torch wheels on the exact Spark/N1X path this PR exists for, failing
the later torch.cuda probe. The forwarded env now appends
/usr/lib/wsl/lib to PATH (appended, so a PATH nvidia-smi still wins),
and install.sh's _has_usable_nvidia_gpu and torch-index _smi resolution
gained the same location fallback for direct WSL runs.
Three WSL failure paths in install.ps1 (WSL-not-installed deferral, the
download sentinel, and the final torch.cuda failure) set LASTEXITCODE
and returned, bypassing the round-eight Exit-InstallFailure fix, so
powershell -Command automation using the published pipe form still saw
success on those failures. All three now route through
Exit-InstallFailure, which restores the rolled-aside venv and fails the
process in every invocation mode.
The uninstall.ps1 WSL cleanup removed /root/.unsloth before killing and
matched only full argv, so cmake/nvcc children of a live CUDA build
(relative argv after the provisioner cds into the tree) survived the rm
and recreated files. The cleanup now signals each matched PID's whole
process group (guarded against the shell's own pgid, direct children
via pkill -P as fallback) before any rm; the /proc cmdline greps are
unaffected by kill order since they read process state, not files.
The round-eight same-group fallback called pkill -P without a guard;
under this script's set -e a matched provisioner with no children at
that instant (TERM pass already reaped them) aborted the whole
uninstaller before any cleanup. Now || true, like the kill beside it.
Reproduced in a dash sandbox with set -e: a childless matched PID
previously killed the harness, now dies cleanly while setsid-group and
same-group scenarios keep passing.
Verified: bash -n on both shell scripts, sh -n on the extracted WSL
clean snippet, PowerShell AST parse on both ps1 files, the three-
scenario kill sandbox, gpu-detection and installer-index pytest suites
pass, and the sh battery matches the branch baseline.
Cross-platform staging CI surfaced five Repo tests (CPU) failures where
main's tests assert on script internals this branch legitimately
changed; each reconciled on its merits.
The WoA native-wheel probe still used uv's deprecated --index-url alias
that main's test suite now forbids in favor of --default-index (same
semantics, and --default-index is what overrides inherited uv index
defaults); the probe now matches the convention.
The CUDA provision gate spelled its linked-dir guard with the
:-false default form that main's prune-refactor test blacklists
file-wide. The variable is unconditionally initialized far above, so the
guard now uses the plain spelling with identical semantics. The variable
guard itself stays: unlike a symlink test, it also covers the
canonical-location reuse case where the linked dir is not a symlink.
The gpu-detection tests extract named shell functions into a sandbox,
so _setup_has_usable_nvidia_gpu's new _resolve_nvsmi dependency made the
sandboxed helper die on command-not-found and report not_usable for
usable cases; the extraction list now includes the resolver, and the
driver-version hardening assertion tracks the resolved-path spelling
while still requiring the timeout wrapper. Also hardened the resolver
assignment with an explicit empty fallback so a future non-condition
call site cannot trip set -e.
The staging run also showed the Mac Studio Update uninstall step dying
mid-run, consistent with the round-seven group kill signalling its own
process group; the round-eight self-pgid guard already fixes that and
this push carries it to CI.
Verified: the five failing tests pass locally at this head (the one
remaining local red, test_negative_control_no_tokenizers, fails
identically with these changes stashed and did not fail in CI), bash -n,
PowerShell AST parse, and the sh battery matches the branch baseline.
Eighth review round; each item reproduced before fixing.
Exit-InstallFailure under irm-pipe-iex set LASTEXITCODE and returned, so
powershell -Command automation using the published pipe form exited 0 on
fatal installer errors (verified: a -Command run whose last call only
assigns LASTEXITCODE exits 0, while one that throws exits 1). The iex
branch now sets the var for callers that check it and then raises a
terminating error, matching the pre-existing throw behavior there:
interactive shells survive and print it, automation gets exit 1, and the
-File branch keeps carrying the specific code via exit.
The uninstall group kill could signal the uninstaller's own process
group: in a non-interactive session without job control a lingering
provisioner can share the script's pgid, and kill(-pgid) would TERM the
cleanup mid-run. The helper now compares each match's pgid against its
own and falls back to the PID plus its direct children in that case.
Both scenarios exercised in a sandbox: a setsid provisioner group still
dies whole, and a same-group provisioner dies without taking the
harness.
detect_host in install_llama_prebuilt.py resolved nvidia-smi only via
shutil.which, so the root WSL sessions this PR creates (PATH without
/usr/lib/wsl/lib) classified ARM NVIDIA WSL hosts as non-NVIDIA and took
the CPU prebuilt path before setup's provisioning logic could run. It
now falls back to /usr/lib/wsl/lib/nvidia-smi then /usr/bin/nvidia-smi,
the same order as setup.sh's resolver.
Verified: bash -n, Python AST parse, PowerShell AST parse, the pwsh
exit-code experiments above, the two-scenario kill sandbox, and the sh
test battery matches the branch baseline.
Seventh review round; each item reproduced against the live tree first.
The aarch64 bitsandbytes step gated on a bare nvidia-smi, which root login
shells cannot see under WSL2 GPU-PV (the binary lives only in
/usr/lib/wsl/lib, dropped from PATH by the /etc/profile reset), so Spark
and N1X WSL installs finished with CUDA torch but no 4-bit QLoRA. The
gate now resolves nvidia-smi explicitly with the same PATH,
/usr/lib/wsl/lib, /usr/bin order as setup.sh's resolver.
uninstall.sh's CUDA-build kill matched patterns against argv, but the
provisioner cds into the tree before `cmake --build build`, so cmake and
make children carry relative argv no pattern can match; killing only the
wrapper orphaned them mid-build. Each match's whole process group is now
signalled (TERM then KILL), with a plain PID kill as fallback when the
pgid is unreadable or shared with init. Verified in a sandbox: a child
with unmatchable argv in the wrapper's group dies with it.
The WSL shim dir was appended to user PATH while the native installer
prepends its own %USERPROFILE%\.unsloth\studio\bin, whose unsloth.exe
outlives the venv the fallback rolls aside, so on a native-to-WSL rerun
a new terminal resolved unsloth to the dead native launcher. The shim is
now prepended via Add-ToUserPath (which de-dupes and hoists), and the
dead default-root native shim is removed when the venv binary it targets
is gone; custom-root shims are left alone since the prepend outranks
them.
UNSLOTH_NPM_REGISTRY was not forwarded into the inner WSL shell even
though setup.sh threads it into every npm/bun install, so mirror-required
networks failed the frontend step (and with it the install) while the
outer installer honored the mirror. It is now forwarded with the same
strict http(s) allow-list and single-quoting as UNSLOTH_PYTORCH_MIRROR.
Verified: bash -n on both shell scripts, PowerShell AST parse on
install.ps1, the group-kill sandbox above, resolver smoke tests for the
bitsandbytes gate, and the sh test battery matches the branch baseline.
Sixth review round; every item reproduced against the live scripts first.
The WSL fallback tolerated a nonzero inner exit (the optional llama.cpp
step legitimately fails), so a run whose installer died mid-way could pass
the torch and CLI probes on a stale venv from a previous install and be
reported as success. setup.sh now stamps /root/.unsloth/.install-ok after
the core venv and Studio deps complete, just before its tolerated
llama-only nonzero exit; install.ps1 clears the stamp before the run and
requires it to exist afterwards (existence only, no mtime compare, so
WSL/Windows clock skew cannot bite). uninstall.sh removes the stamp and
the downloaded installer file so the trailing rmdir can still prune.
Root login shells reset PATH via /etc/profile and drop /usr/lib/wsl/lib,
the only location of nvidia-smi under WSL2 GPU-PV, so every bare
nvidia-smi probe in setup.sh and the provisioner could silently misreport
"no GPU". Both now resolve nvidia-smi explicitly (PATH, then
/usr/lib/wsl/lib, then /usr/bin) via a shared-resolver pattern, and the
provisioner's driver-major and compute_cap reads use the resolved path.
My round-five uninstall fix inserted the CUDA-build kill block after the
llama.cpp tree was already removed, so a live cmake/nvcc kept running
against deleted paths; the block now runs before the removal.
uninstall.ps1 gated its legacy marker-less WSL cleanup on the process
PROCESSOR_ARCHITECTURE, which reports AMD64 under an x64-emulated
PowerShell on ARM64, skipping exactly the machines the fallback installs
on. It now uses the same triple detection as install.ps1 (OSArchitecture,
Win32_Processor.Architecture 12, machine-level registry arch), factored
into one helper used at both gate sites.
The nvidia-smi capture helper retried twice with a 60s timeout
everywhere, so off WSL a hung nvidia-smi stalled three successive
detect_host probes for about two minutes each; the generous retry now
applies only under WSL (where GPU-PV load slowness is real) and bare
metal keeps a single short attempt.
The generated WSL Desktop launcher hardcoded port 8888 for launch, health
poll, and browser open, so with Jupyter or a second Studio on 8888 the
poll waited on the wrong server forever; it now scans 8888..8908 with a
TcpListener, mirroring the native launcher's free-port window, and passes
the winner via -p everywhere.
Verified: bash -n on all shell scripts, Python AST parse, PowerShell AST
parse on install.ps1, uninstall.ps1, and the generated launcher; the
launcher port scan exercised free, busy, and exhausted cases; the capture
helper unit-tested for WSL and bare-metal attempt/timeout splits; sh test
battery matches the branch baseline.
Fifth review round; each item traced through the live scripts before fixing.
A provisioner fresh clone that failed to produce a server was left behind as
a markerless git tree; under a custom UNSLOTH_STUDIO_HOME the next run's
ownership assert refuses the unmarked dir and aborts the whole install until
the user deletes it by hand. _restore_prev now removes a clone this script
created when no server came out of it (backed-up dirs restore as before).
The CUDA provision gate ignored --with-llama-cpp-dir linked mode, so a
linked user tree with a CPU-only server could be checked out to a pinned
ref, rebuilt in place, or moved aside entirely and replaced by a fresh
clone. The gate now skips linked local dirs.
uninstall.sh removed the CUDA build artifacts without stopping a running
detached build; _pkill_studio only matches Studio roots, so live cmake/nvcc
kept burning thermals, recreated build files, and defeated the trailing
rmdir. The runner, provisioner, and llama.cpp-path processes now get
TERM-then-KILL with the same escape helper and grace the Studio kill uses.
The worker's memory-fraction guard classified Spark purely from device
props, so UNSLOTH_FORCE_DGX_SPARK=1 on an unlisted name got no fraction
guard (and the fraction env was dead), while FORCE=0 could not disable it;
the guard now honors the same force semantics as the detectors.
UNSLOTH_LLAMA_TAG / UNSLOTH_LLAMA_PR were interpolated into the runner
script's single-quoted exports unvalidated while every sibling forward has
an allow-list; they now get the INSTALL_REF ref allow-list and a digits-only
check respectively (own-machine robustness, not a trust boundary).
On WSL-fallback success with a custom UNSLOTH_STUDIO_HOME, the installer
deleted the rolled-aside custom-root venv right after telling the user that
root is not used by the WSL install; a custom root now restores the previous
venv instead (the WSL shim does not depend on the Windows venv), while the
default root keeps dropping the vestigial backup.
Verified: bash -n on all three shell scripts, AST parse on worker.py,
PowerShell AST parse on install.ps1, icon suites pass, sh battery matches
the branch baseline. Two resurfaced anchors (build/bin backup, --package
forwarding) confirmed already fixed at head.
Fourth review round; each item verified against the live scripts and the CUDA
compatibility documentation before fixing.
The provisioner installed (and, since the stale-toolkit change, preferred)
CUDA 13.3 without ever consulting the driver, but cu13 binaries need a 580+
driver and minor-version compatibility never crosses majors, so a GH200-class
host on a 5xx driver got an unloadable llama-server that the structural
acceptance check then stamped as ready. The driver's supported CUDA major is
now read from nvidia-smi and enforced three ways: the stale-toolkit upgrade
only fires when the driver can run cu13, a fresh install on a sub-13 driver
bails to the existing no-toolkit message instead of installing 13.3, and a
final guard swaps a too-new selected toolkit for the newest one the driver
supports (or refuses to build). Spark-class hosts (580+ drivers) behave
exactly as before; unparseable output keeps the previous behavior.
The WSL install pipeline ended in curl | sh, so a failed download fed sh an
empty stdin and exited 0; on a rerun the stale venv then passed the torch
probe and the installer reported success without ever running. install.sh is
now downloaded to a file with exit 86 as the never-ran sentinel, checked
before any probe (rollback + non-zero). The --package splice moved onto the
file invocation.
When the Studio web-server dep repair failed its re-verify, the installer
still created shims and reported success; the missing set includes typer, so
even the plain unsloth CLI dies. A failed repair now routes to the existing
failure path (rollback + non-zero), mirroring the CLI-missing case.
If all three provision-script resolutions fail (unpackaged wheel + GitHub
unreachable), the provision block silently skipped and, with the CPU build
now deferred on native Spark hosts, the install could report success with no
GGUF server; that case is now marked degraded so the CPU-prebuilt last
resort and failure exit fire.
flex_attention.py's Spark sniff gets the same /usr/lib/wsl/lib/nvidia-smi
fallback as the other two detectors (grep confirms these are the only three),
and uninstall.sh removes the remaining WSL-side build artifacts
(run_llama_build.sh, llama_cuda_build.log, the shortcut-skip marker) so the
.unsloth directory can actually be removed.
Verified: bash -n on all three shell scripts, AST parse on flex_attention.py,
PowerShell AST parse on install.ps1, the toolkit-picker awk exercised against
a fake /usr/local tree (driver 12 picks cuda-12.8 over 13.0, driver 11 picks
none), icon suites pass, sh test battery matches the branch baseline.
The Spark detectors (library _is_dgx_spark_no_cuda_init and the worker's
pre-CUDA sniff) called bare nvidia-smi, but the WoA shim execs the venv
binary directly with no login shell, where /usr/lib/wsl/lib can be off PATH;
both now resolve WSL's nvidia-smi path explicitly when the bare name is not
found, so the allocator setup works on plain 'unsloth ...' launches.
An explicit UNSLOTH_PYTHON pin was lost across the WSL boundary (Windows env
vars do not cross into the distro), so the inner install.sh built the venv on
its default Python while the installer reported success; the pin is now
forwarded, gated on a strict X.Y[.Z] shape before splicing into bash -lc.
The WSL2 probe/conversion only ran for pre-existing distros; a fresh install
relied on wsl --set-default-version 2 succeeding silently and could proceed
on WSL1 all the way to the final torch.cuda failure. The probe and in-place
conversion now run for freshly installed distros too.
The fourth resurfaced item (complete Studio dependency repair set) is already
fixed at head: the repair list includes sqlite-vec, pymupdf, and python-docx.
Third review round; each item re-verified against the live scripts.
A stale CUDA < 13 toolkit was kept forever: the 13.3 install was gated on
nvcc being absent, so a host with CUDA 12.x failed the sm_121 configure (or
the glibc >= 2.41 rsqrt clash) on every rerun and always exited with the CPU
server. When apt can provide 13.3 the provisioner now installs it alongside a
stale toolkit; find_nvcc's sort -V prefers the new install, and a failed
install leaves the old toolkit as the last resort, so non-Spark hosts that
build fine on cu12x are unaffected.
llama.cpp pins only applied to fresh clones; an existing checkout rebuilt
whatever commit it had while the log claimed a release pin. Existing
checkouts now fetch and check out the pinned (or resolved-latest) ref, best
effort with the current commit as fallback, and the UNSLOTH_LLAMA_PR handling
moved out of the fresh-clone branch so it applies to both paths.
The WSL fallback silently dropped a non-default --package and reported
success with stock unsloth; it is now spliced into the curl | sh invocation
(the name is regex-validated at parse time).
setup.sh's CUDA provision gate used raw nvidia-smi and ignored the
_setup_nvidia_usable computation that honors CUDA_VISIBLE_DEVICES=""/-1, so a
mixed-GPU host that hid its NVIDIA card still got a system CUDA install; the
gate now requires the flag. On native Linux Spark hosts without nvcc, setup.sh
also no longer does the multi-minute CPU source build that the CUDA provision
in the same run immediately replaces (mirroring the existing WSL deferral
arm); provision failure still cascades to the CPU-prebuilt last resort.
uninstall.ps1's distro extraction truncated quoted names at the first space
(-d "Ubuntu Preview" matched as "Ubuntu"), wrongly keeping or removing
shortcuts; the regex now matches a full quoted token first. And the profile
icon (%USERPROFILE%\.unsloth\unsloth.ico) was removed unconditionally while
the sweep above deliberately keeps launchers for non-evidenced WSL installs,
blanking their icons; removal is now gated on no surviving Unsloth shortcut,
mirroring uninstall.sh's _drop_shared_icon_if_unused guard.
Verified: bash -n on both shell scripts, PowerShell AST parse on both ps1
files, the new distro regex proven on spaced and unspaced names, icon suites
pass, sh test battery matches the branch baseline.
Second review round on the Windows-on-ARM + NVIDIA path; each item verified
against the live code (and torch where relevant) before fixing.
The native-CUDA probe ran uv --dry-run against the venv interpreter without
checking its architecture. uv resolves for the interpreter's platform tags, so
an x64-emulated python resolved existing win_amd64 CUDA wheels and "proved" a
native wheel WoA cannot use, skipping the WSL fallback entirely. The probe now
requires platform.machine() ARM64 from the venv python first; anything else
keeps the WSL routing.
The Studio worker appended PYTORCH_CUDA_ALLOC_CONF next to its memory-fraction
logic, 550 lines after detect_hardware() had already initialized CUDA, where
the allocator config is latched (verified on torch 2.9.1: expandable_segments
set after get_device_properties is a no-op in memory snapshots). The CUDA-free
Spark sniff now runs immediately before detect_hardware(), and it honors the
documented UNSLOTH_FORCE_DGX_SPARK=1/0 override the library detectors support,
closing the older force-flag item on the same block.
setup.sh's _have_cuda_llama_server accepted any co-located libggml-cuda.so,
re-opening the interrupted-relink hole the provisioner's completion stamp was
added to close: in exactly that state setup.sh skipped provisioning and
reported CUDA ready over the old CPU binary. The split-.so branch now also
requires the stamp; monolithic ldd-linked builds are unaffected.
The provisioner builds llama-quantize but never created the repo-root shim
that unsloth_zoo's check_llama_cpp needs (it only searches the root, which is
why setup.sh symlinks it in all three of its own paths). The success branch
now mirrors that symlink.
CMAKE_CUDA_ARCHITECTURES=native needs CMake >= 3.24, but this script installs
distro cmake (Ubuntu 22.04 apt ships 3.22), so the N/A-compute_cap fallback
aborted configure, wiped build/, and aborted again. The fallback now omits the
flag and lets ggml's version-guarded CMake defaults pick the arches.
Fresh clones tracked ggml-org master, bypassing setup.sh's newest-release pin
policy (its own header warns master bypasses the pin). An unset or "latest"
ref now resolves to the newest release tag via the GitHub API, keeping the
default-branch clone as the best-effort fallback when the API is unreachable.
install.sh writes the WSL shortcut icon to the Windows profile
(%USERPROFILE%\.unsloth\unsloth.ico) because the WoA icon broker cannot read
AppData\Local, but both uninstall.sh cleanup sites only cleaned the
AppData\Local icon. Both now clean the profile icon and drop the directory
when empty.
Verified: bash -n on all four shell scripts, AST parse on worker.py,
PowerShell AST parse on both ps1 files, the icon suites pass, and the sh test
battery matches the branch baseline (test_install_host_defaults.sh fails
identically on the clean tree).
Review round on the Windows-on-ARM + NVIDIA WSL2 path; each item reproduced
against the live scripts before fixing.
provision_llama_cuda.sh now serializes with install_llama_prebuilt.py on the
same <parent>/.<name>.install.lock file (its filelock backend is flock(2), so
shell flock interoperates; append-mode open so the Python O_EXCL fallback's
PID file is never truncated). The detached background builder could otherwise
race an installer rerun or `unsloth studio update`, both of which mv/rm -rf
inside the llama.cpp dir. Losing the 2h wait exits 0: another provisioner is
already doing the job.
The step-0 early-skip trusted a co-located libggml-cuda.so alone, which
wrongly skips one case: an in-place rebuild interrupted after the .so links
but before llama-server relinks leaves new .so + old CPU server. A completion
stamp (build/bin/.unsloth-cuda-ok) written only after the script's own final
CUDA check closes that window; skip now requires ldd evidence or the stamp.
The rejected functional --list-devices probe stays rejected: the stamp does
not gamble thermals on an env-fragile probe.
The WSL shortcut skip (install.ps1 owns the canonical WoA .lnk) was only a
transient env var, so the first `unsloth studio update`, whose wsl.exe shim
carries no env into install.sh --shortcuts-only, recreated the duplicate
blank-icon shortcut. The skip is now also persisted as
/root/.unsloth/.skip-wsl-windows-shortcut, checked by install.sh and removed
with the install by both uninstallers.
--with-llama-cpp-dir (and UNSLOTH_LOCAL_LLAMA_CPP_DIR) were parsed but
silently ignored on the WSL fallback path, which builds its own llama.cpp
inside the distro. Reject with guidance (UNSLOTH_LLAMA_TAG / UNSLOTH_LLAMA_PR
pin the WSL-side build), mirroring the --local reject.
uninstall.sh's Windows shortcut sweep only removed wsl.exe-target .lnks, so
the WoA fallback shortcuts (powershell.exe + launch-studio-wsl.ps1) survived
while their launcher dir was deleted, leaving dangling shortcuts. The
owner-matched cleanup now removes them first.
uninstall.ps1 swept every "Unsloth Studio (*.lnk" as legacy, but install.sh
creates exactly that per-distro name for current WSL installs, and the WSL
cleanup below only removes evidenced distros. The sweep now keeps a live
wsl.exe launcher whose distro is not in the same evidence set, so a surviving
WSL install keeps its shortcut; everything else is still swept.
Verified: bash -n on all three shell scripts, PowerShell AST parse on both
ps1 files, flock mutual-exclusion and stamp skip/rebuild decisions exercised
standalone, and the uninstall icon suites (sh + ps1) pass. The
test_install_host_defaults.sh failure pre-exists on the branch merge base.
On Windows-on-ARM + NVIDIA, an existing native Studio venv is rolled
aside (Start-StudioVenvRollback) before the WSL-routing block. The
TauriMode, --local, and invalid-UNSLOTH_INSTALL_REF rejects returned
without calling Restore-StudioVenvRollback, orphaning the user's
previous venv backup. Restore it on all three early exits, matching
the deferred-reboot / WSL1-conversion / final-failure paths that
already do. Restore-StudioVenvRollback no-ops when nothing was rolled
aside, so the fresh-install case is unaffected.
Addresses Codex review (venv-rollback ordering on the WoA reject paths).
Three valid findings from the 06-22 Codex review:
1. provision_llama_cuda.sh: when $LLAMA_DIR holds a .git checkout (a prior CPU
source build), the whole-dir backup was skipped, so a failed CUDA rebuild's
'rm -rf build' destroyed the working CPU server with nothing to restore --
leaving NO llama-server despite the 'keeps the existing server' promise (a
thermal shutdown mid-build is a real failure mode on this hardware). Back up
build/bin before the rebuild and restore it on total failure; idempotent and
self-cleaning (never overwrites a freshly built server). Verified both paths.
2. uninstall.ps1: 'fuser -k 8888/tcp' killed ANY listener on 8888 (Jupyter et al.
default to it), not just Studio. Now only kills a PID whose /proc/cmdline is
under /root/.unsloth -- matching the adjacent pkill scoping.
3. setup.sh: the 'defer to background CUDA build' branch fired even on a direct
in-WSL 'unsloth studio update', where install.ps1 never launched a background
builder -- so the footer claimed a build was running while nothing built. Gate
it on UNSLOTH_WSL_LLAMA_DEFERRED=1 (set only by install.ps1, and already read
elsewhere in setup.sh); a direct run now falls through to a real CPU build.
bash -n + PS parse clean; the common install.ps1 WoA path (prebuilt success,
deferred flag set) is unaffected.
Address three Codex P2s on the Windows-on-ARM + NVIDIA WSL2 fallback, all cases
where the branch silently ignored a Windows-side option while reporting success:
1. UNSLOTH_INSTALL_REF was spliced raw into the inner 'bash -lc' twice (an export
and a GitHub raw URL); a ref with shell metacharacters (;, &, ', space) would
break or inject the command. Validate against a strict git-ref allow-list
(^[A-Za-z0-9][A-Za-z0-9._/-]*$) and reject loudly. Real git refs always pass.
2. --local (editable install of the Windows checkout) can't be honored by the WSL
tunnel, which installs from PyPI/a git ref and never mounts $RepoRoot -- it would
silently install the published package. Reject it up front and point at the
supported pre-merge path (push the branch + UNSLOTH_INSTALL_REF).
3. A custom UNSLOTH_STUDIO_HOME / STUDIO_HOME only applies to the native Windows
layout; the WoA install lives in WSL at /root/.unsloth. Warn clearly so the user
isn't misled into thinking Studio landed at their custom path.
All three guard rare conditions; the default install path (no --local, default
root, normal branch/tag ref) is unaffected. install.ps1 parses clean; ref guard
verified against valid refs + metacharacter-injection cases.
The cuda_server_probe() added in 27bc44c gated the step-0 rebuild-skip on a
runtime 'llama-server --list-devices' check. In a real cold install on the N1X
this BACKFIRED: the background provision runs step-0 while the install is still
under heavy load (torch download, frontend build), and under WSL2 GPU-PV the
CUDA backend's init transiently fails under load (the same flakiness cycle-21
worked around for nvidia-smi). --list-devices then enumerated devices but no
CUDA, so the probe declared the freshly-validated PREBUILT 'broken', wiped it
(rm -rf build), and kicked off a CUDA-13.3 toolkit install + source build -- the
exact thermal-risk + wasted-prebuilt outcome cycle-21 eliminated. (Confirmed the
prebuilt is fine: --list-devices shows CUDA0 in a normal shell, even with
LD_LIBRARY_PATH stripped -- the probe failure was purely load-induced.)
Restore the load-insensitive structural check: a co-located libggml-cuda.so* is
trusted, because the prebuilt resolver validates what it installs and an
interrupted SOURCE build is already caught by the build-failure wipe+rebuild in
section 6. The Codex P2's half-linked-.so concern is real but narrow, and a
runtime probe that can gamble the machine's thermals on an env/load-fragile GPU
call is the wrong trade on this hardware.
The empty-dir sweep of ~/.unsloth ran before the WoA-fallback block removes
~/.unsloth\unsloth.ico, so on a Windows-on-ARM install the still-present icon
kept the dir non-empty at sweep time and it was skipped -- leaving an empty
~/.unsloth behind after a full uninstall. Re-attempt the empty-only removal
right after the icon is deleted (the last default-mode child). uninstall.sh is
unaffected: its rmdir runs as the final step.
is_cuda_server() treats a co-located libggml-cuda.so* as proof the server is
CUDA-ready. That's normally true (llama.cpp dlopens the backend from beside the
binary), but an *interrupted* build (thermal/power shutdown -- common on the
NVIDIA-ARM laptops this path targets) can leave a half-linked libggml-cuda.so
next to the server: present, so is_cuda_server() matches, yet the backend fails
to load at runtime. The post-build path already wipes+rebuilds such a partial
.so, but the step-0 early-skip trusted it and never rebuilt -- so Studio could
report GGUF CUDA inference ready while running a broken/non-CUDA backend.
Gate the early-skip with cuda_server_probe(): 'llama-server --list-devices'
enumerates backends and exits (cheap, no server spin-up). Only a definitive
'flag supported, ran, but no CUDA device' triggers a clean rebuild; a timeout or
an old pin without --list-devices stays inconclusive and keeps trusting the .so,
so we never force a needless, thermally-expensive rebuild. Probe logic verified
against healthy/broken/unsupported/timeout stubs (0/1/2/2).
Addresses Codex review P2 (provision_llama_cuda.sh).
The WoA+NVIDIA WSL2 fallback bridges UNSLOTH_NO_LLAMA_CUDA / UNSLOTH_PYTHON /
UNSLOTH_SKIP_WSL_WINDOWS_SHORTCUT into the distro, but not UNSLOTH_PYTORCH_MIRROR.
install.sh's get_torch_index_url() reads it (as does install.ps1's own native
Get-TorchIndexUrl), yet Windows env vars don't cross into WSL -- so a
mirror-required / restricted-network install silently fell back to
download.pytorch.org inside the distro even though the outer installer honored
the mirror.
Forward it alongside the other vars, guarded by a strict http(s)-URL allow-list
(no shell metacharacters) and single-quoted so the value can't break out of the
bash -lc string. Verified: legit mirror URLs (incl. host:port and query strings)
forward; space/';'/$()/quote-injection and non-http schemes are rejected.
Addresses Codex review P2 (install.ps1).
The foreground source build in setup.sh used -j(nproc), which on the
lightly-cooled NVIDIA-ARM boxes this WoA/WSL path targets (DGX Spark /
GB10, N1X RTX Spark laptops) draws enough sustained power during the
nvcc compile to trip a thermal shutdown -- the exact reason
provision_llama_cuda.sh already caps its background build.
Mirror that cap for the foreground build (only reached when no prebuilt
llama.cpp was available and a CUDA toolkit is present): gate on
aarch64/arm64 + GPU_BACKEND=cuda, then use ~half the cores, also bounded
by ~1.5 GB/nvcc job. Other platforms and CPU builds keep full -j(nproc).
Override anywhere with UNSLOTH_LLAMA_BUILD_JOBS=N.
Verified: nproc=20/29GB box -> -j10; override=6 -> -j6; CPU build and
x86_64 stay uncapped.
Cosmetic follow-up to a4ad50e (review nit): the retry loop slept 2s even
after the last attempt, adding ~2s only when nvidia-smi is permanently hung.
Sleep only between attempts.
On Windows-on-ARM + NVIDIA the Studio install runs inside WSL2, where
nvidia-smi is served over GPU-PV and can take far longer than its usual
sub-second response when the host is under heavy CPU load (the concurrent
pip / frontend / cmake work during install). detect_host probed nvidia-smi
with a single 20s timeout; under that load it raised TimeoutExpired, the GPU
was treated as ABSENT, and the host was misrouted to the ggml-org CPU prebuilt
-> rejected on an NVIDIA host -> slow (and on thermal-limited laptops, risky)
CUDA source build, even though a usable arm64 CUDA prebuilt was published.
Add _nvidia_smi_capture(): retry the three detect_host nvidia-smi probes with
a generous 60s per-attempt timeout. It is only reachable when nvidia-smi exists
on PATH, so CPU-only hosts incur no extra wait. Measured: nvidia-smi took
42-59s under a -j20 build on an N1X; with the fix the probe rides it out and
detect_host correctly reports has_usable_nvidia + compute_cap, so the CUDA
prebuilt is selected (no source build).
- find_nvcc now prefers the highest /usr/local/cuda-<ver> toolkit so a stale
unversioned `cuda` symlink or an older nvcc earlier on PATH can't win and
rebuild with CUDA 12.x (re-hitting the glibc>=2.41 / Blackwell clash this
script avoids); falls back to a PATH nvcc only when no versioned toolkit.
- Validate the GPU compute_cap is purely numeric before using it as
CMAKE_CUDA_ARCHITECTURES: some WSL GPU-PV / driver combos report "N/A",
which CMake rejects (aborting an otherwise-usable build) instead of letting
"native" autodetect.
- Gate the native-Linux aarch64 provisioner on _SKIP_GGUF_BUILD: when a non-root
user declines the sudo prompt (or lacks sudo) for GGUF deps, don't then run a
provisioner that does its own sudo apt-get installs.
_run_update imports routes.inference.get_llama_cpp_backend, which on a
fully-installed host pulls a real Studio singleton and blocks on its load
lock. Default the autouse fixture to a no-backend stub (the fail-open
path); the load-coordination tests still inject their own backend over it.
The WoA WSL path gated success solely on torch.cuda.is_available(), but
install.sh can exit after PyTorch yet before the `unsloth` package/console
script (e.g. a transient `uv pip install unsloth`). torch would still import,
so the installer wrote a Windows shim pointing at
/root/.unsloth/studio/unsloth_studio/bin/unsloth and reported success even
though that binary was absent -- `unsloth studio` then fails "no such file".
$wslRc can't distinguish this (it also goes non-zero on the optional llama
prebuilt step). Now `test -x` the exact shim target; if missing, fall through
to the existing failure path (restore rollback + non-zero exit) instead of
creating a dangling shim. Verified on N1X: present->exit 0 (success kept),
absent->exit 1 (fails). Addresses Codex review 4494521902.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On the WoA path install.ps1 already creates one canonical "Unsloth Studio.lnk"
with a %USERPROFILE%\.unsloth icon (renders on WoA). install.sh's
create_studio_shortcuts ALSO made a second "Unsloth Studio (WSL - <distro>).lnk"
whose icon lived under %LOCALAPPDATA% -- which the WoA shell icon broker can't
read, so it rendered BLANK. Net: two shortcuts, one blank ("blank for both").
- install.ps1: export UNSLOTH_SKIP_WSL_WINDOWS_SHORTCUT=1 into the WSL install so
install.sh skips its own Windows .lnk (install.ps1 owns the WoA shortcut).
- install.sh: honor that flag (skip the WSL .lnk branch); and move the WSL
shortcut icon from %LOCALAPPDATA%\Unsloth Studio to %USERPROFILE%\.unsloth so a
DIRECT native-WSL install (no install.ps1) also renders instead of going blank.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- install.ps1: force WSL2 (`wsl --set-default-version 2`) before installing a
NEW distro, so a host whose default is WSL1 doesn't get a GPU-less distro
that fails only at torch.cuda (the pre-existing-distro branch already
probes/converts).
- install.ps1: forward `UNSLOTH_PYTHON` into the WSL install (install.sh reads
it; a Windows env var isn't visible inside WSL otherwise). Numeric-only guard
rejects shell injection.
- install.ps1: add sqlite-vec / pymupdf / python-docx to the cut-short-install
server-deps repair so RAG/knowledge-base features aren't left broken.
- uninstall.sh: gate the Windows %LOCALAPPDATA%\Unsloth shim removal on the
current distro owning the fallback (wsl-distro.txt), so uninstalling Studio
from a different WSL distro no longer breaks the still-installed shim.
Disproved (no change): worker.py Spark name match is already whole-token
(commit 4cebfab, not substring); the shim's non-login WSL exec DOES have
/usr/lib/wsl/lib on PATH (nvidia-smi resolves -> Spark detector returns True),
verified on N1X.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Windows uninstaller removed only the exact name "Unsloth Studio.lnk",
orphaning legacy "Unsloth Studio (WSL - <distro>).lnk" shortcuts left by
pre-release dev builds. Glob "Unsloth Studio (*.lnk" across Desktop + Start
Menu so the documented "remove the shortcuts" contract holds regardless of
suffix. Validated on N1X: both canonical and suffixed .lnk removed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A loose substring match ("GB10" in name) misdetected a discrete Grace+Blackwell
datacenter GPU (e.g. nvidia-smi name containing "GB100") as a unified-memory DGX
Spark, applying the UMA tuning (pin_memory off, vLLM disabled, allocator capped to
0.80) and regressing that hardware. Match each device-name token with non-alphanumeric
boundaries instead. Found by a platform x device-name gating simulation; the real N1X
(JMJWOA-Generic-GPU) still detects, GB100/B100/GB200/GH200/B200 now correctly reject.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolved 2 conflicts:
- studio/setup.sh: main restructured the CUDA-toolkit branch (new
driver-vs-toolkit major-version compatibility check + an
_CUDA_TOOLKIT_ALLOWED guard that now owns -DGGML_CUDA=ON and the
CUDA_ARCHS detection). Took main's structure and re-injected our
glibc>=2.41/CUDA<13.3 rsqrt diagnostic inside the guarded block so it
runs against the final _NVCC_VER (after main's driver-compat swap).
- scripts/uninstall.sh: main expanded ~/.unsloth cleanup (.cache,
.staging, librocdxg, rocm-smoketest, rmdir) and rewrote the WSL
Windows-shortcut removal to per-distro, wsl.exe-target-filtered
matching. Took main's superset + kept our provision_llama_cuda.sh
removal and our %LOCALAPPDATA%\Unsloth shim+PATH cleanup (appended
after main's per-distro .lnk loop).
The in-distro cleanup probed a hardcoded candidate set ('', Ubuntu,
Ubuntu-24.04, Ubuntu-22.04, Debian) on every Windows uninstall, wiping
/root/.unsloth in any reachable distro even when the WoA fallback never
ran -- on an x86 AMD box this deletes a ROCm-on-WSL Studio the AMD flow
installed. Use the evidence the installer already records: clean only
the wsl-distro.txt marker distro or UNSLOTH_WSL_DISTRO; keep the broad
candidate probe solely for legacy marker-less installs, which can only
exist on ARM64 hosts.
Addresses the open Codex P1 on this path. Verified gating matrix:
x86+no-marker -> no cleanup; marker/env -> that distro only;
ARM64+no-marker -> legacy broad probe unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Real fixes:
- uninstall.ps1: scope the WSL process kill to argv referencing
/root/.unsloth/ (the fallback's install dir, which its Studio server,
llama-server, and build runner all reference) instead of the bare
'[l]lama-server' / '[u]nsloth_studio' name patterns -- uninstalling the
Windows shim must not kill a user's own unrelated llama.cpp server or a
/home Studio in a probed distro. Proven live: the path pattern matched
exactly the three fallback processes while a planted /tmp/llama-server
decoy matched the old pattern and not the new one. The backslash in
'/root/\.unslot[h]/' keeps the pattern from matching the cleanup
command's own argv.
- install.ps1: bridge UNSLOTH_LLAMA_TAG / UNSLOTH_LLAMA_PR into the
background CUDA-build runner -- the provisioner honors both pins, but
Windows env vars don't cross into WSL on their own, so a user's pin was
silently ignored by the deferred build. (Deliberately NOT forwarded into
the inner install.sh env: setup.sh skips its deferral when a PR pin is
visible there, which would CPU-build the pin in the foreground.)
- kernels/flex_attention.py: make _flex_is_dgx_spark() CUDA-free
(nvidia-smi device names, mirroring _is_dgx_spark_no_cuda_init) -- it
runs at module import and called torch.cuda.get_device_name(), which
initializes the CUDA allocator before patch_dgx_spark_memory_config()
can set PYTORCH_CUDA_ALLOC_CONF on exactly the Spark hosts it targets
(reachable via vision.py importing ..kernels before ._utils). Verified
on the N1X: detects the machine with torch.cuda.is_initialized() still
False.
- _utils.py: the TrainingArguments __post_init__ wrapper now forwards
*args/**kwargs (robustness against future InitVar signatures).
Disproven on hardware (no change): the five "high" PS-5.1 claims --
String.TrimEnd('\', '/') with multiple char args binds fine to
params char[] (verified on PS 5.1.28000.1737, and the uninstaller's PATH
cleanup using exactly that code ran successfully this same day), and
[Text.Encoding] resolves via the System namespace prefix (the background
build dispatch using it has run in every install this week). The worker
"_sp possibly undefined" claim is false: `import subprocess as _sp` is at
worker.py line 25.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- install.sh: gate the new aarch64 bitsandbytes block on SKIP_TORCH=false --
with --no-torch/UNSLOTH_NO_TORCH (GGUF-only install) it would have pulled
torch back into the venv through bitsandbytes' dependencies.
- studio worker: in the new Spark OOM-guard section, decide
PYTORCH_CUDA_ALLOC_CONF (expandable_segments) BEFORE the guard's first CUDA
touch -- get_device_properties initializes the CUDA allocator, after which
the env var is ignored, and the later `import unsloth`
(patch_dgx_spark_memory_config) is too late for the worker process. Uses
the same CUDA-free nvidia-smi name sniff, append-don't-override, and
UNSLOTH_NO_EXPANDABLE_SEGMENTS opt-out as the library patch. Live-verified
on the N1X: env set while torch.cuda.is_initialized() is still False.
- uninstall.ps1: only run `fuser -k 8888/tcp` in a probed WSL distro when an
Unsloth install actually exists there (checked BEFORE the rm deletes the
marker) -- an unrelated listener on 8888 (e.g. Jupyter) in a clean distro
must survive a Windows-side uninstall. The Unsloth-specific pkills stay
unconditional.
- install.ps1 + uninstall.ps1: persist the chosen WSL distro to
%LOCALAPPDATA%\Unsloth\wsl-distro.txt at install; uninstall reads it
(before removing the directory) and prepends it to the cleanup candidates,
so a custom UNSLOTH_WSL_DISTRO install is cleaned without the env var
being set again at uninstall time.
- provision_llama_cuda.sh: honor UNSLOTH_LLAMA_PR (numeric-validated,
best-effort fetch of pull/N/head after clone) so a provisioned tree
matches a PR pin the way setup.sh does; and require only llama-server in
the main cmake build (mirroring setup.sh), building the helper targets
(llama-cli/quantize/mtmd-cli/gguf-split) best-effort afterwards -- an
older UNSLOTH_LLAMA_TAG pin lacking a newer helper target no longer fails
the whole provision.
Not changed: the "--tauri rejection doesn't restore the venv rollback"
comment is incorrect -- the rejection returns through Exit-InstallFailure,
which itself calls Restore-StudioVenvRollback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The base unsloth package does not depend on bitsandbytes and the cuXXX
extras that normally add it are x86_64-oriented, so the Spark-class install
path (DGX Spark / GB10 / N1X, native or WSL) produced a venv where
FastLanguageModel.from_pretrained(..., load_in_4bit=True) fails with
ModuleNotFoundError -- found while benchmarking the UMA training knobs on
the N1X. bitsandbytes ships working aarch64 manylinux wheels (0.49.2
verified on sm_121 Blackwell: 4-bit Linear4bit forward runs on GPU via PTX
JIT), so install.sh now adds it best-effort on Linux aarch64 + NVIDIA after
the unsloth install, using the same version constraint as pyproject
(>=0.45.5,!=0.46.0,!=0.48.0). Platforms without a wheel just keep 16-bit
LoRA / full finetuning, with a substep saying so.
Live-tested on the N1X: with bitsandbytes removed from the venv, the block
reinstalls and imports it; benchmark suite then ran 4-bit QLoRA training in
7 configs without error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PR #5301 protects ROCm unified-memory APUs (Strix Halo gfx1150/gfx1151) with
a default set_per_process_memory_fraction(0.80) at training-worker startup,
because exhausting a shared GPU+OS memory pool can stall the whole box
instead of raising a catchable OutOfMemoryError. NVIDIA Spark-class parts
(DGX Spark / GB10, N1X "RTX Spark") have the same pool topology and the same
failure mode, but only had an opt-in cap (UNSLOTH_SPARK_MEM_FRACTION).
- worker.py: new _nvidia_classify_spark_unified_memory(props) mirroring
_rocm_classify_unified_memory: is_integrated property first (authoritative
on native Linux), then Spark device-name tokens -- WSL2's GPU
paravirtualization masks is_integrated to 0 and renames the device (the
N1X reports "JMJWOA-Generic-GPU"; verified on hardware), so the property
alone misses Spark-under-WSL. Section 1h applies the 0.80 cap on match;
UNSLOTH_SPARK_MEM_FRACTION overrides it and any value outside (0, 1]
disables the guard. Discrete NVIDIA GPUs and CPU-only hosts are untouched.
The existing generic OOM handler in the training loop surfaces the
resulting OutOfMemoryError.
- _utils.py: range-validate the opt-in UNSLOTH_SPARK_MEM_FRACTION -- "0"
previously called set_per_process_memory_fraction(0.0), which makes every
subsequent CUDA allocation OOM.
- tests: test_spark_oom_guard.py mirroring test_rocm_oom_guard.py (property
path, WSL name-token path, discrete negatives). 47/47 pass alongside the
ROCm suite.
Live-verified on the N1X (WSL2): classifier matches via JMJWOA, and with the
cap set an over-allocation raises catchable torch.OutOfMemoryError instead
of stalling the box; allocations recover after the error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two Codex round-3 review fixes (both reproduced empirically before fixing):
- Exit-InstallFailure (and the WoA deferred-WSL / WSL-install-failed return
paths) now `exit $Code` when the script runs from a file (powershell -File
or .\install.ps1): a plain return exits the process with 0 regardless of
$global:LASTEXITCODE, so automation treated fatal failures -- including the
"enable WSL + reboot" deferred state -- as completed installs. Under
`irm | iex` $PSCommandPath is empty and `exit` would kill the user's shell,
so that context keeps the return + $LASTEXITCODE behavior. Verified: the
old pattern exits 0 under -File, the new one exits 1, and an iex run
survives with the session intact. Tauri behavior is unchanged (already
exited). All Exit-InstallFailure call sites are body-level in
Install-UnslothStudio followed by nothing but the trailing invocation, so
control flow is unchanged -- only the process exit code.
- The detached background CUDA-build launcher now passes $_distroArg instead
of the raw distro name: PS 5.1's Start-Process joins -ArgumentList with
spaces WITHOUT quoting (verified: 'Ubuntu Preview' arrives as two args), so
a spaced UNSLOTH_WSL_DISTRO never started the background builder while the
install reported it running. $_distroArg is pre-quoted only when the name
contains spaces, since wsl.exe rejects a quoted space-free name.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
wsl.exe parses its raw command line itself: invoked from the generated
unsloth.cmd shim, `wsl.exe -d "Ubuntu-24.04"` fails with
WSL_E_DISTRO_NOT_FOUND -- the quotes are treated as part of the name
(reproduced live on WSL 2.x). The blanket quoting added in 8e51d18 for
spaced UNSLOTH_WSL_DISTRO values therefore broke the shim for every
standard distro name.
Quote the name only when it actually contains whitespace: bare names keep
the proven working form, and spaced names get quoting (bare would split
after -d, so quoting is their only viable form). Applied to the shim and
the copy-paste hint commands via a single $_distroArg.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- install.ps1: detect a PRE-EXISTING WSL1 distro up-front (kernel string +
libcuda probe inside the distro; encoding-proof vs UTF-16 `wsl -l -v`) and
convert it with `wsl --set-version 2`, failing early with instructions if
conversion does not take -- instead of completing a full install that only
fails at the final torch.cuda check (no GPU passthrough under WSL1).
- install.ps1: quote the distro name in the generated unsloth.cmd shim and in
the copy-pasteable hint commands so UNSLOTH_WSL_DISTRO values with spaces
("Ubuntu Preview") keep working.
- install.ps1: forward UNSLOTH_NO_LLAMA_CUDA=1 into the WSL install env; the
inner setup.sh otherwise defers its llama.cpp build to a background builder
this script then never dispatches (the same opt-out skips it), leaving no
llama-server and a misleading "building in background" footer. Also add
libcurl4-openssl-dev to the WSL bootstrap apt line.
- provision_llama_cuda.sh: install libcurl4-openssl-dev with the base tools --
_cmake_configure forces -DLLAMA_CURL=ON and on the deferred WSL path this
script is the only build path (setup.sh's GGUF dep install was skipped), so
configure failed on fresh hosts without the headers.
- provision_llama_cuda.sh: keep the pre-existing llama.cpp backup until the
fresh build is CONFIRMED (was: dropped right after a successful clone), and
restore it on configure/build failure or when no server binary was produced
-- a failed CUDA build no longer destroys a previously working (CPU) server.
- setup.sh: when provisioning fails and NO llama-server is present, set
_LLAMA_CPP_DEGRADED=true so the arm64 CPU-prebuilt last resort and the
installer failure exit fire instead of reporting a working install.
Round-2 comments verified already fixed in ad77ae6 (anchored to its parent
d161ff5): the torch probe already passes --reinstall; the WSL uninstall is
already scoped to /root only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
install.ps1 (WoA WSL fallback):
- report failure (non-zero) + restore the rolled-aside venv when the WSL GPU
install fails (torch.cuda absent) or when WSL needs enabling+reboot, instead of
returning success — so -File/Tauri callers don't see a broken install as complete
- on WSL success, Complete-StudioVenvRollback so the previous-venv backup isn't orphaned
- refuse under --tauri with a clear "use the CLI installer" message (the desktop
launcher resolves a Windows-venv backend, which a WSL-only install can't provide)
- reset $LASTEXITCODE before each wsl.exe / python probe (a stale 0 could mark WSL
ready / torch OK if the native command fails to launch)
- torch-availability probe: --reinstall so an already-installed CPU torch in a
migrated venv isn't accepted as "satisfied" (would wrongly skip the WSL path)
- treat a null HKCU PATH as empty (fresh profile) so shim PATH update can't throw
- keep apt stderr visible inside WSL (only stdout -> /dev/null) for diagnosability
scripts/uninstall.ps1:
- scope WSL cleanup to /root (the fallback's install location); stop deleting
/home/*/.unsloth, which could erase an unrelated WSL user's own Unsloth/cache
studio/setup.sh:
- direct (non-install.ps1) WSL installs now provision CUDA llama.cpp themselves
instead of being left with no GGUF server: install.ps1 exports
UNSLOTH_WSL_LLAMA_DEFERRED=1, and the aarch64+NVIDIA provision block runs under
WSL only when that marker is absent
- mark a provisioner-built llama.cpp as Studio-owned in custom-STUDIO_HOME mode so
the next setup's _assert_studio_owned_or_absent doesn't abort
- glibc>=2.41 check: also match a future major>2 (e.g. 3.0)
studio/scripts/provision_llama_cuda.sh:
- install base tools (cmake/git/curl) in their own apt transaction before the
best-effort gcc-14/g++-14 (unavailable on Ubuntu 22.04 / Debian 12, where bundling
them aborted the whole transaction and left no build tools)
- back up an existing (e.g. CPU-only) llama.cpp before the destructive clone and
restore it on clone failure, so a failed clone doesn't leave the user with no server
- honor a pinned llama.cpp ref via UNSLOTH_LLAMA_TAG instead of always tracking main
unsloth/models/_utils.py:
- set PYTORCH_CUDA_ALLOC_CONF (expandable_segments) via a CUDA-free Spark detector
(nvidia-smi, not torch.cuda.get_device_name) so it takes effect before CUDA/the
caching allocator initialize — previously it was a silent no-op on auto-detected Spark
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause (diagnosed live on an N1X WoA box, confirmed by on-screen checks):
the Windows shell's sandboxed icon-extraction broker cannot read a standalone
.ico stored under %LOCALAPPDATA% (it gets a redirected/virtualized view), so
the Desktop + Start Menu shortcuts render BLANK -- regardless of icon format
(BMP vs PNG frames), ACLs, icon cache, or shortcut-creation method, all of
which were ruled out. The IDENTICAL .ico renders correctly from a path under
the user profile. Fix: write unsloth.ico to %USERPROFILE%\.unsloth instead of
%LOCALAPPDATA%\Unsloth (shim/launcher stay in %LOCALAPPDATA%). uninstall.ps1
removes the icon at the new location. Also drops the speculative SHGetFileInfo
"image-list prime" block added while chasing the wrong (format/cache) theory.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diagnosis this cycle: the .ico is well-formed (6 BMP frames 16-256px) and the
shell resolves the logo at every size (verified via SHGetFileInfo +
SHGetImageList/ImageList_GetIcon on the system image list, all sizes incl. the
256px jumbo slot the desktop draws). The residual blank is a first-paint race:
Explorer lazily extracts a .lnk's icon and a miss (icon not yet flushed, cache
just cleared) gets cached blank. Force the extraction at install time via
SHGetFileInfo(SHGFI_SYSICONINDEX) per .lnk, populating the per-session image
list both Desktop and Start Menu draw from. WoA path only, try/catch-wrapped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A same-name "Unsloth Studio.lnk" recreated across reinstalls keeps Explorer's
stale (blank) iconcache_*.db entry; -show rebuilds but does not purge, so add
-ClearIconCache first (matches PR #5940). The per-.lnk SHChangeNotify remains
the primary fix. WoA-path only -- no effect on other installs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A non-numeric or 0 override silently fell through to `cmake -j0`, which
builds with ALL cores -- the opposite of the thermal-headroom default and a
shutdown risk on NVIDIA-ARM laptops. Validate it's a positive integer; ignore
anything else and auto-compute.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`curl https://unsloth.ai/install.sh | sh` runs install.sh from a pipe, so the
script *is* the shell's stdin. A Windows process launched via WSL interop
(powershell.exe / cmd.exe) inherits that stdin and drains the remaining
piped script, truncating it -- dash then aborts parsing the tail with
"Syntax error: Unterminated quoted string". This surfaced as a non-fatal
"sh: <line>: Unterminated quoted string" near the end of every non-tty
install (e.g. the WoA install.ps1 -> curl|sh flow). Add `</dev/null` to the
three Windows-exe invocations (create_studio_shortcuts' powershell .lnk
writer + the two browser-open helpers) so they cannot consume the script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An interrupted CUDA build (e.g. a thermal/power shutdown mid-compile -- which
this machine class hits) can leave a partially-linked libggml-cuda.so. On the
next run cmake does not relink it, so linking llama-server fails with undefined
ggml_cuda_op_* references and the script gives up with no server. Mirror the
existing configure retry-clean: if `cmake --build` fails, wipe build/,
reconfigure, and rebuild clean once before giving up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A full -j(nproc) CUDA build trips power/thermal shutdowns on thermally
constrained NVIDIA-ARM laptops (observed on the N1X "RTX Spark": a full-core
build, especially alongside other load, shuts the machine down). nice lowers
CPU *scheduling* priority but not heat -- power/heat scale with the number of
active compile jobs -- so default to ~half the cores instead: still ~2.5x
faster than a tiny -j4, but leaves real headroom. Still mem-capped (~1.5 GB
per nvcc job) and overridable via UNSLOTH_LLAMA_BUILD_JOBS (raise on a
well-cooled box, lower if it still trips). Tiny boxes (<=4 cores) use all.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Shorten the multi-line rationale comments added by this PR across the
remaining changed files to 1-2 lines each, preserving intent (gotchas,
workarounds, why-notes). Comment-only changes; no code, strings, or
behavior altered. Verified: PowerShell AST parser, bash -n, and python
ast.parse all pass; diffs confirmed comment-only.
Files: install.ps1, scripts/uninstall.ps1, scripts/uninstall.sh,
studio/setup.sh, unsloth/models/_utils.py, unsloth/kernels/flex_attention.py
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Building at -j(nproc) saturates every core (load ~25 on a 20-core box),
which starved a concurrently launched `unsloth studio` / training session
during the build's few-minute window. Wrap the cmake build in
`nice -n 19` (+ `ionice -c 3` when available): full speed when the box is
idle, but instant yield to foreground work. Also trims this file's comments.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
unsloth_zoo is an optional extra (not a base dependency), and install.sh
always runs the studio deps step with SKIP_STUDIO_BASE=1 (which skips the
base.txt install that would otherwise add it). Every other install path
names unsloth-zoo explicitly; the pre-merge UNSLOTH_INSTALL_REF git path
did not, so a branch build left unsloth_zoo missing and `import unsloth`
failed with "Please install unsloth_zoo". Name it explicitly here too.
The default PyPI path is unaffected (released unsloth carries zoo as a
base dep). Also trims the verbose comment on this block.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When install.ps1 is launched from inside a cloned unsloth repo, the WSL
subprocess inherited the Windows cwd (/mnt/c/.../unsloth). Python then
prepended that dir to sys.path and `import unsloth` resolved the LOCAL
CLONE instead of the installed package, so `unsloth studio update`'s
_find_setup_script() returned the clone's studio/setup.sh -- which has
CRLF line endings on a Windows checkout. bash aborted on line 4
($'\r': command not found / set: pipefail: invalid option name), the
deps + frontend step never ran, and Studio was left unusable (missing
packaging/structlog/fastapi).
Fix: pass `--cd /root` to the WoA-branch wsl.exe invocations (install,
self-heal repair, torch/server verifications, the desktop launcher, and
the background CUDA build) so the WSL side never starts in /mnt/c and
always imports the installed package -> resolves the LF setup.sh in
site-packages. The native `unsloth` shim is intentionally left without
--cd so relative model-path args keep resolving against the user's cwd
(the console-script entry point does not cwd-shadow at runtime).
Also: use SHCNF_IDLIST (0) for the global SHCNE_ASSOCCHANGED notify
(items are unused for that event) instead of SHCNF_PATHW.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The WSL distro-probe loop tries a candidate list that intentionally
includes distros that may not exist; the last failed `wsl -d <name> -- true`
probe left $LASTEXITCODE=255, so `& .\uninstall.ps1` returned non-zero even
when every cleanup step succeeded. Reset $global:LASTEXITCODE=0 at the end
(not `exit 0`, so the `irm ... | iex` usage does not kill the caller shell).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The WSL-fallback Desktop/Start-Menu shortcuts could render blank because
the block only downloaded the .ico from GitHub (no fallback, no validation)
and only fired a single global shell notify. Now it:
- prefers the icon bundled in the local clone (instant, reliable) and
only falls back to a GitHub download when no bundle is present;
- validates the ICO header (00 00 01 00) before attaching, so a partial/
empty/404 download can never leave a non-icon attached;
- sets IconLocation as "<path>,0" (explicit index);
- issues a per-.lnk SHCNE_UPDATEITEM (SHCNF_PATHW) notify in addition to
the global SHCNE_ASSOCCHANGED, forcing Explorer to re-read each new
shortcut icon immediately and clear any stale blank cache entry.
Mirrors the icon handling already used by the native New-StudioShortcuts
path, plus the ie4uinit refresh approach from PR #5940.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The CUDA llama.cpp compile is the slow step of the WSL GPU setup. The job
count now defaults to the full core count (nproc) instead of being capped,
which is ~5x faster on a 20-core box (-j4 -> -j20). To stay safe on
unified-memory machines, where nvcc jobs (~1.5 GB each) could OOM-kill a
full-parallel build, jobs are capped at mem/1.5GB when that is lower than
nproc. UNSLOTH_LLAMA_BUILD_JOBS=N still overrides for thermal throttling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous Start-Process passed `@(... 'bash','-lc',$buildCmd)` where $buildCmd
contained spaces (`env VAR=N bash provision.sh > log`). Start-Process's ArgumentList
array mis-quotes a space-containing element, so wsl ran just `env` -- which dumped
the environment to the log and exited; no build, only a CPU server left behind.
Fix: build a tiny runner script here, ship it as base64 (dodges every quoting
layer), and Start-Process invokes `bash /root/.unsloth/run_llama_build.sh` with
ONLY space-free args. The runner also restores PATH (/usr/lib/wsl/lib for
nvidia-smi, /usr/bin for apt) so the non-login detached shell doesn't make
provision early-exit "no nvidia-smi", then caps jobs and runs provision with
logging. Verified on a cold distro: the detached build survives install.ps1's
exit and provision runs correctly (toolkit install + CUDA build), log shows real
provision output.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous attempt passed install.sh `--package git+https://...@ref`, but
install.sh validates --package and rejects URL characters ("invalid characters")
-> the WSL install aborted (exit 127). Fix it properly and symmetrically:
- install.sh: add a gated UNSLOTH_INSTALL_REF path that installs
`unsloth @ git+https://github.com/unslothai/unsloth@<ref>` via uv. Gated to the
default package ("unsloth") and a non-"main" ref, so released-PyPI behavior is
unchanged by default. Bypasses the --package name validation (fixed literal URL,
no injection surface).
- install.ps1: when UNSLOTH_INSTALL_REF is a branch, fetch THAT ref's install.sh
(which honors the env) and export UNSLOTH_INSTALL_REF, so the WSL studio venv
carries the branch's studio/setup.sh + unsloth Python (e.g. the WSL CPU-build
skip is actually exercised pre-merge). Default (ref = main) is byte-identical:
`curl https://unsloth.ai/install.sh | sh`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
So a branch can be tested end-to-end pre-merge: when UNSLOTH_INSTALL_REF is set
(not main), pass install.sh `--package git+https://github.com/unslothai/unsloth@<ref>`
so the WSL studio venv carries THAT ref's studio/setup.sh + unsloth Python patches
(otherwise install.sh installs released PyPI unsloth and the branch's setup.sh --
e.g. the WSL CPU-build skip -- never runs). Default (ref = main) is byte-identical
to before. The git URL has no spaces, so it survives PowerShell -> wsl.exe -> bash -lc.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three coupled fixes so the Windows-on-ARM + NVIDIA (DGX Spark / N1X) WSL path
builds the GPU llama-server reliably and never wastes time on a CPU build:
1. install.ps1 -- the post-install CUDA provision was launched as a WSL-side
`nohup setsid ... &`. That does NOT survive: WSL shuts the distro's VM down
once the launching wsl.exe session exits, killing the detached build (observed
on a fresh distro: no build log, only a CPU server left behind). Fetch the
provision script in a quick session, then run the build anchored to a DETACHED
Windows-side process (Start-Process wsl.exe, no -Wait) that holds the VM up for
the whole build while install.ps1 returns immediately.
2. provision_llama_cuda.sh -- a pre-existing build/ can carry an incompatible
CMake cache (the Studio installer stages its build in llama.cpp.build.NNNN then
relocates it, leaving a cache with stale absolute source/build paths and
GGML_CUDA=OFF), so reconfiguring for CUDA fails ("CMakeCache directory is
different" / "source does not match"). Try to reuse build/ first (incremental
resume), and if configure fails, wipe build/ and configure clean once. Verified
live on the failing scenario: stale cache detected, wiped, clean CUDA configure.
(setup.sh's skip of the CPU source build on this path is the companion commit;
together the fresh-install path builds only the CUDA server, in the background.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On Windows-on-ARM + NVIDIA (DGX Spark / N1X "RTX Spark"), install.ps1 routes the
install through WSL2 and, after setup finishes, launches provision_llama_cuda.sh
in the BACKGROUND to install the CUDA toolkit + gcc-14 and build the real sm_121
CUDA llama-server, replacing whatever section 9 produced.
On a fresh WSL distro there is no nvcc yet, so section 9 could only ever build a
CPU-only server ("building (CPU, CUDA driver found but nvcc missing)") that the
background CUDA build immediately throws away -- slow and wasteful.
Skip the section-9 source build entirely on this exact path. Introduce a distinct
_LLAMA_CPP_DEFERRED state (NOT _LLAMA_CPP_DEGRADED) so:
- the footer reports "GGUF engine: CUDA build running in background" (success),
not "limited: llama.cpp unavailable";
- the arm64 CPU-prebuilt last-resort does NOT fire (it gates on DEGRADED=true);
- the install-failure exit 1 does NOT fire (it gates on DEGRADED=true).
Strictly gated -- defers only when ALL hold: WSL (grep microsoft /proc/version),
aarch64/arm64, an NVIDIA GPU is listed by nvidia-smi, nvcc is missing (PATH and
/usr/local/cuda*/bin), UNSLOTH_NO_LLAMA_CUDA != 1, no forced compile, no pinned
PR. Every other host (x86_64, native-Linux aarch64, nvcc-present, opt-out,
ROCm, macOS, non-NVIDIA) is byte-for-byte unaffected and still builds via
section 9 as before. install.ps1 is unchanged; it still builds CUDA in the
background, but now with no wasted CPU build first.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The WSL cleanup ran `pkill -9 -f unsloth_studio` then `pkill -9 -f llama-server`,
but the `bash -lc <cmd>` shell's own argv contains those literal patterns, so the
first pkill SIGKILLed the shell before the llama-server pkill (and trailing `true`)
ever ran -- leaving a running llama-server (dynamic port, not covered by
`fuser -k 8888`) alive after uninstall. Use the [x]-regex self-exclusion trick
('[u]nsloth_studio' / '[l]lama-server') so the shell's argv no longer contains the
matched substring; real target processes still match. Verified in WSL: shell
survives, both dummy processes are killed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Current llama.cpp ships the CUDA backend as a dynamically-loaded plugin
(libggml-cuda.so* next to the binary), NOT a load-time dependency, so
ldd llama-server | grep libggml-cuda is a false negative: it reports no
CUDA on a perfectly good CUDA build. That made both is_cuda_server()
(provision_llama_cuda.sh) and _have_cuda_llama_server() (setup.sh) force a
needless full rebuild every run.
Fix both: keep the ldd check (old monolithic builds) and additionally treat
the presence of libggml-cuda.so* beside the binary as the CUDA signal. A
CPU-only build has no such backend, so this stays correct for the CPU case.
Verified on an N1X/sm_121 WSL build: llama-server --list-devices shows
CUDA0 JMJWOA-Generic-GPU and serves on the GPU, while ldd lists no
libggml-cuda; the new check correctly returns CUDA-present.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
install.ps1: fetch repo-versioned WSL-fallback assets (provision_llama_cuda.sh,
unsloth.ico) from a configurable git ref via new UNSLOTH_INSTALL_REF env var
(defaults to main, so existing users are byte-for-byte unaffected). Lets the
ARM64+NVIDIA WSL-fallback GPU path be exercised end-to-end on a branch before it
merges (provision_llama_cuda.sh does not exist on main until then).
uninstall.ps1: the WSL cleanup rm -rf'd /root/.unsloth but left the
~/.local/bin/unsloth launcher symlink dangling, so `unsloth` still resolved on
PATH after an uninstall. Also remove /root/.local/bin/unsloth and
/home/*/.local/bin/unsloth.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When the installer is launched from a Windows shell, WSL interop leaks the Windows
PATH (/mnt/c/... entries, with spaces) into the build environment, which can make
cmake/gcc/git resolve to Windows tools or otherwise confuse the CUDA build. Prepend
the CUDA toolkit + standard Linux dirs so the Linux toolchain always wins; keep the
original PATH after so nvidia-smi etc. still resolve.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The server-deps self-heal passed specs like "structlog>=24.1.0" with embedded
double-quotes through PowerShell -> wsl.exe -> bash -lc; PowerShell's native-arg
quoting drops the quotes, so bash parsed >= as a redirection and the whole install
failed ('could not auto-install Studio server deps'). Use bare package names (uv
resolves latest, satisfying the studio.txt minimums) -> no embedded quotes, no
redirection. Verified: bare-name uv install populates fastapi/uvicorn/structlog/...
and Studio starts (HTTP 200).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RuntimeInformation::OSArchitecture (and $env:PROCESSOR_ARCHITECTURE) report
X64/AMD64 when install.ps1 runs under an x64-emulated PowerShell on a Windows-on-ARM
host, which mis-skips the WSL fallback and then fails the native win_arm64 torch
install. Add additive fallbacks (Win32_Processor.Architecture=12 ; machine-level
PROCESSOR_ARCHITECTURE) that read the true OS arch even under emulation. Only turns
the ARM64 path ON for genuine ARM64 hosts; x86_64/native detection unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The native-CUDA-torch viability probe used a bare 'uv pip install --dry-run torch',
but the real native install pins 'torch>=2.4,<2.11.0'. The cu130 index can carry an
out-of-range torch (e.g. <2.4 or a >2.11 nightly) with a win_arm64 wheel, so the
bare probe PASSED while the pinned install FAILED -> the WSL fallback was skipped and
the install died at 'Failed to install PyTorch' on Windows-on-ARM. Use the same
pinned spec in the probe so its result exactly predicts the native install.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Windows env vars don't cross into WSL by default, so the background
provision_llama_cuda.sh always built at -j(nproc). Forward
UNSLOTH_LLAMA_BUILD_JOBS via 'env' so thermally/power-limited laptops can cap the
build's parallelism (harmless no-op passthrough when unset).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The WSL-distro cleanup parsed 'wsl --list --quiet', whose UTF-16 output PowerShell
often mis-parses into an EMPTY list, so the WSL install (/root/.unsloth + CUDA
llama build) was silently never removed. Probe a candidate set ('' = default
distro, Ubuntu, Ubuntu-24.04, ...) by 'wsl -d <d> -- true' exit code instead
(encoding-proof; same idiom install.ps1 uses), then run the idempotent cleanup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- uninstall.ps1: the WSL-distro cleanup ran 'pkill -f "unsloth studio"' before the
rm inside a single bash -lc, but that pattern matches the bash -lc's own argv ->
pkill SIGKILLs the shell before rm runs, so /root/.unsloth survived. Reorder: rm
FIRST (guaranteed), then non-self-matching fuser -k 8888/tcp + pkill best-effort;
also remove the fetched provision script + build log.
- uninstall.sh: also remove ~/.unsloth/llama.cpp (CUDA build from provision on
native-Linux Spark) + the fetched provision_llama_cuda.sh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
setup.sh runs during install.sh, so on WSL the new aarch64+NVIDIA provision block
would foreground-build CUDA llama.cpp during the install -- blocking it and
duplicating install.ps1's WSL background provision. Exclude WSL (grep microsoft
/proc/version, same idiom setup.sh already uses) so this block is native-Linux
(DGX Spark/GB10) only; WSL stays handled by install.ps1's background path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Native-Linux (non-WSL) aarch64+NVIDIA hosts (DGX Spark / GB10 / N1X "RTX
Spark") had a GGUF *inference* gap the Windows WSL2 fallback already closes:
setup.sh's source build only emits a CUDA llama-server when a CUDA toolkit
(nvcc) is already present. A fresh Spark ships only the driver + nvidia-smi,
so the build silently dropped to a CPU-only llama-server and Studio GGUF
inference ran without GPU.
Mirror the Windows path in the shared Linux installer (studio/setup.sh) so
ALL native Linux installs benefit, not just the Windows-specific file:
* setup.sh: after the source build, on Linux aarch64/arm64 WITH an NVIDIA GPU
AND when no CUDA-linked llama-server exists yet, invoke the existing
provision_llama_cuda.sh (installs CUDA 13.3 + gcc-14, builds a CUDA server
into the same $LLAMA_CPP_DIR setup.sh validates). Best-effort, never aborts
setup; opt out with UNSLOTH_NO_LLAMA_CUDA=1; build load via
UNSLOTH_LLAMA_BUILD_JOBS. Resolves the script from the packaged copy, the
local-dev repo, or the pinned GitHub raw URL (matches install.ps1).
* pyproject.toml: ship studio/scripts/*.sh in the wheel (package-data) so the
normal `curl | sh` install has provision_llama_cuda.sh locally.
Strictly gated + additive: x86_64 NVIDIA, ROCm/AMD, Intel, macOS/MLX,
Windows-native, WSL, CPU-only ARM, and any ARM host that already built a CUDA
server are byte-for-byte unaffected. Studio web-server deps + pip seeding are
already complete on native Linux via install_python_stack.py (studio.txt
step 8 + ensurepip/uv bootstrap step 2), and the Linux .desktop launcher is
already created by install.sh create_studio_shortcuts() -- so no duplicate
self-heal/launcher was added.
bash -n setup.sh / provision_llama_cuda.sh / install.sh: pass.
pyproject.toml: valid TOML.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Four gated, is_dgx_spark()-only extensions (strict no-op on x86 NVIDIA, AMD/ROCm,
Mac/MLX, Intel, discrete aarch64, normal WSL/Windows; no computed value changes):
1. max_autotune=False on Spark, in BOTH torch_compile_options dicts
(models/_utils.py + kernels/flex_attention.py). This 48-SM GPU is below
inductor's hardcoded 68-SM is_big_gpu threshold, so max_autotune_gemm is
already skipped (the 'Not enough SMs to use max_autotune_gemm mode' warning) --
dropping it only avoids the wasted compile-time autotuning search; the produced
Triton/inductor kernels are identical (same accuracy + steady-state speed).
2. dataloader_pin_memory=False on Spark, via an idempotent
TrainingArguments.__post_init__ wrap (covers SFT + all TRL trainers). Pinned
host memory is pointless on unified memory (no separate device memory) and only
reserves non-pageable RAM from the shared pool. Mirrors transformers' own
. Opt out:
UNSLOTH_SPARK_KEEP_PIN_MEMORY=1.
3. UNSLOTH_DISABLE_DOUBLE_BUFFER defaulted on Spark (setdefault): unsloth-zoo's
gradient-checkpointing double-buffer is gated on mem_get_info (undercounts on
UMA) and overlaps a host<->device copy that is free on a shared pool.
4. Opt-in UNSLOTH_SPARK_MEM_FRACTION -> torch.cuda.set_per_process_memory_fraction
safety valve (default unset = no cap, no capacity loss), so an over-allocation
raises a catchable OOM instead of wedging the box.
Findings from a 5-agent code+web review (transformers/unsloth/zoo/trl + NVIDIA
DGX-Spark playbooks). Higher-impact-but-needs-validation items (device_map
max_memory sizing, GC offload short-circuit, drop_caches) deferred.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A full -j(nproc) CUDA build is power/thermal-heavy on laptops (e.g. N1X) and can
trip a thermal/power shutdown mid-build. Allow lowering the job count; cmake
--build is incremental so re-running resumes from where it stopped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds patch_dgx_spark_memory_config() (models/_utils.py), applied at import: on
Spark-class machines it sets PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True so
the CUDA caching allocator grows segments in virtual address space instead of
fragmenting the shared unified-memory pool. More of the pool stays usable for
weights/activations -> fewer fragmentation OOMs and headroom for larger models /
longer sequences. Pure memory management: computed values are unchanged, so
accuracy is unaffected (verified: gemma-3-270m losses identical with/without it).
Regression-safe: gated by is_dgx_spark() (strict no-op on x86 NVIDIA, AMD/ROCm,
Intel, Mac/MLX, discrete aarch64). Uses setdefault semantics -- only appends when
expandable_segments is absent, never overrides a user's PYTORCH_CUDA_ALLOC_CONF;
opt out with UNSLOTH_NO_EXPANDABLE_SEGMENTS=1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Runtime training support for NVIDIA Blackwell unified-memory (UMA) machines --
DGX Spark (GB10) and the N1X "RTX Spark" laptop. Gated so it is a strict no-op
on every non-Spark platform (x86 NVIDIA, AMD/ROCm, Intel/XPU, Mac/MLX, discrete
aarch64 GH200/GB200): those are not aarch64 and/or report non-matching device
names, so behaviour there is byte-for-byte unchanged.
models/_utils.py:
- Add is_dgx_spark(): aarch64 + NVIDIA CUDA + a known Spark device-name token
(GB10 / JMJWOA / N1X / ...). @lru_cache; overridable via UNSLOTH_FORCE_DGX_SPARK.
One shared detector that also catches the N1X laptop, which reports
"JMJWOA-Generic-GPU" rather than "NVIDIA GB10".
- Add patch_dgx_spark_caching_allocator_warmup(), applied at import: no-ops
transformers.modeling_utils.caching_allocator_warmup on Spark. HF sizes a GPU
pre-allocation from cudaMemGetInfo() to warm the caching allocator; on Spark
UMA cudaMemGetInfo undercounts free memory (reclaimable buffer cache shows as
unavailable), so the warmup torch.empty() raises
`AcceleratorError: invalid argument` and aborts any bitsandbytes 4/8-bit load.
The warmup is only a speed hint -> dropping it on Spark lets quantized loads
succeed. Idempotent; single call site (modeling_utils.py:4212) confirmed.
(Patch credited to Roland [UnAI] / Daniel, Unsloth Discord.)
models/loader.py:
- Replace the two inline `"NVIDIA GB10" in get_device_name()` checks (which
disable the currently-broken vLLM fast_inference) with is_dgx_spark(), so the
N1X is covered too. Same behaviour on DGX Spark; no change off-Spark.
torch.compile + Triton are verified WORKING on the N1X (Triton 3.6.0; a real
gemma-3-270m-it 4-bit finetune with compile ON trains and emits the full compiled
cache), so nothing is disabled -- UNSLOTH_COMPILE_DISABLE is not set.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A clean-slate reinstall on an ARM64+NVIDIA box surfaced three follow-on gaps in
the WSL path. All fixes are additive, best-effort, and confined to the $torchOk
success branch of the WSL fallback, so they only run on the ARM64+NVIDIA machines
that reach it -- no other platform is affected.
install.ps1:
- Self-heal Studio's web-server deps. install_python_stack.py installs the Studio
UI deps (fastapi/uvicorn/structlog/starlette) in a late step; if that run is cut
short, torch+unsloth land but the server stack is missing and `unsloth studio`
dies at launch on ModuleNotFoundError. Import-check the stack after the torch.cuda
probe and, if absent, install it WITHOUT re-pinning huggingface-hub/transformers/
datasets, so the verified GPU torch path is never disturbed.
- Seed pip into the (uv-managed, pip-less) venv via ensurepip so save_pretrained_gguf
-> check_pip() works regardless of how Studio is launched.
- Auto-build a CUDA llama-server for GGUF inference in the background via the new
provision script (below), so GGUF chat/tool-calling lights up a few minutes after
install with zero manual steps. Opt out with UNSLOTH_NO_LLAMA_CUDA=1.
studio/scripts/provision_llama_cuda.sh (new):
- Idempotent, best-effort (always exits 0). Builds a CUDA llama.cpp into
~/.unsloth/llama.cpp (Studio's resolver path). Generic across NVIDIA Linux/WSL
incl. aarch64 (DGX Spark, N1X): derives the arch from the GPU's compute_cap,
installs gcc-14 + CUDA 13.3 only when nvcc is missing (gcc-15 is rejected by nvcc;
CUDA <13.3 hits the glibc>=2.41 rsqrt header clash), and builds the full target set
(llama-server llama-cli llama-quantize llama-mtmd-cli llama-gguf-split) so it
satisfies both Studio inference and save_pretrained_gguf without a later rebuild.
Validated on an NVIDIA N1X (sm_121): training, GPU inference, GGUF q4_k_m export,
`unsloth studio` via both Desktop + Start Menu shortcuts (HTTP 200), GGUF chat at
121 tok/s (BLACKWELL_NATIVE_FP4=1) and OpenAI-style tool-calling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- install.ps1: refresh the shell icon cache (ie4uinit -show) right after creating the
Desktop/Start Menu shortcuts, so the (valid) .ico renders immediately instead of showing
a blank icon (Explorer caches per-.lnk icons; programmatically-created links need a poke).
- scripts/uninstall.ps1 + scripts/uninstall.sh: also remove the WSL-fallback artifacts the
native uninstall missed -- the %LOCALAPPDATA%\Unsloth shim/launcher/icon dir, its user-PATH
entry, and the real Studio install inside each WSL distro (rm ~/.unsloth + any CUDA llama
build). Previously the native uninstaller only cleaned the (empty) native venv + .lnk files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The fallback returns before install.ps1's native shortcut code, so it created no
shortcuts. Add a WSL launcher (launch-studio-wsl.ps1) plus Desktop and Start Menu
.lnk shortcuts that start `unsloth studio` inside WSL and open http://localhost:8888
in the browser once the backend is healthy. Best-effort + wrapped in try/catch so a
shortcut failure never aborts the install.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Future-proof: probe whether a CUDA torch wheel is installable natively for win_arm64
(uv pip install --dry-run). If it resolves (NVIDIA ships the wheel) keep the NATIVE
install; otherwise fall back to WSL. WSL is used ONLY when native genuinely can't.
- Create a native Windows unsloth.cmd shim (on user PATH) that forwards every
"unsloth ..." into the WSL GPU env, so "unsloth studio" / "unsloth studio run" typed
in PowerShell run inside WSL and stream output + the http://localhost:8888 URL back.
- Run the WSL install under Continue-EAP and verify torch.cuda before reporting success
so the optional (aarch64) llama.cpp prebuilt failure cannot abort or mis-report.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two independent, purely-additive changes for Grace-Blackwell aarch64 (NVIDIA N1X
"RTX Spark" and DGX-Spark-class) and new-glibc hosts. 78 insertions, 0 deletions.
studio/setup.sh: when the GPU llama.cpp source build runs on glibc >= 2.41 with a
CUDA toolkit < 13.3, nvcc fails on the rsqrt/rsqrtf <crt/math_functions.h> exception-
spec clash (fixed upstream in CUDA 13.3 via _NV_RSQRT_SPECIFIER) and the build silently
falls back to CPU. Add a clear diagnostic recommending CUDA >= 13.3. Diagnostic only,
strictly inside the existing NVIDIA CUDA branch (Metal/ROCm/CPU/x86 unaffected).
install.ps1: native Windows-ARM64 has no CUDA PyTorch / Triton wheels, so the native
install can't deliver GPU. When ARM64 + NVIDIA is detected, automatically set up WSL2
and run the Linux installer there (full GPU), then print the launch command. Strictly
gated on ARM64 && NVIDIA && not --no-torch; x86_64 Windows (NVIDIA/AMD) and
ARM64-without-NVIDIA are byte-for-byte unchanged. Opt out: UNSLOTH_NO_WSL_FALLBACK=1;
distro: UNSLOTH_WSL_DISTRO. Encoding-proof distro detection via 'wsl -d <d> -- true'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 03:59:47 -07:00
15 changed files with 1909 additions and 35 deletions
step"wsl""Windows on ARM + NVIDIA, native CUDA unavailable -- routing GPU setup through WSL2"
substep"no win_arm64 CUDA PyTorch/Triton yet; WSL2 delivers full GPU (DGX Spark / RTX Spark path).""Yellow"
# The Tauri desktop app launches its backend from a Windows venv, not WSL, so a
# WSL-only install would start nothing -- send those users to the CLI installer.
if($TauriMode){
# A prior native Studio venv was rolled aside (Start-StudioVenvRollback) before
# here; restore it so rejecting this path doesn't orphan the user's working
# install. No-op when nothing was rolled aside.
Restore-StudioVenvRollback
return(Exit-InstallFailure"Windows-on-ARM + NVIDIA GPU needs the WSL2 GPU install, which the desktop app can't launch yet. Install from PowerShell instead: irm https://unsloth.ai/install.ps1 | iex"1)
}
# --local installs the Windows checkout editably, but the WSL tunnel installs from
# PyPI / a git ref and never mounts $RepoRoot -- so --local here would silently
# install the published package in WSL and report success. Reject it and point at
# the supported pre-merge mechanism (push the branch + UNSLOTH_INSTALL_REF).
if($StudioLocalInstall){
Restore-StudioVenvRollback# see TauriMode note above: don't orphan a rolled-aside venv
return(Exit-InstallFailure"--local can't be honored on Windows-on-ARM + NVIDIA: the GPU install runs inside WSL2 and installs from a published/git ref, not this Windows checkout. For pre-merge testing, push your branch and set UNSLOTH_INSTALL_REF, e.g.: `$env:UNSLOTH_INSTALL_REF='<branch>'; irm https://unsloth.ai/install.ps1 | iex"1)
}
# A custom Studio root only applies to the native Windows layout; the WoA GPU
# install lives in WSL at /root/.unsloth with fixed shim/verification paths. Warn
# rather than pretend to honor it (the uninstaller still cleans the WSL install).
if($envOverride){
substep"note: $envOverrideVar='$envOverride' is not used for the Windows-on-ARM WSL install -- Studio installs inside WSL at /root/.unsloth.""Yellow"
}
# --with-llama-cpp-dir names a Windows-side llama.cpp, but this install runs it
# inside WSL2 and would silently ignore the choice. Reject like --local and point
return(Exit-InstallFailure"--with-llama-cpp-dir / UNSLOTH_LOCAL_LLAMA_CPP_DIR can't be honored on Windows-on-ARM + NVIDIA: llama.cpp runs inside WSL2 and can't use a Windows path. Remove it, or pin the WSL-side build with UNSLOTH_LLAMA_TAG or UNSLOTH_LLAMA_PR instead."1)
Restore-StudioVenvRollback# see TauriMode note above: don't orphan a rolled-aside venv
return(Exit-InstallFailure"UNSLOTH_INSTALL_REF='$_instRef' is not a valid git ref (allowed: letters, digits, '.', '_', '/', '-'). Set it to a real branch or tag name."1)
}
# UNSLOTH_WSL_LLAMA_DEFERRED=1: setup.sh skips its foreground CUDA llama.cpp build;
# we build it in the background. apt stderr stays visible so failures are diagnosable.
# Forward UNSLOTH_NO_LLAMA_CUDA (it also skips the dispatch below, so unforwarded
# setup.sh would defer to a background builder that never starts).
substep"WSL install did not complete its core steps this run (no completion stamp; inner exit $wslRc) -- the venv passing the probes is from a previous install.""Yellow"
