The pin flipped exactly as designed: it looks for _unsloth_main in the installer
it actually ran, and #7548 put it there. Delete the pin and the CLI waiver, and
assert the opposite instead.
WSL is the only platform whose install shells out to Windows interop mid-script,
and interop relays the stdin it inherited, so this job is the one that can catch
the pipe being drained again. A truncation here is now a hard failure.
* Fix bitsandbytes zombie module breaking test collection
A partially failed `import bitsandbytes` leaves the package half-imported:
CPython evicts only the parent from sys.modules and keeps every submodule it
had already loaded. The next import re-executes __init__ but every
`from .x import y` is served from cache, so the submodule attributes are never
rebound. The package imports "successfully" while `bnb.functional` is gone.
Bind the submodule via `import bitsandbytes.functional as bnb_functional`,
which reads sys.modules directly and survives that state, and import
bitsandbytes in tests/conftest.py on the real CPU path before
torch.cuda.is_available() is mocked, so the half-imported state is never
created in the first place.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Clear ALLOW_BITSANDBYTES when the bitsandbytes native kernels are not real
From bitsandbytes 0.46 a wheel whose native library never loaded still imports and
resolves every ctypes handle: BNBNativeLibrary.__getattr__ returns a throw_on_call
closure, and a dead library is replaced wholesale by ErrorHandlerMockBNBNativeLibrary,
which does the same for every name. Nothing raises while kernels/utils.py binds them
at module scope, so device_type.py's guarded import sees a healthy wheel,
ALLOW_BITSANDBYTES stays true, loader.py forwards the default load_in_4bit=True and
the run dies inside a kernel instead of degrading to 16bit.
Probe the handles the kernels actually bind and clear the flags when they are not
native. A real handle is a ctypes function pointer and carries restype; a deferred
failure is a Python function and does not.
Scoped to the capability flags on purpose. The module stays bound and get_ptr keeps
pointing at bitsandbytes, because these shapes import perfectly well and treating
them as absent would disable a wheel whose Python side works - a CPU-only install is
exactly that shape.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Only clear the flags when the native library is dead, not partially exporting
ALLOW_BITSANDBYTES gates 8bit as well as 4bit - loader.py:505-510 clears both - so
failing the check on one missing 4bit symbol would silently downgrade an otherwise
valid LLM.int8 request to 16bit. A library that exports some of these handles is
alive; only one where none of them is a ctypes function pointer is dead, which is
the CPU-only and ErrorHandlerMockBNBNativeLibrary case this exists for.
A genuinely missing symbol raises where kernels/utils.py binds it, so it is a crash
no capability flag can rescue and not something to trade 8bit for.
* Gate the bitsandbytes ctypes binds on the same verdict as the flags
Clearing ALLOW_BITSANDBYTES is not enough on its own. kernels/utils.py guarded
the bnb.functional.lib.* binds on `bnb is None` alone, so an importable but dead
wheel still reached them at module scope: bitsandbytes 0.45.5, the floor in
pyproject.toml, sets functional.lib = None when the native library fails to load,
and None.cdequantize_blockwise_fp32 raises right there. That kills import unsloth
outright instead of degrading to 16bit, which is the fallback the cleared flag
exists to reach.
Reuse native_kernels_ready so the bind path and the flag path agree, and take the
_bnb_required branch when they say the library is dead. Touches only the guard
expression, not the binds themselves.
* Tighten the comments on the bitsandbytes kernel readiness probe
* Require every probed handle, and license the module Apache like the rest of unsloth
The readiness verdict now gates the module-scope ctypes binds as well as the flags,
so "at least one handle is native" is no longer the right question. A library that
resolves one symbol and not another passed the probe and then raised AttributeError
at the bind the probe exists to prevent. Require all of them.
That costs 8bit in the partial case, since ALLOW_BITSANDBYTES gates both, but a wheel
missing a symbol is a shape no flag can make safe and refusing it beats crashing on
it. Flipped the test that encoded the old behaviour and added the more realistic
shape: the library loaded, one symbol is still a deferred-failure closure.
LICENSE:190 assigns files under unsloth/* to Apache 2.0, and 87 of the 90 modules
there carry that header, so use it here rather than AGPL.
* State the all-handles rule once instead of three times
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Installer: wrap install.sh in a function so a piped install cannot report curl (56)
`curl -fsSL https://unsloth.ai/install.sh | sh` makes sh the READER of a pipe.
The file is ~150KB, far more than a pipe buffer holds, so a top-level `exit` left
sh dead with thousands of lines unread. The write end then failed and curl
appended
curl: (56) Failure writing output to destination, passed 16357 returned 0
after the installer's own message, which reads as a download failure rather than
the real diagnosis. 29 of the 35 exits are in the first half of the file, so every
early failure on every platform looked like a bad download.
Measured, piping this file into sh and forcing an early exit:
before: writer rc=141 (SIGPIPE) reader rc=1
after: writer rc=0 reader rc=1
Through a real curl against a local server, curl rc went 23 -> 0 while the
installer's own exit code kept propagating.
Defining a function forces sh to parse to the closing brace before running
anything, so the pipe is always drained. install.ps1 has always had this shape
(Install-UnslothStudio invoked at the end of the file); this brings install.sh
into line.
Deliberately not reindented. Shell ignores leading whitespace, so the diff stays
two hunks instead of 4400 reflowed lines, and `exit` still exits the shell from
inside a function, so no control flow changes.
tests/sh/test_install_pipe_safety.sh pins both halves of the contract: the writer
must survive, and the installer's real exit code must still reach the caller. It
fails against the unwrapped file (writer rc=141).
* Tighten the pipe-safety comments
Compress the install.sh wrapper rationale and the test header down to the
parts that are not obvious from the code. Comments only, the parsed command
tree of both files is byte identical.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
The WSL leg dies at install.sh:2082 with an unterminated quoted string.
Nothing is wrong with that line: piping the script into sh is not atomic.
dash reads it from the pipe in 8192-byte blocks and runs each command as
it parses, and install.sh:2007 calls _maybe_reroute_strixhalo_to_2404,
which on WSL alone shells out to Windows interop; interop relays the
stdin it inherited and drains the pipe. dash has 11 blocks buffered at
that point, ending at byte 90112, which falls inside
"$STUDIO_LOCAL_INSTALL" on line 2082. Truncating install.sh at 90112
and parsing it reproduces the message verbatim, and running the whole
file under a stdin-draining interop stub reproduces the exit code too.
#7548 wraps the body in _unsloth_main so sh parses everything before
running anything, and the same reproduction against its head is clean.
The eight green staging runs cited when this job's continue-on-error came
off were all on trees that already carried #7548, so that evidence never
covered this branch. Pin the exact signature instead: exit 2 plus the
shell's own unterminated-quoted-string error, with the _unsloth_main
marker read back out of the distro as the flip condition.
Pin winget=masked the same way. studio/setup.ps1:1655-1669 gates on git
unconditionally and can only fetch it through winget, so masking winget
leaves no way to satisfy it. #7549 relaxes the gate, and its wording
appearing in the tree retires the pin.
The Linux rows already pin the shipped bundle's own install.sh exiting 2 at the
NEED_SUDO handshake. macos-15 and macos-26 fail the same way for the same reason:
desktop-v0.1.50-beta predates #7547, so the bundled installer still hard-exits on
the Xcode CLT gate that #7547 turned into a warning.
Accept exit 1 plus that exact gate line, and nothing else. _check_macos_deps is
the function #7547 added, so its presence in the bundle means the release caught
up and the block errors out asking for the pin to be deleted.
The macho check asserted a valid signature for every Mach-O under the studio
home, and failed the macos-15 mask/pipe leg on 29 files: lxml, charset_normalizer,
cygrpc, _upb, fontTools, caio, brotli and a bundled libportaudio.dylib. Those are
MH_BUNDLE and MH_DYLIB images dlopen'd into a process without library validation,
they ship unsigned in the wheels, and the same run had already installed and
imported them with the installer exiting 0.
Key the signature half off the Mach-O filetype and run it only on main
executables. Report an absent seal separately from one that fails to verify, and
capture codesign output instead of piping it into grep, which returned the
unsigned exit status through pipefail and called every unsigned binary broken.
The architecture half is unchanged and still a hard failure: it is what closes
the Rosetta 2 gap. The zero-Mach-O guard is unchanged. The .venv_t5_* sidecars
stay in scope; setup.sh creates them during a normal install and
transformers_version.py puts them on sys.path, so they are payload.
Three red checks, two of which test something this branch does not own.
desktop linux deb / appimage run the SHIPPED bundle's own install.sh, and
desktop-v0.1.50-beta was cut on 2026-07-21, before #7547 merged on 07-29. That
bundle still carries the old optional-dependency gate, so on a stripped runner it
exits 2 at [TAURI:NEED_SUDO] cmake git build-essential libcurl4-openssl-dev and
never creates a venv. Current main's _check_linux_deps runs the same set through
_SMART_APT_OPTIONAL, which suppresses every escalation path, so only a new release
can change this. The step now pins that exact outcome: the exit code must be 2 and
the log must carry exactly that package list, anything else still fails, and
finding _SMART_APT_OPTIONAL in the extracted install.sh (the guard #7547 added)
turns into a hard error saying to delete the pin. The venv and torch assertions
stay and still run whenever the installer succeeds.
win windows-11-arm gets a native ARM64 CPython, and torchaudio publishes no
win_arm64 wheel at any version, so the PyTorch step cannot resolve. The fix is in
install.ps1 on #7549, still open. Same treatment: the Install step is
continue-on-error and a new step requires all three of the PyTorch step, the
torchaudio resolution error and the missing win_arm64 platform tag, so any other
failure is red. The row leaves experimental so the job is required, and the pin
errors out as soon as the venv interpreter reports anything but win-arm64, which
is what #7549 landing looks like.
Adds the virgin Windows container lane as two jobs here rather than a sibling
workflow: same premise as the win legs, same path filters, and masked-versus-real
reads better side by side. The hosted Windows legs cannot test the VC++
2015-2022 runtime (it ships in the runner image's System32) or a Windows with no
Microsoft Store, and a servercore:ltsc2022 container on windows-2022 answers both.
The probe asserts no python, py, git, cmake, cl, winget or uv on PATH, on disk or
in the registry, and now also asserts vcruntime140.dll, vcruntime140_1.dll and
msvcp140.dll are absent, which is the one thing the hosted runner cannot un-ship.
Both container install rows stop at studio/setup.ps1's winget-only git gate on
this branch, since #7549 is what relaxes it, so both are pinned the same way. The
overlay row additionally requires the UNSLOTH_CI_SOURCE_OVERLAY hook to have
fired, unconditionally: without that it would be indistinguishable from the
released-wheel row, and the hook is this branch's own feature.
Container notes carried over from the spike: never docker pull when the image is
cached, since MCR has shipped an image ahead of the runner host before; wait for
the Docker daemon, because one leg died in 21s on npipe:////./pipe/docker_engine
and that flake misreads as "Windows containers unavailable"; drive docker from a
run: step, because the job-level container: key is Linux-only. The root CA store
is seeded after the virginity assertion, restoring what a real Windows already
has, because studio/install_node_prebuilt.py downloads Node with bare
urllib.request.urlopen and hits CERTIFICATE_VERIFY_FAILED against the empty
container ROOT store. That product bug is left alone here.
Assert arch and signature on every downloaded Mach-O. This is the one genuine
gap the simulation had: Rosetta 2 is preinstalled on hosted runners and absent
from a factory-fresh Mac, so an x86_64-only llama.cpp, whisper.cpp, Node or uv
payload runs green here and dies with "bad CPU type in executable" for the
user. llama-server launching under `assert-llama-loads.sh` does not rule that
out, because Rosetta makes it launch. The new `macho` check reads `file -b`
(`lipo` is an xcrun shim and is gone after masking, as the desktop lane already
notes) and keys the expected arch off `uname -m`, so macos-15-intel expects
x86_64. It also requires at least an ad-hoc signature on arm64, which closes
the AMFI "Killed: 9" class that uv has already been bitten by; the check is
skipped on x86_64, where unsigned code loads fine and so is not the same
defect. It fails when the scan finds nothing, since an empty scan reads exactly
like a clean one.
Make absence real rather than PATH-hidden. uv probes well-known interpreter
locations and the framework loader ignores PATH entirely, so hiding the
toolcache only hid it from `command -v`. Empty /usr/local (it EXISTS on a
factory-fresh Mac as a SIP-exempt firmlink, and is empty; it is /usr/local/bin
that is absent, so the directory itself stays), move the hosted toolcache and
/Library/Frameworks/Python.framework aside, and clear the developer dotdirs and
caches. A populated uv or pip cache can also satisfy a resolution that would
fail on a user's machine. Every removal goes through --remove and is recorded
in the generated restore.sh, guarded so a path the install recreated is not
buried inside its own restore.
Unset CI, GITHUB_* and RUNNER_* for the installer process only. An installer
branching on CI=true is a hidden dependency no consumer exercises. Scoped to
the child so the step's own $GITHUB_OUTPUT still resolves.
Record spctl --status and csrutil status. Neither is documented for these
images and both change what a binary is allowed to do.
* Tests: import bitsandbytes before the GPU-free harness spoofs CUDA
The CPU test harness patches torch.cuda.is_available to return True so
device_type.py's cache captures "cuda" on a GPU-less runner. bitsandbytes
reads the same flag at import time to decide whether to load its CUDA
backend, and that backend reads torch._C._cuda_getCurrentRawStream, which
a CPU-only torch build does not expose. An import landing inside the spoof
window therefore raises, Python drops bitsandbytes from sys.modules while
leaving its submodules cached, and every later import returns a module with
no .functional, so unsloth/kernels/utils.py dies at module scope.
Import bitsandbytes before the window so it stays on its CPU backend and
remains fully usable, rather than being degraded to unavailable.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Desktop: ask before quitting on top of a running install
This is the trigger neither #7492 nor #7490 addresses -- both start from a venv
that is already broken. Confirmed: neither PR touches cleanup_child_processes.
Quitting runs cleanup_child_processes -> install::stop_install, which SIGTERMs the
installer's process group. In the reported session that landed at "5/10 studio
deps", so the venv kept the CLI's dependencies and lost the server stack, and the
next launch died on `import structlog`. Three minutes of installing, destroyed
with no warning and no way back.
So ask. Only from the tray Quit item -- a deliberate action with a UI present. The
RunEvent::Exit path (OS shutdown, SIGTERM) is left alone: it must never block on a
dialog nobody can answer. The call already runs off the menu callback thread,
which is also what blocking_show requires.
Closing the window was already safe (it hides to tray); this closes the remaining
way to lose an install by accident.
* Tighten comments in desktop quit-during-install guard
* Condense comments in quit-during-install guard
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Installer: stop requiring a developer toolchain on the consumer path
A brand new Mac cannot install Studio at all. install.sh gates on
`xcode-select -p` and exits 1 with 'Xcode Command Line Tools are required',
and Linux exits 1 on any non-apt distro over cmake/gcc/git/libcurl headers.
Nothing under either gate needs a toolchain. uv is a prebuilt binary, CPython
comes from uv's managed python-build-standalone, llama.cpp and whisper.cpp are
prebuilt downloads, Node is a pinned nodejs.org archive, and triton is skipped on
macOS. unslothai/llama.cpp b10107-mix-1911198 publishes macos-arm64, macos-x64,
linux-x64 and linux-arm64 builds covering cpu, cuda12, cuda13, rocm and vulkan.
PR #6617 already dropped the Homebrew/cmake stop on macOS for this reason and
just left the CLT stop behind.
macOS: warn and continue when the CLT are absent. Linux: only a download
transport (curl or wget) is fatal; build tooling warns. Both keep a hard git
requirement for --local, which installs unsloth-zoo from a git+https URL.
Both gates move into functions so tests/sh can extract them. The old inline form
could not be reached by the tests/sh convention, which is why this shipped broken
and stayed broken. test_macos_clt_gate.sh (19 assertions) and
test_linux_deps_gate.sh (25) cover the clean machine, the CLT-stub shape where
/usr/bin/git exists but fails, the non-apt distro, and the --local paths.
Writing the Linux test caught a latent bug: the gate trimmed its list with
$(echo ... | sed ...), so on a minimal image without sed the substitution yields
empty and it reports 'all system dependencies found' on a machine with none of
them. Replaced with parameter expansion.
Also caps av<16 in the single-env constraints. av 16+ ships no cp313 macOS arm64
wheel, and it is a C extension over FFmpeg, so uv would silently fall back to a
source build needing both a compiler and FFmpeg headers.
Verified on GitHub-hosted macOS runners with /var/db/xcode_select_link,
/Library/Developer/CommandLineTools, /Applications/Xcode*.app and Homebrew moved
aside. macos-14, macos-15 and macos-26 fail on main and install cleanly with
this; the recorded tool-invocation trace for the whole install is a single
`xcode-select -p`, so nothing compiled and nothing installed a toolchain.
* Linux: auto-install git rather than dropping it, and skip triton kernels without it
Making git optional on Linux was too broad. studio/backend/requirements/
triton-kernels.txt line 2 is a git+https URL, so step 6/14 died with 'Cannot find
command git' and failed the whole setup on ubuntu2404-root, ubuntu2404-arm-root
and fedora41, all of which had been passing. The claim that nothing on the
consumer path needs git holds on macOS, where triton is skipped, but not here.
install.sh now auto-installs git through apt with the other optional tooling, so
Debian and Ubuntu are unchanged. The triton kernels step skips with a message
when git is absent instead of failing: they are a training speedup, not a boot
requirement, and a GGUF chat install has no use for them.
Six more assertions pin both halves.
* macOS Intel: skip the one package with no x86_64 wheel
The Intel clean-machine leg installed with the toolchain masked, then died in
studio setup:
subprocess.CalledProcessError: Command '['cmake', ...]' returned non-zero
ERROR: Failed building wheel for pytorch_tokenizers
pytorch_tokenizers publishes wheels for macOS arm64, linux x86_64, linux aarch64
and windows, but none for macOS x86_64 at any Python version, so uv falls back to
an sdist that shells out to cmake. Nothing passes --only-binary, so the
compiler-free property was an assumption rather than a contract, and Intel is
where it broke.
Marked so it installs everywhere except Intel macOS. Apple Silicon is unaffected.
* Stop the optional dep gate from aborting the install
_smart_apt_install exits rather than returns, and `|| true` does not catch an
exit, so a box missing cmake or git aborted at the gate added to let it
continue. Verified in sh, dash and bash. Run it in a subshell and re-raise only
code 2, the NEED_SUDO handshake install.rs answers with an elevation prompt.
install.sh treats a present-but-broken git as missing, but the Python side
tested only shutil.which, so it promised to skip the git+https triton
requirement and then fetched it anyway. Same check on both sides now.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Never elevate for optional build tools
Re-raising code 2 turned the optional set into a NEED_SUDO handshake, so a box
missing cmake or gcc got the desktop's mandatory permission dialog, whose Cancel
drops back to not-installed. That re-imposes through a prompt the build-tool
requirement this gate removes, and none of those tools are needed to run.
Suppress the handshake for optional callers; a required package still elevates.
Verified in sh, dash and bash.
Also advance the progress bar on the no-git triton skip, which otherwise ends at
14/15.
* Tighten the comments on the dependency gate
* Correct why the PyAV cap is needed
16.0.0 does ship cp313-cp313-macosx_14_0_arm64; the comment claimed no cp313
wheel exists. The actual reason is the deployment target: 15.1.0 is macosx_13_0
and 16+ is macosx_14_0, so the cap is what keeps macOS 13 off a source build.
* Tighten the installer gate comments
* Cap cryptography on x86_64 macOS so the consumer install needs no Rust
cryptography 49.0.0 (2026-06-12) dropped the macosx_10_9_universal2 wheel
and now ships macosx_11_0_arm64 only, so x86_64 macOS has no wheel and uv
falls back to the sdist. That build calls maturin, which pulls Rust and
then fails at 'linking with cc failed' on a clean Mac without the Xcode
Command Line Tools. It surfaced in the clean-machine leg mac macos-15-intel
/ mask / file, several minutes into the studio dependency step, which is
exactly the up-front toolchain requirement this branch removes.
48.0.1 is the newest release carrying a universal2 wheel, and its
cp39-abi3 / cp311-abi3 tags cover the 3.12 and 3.13 interpreters the
installer creates. The cap is marker-scoped to darwin + x86_64, so arm64
macOS and every other platform still resolve to the latest. Lift it when
cryptography ships an x86_64-capable macOS wheel again.
Resolution of studio/backend/requirements/studio.txt under this
constraints file gives 48.0.1 on x86_64-apple-darwin and 49.0.0 on
aarch64-apple-darwin and x86_64-unknown-linux-gnu, on both 3.12 and 3.13.
* Correct the av note now that cryptography also compiles on macOS
* Never escalate for optional apt packages outside Tauri mode
The optional bypass sat inside the TAURI_MODE branch, so a plain curl | sh
install on a non-root Debian or Ubuntu box still fell through to the
escalation branch and showed the default-yes permission prompt for cmake,
GCC and the libcurl headers. That is exactly the toolchain this change set
declared unnecessary on the consumer path, so the prompt asked for a
password to install packages nothing here uses, and a headless run failed
the same way instead of falling through to prebuilt llama.cpp.
Move the check above the mode split so optional callers return 2 in both
modes. Required packages such as curl still escalate unchanged.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: free the llama-server slot when a chat stream reaches [DONE]
* Release the slot before yielding, only on a completed decode
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the comments added by this PR
* Inline the done-sentinel check and use plain bools for the decode flags
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: show a Nudging tool calls badge while the tool-call re-prompt runs
* Guard the nudge status ordering assertion against index 0
* Tighten the nudge status comments
* Announce the nudge text instead of the generic spinner label
* Trim the nudge status comments
Collapse the multi-line notes to fewer lines and drop one that restated the assert below it. The blank-before-badge ordering reason and the keep-in-sync contract are preserved.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* fix(studio): show the current artifact's source after switching artifacts
The canvas source view feeds one Streamdown a fence built from the selected
artifact's code, but never keys it. Streamdown does not revise a block it has
already committed, so the panel keeps rendering the previous artifact's source.
Key the source view on the artifact ID plus a hash of its code: tool artifact
IDs are derived from the tool call, not the code, so the ID alone does not
change when a tool artifact is updated in place.
* Name the real root cause and make the source-key test load-bearing
The remount is needed because Streamdown memoizes a fenced code block on its
hast node's line/column span, which ignores the text inside the fence, so two
canvases of equal line count compare equal and the old source stays on screen.
Verified in Chromium against streamdown 2.5.0: unkeyed, 70 lines -> 70 lines
renders the previous artifact, 70 -> 71 and 70 -> 90 render correctly.
Move the key expression into the source branch so it costs nothing while the
artifact is streaming and the view is unmounted, and export the helper from
types.ts so the test exercises the shipped code instead of a local copy of the
formula (it passed before even with the key removed from the component).
* Assert the source view's Streamdown key wiring, not just the helper
The suite exercised buildArtifactSourceKey but never the component, so deleting
key={buildArtifactSourceKey(artifact)} from the Streamdown left every test
green. There is no DOM renderer available to these tests, so parse
artifact-surface.tsx with the TypeScript compiler API (already a devDependency)
and assert the source view's Streamdown carries that key.
Mutation-checked: removing the key fails 1 test, swapping it for artifact.id
fails 1, and making the helper ignore code fails 2.
* Tighten the comments added by this PR
* amd: require bitsandbytes>=0.50.0 in the amd extra
bnb <= 0.49.2 NaNs at decode shape on every AMD GPU. The ROCm 4-bit GEMV
fix (bnb PR #1887) first ships in 0.50.0, on PyPI since 2026-07-24, so the
old >=0.49.1 floor could still resolve the broken range.
Mirrors the same change made on the pip release branch in #7278.
* amd: cite the 0.50.0 ROCm work accurately in the bnb floor comment
The comment credited bnb PR #1887 as "the ROCm 4-bit GEMV fix" for every
AMD GPU. #1887 decouples blocksize from warp size and fixes a hardcoded
warp size of 32 in kgemm_4bit_inference_naive, which is a CDNA problem by
construction. The RDNA-side work is #1979 (fused 4-bit SIMT GEMM) and
#2012 (RDNA3/4 workgroup resonance). All three first ship in 0.50.0, so
the >=0.50.0 floor is unchanged; only the justification was wrong.
* amd: raise the installer bitsandbytes fallback floors to 0.50.0
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* amd: stop reporting the bitsandbytes PyPI fallback as broken
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten AMD bnb floor comments
* Keep the amd extra citation and the AMD install guide reference
* amd: do not promise aarch64 a ROCm 4-bit backend it never gets
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* amd: fall back to the PyPI bitsandbytes floor on Windows ROCm too
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Every desktop-v* release in this repo is a draft, and GitHub lists drafts only
to a token with push access, which is why resolving one needs contents: write.
A pull request from a fork receives a read-only token no matter what the
workflow declares, so on those runs the resolver cannot see any release and the
job died on "no desktop-v* release visible", accusing the repo of having no
bundle when the real cause is the trigger.
This workflow runs on pull_request for changes to itself and the stripping
scripts, so an outside contributor editing either would have hit that. Guard the
three jobs on the head repo not being a fork. A skipped job is honest here: it
does not claim to have tested a bundle it was never able to download, and it is
not reported as a pass.
All three desktop legs died at the download step with an empty REL_TAG. The
resolver passed --exclude-drafts while REL_REPO now defaults to
github.repository, and every desktop-v* release in unslothai/unsloth is a draft:
desktop-v0.1.50-beta and desktop-v0.1.471-beta are both drafts carrying the .dmg,
.deb, .AppImage and setup.exe, while only the non-desktop tags like v0.1.501-beta
are published. Excluding drafts therefore matched nothing and no leg could ever
run against a production bundle.
Drop --exclude-drafts so the newest desktop-v* release is found. A draft has no
tag ref, so releases/tags/<tag> 404s for one, but gh resolves drafts over GraphQL
and gh release download <tag> fetches their assets normally, so the download call
is unchanged. Listing drafts requires push access, which for GITHUB_TOKEN means
contents: write, so the workflow permission is raised from read and annotated.
When nothing resolves the leg still fails hard rather than skipping: with no
bundle to install there is nothing to prove, so a green run would be a lie. The
error now names both causes, no release cut yet or a token that cannot see drafts.
Also stop the restore step swallowing its own failure. `bash
.clean-machine/restore.sh || true` printed "No such file or directory" whenever an
earlier step failed before the toolchain was stripped, and hid a genuinely broken
restore just the same. Skip explicitly when the file is absent and let a real
restore failure surface. Same fix in clean-machine-install-ci.yml, which had the
identical line.
* feat(studio): share the llama-server --parallel bounds as PARALLEL_MIN/MAX
The per-load parallel-slots field needs the same 1..64 range the CLI flag
validates, but models/inference.py cannot import run.py (run.py builds the
app that imports routes that import models). Promote the bounds into this
dependency-free module, which already owns the -np/--parallel semantics, and
record the deliberate mirrors that cannot import it (run.py, the unsloth CLI,
the web UI). The denylist entry stays: the first-class field is now the single
write path for the slot count, so a pass-through would still desync the
committed bookkeeping from llama-server.
* feat(studio): note the per-load override in the --parallel help text
--parallel is now the server-wide default that a per-load n_parallel (the
Studio Parallel Slots run setting) can override, not the definitive slot
count. Point at the new control so a user does not conclude a restart is the
only way to change slots, and record the shared PARALLEL_MIN/MAX mirror
alongside the existing CLI one.
* feat(studio): add n_parallel to LoadRequest and echo the slot counts
LoadRequest.n_parallel (optional, PARALLEL_MIN..PARALLEL_MAX) lets a load pick
its own llama-server --parallel count; omitted, the server-wide launch default
applies. ValidateModelRequest carries it too so the training-coexistence
estimate sizes the KV cache like the follow-up load rather than passing on a
smaller footprint.
LoadResponse and InferenceStatusResponse gain both requested_parallel_slots
(what the load was invoked with) and parallel_slots (what llama-server
actually runs after the fitter's slot reduction), so a client can tell an
honored request from a reduced one. Both are None where --parallel has no
meaning: non-GGUF loads and the diffusion runner.
* feat(studio): record the requested parallel-slot count on the backend
The auto GPU-memory fit may launch fewer slots than requested to keep the
model fully on GPU, so the committed effective count cannot answer "is the
live server what this request asked for?". Store the invoked count separately
(mirroring the _requested_n_ctx pattern) from the pre-reduction pending
kwargs, expose it as requested_parallel_slots, and have _already_in_target_state
compare requested-vs-requested: comparing against the effective count would
reload -- and re-reduce -- forever on an identical Apply.
The comparison sits in the non-diffusion branch, since the diffusion runner
ignores --parallel entirely. The requested value shares the effective count's
lifecycle, so every unload/kill path clears it and a stale count cannot
poison the next load's dedupe.
* feat(studio): honor a per-load parallel-slot count in /load and /validate
Resolve the slot count once per load -- the request field if set, else the
server-wide launch default -- and feed it to every consumer that must agree:
the training-coexistence guard, the llama-server load kwargs, and the reload
dedupe. Without the dedupe comparison a changed slot count would be swallowed
as already_loaded; it compares requested-vs-requested and skips the diffusion
runner, which ignores --parallel.
app.state.llama_parallel_slots is deliberately never written: it stays the
launch intent and the admission-queue fallback, so one load's override cannot
leak into later loads. /validate resolves the same way so its estimate cannot
undercount what the load then allocates.
Both /load returns and /status echo the counts through one helper, which
reports None for diffusion -- its load never commits a count, so echoing the
reset placeholder would fabricate an "invoked with 1 slot".
* feat(studio): accept nParallel in the chat-preset load config
ChatPresetLoadConfig is extra="forbid", so a preset carrying the new parallel
slots knob would 422 the whole settings sync without this field. Bounds come
from the shared PARALLEL_MIN/MAX rather than literals, so a future range
change cannot start rejecting presets the UI still allows.
* test(studio): cover the per-load parallel-slots knob
Pins the behaviors a regression would silently break: the requested-vs-effective
dedupe (comparing against the reduced count would reload forever), the diffusion
skip and its None echo, the requested count's reset lifecycle, and its commit
from the pre-reduction pending kwargs.
Also pins the three bounds mirrors that cannot import PARALLEL_MIN/MAX (run.py,
the unsloth CLI, the web UI) plus the preset model that can, so a range change
cannot leave one of them clamping or rejecting at the old limit.
* test(studio): refresh the --parallel denylist comments for the UI knob
The pinned rationale said the typer flag owns the slot count and pointed users
at a Studio restart. Parallel Slots / LoadRequest.n_parallel is now the other
managed writer, and the 1..64 guard is the shared PARALLEL_MIN/MAX -- a reader
following the old comments would conclude the UI control does not exist.
* feat(studio): note the per-load override in the CLI --parallel help
Both the plain-serve and `unsloth studio run` flags now describe a server-wide
default the Studio Parallel Slots run setting can override per load, matching
the backend help text.
* feat(studio): remember a per-model Parallel Slots override
nParallel joins the per-model config with the same null-means-follow-the-default
convention as the other knobs: null keeps the server-wide --parallel count, so
a blank control never pins a number and isDefaultConfig still deletes an
otherwise-untouched config instead of storing it.
The value is re-clamped to N_PARALLEL_MIN/MAX on every localStorage read and
write (the store is user-editable), and listing it in STORED_CONFIG_FIELDS
keeps it from being dropped as an unknown key. Legacy blobs predate the knob,
so their migration carries null. No schema-version bump: an additive optional
field, like the GPU fields before it.
* feat(studio): bridge nParallel between the per-model config and the store
The config->store, store->config and equality helpers all need the new field:
without the equality arm a slots-only edit reads as unchanged, so Apply is
dropped and the dirty state never lights up.
* feat(studio): track the parallel-slot override in the chat runtime store
nParallel holds the editable override and loadedNParallel the value the last
successful load sent, which the failed-switch rollback re-sends. Both are
per-model: they clear on unload and on a model switch, unlike the standing
preferences (GPU memory mode, speculative type) that survive one.
There is deliberately no backend-echo field for the control: the echo is the
resolved count, so adopting it would pin a blank "follow the server default"
input to an explicit number.
* feat(studio): type n_parallel and the slot-count echoes
The load request gains the optional per-load slot count, and both the load
response and the status payload gain requested_parallel_slots (invoked) and
parallel_slots (actually running after the fitter's reduction). Keys stay
snake_case: the payload is serialized as-is, with no case conversion.
* feat(studio): forward n_parallel to the validate preflight
validateModel builds its own body rather than forwarding the load payload, so
the slot count has to be listed explicitly. Slots scale the KV estimate, and
the preflight exists to refuse a load the training guard would then 409 -- an
unforwarded count would validate a smaller footprint than the load allocates.
* feat(studio): include nParallel in the active model's config
The sidebar assembles the active model's config from individually subscribed
store fields; an unsubscribed field would leave the form showing a stale value
after any external change.
* feat(studio): add the Parallel Slots control to the run settings
A numeric input in the GGUF advanced section, blank meaning "follow the server
default". It clamps on change like the Draft Tokens field rather than using
NumericValueInput, so there is no blur-draft to lose when the user types a
value and immediately clicks Load.
hasNonDefaultAdvanced counts it too, so a remembered override reopens the
advanced section instead of hiding the setting that is actually in effect.
* feat(studio): key the sidebar config form on nParallel too
The signature drives the remount that re-seeds the form; without the new field
an externally changed slot count would leave the sidebar showing the old one.
* feat(studio): send the Parallel Slots override on load
performLoad snapshots the slot count at click time (staged run-settings config
first, else the store) and sends it on both the validate preflight and the
load, so the two size the same footprint. A cross-model switch re-baselines it
like the other per-model knobs -- the previous model's count must not follow
onto the next one -- and the failed-switch rollback re-sends the previous
model's value so a rescue reload cannot silently drop to the server default.
The success path keeps the click-time value rather than the response echo: the
echo is the count the fitter resolved, so adopting it would turn a blank
"follow the server default" control into an explicit pin. Slots are GGUF-only,
so a transformers load sends and records null instead of a phantom override.
* feat(studio): carry the slot override through the compare-pane load
The compare pane builds its own load request, so it needs the field explicitly
or a pane with a remembered override would load at the server default. Its
validate preflight sends the same count, matching the comment above it that
promises validation is sized exactly as the load below.
GGUF-gated on both calls, and the store adopts the pane's own click-time value
rather than the resolved echo, mirroring the single-model path.
* feat(studio): honor the remembered slot override on startup auto-load
The auto-load path reads the per-model config and forwards every other
remembered knob, so a remembered Parallel Slots value was the one setting lost
on the "load last used model" path: llama-server came back at the server-wide
default with the control showing blank, and the first manual Apply afterwards
then forced a needless reload because the counts disagreed.
* feat(studio): seed the slot baseline from the status echo
Only the rollback baseline is seeded, never the editable control: the echo is
the resolved count, so adopting it would pin a blank "follow the server
default" input to a number. Without the seed, loadedNParallel stayed null
after a tab reload or a second tab adopting the running model, and a failed
switch then rolled the previous model back at the server default while every
other knob was restored.
* feat(studio): capture Parallel Slots in chat presets
The knob joins the preset load config end to end: captured from the store,
re-clamped when read back (persisted presets are untrusted input), applied on
switch, and summarized in the preset chip. Its default is null, so
coalesceDefaultLoadKnobs keeps a default-only preset empty rather than
persisting a no-op override.
* feat(studio): re-derive the preset state when Parallel Slots changes
Both preset memos snapshot the store through capturePresetLoadConfig, so
without the new dependency a slots-only edit left the unsaved-changes flag and
the load summary showing the previous value.
* test(studio): pin the Parallel Slots wiring end to end
Source-contract coverage for the hops a refactor can silently drop: the three
/load builders (interactive, compare pane, startup auto-load) and their
validate preflights, per-model persistence and clamping, the UI row, and the
status seed -- including the negative assertion that hydration seeds only the
rollback baseline, never the control, so the resolved echo cannot pin a blank
"server default" input.
* test(studio): pin nParallel in the preset load config
Covers capture, clamped read-back and apply on the frontend, plus the backend
field itself: ChatPresetLoadConfig is extra="forbid", so a missing or drifted
field 422s every settings sync that carries a preset.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fall back to one slot when llama-server lacks --kv-unified for PR #7447
Without --kv-unified an explicit --parallel N makes llama-server give each slot -c/N, so on a build without the flag choosing N slots silently shrinks every context window for a feature that build cannot serve. Clamp to one slot and log why, placed after the requested count is captured so the echo still reports it and before the KV estimates so the fit matches what actually launches.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Clear the slot control on load paths that never send it, and size the training guard for diffusion
Four review findings on the per-load Parallel Slots knob.
The editable nParallel control means "follow the server default" when null, so
any success path that does not send a slot count has to clear it. Three paths
kept a value staged for a different model:
- chat-adapter.ts, cached non-GGUF auto-load: the interactive and compare
builders already clear both fields for a non-GGUF response, this third one
did not. The field never renders for a non-GGUF target, so the stale count
was invisible and unclearable from the UI yet still persisted, and it flips
isDefaultConfig so a user with no overrides silently gets a stored entry.
- chat-adapter.ts, fresh-model fallback: its request omits n_parallel but its
success state resynced every other knob and left the slots alone, so a staged
edit survived against a server running the default and the next Apply
reloaded at a count that load never sent.
- apply-inference-status-to-store.ts: on a model change underneath the tab
every sibling knob adopts the new model's status, but nParallel updated only
its baseline, so the previous model's explicit count followed onto the new
model and saving or reloading there pinned it. Clear the control and keep
seeding the baseline for the rollback.
The training-coexistence guard sized a diffusion GGUF with the requested slot
count. _estimate_kv_cache_bytes scales the SWA cache with slots
(swa_limit = swa * slots + ubatch), but load_model hands a diffusion target to
_start_diffusion_server before the slot plumbing, so that runner is always
single-slot. At the new default of 4 this inflated the estimate and could 409 a
load that fits. An unclassified GGUF keeps the requested count.
Backend base KV depends on -c alone, not on --parallel, which is why only the
SWA term is affected: llama.cpp PR 14363 and discussion 4130.
Tests: three training-guard cases in test_parallel_slots_per_load.py and one
source contract in test_model_picker_contracts.py, each mutation-checked.
174 passed across the backend slot/admission/training suites, 56 across the
frontend contract suites.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the slot control when re-adopting the running model, and never record slots for a diffusion load
Two follow-ups from the latest review round.
The first is a regression from c796393. That commit cleared the slot control
whenever hydratingExistingModel was set, to stop model A's count following onto
model B. But that flag is also set on the resident-model adopt path: when the
store checkpoint is an external provider id and the user re-picks the still
loaded local model, applyActiveModelStatusToStore is called with the external
id as previousCheckpoint, so the flag is unconditionally true. The clear then
wiped the config applyPerModelConfigToRuntime had restored two lines earlier,
and it was the only knob that did, because the siblings re-adopt the status
echo while this one cleared. Gate the clear on the tab's own baseline no longer
matching the running count: a genuine A to B swap still clears, re-adopting the
same model keeps its value.
The second revises an earlier call of mine. I rejected the diffusion phantom as
cosmetic because the backend ignores the value on every send. The sharpened
report is right and my rejection was wrong: capturePresetLoadConfig records
nParallel with no model gate, a Preset carries no model id, and applying one
writes nParallel for whatever model is current. So a count recorded against a
diffusion model, which the backend never applied, rides a saved preset onto a
text GGUF and becomes a real override the user never chose. Record slots only
when the load actually committed them, on all three load builders.
Tests: two source contracts in test_model_picker_contracts.py, both mutation
checked. Frontend typecheck clean, 58 passed across the contract and preset
suites.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Clear the slot baseline when status reports a model without slots
Hydrating from a GGUF to a slotless model left loadedNParallel at the previous
model's count: the seed only runs when the echo is non-null, and the control
clear added earlier touches nParallel alone. The stale baseline is what a
failed-switch rollback re-sends, and preset capture reads it, so it could claim
slots for a model that never used them.
Clear it when status describes a model that cannot have slots. /status omits
the echo entirely for non-GGUF and sends an explicit null for the diffusion
runner, so keying on is_gguf === false or an explicit null covers both while an
absent field on a GGUF, which is how an older backend reports one, still leaves
the baseline alone.
Test mutation checked; frontend typecheck clean against a fresh npm ci.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Distinguish a same-model re-adopt from a model swap, and size the training guard at the slots that launch for PR #7447
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the blank slot control across a failed-switch rollback for PR #7447
* Restore a remembered slot override when hydrating a fresh store for PR #7447
* Tighten comments for PR #7447
* Restore a remembered slot override on a model switch too for PR #7447
* Tighten comments and docstrings for PR #7447
* Take the rollback slot intent from the picker's pre-switch snapshot for PR #7447
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
The desktop job's Windows masking renamed the toolcache Python, scrubbed the
Machine and User registry PATH, and probed `py`, but nothing checked that
`python`, `git`, `cmake` or `cl` were gone. The drop list is heuristic path
fragment matching, so a runner image that moves any of those outside those
fragments leaves the bundled install.ps1 reusing hosted developer tooling while
the job still reports a clean machine. PATH written to $GITHUB_ENV only applies
to later steps, so the check has to live in a step of its own; it carries the
same event gate as the strip, exempts `py` (it lives in C:\Windows and stays,
which is why the start probe is the real evidence), and resets $LASTEXITCODE
before exiting 0 so an intentionally failing probe cannot fail a clean machine.
Also correct the no-winget matrix note: that leg is not failing for an unfixed
product reason. It stops at the unconditional git gate in setup.ps1 only on this
ref, and with that gate relaxed it passes along with every other leg, so the row
is a merge order dependency and stays required.
The Windows Install step ran `& $script` inside a `shell: pwsh` step, so
install.ps1 was executing under PowerShell 7. A genuinely clean Windows box
does not have PowerShell 7: Windows ships powershell.exe (Windows PowerShell
5.1) and pwsh is a separate install that the hosted runner image happens to
preinstall. So the one workflow whose premise is a machine that has never seen
a developer toolchain was testing the installer under a shell that machine
would not have, and no other Windows job anywhere in .github exercises
install.ps1 under 5.1.
Invoke it the way the desktop does (install.rs:325-339, and the bundled
installer step in desktop-app-clean-machine-ci.yml): powershell.exe with
-NoLogo -NoProfile -NonInteractive -ExecutionPolicy Bypass -File. The pwsh
step wrapper stays, since it is only the installer that has to be under 5.1.
Calling powershell.exe with `&` keeps the output in the pipeline, so
Tee-Object still fills logs/install.log, and $LASTEXITCODE after the pipeline
is the child's real exit code, so $rc and `exit $rc` are unchanged.
install.ps1 and studio/setup.ps1 hold no PowerShell 7-only constructs: no
`#Requires` above 5.1, no `&&`/`||` chain operators, no ternary, no
null-coalescing, no ForEach-Object -Parallel, no $IsWindows/$PSStyle, and no
6+ cmdlets or parameters. setup.ps1 declares `#Requires -Version 5.1`, and its
three $PSVersionTable branches gate a 7-only preference on the 7 side with a
5.1 fallback. Every Invoke-WebRequest already passes -UseBasicParsing, which
5.1 needs because it otherwise reaches for the IE engine.
The desktop workflow claims all three platforms are stripped, but only macOS
and Windows had a strip step and the Windows one scrubbed the process PATH
only. Both gaps let a bundle that needs a developer toolchain pass the one
workflow whose premise is that it must not.
Linux: the job ignored strip_toolchain entirely and ran the bundled install.sh
with the runner's git, gcc, cmake and make in /usr/bin. clean-machine-env.sh
now has a Linux --remove branch that moves the resolved tool binaries aside,
recorded in restore.sh, and the job calls it plus `assert absent` after the apt
step (the .deb install needs dpkg) and before the bundled installer, with a
restore step to match macOS. The loop repeats per tool so a name present in
both /usr/bin and /usr/local/bin is fully masked rather than half masked.
Windows: rewriting $env:PATH does not survive the bundled install.ps1, which
calls Refresh-SessionPath (318-337) and rebuilds $env:Path from the Machine and
User registry values, and py.exe in C:\Windows reaches the toolcache whatever
PATH says. Ported the on-disk toolcache rename, the Machine/User registry scrub
and the py -3.11/-3.12/-3.13 start probe from clean-machine-install-ci.yml, so
the strip is proven rather than assumed.
Windows preflight: the log step was Test-Path, Get-Content and Select-String,
none of which can fail, so an app that hangs before preflight passed on the
90 second liveness check alone. It now asserts a tauri.log exists and carries a
`desktop_preflight completed disposition=` line, the same unconstrained check
macOS and Linux already make. The disposition VALUE is deliberately not
constrained: ManagedReady over an unbootable venv is the reported bug.
installer_source on macOS: only the pipe delivery branched on it, so a
`published` dispatch ran the checked-out script on six of the eight macOS rows
while the run was labelled published. The script is now resolved once at the
top of the Install step and used by the file and tauri deliveries; pipe still
re-fetches through the live transport, because that is half of what it tests.
Linux, WSL and Windows already honoured the input.
Also shortened the comments across the changed files, keeping the reasoning
that says why each check exists.
Four things that let a leg go green while proving nothing.
The desktop Windows job installed the bundle and launched it, and that was all.
On a fresh profile preflight reports not_installed and the app sits on the
install screen waiting for a click, so the process happily stays alive for 90
seconds without the bundled install.ps1 ever running. A bundle that shipped no
install.ps1 resource, or a broken one, passed this job -- which is the packaged
app failure the workflow exists to catch. macOS and Linux already invoke their
bundled script directly; Windows now does the same, via the resource NSIS laid
down next to the exe, invoked the way install.rs invokes it, then asserts the
managed venv exists and can import torch. Its timeout goes to 60 minutes
because a full torch install on a Windows runner is the slowest of the three.
A manual run that selects installer_source: published only redirected the macOS
and Linux jobs. WSL kept copying the checked-out install.sh and Windows kept
running the checked-out install.ps1, so a run asking whether the script on
unsloth.ai works reported on this ref under the published label. Both now honor
the selection; install.ps1 advertises its own unsloth.ai URL, so published has a
meaning on Windows too. Both branches stay empty on pull_request and push, so
automatic runs are unchanged.
The push-to-main filter listed only install.sh, install.ps1 and this workflow,
while the PR filter also covers setup.sh, setup.ps1, install_python_stack.py and
the clean-machine helpers. A direct push touching those skipped the workflow
entirely, so the post-merge backstop never ran for the files the source overlay
was added to cover. The two lists now match.
Neither filter covered studio/backend/requirements, even though the overlay
exists precisely so a constraints change is resolved on a machine with no
compiler and no cached wheels. The update-smoke workflows cannot stand in: they
start from a preinstalled Python and full developer tooling.
* Pin the newer-mapper FP8 probe with tests that can fail
The two identity assertions added in #7478 compare the returned FP8 tables
against the installed ones, but the fixture serves the same mapper.py as both
the installed and the fetched source and exec always allocates fresh dicts, so
they pin allocation rather than provenance and hold for any new dict.
Replace them with two tests that drive get_model_name end to end: one splices an
FP8 entry into the fetched source only and asserts the upgrade error still fires,
the other serves a mapper.py with no FP8 tables and asserts the 4bit half of the
probe survives, which is the regression #7497 fixed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the resolver stub for PR #7516
- Restore the fp8_block/fp8_row identity assert alongside the new provenance
test. It is weak, not vacuous: it still catches a probe that hands back the
installed table objects, and it costs nothing to keep.
- Bind Version and transformers_version in the stub namespace. Both are
unreached under the current gates, so a change to either would fail with a
bare NameError instead of the assertion.
Merged main, which clears the unrelated test_runtime_text_encoding failure the
branch inherited from its base.
* Cover the FP8 row-scaling path instead of duplicating the block one
The two tests this PR originally added were already covered by
tests/test_new_mapper_fetched_fp8.py from #7497. An 8-mutant matrix over
loader_utils.py found nothing they caught that the existing file did not, so
they are dropped and test_new_mapper_no_global_leak.py goes back to main.
Two real gaps were open, both on the row branch that load_in_fp8 = True plus
UNSLOTH_HAS_FBGEMM selects ahead of block:
- the FBGEMM row branch in __get_model_name could be deleted outright with
every test still green
- _resolve_with_mappers could ignore its fp8_row argument and silently fall
back to the installed row table
Adds two tests to the existing file, reusing its _load_resolver rather than a
second harness. The row-only fixture splices into the fetched row table alone,
since an entry the block table also knows lets the block branch answer and
masks the regression.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
The two ubuntu2404 root legs went red at "Assert no source build" reporting
triton-kernels. That is not a regression in what the installer does. Those
builds have always happened; they only became visible now that pip_install
stopped discarding uv's output on success, which is what finally let the
nobuild check read the dependency phase at all.
So the question was whether each build actually needs a compiler. Checked
against the real artifacts rather than assumed:
openai-whisper 20250625, randomname 0.2.1, argbind 0.3.9 -- no version of
any of the three has ever published a wheel; antlr4-python3-runtime is
pinned at 4.9.3, below the first release that ships one. All four sdists
use setuptools.build_meta, declare no ext_modules, and contain no
.c/.cpp/.pyx/.rs file. Already allowlisted, correctly.
triton-kernels is the same category and was the only name failing. It is
pinned to the triton repo's python/triton_kernels subdirectory; that tree
is 75 files of Python, a four-line pyproject.toml, no setup.py and no
native source at all. The kernels are Triton DSL compiled at runtime, not
at install time. It is also a direct URL the installer names itself rather
than something resolution picked, and only Linux reaches it. It belongs in
the allowlist, so add it with that reasoning written down.
The allowlist match is now lowercased and underscore-folded on both sides.
The requirement spells the package triton_kernels while uv prints
triton-kernels, and an allowlist that matched only one spelling would pass
by luck rather than by intent. A plain pyarrow sdist is still caught.
The two data-designer @ file:// plugin builds needed nothing: they are
in-tree local paths, already dropped by the same rule that exempts the
source overlay's own build.
Separately, the windows-11-arm leg fails for a real reason and should keep
failing. The ARM handling itself works, the log shows torchaudio being
skipped and torch plus torchvision installing from wheels. What stops it is
that pyarrow and hf-transfer publish no win_arm64 wheel at all, so uv falls
back to their sdists and they fail on CMake configure and on openssl-sys
wanting perl. That is a product gap on the platform, not a gap in the
simulation, so the leg stays experimental and keeps reporting it. Record
that above the matrix entry so the next reader does not re-diagnose it.
* fix: add XPU device support and update hardcoded CUDA selections
* fix: add XPU device support for pytest CUDA skipped tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix device handling for PR #7401
- perplexity_eval.py: use DEVICE_TYPE_TORCH, not DEVICE_TYPE. The latter can
be "hip" or "mlx", which .to() rejects, so this regressed ROCm.
- test_batched_leftpad_generation_gpu.py: XPU diverges here today, so mark it
non-strict xfail on XPU instead of reverting to a CUDA-only guard. Keeps the
real XPU gap visible and turns green once it is fixed.
- Guard torch.xpu.is_available() with hasattr, matching device_type.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Re-enable the flash varlen attention test in CI for PR #7401
attention_dispatch.py now predefines flash_attn_func / flash_attn_varlen_func
as None, so test_run_attention_flash_varlen_receives_window_and_softcap no
longer needs flash_attn importable to be monkeypatched. Verified on a runner
shaped like the CPU-only one: the test fails against main's attention_dispatch
and passes at this head, so the deselect is now dead weight.
* Tighten comments for PR #7401
Drop the hasattr rationale: torch.xpu has existed since torch 2.3 and the
dependency floor is 2.4, so no supported build predates the namespace. The
guard stays as cheap defence, but the comment claimed something untrue.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
install.sh and install.ps1 come from the ref under test, but they install
unsloth from PyPI, which is the consumer path and has to stay that way. That
left everything Python-side coming out of the released wheel: studio/setup.sh,
studio/setup.ps1, studio/install_python_stack.py, and every requirements and
constraints file those resolve through Path(__file__). A branch that changes
constraints.txt or setup.ps1 therefore got a green run that proved nothing
about the change, and some legs proved less than they looked. The Fedora
assertion was already carrying a hand-written workaround for exactly this,
tolerating a triton/git failure on the grounds that the released package lags
the ref.
Legs marked overlay: true now re-point the venv at the ref just before studio
setup runs, through UNSLOTH_CI_SOURCE_OVERLAY: a --no-deps editable install of
the checkout. That makes import studio resolve to the working tree, so the
existing setup-script lookup finds the ref's setup.sh / setup.ps1 and
install_python_stack reads the ref's constraints, with no other change to
either installer.
Not --local: --local additionally installs unsloth-zoo from a git+https URL,
which genuinely needs git, and git absence is the whole point of the masked
legs. The overlay resolves no dependencies and clones nothing, so it holds up
with git, cmake and the compilers all gone. It is not a consumer knob either:
no flag, no usage entry, ignored unless the variable names a directory with a
pyproject.toml in it.
Four legs stay on the released package deliberately, each for its own reason,
recorded in the header: the mac pipe legs keep an end-to-end signal on what a
user actually runs; the trace leg would otherwise answer its own question,
since the editable build calls git through setuptools-scm's file finder; the
non-root Linux leg dies before a venv exists; and WSL only ever receives
install.sh, not a source tree.
Two supporting fixes the overlay depends on or exposes:
install_python_stack.py discarded uv's output whenever a step succeeded, so
the nobuild assertion, which reads the install log, could not see a source
build in the dependency phase at all. That is the phase that installs
studio.txt, where an sdist-only dependency actually turns up, and it reported
"built: none" regardless. It now echoes successful output under
UNSLOTH_VERBOSE, matching what install.sh's run_install_cmd already does.
nobuild now ignores "Building <name> @ file://" lines. A local-path build is
something the caller pointed at, never a dependency resolution chose, and
index dependencies always print <name>==<version>, so a real sdist from PyPI
is still caught, including one named unsloth.
Each overlaid leg also asserts it really was overlaid, so an unset variable
cannot quietly put the whole matrix back on the released wheel.
The appimage row invoked the extractor by bare filename, and a command word
with no slash is resolved through PATH rather than the working directory, so
the extraction exited 127 and the bundled-installer assertion below it never
ran. Prefix it with ./ so the row exercises what it claims to.
The Linux log step also asserted nothing: it skipped a missing log with
continue and discarded the grep with || true. The launch step only proves the
process stayed alive for 90 seconds, and the bundled-installer checks do not
exercise the Rust preflight path, so an app that hung before preflight
completed passed both required Linux rows. Require the same
desktop_preflight completed disposition= record the macOS rows already do.
All three Windows legs failed "Verify the simulation took effect" with no
::error:: printed at all. The check itself was right: the mask step logged
"masked toolcache python: C:\hostedtoolcache\windows\Python", python/git/cmake/cl
were ABSENT, no `py -3.x` probe started an interpreter, and the winget assertions
were satisfied. The step still exited 1.
The cause is $LASTEXITCODE leaking out of the step. The last external command is
the `py -3.13` probe, which is SUPPOSED to fail; Get-Command and Write-Host are
cmdlets and never reset $LASTEXITCODE, and the runner appends
`if ((Test-Path -LiteralPath variable:\LASTEXITCODE)) { exit $LASTEXITCODE }`
to every pwsh step (actions/runner#351). So a clean machine reported failure,
and because this step runs before Install, no Windows leg has ever reached the
installer. Clear $LASTEXITCODE after the probe loop and end with an explicit
exit 0. The leak detection is untouched: a surviving python/git/cmake/cl, or a
`py -3.x` that actually starts, still exits 1.
Also print each probe's exit code and output, so the next failure here explains
itself instead of being silent, and label `py -0p` as what it is. The launcher
reads the registry, which the on-disk toolcache rename cannot rewrite, so -0p
keeps naming paths that no longer exist. Unlabelled it reads like a leak.
Accept the Fedora leg's real outcome instead of a message that can be absent
The fedora assertion only accepted the unsupported-package-manager hard exit.
That is still what this ref's install.sh does, but the pending installer change
replaces it with a warning that lets the install continue, at which point the
old grep matches nothing and the step fails for the wrong reason.
Handle both, strictly. If the log shows the newer "using prebuilt llama.cpp
(missing:" warning, the Linux gate demonstrably did not hard-stop, and the only
tolerated failure past that point is release lag: install.sh comes from this ref
while unsloth comes from PyPI, and the released studio/install_python_stack.py
has no "skip triton kernels when git is missing" guard, so it still fetches the
git+https triton_kernels requirement on a machine with no git. Anything else
after that warning fails the step. Otherwise the old hard-exit message is still
required. A missing log, a bootstrap outage or any unrecognised failure all
remain errors, and the step retires to a plain success assertion once a release
ships the no-git skip.
The manifest-removal guard added in #7492 exits with a bare 'exit 1', so in
Tauri mode the installer never emits the [TAURI:ERROR] line and the desktop
UI falls back to a generic failure instead of naming the cause. Every other
failure path in studio/setup.ps1 goes through Exit-SetupFailure, and
tests/sh/test_tauri_retry_failure_context.sh asserts that invariant, so
'Repo tests (CPU)' has been red on main since that merge.
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Bound how many approvals may park, against the executor
#7455 landed parking, which is the right shape and supersedes what this branch
was carrying. It is unbounded, though, and the thing it is unbounded against is
not the GPU.
A run stopped on an approval prompt is blocked inside the to_thread(next, gen)
call that drives it, so it holds one of asyncio's default min(32, cpu + 4)
executor threads until the user answers. The slot cap used to bound that.
Parking hands the slot back, which admits another run that can park too, so the
ceiling became the wait line: 64 deep on a 1-slot backend. Long before that, the
executor is full and nothing else in the backend runs, including generation
steps for chats that already hold slots and the stream teardown that would clean
up after a disconnect.
The pool already permits `capacity` pending prompts, and each park adds one
more, so the budget is what the executor has left after the cap and a reserve of
4. On this machine (32 workers) --parallel 4 gets 8 parks and 20 free threads,
--parallel 24 gets 4 and 4, and --parallel 28 or higher gets none: there the
prompt keeps its slot and behaves exactly as it did before parking existed.
Counted process-wide rather than per queue. There is one executor, but a
per-queue budget is the same allowance again for every backend, and base_url
carries a fresh port on every model load, so a reload would mint a queue that
knows nothing about the approvals still parked on the old one. A reset clears it
too, or a leaked claim shrinks the budget for the life of the process.
park() reports whether it took the budget, and a refusal costs nothing to undo
because the slot never left its holder. The stream reads that answer rather than
recording a refused park as parked, which would make it skip the park for every
later approval in the same run even once the budget freed up.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Size the park budget from the executor's own CPU count
Two review findings, both real.
The budget read os.cpu_count(). 3.13 sizes ThreadPoolExecutor from
os.process_cpu_count(), which honours CPU affinity and cgroup quotas, and
asyncio's default executor is a plain ThreadPoolExecutor(), so a container
pinned to one core on a 64-core host got a 5-thread executor and a budget
computed from 64. The bound was then looser than no bound at all in exactly the
environment that can least afford it. It asks the same source the executor does,
and the test compares against a real ThreadPoolExecutor rather than restating
the formula, so it stays right on 3.12 as well.
The reserve was a flat 4, which on that same 5-thread executor left nothing to
budget and turned parking off entirely. Small hosts are where a chat most needs
to keep moving while another sits on a prompt. It scales now, and the ceiling
has a floor of two: a quarter of five is one, and one park cannot cover two
chats on prompts at once, which is what #7455's own two-approvals test needs.
Without that floor, that test fails on a one or two CPU runner. `spare` still
takes the budget to zero when the pool already fills the executor, so nothing
about a 32-worker machine changes: --parallel 4 still gets 8 parks, 24 gets 4,
28 gets none.
The two behavioural budget tests pin the worker count rather than reading it off
the runner, and the property test sweeps executor sizes from one CPU to 64
instead of asserting against whatever the host happens to have. The whole suite
passes with the CPU count faked to 1, 2 and 4, which is how both of these were
reproduced.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Size the park budget from every live backend, and free it on the answer
Two review findings, both real.
The budget was global but sized from one queue's capacity. A reload mints a
queue on a new port while the old one drains, so both are live, and prompts on
both park executor threads. Eight parks on an old 1-slot queue plus a new
24-slot backend is 32 threads on a 32-thread executor, with the new backend's
prompts refused and holding their slots, which is the state the reserve exists
to prevent. It sums the capacity of every backend still serving instead. Idle
queues are skipped: those are the ones the registry is about to evict, and they
are holding nothing.
The budget also outlived the wait it was paying for. unpark_async only dropped
it after reacquiring a slot, but the generator yields its post-approval event
first, so the executor thread is already back in the pool while the resume
queues. An approved chat waiting on a slot would refuse a different chat's park,
and that chat then keeps the slot the resumer is waiting for, so an unanswered
prompt strands chats that were already approved. The budget is released when the
prompt wait ends now, and the queue's parked count still runs until the slot is
back, which is what guards idle eviction and the resume ordering.
Both are separate counters on the lease as a result, and every exit from a park
drops the budget: unpark, unpark_async and release. That last one was the mutant
that came back missed, since a client disconnecting on a prompt releases
straight out of parked and would otherwise lose a budget slot for good.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the comments on the park budget
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The `notools` check reads an absence: it passes when the trace file contains no
compiler, git or brew invocation. A shim directory that never reached PATH
produces exactly the same empty file as an installer that touched nothing, so
the single leg carrying that assertion would stay green no matter what the
installer did. "Verify the simulation actually took effect" only ran for mask
mode, which left the trace leg with nothing checking its own instrumentation.
Call git explicitly after sourcing the environment and require it to appear in
the trace, then truncate the file so the self-test entry does not count against
the install. The call has to be explicit because macOS reaches _has_working_git
only under STUDIO_LOCAL_INSTALL (install.sh:2026), so no consumer leg on that
platform probes git on its own.
Re-run `absent` after the install on the masked macOS legs. It only ran
before, so an installer that quietly selected the Xcode CLT or installed a
compiler left the leg green while every later source build could succeed,
which is the one thing clean-machine-assert.sh says `absent` guards the whole
run against.
Fail the Windows simulation when py.exe can still start an interpreter. The
launcher binary itself may stay, but Find-CompatiblePython probes `py` first
(install.ps1:1130-1153), so an interpreter registered outside the two renamed
toolcache directories gets reused and Python bootstrap is never exercised.
Exempting `py` without ever running it left that unchecked.
Propagate the WSL installer exit code. It was printed and discarded, and the
CLI check does not compensate: install.sh links the `unsloth` shim (4174-4182)
before it reports a failing studio/setup.sh (4219-4230), so a late setup
failure leaves a shim whose --version succeeds.
Run the bundled installer in the Linux desktop jobs. The launch step only
proves the process stayed alive, and on a fresh home preflight reports
not_installed and the app waits on the install screen, so both required rows
passed after 90 seconds without ever touching the shipped install.sh. Locate
the resource in the deb payload or the extracted AppImage, run it the way
install.rs does, and require a managed venv that can import torch.
REL_REPO defaulted to unsloth-test/unsloth-test, which holds one release frozen
at 2026-07-27, while release-desktop.yml publishes into github.repository. The
schedule was re-testing the same fixture forever and could never see a broken
production bundle.
The windows job carried a blanket continue-on-error, so its NSIS assertions
could not gate. lipo -archs prints and exits 0 for a thin binary and `|| true`
swallowed even that, so the architecture was never checked; fall back to file,
which survives the CLT mask. And require the preflight disposition line rather
than the mere existence of tauri.log, which setup_logging creates at process
start regardless.
Compress the comment blocks across the clean-machine workflows and
scripts. The explanations of why each check is written the way it is
stay; the padding, restatement and duplication go.
No code or workflow logic changes.
