Comments only, no assertion logic, pins or leg definitions touched.
Reflowed every rationale block to denser wording and removed the
duplication that had built up across repeated steps: the desktop
workflow repeated the fork-PR skip, the desktop-v* tag resolution and
the restore-runner note once per platform, and the installer workflow
repeated its path-filter rationale in both the pull_request and push
blocks. Those now point at the first copy.
Every WHY is kept: why the masked legs avoid install.sh --local, what
UNSLOTH_CI_SOURCE_OVERLAY is for, why `absent` tests "must not work"
rather than command -v, why the .venv_t5_* sidecars are in the macho
scan scope, why the signature check is main-executables-only, why each
nobuild allowlist entry is a pure-Python sdist, why the WSL job gates
and what the pipe truncation was, and why the virgin container's
overlay=false row is still pinned.
Proved comments-only three ways: both workflow revisions parsed with
yaml.safe_load_all and every leaf walked (only `run:` scalars differ);
every changed bash body and .sh compared byte-for-byte after
`bash --pretty-print -n`; every changed pwsh body and .ps1 compared as
a token stream with Comment and NewLine tokens dropped. A negative
control injecting one non-comment line into each layer makes all of
them fail.
The macho check asserted a valid signature for every Mach-O under the studio
home, and failed the macos-15 mask/pipe leg on 29 files: lxml, charset_normalizer,
cygrpc, _upb, fontTools, caio, brotli and a bundled libportaudio.dylib. Those are
MH_BUNDLE and MH_DYLIB images dlopen'd into a process without library validation,
they ship unsigned in the wheels, and the same run had already installed and
imported them with the installer exiting 0.
Key the signature half off the Mach-O filetype and run it only on main
executables. Report an absent seal separately from one that fails to verify, and
capture codesign output instead of piping it into grep, which returned the
unsigned exit status through pipefail and called every unsigned binary broken.
The architecture half is unchanged and still a hard failure: it is what closes
the Rosetta 2 gap. The zero-Mach-O guard is unchanged. The .venv_t5_* sidecars
stay in scope; setup.sh creates them during a normal install and
transformers_version.py puts them on sys.path, so they are payload.
Assert arch and signature on every downloaded Mach-O. This is the one genuine
gap the simulation had: Rosetta 2 is preinstalled on hosted runners and absent
from a factory-fresh Mac, so an x86_64-only llama.cpp, whisper.cpp, Node or uv
payload runs green here and dies with "bad CPU type in executable" for the
user. llama-server launching under `assert-llama-loads.sh` does not rule that
out, because Rosetta makes it launch. The new `macho` check reads `file -b`
(`lipo` is an xcrun shim and is gone after masking, as the desktop lane already
notes) and keys the expected arch off `uname -m`, so macos-15-intel expects
x86_64. It also requires at least an ad-hoc signature on arm64, which closes
the AMFI "Killed: 9" class that uv has already been bitten by; the check is
skipped on x86_64, where unsigned code loads fine and so is not the same
defect. It fails when the scan finds nothing, since an empty scan reads exactly
like a clean one.
Make absence real rather than PATH-hidden. uv probes well-known interpreter
locations and the framework loader ignores PATH entirely, so hiding the
toolcache only hid it from `command -v`. Empty /usr/local (it EXISTS on a
factory-fresh Mac as a SIP-exempt firmlink, and is empty; it is /usr/local/bin
that is absent, so the directory itself stays), move the hosted toolcache and
/Library/Frameworks/Python.framework aside, and clear the developer dotdirs and
caches. A populated uv or pip cache can also satisfy a resolution that would
fail on a user's machine. Every removal goes through --remove and is recorded
in the generated restore.sh, guarded so a path the install recreated is not
buried inside its own restore.
Unset CI, GITHUB_* and RUNNER_* for the installer process only. An installer
branching on CI=true is a hidden dependency no consumer exercises. Scoped to
the child so the step's own $GITHUB_OUTPUT still resolves.
Record spctl --status and csrutil status. Neither is documented for these
images and both change what a binary is allowed to do.
The desktop workflow claims all three platforms are stripped, but only macOS
and Windows had a strip step and the Windows one scrubbed the process PATH
only. Both gaps let a bundle that needs a developer toolchain pass the one
workflow whose premise is that it must not.
Linux: the job ignored strip_toolchain entirely and ran the bundled install.sh
with the runner's git, gcc, cmake and make in /usr/bin. clean-machine-env.sh
now has a Linux --remove branch that moves the resolved tool binaries aside,
recorded in restore.sh, and the job calls it plus `assert absent` after the apt
step (the .deb install needs dpkg) and before the bundled installer, with a
restore step to match macOS. The loop repeats per tool so a name present in
both /usr/bin and /usr/local/bin is fully masked rather than half masked.
Windows: rewriting $env:PATH does not survive the bundled install.ps1, which
calls Refresh-SessionPath (318-337) and rebuilds $env:Path from the Machine and
User registry values, and py.exe in C:\Windows reaches the toolcache whatever
PATH says. Ported the on-disk toolcache rename, the Machine/User registry scrub
and the py -3.11/-3.12/-3.13 start probe from clean-machine-install-ci.yml, so
the strip is proven rather than assumed.
Windows preflight: the log step was Test-Path, Get-Content and Select-String,
none of which can fail, so an app that hangs before preflight passed on the
90 second liveness check alone. It now asserts a tauri.log exists and carries a
`desktop_preflight completed disposition=` line, the same unconstrained check
macOS and Linux already make. The disposition VALUE is deliberately not
constrained: ManagedReady over an unbootable venv is the reported bug.
installer_source on macOS: only the pipe delivery branched on it, so a
`published` dispatch ran the checked-out script on six of the eight macOS rows
while the run was labelled published. The script is now resolved once at the
top of the Install step and used by the file and tauri deliveries; pipe still
re-fetches through the live transport, because that is half of what it tests.
Linux, WSL and Windows already honoured the input.
Also shortened the comments across the changed files, keeping the reasoning
that says why each check exists.
The two ubuntu2404 root legs went red at "Assert no source build" reporting
triton-kernels. That is not a regression in what the installer does. Those
builds have always happened; they only became visible now that pip_install
stopped discarding uv's output on success, which is what finally let the
nobuild check read the dependency phase at all.
So the question was whether each build actually needs a compiler. Checked
against the real artifacts rather than assumed:
openai-whisper 20250625, randomname 0.2.1, argbind 0.3.9 -- no version of
any of the three has ever published a wheel; antlr4-python3-runtime is
pinned at 4.9.3, below the first release that ships one. All four sdists
use setuptools.build_meta, declare no ext_modules, and contain no
.c/.cpp/.pyx/.rs file. Already allowlisted, correctly.
triton-kernels is the same category and was the only name failing. It is
pinned to the triton repo's python/triton_kernels subdirectory; that tree
is 75 files of Python, a four-line pyproject.toml, no setup.py and no
native source at all. The kernels are Triton DSL compiled at runtime, not
at install time. It is also a direct URL the installer names itself rather
than something resolution picked, and only Linux reaches it. It belongs in
the allowlist, so add it with that reasoning written down.
The allowlist match is now lowercased and underscore-folded on both sides.
The requirement spells the package triton_kernels while uv prints
triton-kernels, and an allowlist that matched only one spelling would pass
by luck rather than by intent. A plain pyarrow sdist is still caught.
The two data-designer @ file:// plugin builds needed nothing: they are
in-tree local paths, already dropped by the same rule that exempts the
source overlay's own build.
Separately, the windows-11-arm leg fails for a real reason and should keep
failing. The ARM handling itself works, the log shows torchaudio being
skipped and torch plus torchvision installing from wheels. What stops it is
that pyarrow and hf-transfer publish no win_arm64 wheel at all, so uv falls
back to their sdists and they fail on CMake configure and on openssl-sys
wanting perl. That is a product gap on the platform, not a gap in the
simulation, so the leg stays experimental and keeps reporting it. Record
that above the matrix entry so the next reader does not re-diagnose it.
install.sh and install.ps1 come from the ref under test, but they install
unsloth from PyPI, which is the consumer path and has to stay that way. That
left everything Python-side coming out of the released wheel: studio/setup.sh,
studio/setup.ps1, studio/install_python_stack.py, and every requirements and
constraints file those resolve through Path(__file__). A branch that changes
constraints.txt or setup.ps1 therefore got a green run that proved nothing
about the change, and some legs proved less than they looked. The Fedora
assertion was already carrying a hand-written workaround for exactly this,
tolerating a triton/git failure on the grounds that the released package lags
the ref.
Legs marked overlay: true now re-point the venv at the ref just before studio
setup runs, through UNSLOTH_CI_SOURCE_OVERLAY: a --no-deps editable install of
the checkout. That makes import studio resolve to the working tree, so the
existing setup-script lookup finds the ref's setup.sh / setup.ps1 and
install_python_stack reads the ref's constraints, with no other change to
either installer.
Not --local: --local additionally installs unsloth-zoo from a git+https URL,
which genuinely needs git, and git absence is the whole point of the masked
legs. The overlay resolves no dependencies and clones nothing, so it holds up
with git, cmake and the compilers all gone. It is not a consumer knob either:
no flag, no usage entry, ignored unless the variable names a directory with a
pyproject.toml in it.
Four legs stay on the released package deliberately, each for its own reason,
recorded in the header: the mac pipe legs keep an end-to-end signal on what a
user actually runs; the trace leg would otherwise answer its own question,
since the editable build calls git through setuptools-scm's file finder; the
non-root Linux leg dies before a venv exists; and WSL only ever receives
install.sh, not a source tree.
Two supporting fixes the overlay depends on or exposes:
install_python_stack.py discarded uv's output whenever a step succeeded, so
the nobuild assertion, which reads the install log, could not see a source
build in the dependency phase at all. That is the phase that installs
studio.txt, where an sdist-only dependency actually turns up, and it reported
"built: none" regardless. It now echoes successful output under
UNSLOTH_VERBOSE, matching what install.sh's run_install_cmd already does.
nobuild now ignores "Building <name> @ file://" lines. A local-path build is
something the caller pointed at, never a dependency resolution chose, and
index dependencies always print <name>==<version>, so a real sdist from PyPI
is still caught, including one named unsloth.
Each overlaid leg also asserts it really was overlaid, so an unset variable
cannot quietly put the whole matrix back on the released wheel.
Compress the comment blocks across the clean-machine workflows and
scripts. The explanations of why each check is written the way it is
stay; the padding, restatement and duplication go.
No code or workflow logic changes.
The toolchain strip never ran on the automatic triggers: inputs exists only for
workflow_dispatch, and GitHub coerces '' and false alike to 0, so
`inputs.strip_toolchain != false` was false. Confirmed on a pull_request run
where the strip step reports skipped. Gate on the event instead.
Also: scrub the Machine and User registry PATH, since install.ps1 rebuilds
$env:Path from them mid-install and the toolchain came back; stop dropping
WindowsApps unconditionally, which removed winget on the winget=visible leg too;
fail rather than annotate when a bundle ships no installer or no CLI; run the
bundled installer, which a headless launch never reaches; resolve the newest
desktop-v* release instead of a pinned immutable tag; and give the two macOS
matrix rows distinct artifact names.
The tauri leg installs to the legacy root because --tauri refuses a custom
UNSLOTH_STUDIO_HOME. Its install succeeds end to end, but llama.cpp lives at
<root>/llama.cpp while the venv is at <root>/studio, so the assert was pointed one
level too deep.
On macos-15-intel /usr/bin/git keeps working once the CLT are gone, so it is not
CLT-provided there and no masking can remove it, while cc and clang do become
stubs. Calling that 'masking failed' was wrong. That leg allowlists git
explicitly and says why, so the assert stays strict everywhere else.
No job has ever run the installer on a machine without one.
studio-mac-install-matrix.yml is the only macOS installer job and it runs
'bash install.sh --local --no-torch' on runners that already have the Xcode CLT
selected and setup-python preinstalled, so the CLT gate never fires there, and
--local is precisely the mode that legitimately needs git. Repo-wide there was
zero coverage of xcode-select or CommandLineTools outside install.sh itself.
clean-machine-install-ci.yml runs the installer on a genuinely stripped machine.
macOS legs move /var/db/xcode_select_link, /Library/Developer/CommandLineTools,
/Applications/Xcode*.app and Homebrew aside, so xcode-select -p, git, cc and
clang really do fail, and restore unconditionally afterwards. Removing the
select-link alone is not enough: xcode-select falls through to a full Xcode.app
and re-arms /usr/bin/git. Linux legs use containers, which are genuinely clean.
Windows legs cover winget visible and masked, plus windows-11-arm. A WSL leg
covers the 126 lines of WSL-specific install.sh logic that had no runtime test.
Each macOS leg runs four deliveries: pipe (the advertised command, and the shape
that turns an early exit into curl (56)), file (separates installer logic from
pipe delivery), no-torch, and tauri (stdin closed, no tty, as the desktop app
invokes it). One leg records every toolchain invocation and asserts the trace,
which is the real deliverable: proof the installer never reached for a compiler
rather than proof it happened to succeed.
The asserts test that tools do NOT WORK rather than that they are absent from
PATH. On a real virgin Mac /usr/bin/git and /usr/bin/cc exist as CLT stubs, so
'command -v git' succeeds and only running it tells the truth.
desktop-app-clean-machine-ci.yml installs and launches the SHIPPED desktop app
release on a stripped machine, covering Gatekeeper and quarantine on macOS, NSIS
silent install on Windows, and Xvfb with WebKit2GTK on Linux.
Known limit, stated plainly: hosted macOS runners are developer machines. Masking
reproduces this bug and proves the installer does not invoke a toolchain, but it
cannot prove no hidden dependency exists on a truly virgin Mac. An ephemeral-VM
lane is the follow-up.
