ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Make the Linux and Windows desktop legs clean, and honour published on every macOS delivery

Browse source 
The desktop workflow claims all three platforms are stripped, but only macOS
and Windows had a strip step and the Windows one scrubbed the process PATH
only. Both gaps let a bundle that needs a developer toolchain pass the one
workflow whose premise is that it must not.

Linux: the job ignored strip_toolchain entirely and ran the bundled install.sh
with the runner's git, gcc, cmake and make in /usr/bin. clean-machine-env.sh
now has a Linux --remove branch that moves the resolved tool binaries aside,
recorded in restore.sh, and the job calls it plus `assert absent` after the apt
step (the .deb install needs dpkg) and before the bundled installer, with a
restore step to match macOS. The loop repeats per tool so a name present in
both /usr/bin and /usr/local/bin is fully masked rather than half masked.

Windows: rewriting $env:PATH does not survive the bundled install.ps1, which
calls Refresh-SessionPath (318-337) and rebuilds $env:Path from the Machine and
User registry values, and py.exe in C:\Windows reaches the toolcache whatever
PATH says. Ported the on-disk toolcache rename, the Machine/User registry scrub
and the py -3.11/-3.12/-3.13 start probe from clean-machine-install-ci.yml, so
the strip is proven rather than assumed.

Windows preflight: the log step was Test-Path, Get-Content and Select-String,
none of which can fail, so an app that hangs before preflight passed on the
90 second liveness check alone. It now asserts a tauri.log exists and carries a
`desktop_preflight completed disposition=` line, the same unconstrained check
macOS and Linux already make. The disposition VALUE is deliberately not
constrained: ManagedReady over an unbootable venv is the reported bug.

installer_source on macOS: only the pipe delivery branched on it, so a
`published` dispatch ran the checked-out script on six of the eight macOS rows
while the run was labelled published. The script is now resolved once at the
top of the Install step and used by the file and tauri deliveries; pipe still
re-fetches through the live transport, because that is half of what it tests.
Linux, WSL and Windows already honoured the input.

Also shortened the comments across the changed files, keeping the reasoning
that says why each check exists.
This commit is contained in:
danielhanchen 2026-07-28 23:54:57 +00:00
commit b905784f77
7 changed files with 455 additions and 383 deletions
Download patch file Download diff file Expand all files Collapse all files

78
.github/scripts/clean-machine-assert.sh vendored
View file

@ -4,14 +4,14 @@
#
# Assert the clean-machine contract after an install attempt.
#
# absent The toolchain really was absent for the whole run. Guards against a leg
# that "passed" only because masking silently failed, or because the
# installer quietly installed Xcode CLT behind our back.
# absent The toolchain really was absent for the whole run. Catches a leg that
# "passed" only because masking silently failed, or because the installer
# quietly installed Xcode CLT behind our back.
# notools The trace recorded no compiler/git/brew invocation (trace mode).
# nobuild The wheels-only contract: no "Building wheel" from pip, no
# "Building <pkg>==<ver>" from uv. Needs UNSLOTH_VERBOSE=1, else
# run_install_cmd (install.sh:193-243) discards the uv output on success
# and there is nothing here to read.
# "Building <pkg>==<ver>" from uv. Needs UNSLOTH_VERBOSE=1, or
# run_install_cmd (install.sh:193-243) discards uv's output on success
# and there is nothing to read.
#
# Usage: bash .github/scripts/clean-machine-assert.sh absent notools nobuild
set -uo pipefail
@ -28,8 +28,8 @@ for check in "$@"; do
absent)
# Deliberately NOT `command -v`: on a virgin Mac /usr/bin/{git,cc} EXIST as CLT
# stubs, so `command -v` succeeds and only RUNNING them fails ("xcrun: error:
# invalid active developer path"). The honest invariant is: must not WORK.
# stubs, so `command -v` succeeds and only RUNNING them fails ("invalid active
# developer path"). The honest invariant is: must not WORK.
if xcode-select -p >/dev/null 2>&1; then
fail "xcode-select -p still resolves to $(xcode-select -p 2>/dev/null); not a clean Mac"
else
@ -38,10 +38,9 @@ for check in "$@"; do
for tool in git cc clang cmake; do
command -v "$tool" >/dev/null 2>&1 || { ok "$tool not on PATH"; continue; }
if "$tool" --version >/dev/null 2>&1; then
# On Intel runners /usr/bin/git is not CLT-provided and keeps working once
# the CLT are gone, so no masking can remove it. cc and clang do become
# stubs and the macOS consumer path needs no git, so report rather than
# call the simulation broken.
# On Intel runners /usr/bin/git is not CLT-provided and survives their
# removal, so no masking can take it away. cc and clang do become stubs and
# the macOS consumer path needs no git, so report rather than fail.
case " ${UNSLOTH_CLEAN_ALLOW_WORKING:-} " in
*" $tool "*)
echo "[assert] NOTE $tool still works ($(command -v "$tool")); allowed on this runner"
@ -73,10 +72,9 @@ for check in "$@"; do
[ -n "$tool" ] || continue
case " $allow " in *" $tool "*) continue ;; esac
# `xcode-select -p` only ASKS whether a toolchain is selected; the installer
# has to ask, and the point of the fix is that it carries on without one.
# Counting the question as toolchain USE would fail the very leg that proves
# the toolchain was never used. `--install`, which pops the CLT installer,
# stays a hit.
# has to ask, and the fix is that it carries on without one. Counting the
# question as USE would fail the very leg proving the toolchain went
# untouched. `--install`, which pops the CLT installer, stays a hit.
if [ "$tool" = "xcode-select" ]; then
case "$rest" in
-p|--print-path|-v|--version|"") continue ;;
@ -94,44 +92,38 @@ for check in "$@"; do
;;
nobuild)
# "Built an sdist" is NOT "needed a compiler". Every name below was checked
# against its actual sdist: setuptools.build_meta backend, no ext_modules,
# and not one .c/.cpp/.pyx/.rs file in the archive, so the PEP 517 build is
# a pure-Python metadata-and-copy step that completes with no compiler.
# "Built an sdist" is NOT "needed a compiler", so the contract is "nothing
# needing a COMPILER was built". Every name below was checked against its
# actual sdist: setuptools.build_meta backend, no ext_modules, not one
# .c/.cpp/.pyx/.rs file, so its PEP 517 build is a pure-Python copy step.
# openai-whisper, argbind, randomname -- no version ever ships a wheel
# antlr4-python3-runtime==4.9.3 -- pinned below the 4.13.2 wheel
# triton-kernels -- studio/backend/requirements/
# triton-kernels.txt pins it to a git URL under the triton repo's
# python/triton_kernels subdirectory. That tree is 75 files of Python
# with a four-line pyproject.toml and no setup.py; the kernels are
# Triton DSL compiled at runtime, never at install time. It is also a
# direct URL the installer names itself, not something resolution
# chose, and only the Linux legs reach it (install_python_stack.py
# skips the step on Windows and macOS).
# Failing on those is a false alarm, so the contract is "nothing needing a
# COMPILER was built". UNSLOTH_ALLOW_SDIST extends the allowlist.
# triton-kernels -- requirements/triton-kernels.txt pins a git URL under
# the triton repo's python/triton_kernels subdirectory: 75 Python files,
# a four-line pyproject.toml, no setup.py, kernels compiled at runtime.
# A direct URL the installer names itself, not something resolution
# chose, and only the Linux legs reach it (install_python_stack.py skips
# the step on Windows and macOS).
# UNSLOTH_ALLOW_SDIST extends the allowlist.
#
# Lowercased and underscore-folded on both sides, because a project's
# distribution name and the name uv prints can disagree on the separator:
# the requirement says triton_kernels, the build line says triton-kernels,
# and an allowlist that matched only one spelling would silently miss.
# Lowercased and underscore-folded on both sides: a distribution name and the
# name uv prints can disagree on the separator (requirement triton_kernels vs
# build line triton-kernels), and a one-spelling allowlist silently misses.
_allow="$(printf '%s' "openai-whisper argbind randomname antlr4-python3-runtime triton-kernels ${UNSLOTH_ALLOW_SDIST:-}" | tr 'A-Z_' 'a-z-')"
if [ ! -f "$LOG" ]; then
fail "nobuild requested but $LOG is missing"
else
# uv does NOT use pip's phrasing: it prints `Building <name>==<version>` to
# stderr (astral-sh/uv#11165), so the pip-only pattern left _built empty on
# every uv source build. Match both spellings. Requiring `==` or ` @ ` after
# the name keeps this off the installer's own lowercase "building frontend..."
# every uv source build. Match both. Requiring `==` or ` @ ` after the name
# keeps this off the installer's own lowercase "building frontend..."
# progress text. Strip ANSI first so a coloured run (FORCE_COLOR) parses.
#
# `Building <name> @ file://...` is dropped before the names are read: a
# local-path build is something the caller pointed at (install.sh --local,
# or the UNSLOTH_CI_SOURCE_OVERLAY editable overlay the CI legs use to put
# the branch's Python code under test), never a dependency that resolution
# chose. Dependencies from an index always print `<name>==<version>`, so
# this drops no real signal -- a genuine sdist pulled from PyPI is still
# caught, including one named unsloth.
# `Building <name> @ file://...` is dropped first: a local-path build is
# something the caller pointed at (install.sh --local, or the
# UNSLOTH_CI_SOURCE_OVERLAY editable overlay), never a dependency resolution
# chose. Index dependencies always print `<name>==<version>`, so no signal is
# lost: a genuine sdist from PyPI is still caught, including one named unsloth.
_esc=$(printf '\033')
_built="$(sed -E "s/${_esc}\[[0-9;]*[A-Za-z]//g" "$LOG" 2>/dev/null \
| grep -viE "building [a-z0-9._-]+ @ file://" \

49
.github/scripts/clean-machine-env.sh vendored
View file

@ -3,16 +3,16 @@
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
#
# Simulate a virgin developer machine on a GitHub-hosted runner. Two modes, because
# "the tool is absent" and "the installer never called the tool" cannot be simulated
# by the same mechanism:
# "the tool is absent" and "the installer never called the tool" need different
# mechanisms:
#
# mask Make the toolchain genuinely ABSENT: scrub PATH to OS defaults and (with
# --remove) move the real toolchain aside, so `command -v git` correctly
# FAILS, as on a clean Mac. A failing "poison shim" would do the opposite --
# `command -v` finds it and reports the tool as present -- so no shims here.
# trace Leave the toolchain working but route it through logging wrappers that log
# the call then exec the real binary, proving whether the installer ever
# REACHES for a compiler/git without changing behaviour.
# --remove) move the real toolchain aside so `command -v git` correctly
# FAILS. Deliberately no "poison shims": a failing shim is still FOUND by
# `command -v`, which reports the tool as present, the opposite of clean.
# trace Leave the toolchain working behind wrappers that log the call then exec
# the real binary, answering whether the installer ever REACHES for a
# compiler/git without changing behaviour.
#
# Writes shell exports to $CLEAN_ENV_FILE (default ./clean-machine.env) to `source`;
# nothing is exported globally, so other steps keep a normal environment.
@ -81,10 +81,10 @@ if [ "$MODE" = "mask" ]; then
} >> "$ENV_FILE"
if [ "$REMOVE" = "1" ] && [ "$OS" = "Darwin" ]; then
# Best-effort real removal; each step is independent and recorded in restore.sh
# so an `if: always()` step can put the runner back. xcode_select_link is exactly
# what `xcode-select -p` reads, so removing it reproduces a virgin Mac's gate.
# `xcode-select --reset` is NOT enough: it can reselect a full Xcode.app.
# Best effort, each step independent and recorded in restore.sh so an
# `if: always()` step can put the runner back. xcode_select_link is what
# `xcode-select -p` reads, so removing it reproduces a virgin Mac's gate;
# `xcode-select --reset` is NOT enough, it can reselect a full Xcode.app.
if [ -e /var/db/xcode_select_link ]; then
if sudo rm -f /var/db/xcode_select_link 2>/dev/null; then
note "removed /var/db/xcode_select_link"
@ -106,8 +106,7 @@ if [ "$MODE" = "mask" ]; then
# Xcode.app must go too: with the link removed AND CommandLineTools moved,
# `xcode-select -p` still does not fail, it falls through to the image's Xcode
# bundle (observed: /Applications/Xcode_16.4.app/Contents/Developer), which
# re-arms /usr/bin/{git,cc} and silently un-cleans the machine. A rename is
# instant regardless of bundle size: same filesystem, no copy.
# re-arms /usr/bin/{git,cc}. A rename is instant whatever the bundle size.
for app in /Applications/Xcode*.app; do
[ -d "$app" ] || continue
if sudo mv "$app" "${app}.masked" 2>/dev/null; then
@ -128,6 +127,28 @@ if [ "$MODE" = "mask" ]; then
fi
done
fi
if [ "$REMOVE" = "1" ] && [ "$OS" = "Linux" ]; then
# A hosted Linux runner keeps git, gcc, cmake and make in /usr/bin, which the PATH
# scrub has to keep, so absence must be made real: move the resolved binaries
# aside (recorded in restore.sh). Versioned siblings like gcc-11 survive, but a
# consumer install invokes the unsuffixed names, which is what `absent` checks.
for tool in $TOOLS; do
# Repeat per tool: a runner can carry the same name in /usr/bin and
# /usr/local/bin, and moving only the first leaves the second on PATH.
for _ in 1 2 3 4; do
real="$(command -v "$tool" 2>/dev/null || true)"
[ -n "$real" ] && [ -e "$real" ] || break
if sudo mv "$real" "$real.masked" 2>/dev/null; then
note "moved $real aside"
echo "sudo mv '$real.masked' '$real' 2>/dev/null || true" >> "$RESTORE"
else
note "WARN could not move $real"
break
fi
done
done
fi
fi
# ── trace ─────────────────────────────────────────────────────────────────────