From 83c3dba5abfb23b8d985ea645e1398fa357b093d Mon Sep 17 00:00:00 2001 From: danielhanchen Date: Wed, 29 Jul 2026 01:56:34 +0000 Subject: [PATCH] Close the free headroom in the clean-machine simulation Assert arch and signature on every downloaded Mach-O. This is the one genuine gap the simulation had: Rosetta 2 is preinstalled on hosted runners and absent from a factory-fresh Mac, so an x86_64-only llama.cpp, whisper.cpp, Node or uv payload runs green here and dies with "bad CPU type in executable" for the user. llama-server launching under `assert-llama-loads.sh` does not rule that out, because Rosetta makes it launch. The new `macho` check reads `file -b` (`lipo` is an xcrun shim and is gone after masking, as the desktop lane already notes) and keys the expected arch off `uname -m`, so macos-15-intel expects x86_64. It also requires at least an ad-hoc signature on arm64, which closes the AMFI "Killed: 9" class that uv has already been bitten by; the check is skipped on x86_64, where unsigned code loads fine and so is not the same defect. It fails when the scan finds nothing, since an empty scan reads exactly like a clean one. Make absence real rather than PATH-hidden. uv probes well-known interpreter locations and the framework loader ignores PATH entirely, so hiding the toolcache only hid it from `command -v`. Empty /usr/local (it EXISTS on a factory-fresh Mac as a SIP-exempt firmlink, and is empty; it is /usr/local/bin that is absent, so the directory itself stays), move the hosted toolcache and /Library/Frameworks/Python.framework aside, and clear the developer dotdirs and caches. A populated uv or pip cache can also satisfy a resolution that would fail on a user's machine. Every removal goes through --remove and is recorded in the generated restore.sh, guarded so a path the install recreated is not buried inside its own restore. Unset CI, GITHUB_* and RUNNER_* for the installer process only. An installer branching on CI=true is a hidden dependency no consumer exercises. Scoped to the child so the step's own $GITHUB_OUTPUT still resolves. Record spctl --status and csrutil status. Neither is documented for these images and both change what a binary is allowed to do. --- .github/scripts/clean-machine-assert.sh | 49 ++++++++++++++++++- .github/scripts/clean-machine-env.sh | 45 +++++++++++++++++ .../workflows/clean-machine-install-ci.yml | 29 ++++++++--- 3 files changed, 116 insertions(+), 7 deletions(-) diff --git a/.github/scripts/clean-machine-assert.sh b/.github/scripts/clean-machine-assert.sh index d08ba5764b..4fb2b5291f 100755 --- a/.github/scripts/clean-machine-assert.sh +++ b/.github/scripts/clean-machine-assert.sh @@ -12,8 +12,10 @@ # "Building ==" from uv. Needs UNSLOTH_VERBOSE=1, or # run_install_cmd (install.sh:193-243) discards uv's output on success # and there is nothing to read. +# macho Every Mach-O under $MACHO_ROOT is the host architecture and is signed. +# Closes the Rosetta 2 gap, the one divergence masking cannot reproduce. # -# Usage: bash .github/scripts/clean-machine-assert.sh absent notools nobuild +# Usage: bash .github/scripts/clean-machine-assert.sh absent notools nobuild macho set -uo pipefail LOG="${INSTALL_LOG:-logs/install.log}" @@ -151,6 +153,51 @@ for check in "$@"; do fi ;; + macho) + # The one thing masking cannot reproduce: Rosetta 2 is preinstalled on hosted + # runners and absent from a factory-fresh Mac, so an x86_64-only payload runs + # green here and dies with "bad CPU type in executable" for the user. Assert the + # architecture rather than hope the runner lacks Rosetta. + # `lipo` is an xcrun shim and is gone after masking, so read `file -b`, exactly + # as the desktop lane does. Keyed off `uname -m`, since macos-15-intel is x86_64. + root="${MACHO_ROOT:-${UNSLOTH_STUDIO_HOME:-$HOME/.unsloth}}" + want="$(uname -m)" + [ "$want" = "aarch64" ] && want=arm64 + if [ ! -d "$root" ]; then + fail "macho requested but $root does not exist" + else + n=0 bad_arch="" unsigned="" + while IFS= read -r f; do + desc="$(file -b "$f" 2>/dev/null || true)" + case "$desc" in *Mach-O*) ;; *) continue ;; esac + n=$((n + 1)) + # Substring, not equality: a universal binary lists every slice it carries, + # and one that includes the host arch is fine. + case "$desc" in + *"$want"*) ;; + *) bad_arch="$bad_arch $f [$desc]" ;; + esac + # arm64 only: AMFI SIGKILLs unsigned code there ("Killed: 9"), while x86_64 + # loads it happily, so an unsigned x86_64 payload is not the same defect. + # Ad-hoc is enough, which is what the linker emits by default. + if [ "$want" = "arm64" ] && ! codesign -v "$f" >/dev/null 2>&1; then + unsigned="$unsigned $f" + fi + done < <(find "$root" -type f \( -perm -u+x -o -name '*.dylib' -o -name '*.so' -o -name '*.node' \) 2>/dev/null) + if [ "$n" = "0" ]; then + # An empty scan reads exactly like a clean one, so the check would pass on a + # wrong root and prove nothing. + fail "no Mach-O found under $root; the arch/signature assertion proved nothing" + elif [ -n "$bad_arch" ]; then + fail "Mach-O is not $want, so it runs here only under Rosetta 2, which a fresh Mac does not have:$bad_arch" + elif [ -n "$unsigned" ]; then + fail "unsigned Mach-O, which AMFI kills on arm64:$unsigned" + else + ok "$n Mach-O files under $root are $want$([ "$want" = arm64 ] && echo ' and signed')" + fi + fi + ;; + *) fail "unknown check '$check'" ;; diff --git a/.github/scripts/clean-machine-env.sh b/.github/scripts/clean-machine-env.sh index 138ba922d0..435c7f26a9 100755 --- a/.github/scripts/clean-machine-env.sh +++ b/.github/scripts/clean-machine-env.sh @@ -49,6 +49,22 @@ TOOLS="xcode-select xcrun clang clang++ cc c++ gcc g++ git cmake make brew ninja note() { echo "[clean-machine] $*"; } +# Move a path aside and record the reverse in restore.sh. PATH scrubbing only HIDES +# these; uv, the py launcher and framework lookups find them regardless, so absence +# has to be real. The restore line is guarded: the install may have recreated the +# path, and an unguarded `mv` would bury the original inside it. +mask_aside() { + local src="$1" dst="${2:-$1.masked}" as="" + [ -e "$src" ] || return 0 + [ -w "$(dirname "$src")" ] || as="sudo" + if $as mv "$src" "$dst" 2>/dev/null; then + note "moved $src aside" + printf "[ -e '%s' ] || %s mv '%s' '%s' 2>/dev/null || true\n" "$src" "$as" "$dst" "$src" >> "$RESTORE" + else + note "WARN could not move $src" + fi +} + # ── PATH scrub ──────────────────────────────────────────────────────────────── # Keep only OS-default system dirs: drops Homebrew, the hosted Python toolcache, # setup-* shims, pipx, cargo and every other preinstalled developer dir. @@ -116,6 +132,35 @@ if [ "$MODE" = "mask" ]; then note "WARN could not move $app" fi done + # /usr/local EXISTS on a factory-fresh Mac: a SIP-exempt firmlink, and empty. What + # is absent is its CONTENTS, /usr/local/bin included. So empty it rather than + # remove it. Runs before the Homebrew block below so /usr/local/Homebrew is stashed + # once, with one restore line, in the right order. + if [ -d /usr/local ]; then + STASH="$WORK/usr-local" + mkdir -p "$STASH" + for entry in /usr/local/* /usr/local/.[!.]*; do + [ -e "$entry" ] || continue + base="$(basename "$entry")" + if sudo mv "$entry" "$STASH/$base" 2>/dev/null; then + note "emptied /usr/local/$base" + printf "[ -e '/usr/local/%s' ] || sudo mv '%s/%s' '/usr/local/%s' 2>/dev/null || true\n" \ + "$base" "$STASH" "$base" "$base" >> "$RESTORE" + else + note "WARN could not move $entry" + fi + done + fi + # The hosted toolcache and the python.org framework are what a PATH scrub cannot + # reach: uv discovers interpreters by probing well-known locations. + mask_aside "${AGENT_TOOLSDIRECTORY:-$HOME/hostedtoolcache}" + mask_aside /Library/Frameworks/Python.framework + # Developer dotdirs and caches. A virgin $HOME has none of these, and a populated + # uv/pip cache can satisfy a resolution that would fail on a user's machine. + for d in .cargo .rustup .nvm .rbenv .pyenv .local .cache \ + Library/Caches/uv Library/Caches/pip Library/Caches/Homebrew; do + mask_aside "$HOME/$d" + done for brewdir in /opt/homebrew /usr/local/Homebrew; do if [ -d "$brewdir" ]; then if sudo mv "$brewdir" "${brewdir}.masked" 2>/dev/null; then diff --git a/.github/workflows/clean-machine-install-ci.yml b/.github/workflows/clean-machine-install-ci.yml index 961a90bb1c..c7029ba54f 100644 --- a/.github/workflows/clean-machine-install-ci.yml +++ b/.github/workflows/clean-machine-install-ci.yml @@ -155,6 +155,10 @@ jobs: echo "brew : $(command -v brew || echo none)" echo "cmake : $(command -v cmake || echo none)" echo "python3 : $(command -v python3 || echo none)" + # Neither is documented for these images, and both change what a binary is + # allowed to do. One line settles it for anyone reading the artifact. + echo "spctl --status : $(spctl --status 2>&1 || true)" + echo "csrutil status : $(csrutil status 2>&1 || true)" } | tee runner-baseline.txt - name: Simulate a clean machine (${{ matrix.mode }}) @@ -208,6 +212,15 @@ jobs: set -o pipefail rc=0 FLAGS="${{ matrix.flags }}" + # A consumer has no CI=true, no GITHUB_*, no RUNNER_*: an installer branching + # on any of them is a hidden dependency nobody outside CI exercises. Scoped to + # the installer's own process, so $GITHUB_OUTPUT below still resolves. `case` + # rather than `sed`, whose BRE has no \| alternation on macOS. + CLEAN_ENV="" + for v in $(env | cut -d= -f1); do + case "$v" in CI|GITHUB_*|RUNNER_*) CLEAN_ENV="$CLEAN_ENV -u $v" ;; esac + done + echo "unset for the installer:$CLEAN_ENV" # A `published` dispatch asks whether unsloth.ai's script works. Only `pipe` # honoured it, so six of the eight macOS rows ran the checked-out script and # were still reported as published coverage. Resolve it once, here, for every @@ -224,7 +237,7 @@ jobs: file) # Plain file execution isolates "installer logic broken" from # "curl-pipe delivery broken". - bash "$SCRIPT" $FLAGS 2>&1 | tee logs/install.log || rc=$? + env $CLEAN_ENV bash "$SCRIPT" $FLAGS 2>&1 | tee logs/install.log || rc=$? ;; pipe) # The shape users actually run. install.sh is ~150KB of top-level @@ -234,14 +247,14 @@ jobs: # re-fetches rather than piping $SCRIPT: the live transport is half of # what this delivery tests. if [ "${{ inputs.installer_source }}" = "published" ]; then - curl -fsSL https://unsloth.ai/install.sh | sh -s -- $FLAGS 2>&1 | tee logs/install.log || rc=$? + curl -fsSL https://unsloth.ai/install.sh | env $CLEAN_ENV sh -s -- $FLAGS 2>&1 | tee logs/install.log || rc=$? else # `sh -s --` with no further args would pass an empty positional, # so only add the separator when there are flags to pass. if [ -n "$FLAGS" ]; then - cat install.sh | sh -s -- $FLAGS 2>&1 | tee logs/install.log || rc=$? + cat install.sh | env $CLEAN_ENV sh -s -- $FLAGS 2>&1 | tee logs/install.log || rc=$? else - cat install.sh | sh 2>&1 | tee logs/install.log || rc=$? + cat install.sh | env $CLEAN_ENV sh 2>&1 | tee logs/install.log || rc=$? fi fi ;; @@ -252,7 +265,7 @@ jobs: # every other leg relies on must be dropped or the installer exits # before doing any work. The runner is ephemeral, so the real home is # as disposable as the override. - env -u UNSLOTH_STUDIO_HOME \ + env -u UNSLOTH_STUDIO_HOME $CLEAN_ENV \ bash "$SCRIPT" --tauri $FLAGS < /dev/null 2>&1 | tee logs/install.log || rc=$? ;; esac @@ -291,7 +304,7 @@ jobs: UNSLOTH_CLEAN_ALLOW_WORKING='${{ matrix.allow_working }}' \ bash .github/scripts/clean-machine-assert.sh $checks - - name: Assert llama.cpp loads + - name: Assert llama.cpp loads, and every downloaded Mach-O is native and signed if: steps.install.outcome == 'success' run: | set -a; . ./clean-machine.env; set +a @@ -304,6 +317,10 @@ jobs: HOME_DIR="$UNSLOTH_STUDIO_HOME" fi STUDIO_HOME="$HOME_DIR" bash .github/scripts/assert-llama-loads.sh + # Rosetta 2 is on this runner and not on a fresh Mac, so llama-server + # launching above does not prove it would launch for a user. Assert the arch + # of every payload (llama.cpp, whisper.cpp, the Node prebuilt, uv) instead. + MACHO_ROOT="$HOME_DIR" bash .github/scripts/clean-machine-assert.sh macho - name: Restore the runner if: always()