unsloth/tests/sh/test_macos_clt_gate.sh
Daniel Han df63522369
Installer: stop requiring a developer toolchain on the consumer path (#7547)
* Installer: stop requiring a developer toolchain on the consumer path

A brand new Mac cannot install Studio at all. install.sh gates on
`xcode-select -p` and exits 1 with 'Xcode Command Line Tools are required',
and Linux exits 1 on any non-apt distro over cmake/gcc/git/libcurl headers.

Nothing under either gate needs a toolchain. uv is a prebuilt binary, CPython
comes from uv's managed python-build-standalone, llama.cpp and whisper.cpp are
prebuilt downloads, Node is a pinned nodejs.org archive, and triton is skipped on
macOS. unslothai/llama.cpp b10107-mix-1911198 publishes macos-arm64, macos-x64,
linux-x64 and linux-arm64 builds covering cpu, cuda12, cuda13, rocm and vulkan.
PR #6617 already dropped the Homebrew/cmake stop on macOS for this reason and
just left the CLT stop behind.

macOS: warn and continue when the CLT are absent. Linux: only a download
transport (curl or wget) is fatal; build tooling warns. Both keep a hard git
requirement for --local, which installs unsloth-zoo from a git+https URL.

Both gates move into functions so tests/sh can extract them. The old inline form
could not be reached by the tests/sh convention, which is why this shipped broken
and stayed broken. test_macos_clt_gate.sh (19 assertions) and
test_linux_deps_gate.sh (25) cover the clean machine, the CLT-stub shape where
/usr/bin/git exists but fails, the non-apt distro, and the --local paths.

Writing the Linux test caught a latent bug: the gate trimmed its list with
$(echo ... | sed ...), so on a minimal image without sed the substitution yields
empty and it reports 'all system dependencies found' on a machine with none of
them. Replaced with parameter expansion.

Also caps av<16 in the single-env constraints. av 16+ ships no cp313 macOS arm64
wheel, and it is a C extension over FFmpeg, so uv would silently fall back to a
source build needing both a compiler and FFmpeg headers.

Verified on GitHub-hosted macOS runners with /var/db/xcode_select_link,
/Library/Developer/CommandLineTools, /Applications/Xcode*.app and Homebrew moved
aside. macos-14, macos-15 and macos-26 fail on main and install cleanly with
this; the recorded tool-invocation trace for the whole install is a single
`xcode-select -p`, so nothing compiled and nothing installed a toolchain.

* Linux: auto-install git rather than dropping it, and skip triton kernels without it

Making git optional on Linux was too broad. studio/backend/requirements/
triton-kernels.txt line 2 is a git+https URL, so step 6/14 died with 'Cannot find
command git' and failed the whole setup on ubuntu2404-root, ubuntu2404-arm-root
and fedora41, all of which had been passing. The claim that nothing on the
consumer path needs git holds on macOS, where triton is skipped, but not here.

install.sh now auto-installs git through apt with the other optional tooling, so
Debian and Ubuntu are unchanged. The triton kernels step skips with a message
when git is absent instead of failing: they are a training speedup, not a boot
requirement, and a GGUF chat install has no use for them.

Six more assertions pin both halves.

* macOS Intel: skip the one package with no x86_64 wheel

The Intel clean-machine leg installed with the toolchain masked, then died in
studio setup:

    subprocess.CalledProcessError: Command '['cmake', ...]' returned non-zero
    ERROR: Failed building wheel for pytorch_tokenizers

pytorch_tokenizers publishes wheels for macOS arm64, linux x86_64, linux aarch64
and windows, but none for macOS x86_64 at any Python version, so uv falls back to
an sdist that shells out to cmake. Nothing passes --only-binary, so the
compiler-free property was an assumption rather than a contract, and Intel is
where it broke.

Marked so it installs everywhere except Intel macOS. Apple Silicon is unaffected.

* Stop the optional dep gate from aborting the install

_smart_apt_install exits rather than returns, and `|| true` does not catch an
exit, so a box missing cmake or git aborted at the gate added to let it
continue. Verified in sh, dash and bash. Run it in a subshell and re-raise only
code 2, the NEED_SUDO handshake install.rs answers with an elevation prompt.

install.sh treats a present-but-broken git as missing, but the Python side
tested only shutil.which, so it promised to skip the git+https triton
requirement and then fetched it anyway. Same check on both sides now.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Never elevate for optional build tools

Re-raising code 2 turned the optional set into a NEED_SUDO handshake, so a box
missing cmake or gcc got the desktop's mandatory permission dialog, whose Cancel
drops back to not-installed. That re-imposes through a prompt the build-tool
requirement this gate removes, and none of those tools are needed to run.
Suppress the handshake for optional callers; a required package still elevates.
Verified in sh, dash and bash.

Also advance the progress bar on the no-git triton skip, which otherwise ends at
14/15.

* Tighten the comments on the dependency gate

* Correct why the PyAV cap is needed

16.0.0 does ship cp313-cp313-macosx_14_0_arm64; the comment claimed no cp313
wheel exists. The actual reason is the deployment target: 15.1.0 is macosx_13_0
and 16+ is macosx_14_0, so the cap is what keeps macOS 13 off a source build.

* Tighten the installer gate comments

* Cap cryptography on x86_64 macOS so the consumer install needs no Rust

cryptography 49.0.0 (2026-06-12) dropped the macosx_10_9_universal2 wheel
and now ships macosx_11_0_arm64 only, so x86_64 macOS has no wheel and uv
falls back to the sdist. That build calls maturin, which pulls Rust and
then fails at 'linking with cc failed' on a clean Mac without the Xcode
Command Line Tools. It surfaced in the clean-machine leg mac macos-15-intel
/ mask / file, several minutes into the studio dependency step, which is
exactly the up-front toolchain requirement this branch removes.

48.0.1 is the newest release carrying a universal2 wheel, and its
cp39-abi3 / cp311-abi3 tags cover the 3.12 and 3.13 interpreters the
installer creates. The cap is marker-scoped to darwin + x86_64, so arm64
macOS and every other platform still resolve to the latest. Lift it when
cryptography ships an x86_64-capable macOS wheel again.

Resolution of studio/backend/requirements/studio.txt under this
constraints file gives 48.0.1 on x86_64-apple-darwin and 49.0.0 on
aarch64-apple-darwin and x86_64-unknown-linux-gnu, on both 3.12 and 3.13.

* Correct the av note now that cryptography also compiles on macOS

* Never escalate for optional apt packages outside Tauri mode

The optional bypass sat inside the TAURI_MODE branch, so a plain curl | sh
install on a non-root Debian or Ubuntu box still fell through to the
escalation branch and showed the default-yes permission prompt for cmake,
GCC and the libcurl headers. That is exactly the toolchain this change set
declared unnecessary on the consumer path, so the prompt asked for a
password to install packages nothing here uses, and a headless run failed
the same way instead of falling through to prebuilt llama.cpp.

Move the check above the mode split so optional callers return 2 in both
modes. Required packages such as curl still escalate unchanged.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 18:50:38 -07:00

165 lines
6.3 KiB
Bash
Executable file

#!/bin/bash
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
#
# Guards the macOS system-dependency gate in install.sh.
#
# History: the gate was inline top-level code running
# xcode-select -p || { xcode-select --install; exit 1; }
# so a brand-new Mac could not install at all, and being inline rather than a function
# it was out of reach of the tests/sh sed-extraction convention that would have caught
# it.
#
# The contract now: a consumer install must SUCCEED with no Xcode Command Line Tools
# (uv, CPython, llama.cpp/whisper.cpp/Node are all prebuilt, triton is skipped on
# macOS), while `--local` must still fail loudly: unsloth-zoo comes from a git+https
# URL.
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
INSTALL_SH="$SCRIPT_DIR/../../install.sh"
PASS=0
FAIL=0
assert_eq() {
_label="$1"; _expected="$2"; _actual="$3"
if [ "$_actual" = "$_expected" ]; then
echo " PASS: $_label"
PASS=$((PASS + 1))
else
echo " FAIL: $_label (expected '$_expected', got '$_actual')"
FAIL=$((FAIL + 1))
fi
}
assert_contains() {
_label="$1"; _haystack="$2"; _needle="$3"
if echo "$_haystack" | grep -qF "$_needle"; then
echo " PASS: $_label"
PASS=$((PASS + 1))
else
echo " FAIL: $_label (expected to find '$_needle')"
FAIL=$((FAIL + 1))
fi
}
assert_not_contains() {
_label="$1"; _haystack="$2"; _needle="$3"
if echo "$_haystack" | grep -qF "$_needle"; then
echo " FAIL: $_label (found '$_needle' but should not)"
FAIL=$((FAIL + 1))
else
echo " PASS: $_label"
PASS=$((PASS + 1))
fi
}
# ── Extract the functions under test ──
_FN_FILE=$(mktemp)
sed -n '/^_has_working_git()/,/^}/p' "$INSTALL_SH" > "$_FN_FILE"
sed -n '/^_check_macos_deps()/,/^}/p' "$INSTALL_SH" >> "$_FN_FILE"
if ! grep -q '_check_macos_deps()' "$_FN_FILE"; then
echo "FAIL: could not extract _check_macos_deps from install.sh"
echo " (the gate must stay a top-level function so this test can reach it)"
exit 1
fi
# Minimal harness: the output helpers install.sh would otherwise provide.
_HARNESS=$(mktemp)
cat > "$_HARNESS" <<'HARNESS'
C_WARN=''; C_ERR=''; C_OK=''; C_DIM=''; C_RST=''
step() { echo "STEP $1 $2"; }
substep() { echo "SUBSTEP $1"; }
tauri_log() { echo "[TAURI:$1] $2"; }
HARNESS
_BIN=$(mktemp -d)
# Each tool is absent, a working stub, or a broken stub mimicking the Xcode CLT shim
# (exists, exits non-zero).
_mk() { printf '#!/bin/sh\n%s\n' "$2" > "$_BIN/$1"; chmod +x "$_BIN/$1"; }
# PATH is the sandbox and ONLY the sandbox, so unstocked tools are genuinely absent and
# the host's /usr/bin/git cannot leak in. bash must therefore be invoked absolutely.
_SH="${BASH:-/bin/bash}"
_run_gate() {
# $1 = STUDIO_LOCAL_INSTALL
( PATH="$_BIN"; export PATH
"$_SH" -c ". '$_HARNESS'; . '$_FN_FILE'; STUDIO_LOCAL_INSTALL=$1; _check_macos_deps; echo \"RC=\$?\"" 2>&1 )
}
echo "=== clean Mac: no CLT at all (xcode-select missing) ==="
rm -f "$_BIN"/*
_out="$(_run_gate false)"
assert_contains "does not exit 1" "$_out" "RC=0"
assert_contains "says CLT are not required" "$_out" "not required"
assert_not_contains "never claims CLT are required" "$_out" "are required"
echo "=== clean Mac: CLT stubs present but non-functional (the real virgin-Mac shape) ==="
# With no CLT, /usr/bin/git EXISTS and fails when run, so `command -v git` succeeds.
# The gate must not be fooled by that.
rm -f "$_BIN"/*
_mk xcode-select 'exit 1'
_mk git 'echo "xcrun: error: invalid active developer path" >&2; exit 1'
_out="$(_run_gate false)"
assert_contains "consumer install proceeds" "$_out" "RC=0"
assert_contains "reports CLT absent but optional" "$_out" "not required"
echo "=== --local with a non-functional git: must fail loudly ==="
_out="$(_run_gate true)"
assert_contains "fails" "$_out" "RC=1"
assert_contains "explains why git is needed" "$_out" "unsloth-zoo"
assert_contains "names the remedy" "$_out" "xcode-select --install"
assert_contains "emits a machine-readable marker" "$_out" "[TAURI:NEED_XCODE_CLT]"
assert_contains "says a normal install needs none" "$_out" "non---local"
echo "=== --local with a working git: proceeds ==="
rm -f "$_BIN"/*
_mk xcode-select 'exit 1'
_mk git 'echo "git version 2.50.0"; exit 0'
_out="$(_run_gate true)"
assert_contains "--local proceeds when git works" "$_out" "RC=0"
echo "=== CLT installed + cmake present ==="
rm -f "$_BIN"/*
_mk xcode-select 'echo /Library/Developer/CommandLineTools; exit 0'
_mk git 'echo "git version 2.50.0"; exit 0'
_mk cmake 'echo "cmake version 3.30.0"; exit 0'
_out="$(_run_gate false)"
assert_contains "all deps found" "$_out" "all system dependencies found"
assert_contains "rc 0" "$_out" "RC=0"
echo "=== CLT installed, cmake missing: prebuilt path, not fatal ==="
rm -f "$_BIN"/*
_mk xcode-select 'echo /Library/Developer/CommandLineTools; exit 0'
_mk git 'echo "git version 2.50.0"; exit 0'
_out="$(_run_gate false)"
assert_contains "uses prebuilt llama.cpp" "$_out" "using prebuilt llama.cpp"
assert_contains "rc 0" "$_out" "RC=0"
echo "=== the gate never fires the GUI installer on the consumer path ==="
# The dialog needs a GUI session a curl-piped or Tauri-spawned install does not have.
rm -f "$_BIN"/*
_mk xcode-select 'if [ "$1" = "--install" ]; then echo "GUI-DIALOG-FIRED"; fi; exit 1'
_out="$(_run_gate false)"
assert_not_contains "no GUI dialog on consumer path" "$_out" "GUI-DIALOG-FIRED"
echo "=== _has_working_git distinguishes present-but-broken from working ==="
rm -f "$_BIN"/*
_mk git 'exit 1'
_r="$(PATH="$_BIN" "$_SH" -c ". '$_FN_FILE'; _has_working_git && echo yes || echo no")"
assert_eq "broken git stub -> no" "no" "$_r"
_mk git 'echo ok; exit 0'
_r="$(PATH="$_BIN" "$_SH" -c ". '$_FN_FILE'; _has_working_git && echo yes || echo no")"
assert_eq "working git -> yes" "yes" "$_r"
rm -f "$_BIN"/git
_r="$(PATH="$_BIN" "$_SH" -c ". '$_FN_FILE'; _has_working_git && echo yes || echo no")"
assert_eq "absent git -> no" "no" "$_r"
rm -rf "$_BIN" "$_FN_FILE" "$_HARNESS"
echo ""
echo "=== $PASS passed, $FAIL failed ==="
[ "$FAIL" -eq 0 ] || exit 1