ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/tests/sh/test_torch_constraint.sh
Daniel Han d22b2a18f9
 fix: add tokenizers to no-torch deps and TORCH_CONSTRAINT for arm64 macOS py313+ (#4748) 
* fix: add tokenizers to no-torch runtime deps and add TORCH_CONSTRAINT for arm64 macOS py313+

Two installer fixes:

1. Add `tokenizers` to `no-torch-runtime.txt` before `transformers`.
   Without it, `from transformers import AutoConfig` crashes on startup
   because `--no-deps` skips transitive dependencies.

2. Add `TORCH_CONSTRAINT` variable to `install.sh`. On arm64 macOS with
   Python 3.13+, tighten the torch requirement to `>=2.6` since torch
   <2.6 has no cp313 arm64 wheels. The variable replaces the previously
   hard-coded constraint in the uv pip install line.

Includes 66 tests (42 pytest + 24 bash) covering:
- Structural checks on install.sh, install.ps1, no-torch-runtime.txt
- Shell snippet tests with mocked python for 13 platform/version combos
- Mock uv integration verifying correct constraint string
- E2E venv tests on Python 3.12 and 3.13 confirming AutoConfig works
- Negative control proving AutoConfig fails without tokenizers
- Full no-torch sandbox regression guards (safetensors, huggingface_hub)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix incomplete no-torch manifest and align E2E tests with real --no-deps path

- Add missing transitive deps to no-torch-runtime.txt that are required
  under --no-deps: regex, typing_extensions, filelock, httpx, httpcore,
  certifi, idna, anyio, sniffio, h11. Without these, `from transformers
  import AutoConfig` still fails after install.sh --no-torch.

- Change all E2E tests to use --no-deps (matching what install.sh does)
  instead of normal dep resolution. Previous tests passed even with an
  incomplete manifest because uv backfilled transitive deps.

- Rewrite negative control to derive from the real no-torch-runtime.txt
  with tokenizers stripped, proving the specific fix matters.

- Replace GNU-only sed -i with heredoc in shell test for macOS compat.

- Remove unused os/sys imports from Python test file.

- Quote SKIP_TORCH and mock uv paths in bash -c strings.

* Assert install succeeds before checking import results in E2E tests

Address review feedback: test_torch_not_importable and
test_tokenizers_directly_importable in Group 3 now assert that
uv pip install returns 0 before checking import behavior. This
prevents false positives when the install itself fails silently.

* Assert install succeeds in negative control and tighten error check

- Add missing install-success assertion in test_negative_control_no_tokenizers
  to prevent false positives from network/install failures.

- Tighten error message check to look for "tokenizers" in stderr or
  ModuleNotFoundError, rather than the generic "No module" substring
  which could match unrelated import failures.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-04-01 06:12:17 -07:00

266 lines
9.9 KiB
Bash
Raw Permalink Blame History

#!/bin/bash
# Tests for TORCH_CONSTRAINT variable in install.sh and tokenizers in no-torch-runtime.txt.
# Follows the same assertion pattern as test_mac_intel_compat.sh.
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
INSTALL_SH="$SCRIPT_DIR/../../install.sh"
INSTALL_PS1="$SCRIPT_DIR/../../install.ps1"
NO_TORCH_RT="$SCRIPT_DIR/../../studio/backend/requirements/no-torch-runtime.txt"
PASS=0
FAIL=0
assert_eq() {
_label="$1"; _expected="$2"; _actual="$3"
if [ "$_actual" = "$_expected" ]; then
echo " PASS: $_label"
PASS=$((PASS + 1))
else
echo " FAIL: $_label (expected '$_expected', got '$_actual')"
FAIL=$((FAIL + 1))
fi
}
assert_contains() {
_label="$1"; _haystack="$2"; _needle="$3"
if echo "$_haystack" | grep -qF "$_needle"; then
echo " PASS: $_label"
PASS=$((PASS + 1))
else
echo " FAIL: $_label (expected to find '$_needle')"
FAIL=$((FAIL + 1))
fi
}
assert_not_contains() {
_label="$1"; _haystack="$2"; _needle="$3"
if echo "$_haystack" | grep -qF "$_needle"; then
echo " FAIL: $_label (found '$_needle' but should not)"
FAIL=$((FAIL + 1))
else
echo " PASS: $_label"
PASS=$((PASS + 1))
fi
}
# ── Helper: create a mock python that reports a given minor version ──
make_mock_python() {
_minor="$1"
_venv_dir="$2"
mkdir -p "$_venv_dir/bin"
cat > "$_venv_dir/bin/python" <<MOCK_EOF
#!/bin/bash
if echo "\$@" | grep -q "sys.version_info.minor"; then
echo "$_minor"
else
echo "0"
fi
MOCK_EOF
chmod +x "$_venv_dir/bin/python"
}
# ── Helper: run the TORCH_CONSTRAINT snippet with given params ──
run_constraint_snippet() {
_skip_torch="$1"
_os="$2"
_arch="$3"
_py_minor="$4"
_venv_dir="$5"
make_mock_python "$_py_minor" "$_venv_dir"
bash -c "
SKIP_TORCH=\"$_skip_torch\"
OS=\"$_os\"
_ARCH=\"$_arch\"
VENV_DIR=\"$_venv_dir\"
TORCH_CONSTRAINT=\"torch>=2.4,<2.11.0\"
if [ \"\$SKIP_TORCH\" = false ] && [ \"\$OS\" = \"macos\" ] && [ \"\$_ARCH\" = \"arm64\" ]; then
_PY_MINOR=\$(\"\$VENV_DIR/bin/python\" -c \"import sys; print(sys.version_info.minor)\" 2>/dev/null || echo \"0\")
if [ \"\$_PY_MINOR\" -ge 13 ] 2>/dev/null; then
TORCH_CONSTRAINT=\"torch>=2.6,<2.11.0\"
fi
fi
echo \"\$TORCH_CONSTRAINT\"
" 2>/dev/null
}
# ======================================================================
# Structural checks
# ======================================================================
echo "=== Structural: TORCH_CONSTRAINT in install.sh ==="
_SH_CONTENT=$(cat "$INSTALL_SH")
_count=$(grep -c 'TORCH_CONSTRAINT="torch>=2.4,<2.11.0"' "$INSTALL_SH" || true)
assert_eq "default TORCH_CONSTRAINT assignment exists" "1" "$_count"
_count=$(grep -c 'TORCH_CONSTRAINT="torch>=2.6,<2.11.0"' "$INSTALL_SH" || true)
assert_eq "tightened TORCH_CONSTRAINT assignment exists" "1" "$_count"
_count=$(grep -c '"\$TORCH_CONSTRAINT"' "$INSTALL_SH" || true)
_has_var=$([ "$_count" -ge 1 ] && echo "yes" || echo "no")
assert_eq "\$TORCH_CONSTRAINT used in pip install" "yes" "$_has_var"
# Hardcoded torch>=2.4,<2.11.0 should only appear once (the default assignment)
_hardcoded=$(grep -c '"torch>=2.4,<2.11.0"' "$INSTALL_SH" || true)
assert_eq "hardcoded torch>=2.4 appears exactly once" "1" "$_hardcoded"
echo ""
echo "=== Structural: tokenizers in no-torch-runtime.txt ==="
_has_tokenizers=$(grep -c '^tokenizers$' "$NO_TORCH_RT" || true)
assert_eq "tokenizers present as standalone line" "1" "$_has_tokenizers"
# tokenizers before transformers
_tok_line=$(grep -n '^tokenizers$' "$NO_TORCH_RT" | head -1 | cut -d: -f1)
_tf_line=$(grep -n '^transformers' "$NO_TORCH_RT" | head -1 | cut -d: -f1)
_tok_first=$([ "$_tok_line" -lt "$_tf_line" ] && echo "yes" || echo "no")
assert_eq "tokenizers before transformers" "yes" "$_tok_first"
# torch itself NOT in no-torch file
_has_torch=$(grep -c '^torch$' "$NO_TORCH_RT" || true)
assert_eq "torch not in no-torch-runtime.txt" "0" "$_has_torch"
echo ""
echo "=== Structural: install.ps1 unchanged ==="
_PS1_CONTENT=$(cat "$INSTALL_PS1")
_ps1_has_var=$(echo "$_PS1_CONTENT" | grep -c 'TORCH_CONSTRAINT\|TorchConstraint' || true)
assert_eq "install.ps1 has no TORCH_CONSTRAINT variable" "0" "$_ps1_has_var"
_ps1_hardcoded=$(echo "$_PS1_CONTENT" | grep -c '"torch>=2.4,<2.11.0"' || true)
_ps1_has_hc=$([ "$_ps1_hardcoded" -ge 1 ] && echo "yes" || echo "no")
assert_eq "install.ps1 has hardcoded torch constraint" "yes" "$_ps1_has_hc"
# ======================================================================
# Runtime: mocked platform/version combos
# ======================================================================
echo ""
echo "=== Runtime: TORCH_CONSTRAINT with mocked inputs ==="
TMPDIR_BASE=$(mktemp -d)
trap 'rm -rf "$TMPDIR_BASE"' EXIT
# 1. arm64 macOS py3.13 -> tightened
_result=$(run_constraint_snippet false macos arm64 13 "$TMPDIR_BASE/v1")
assert_eq "arm64+macos+py313 -> tightened" "torch>=2.6,<2.11.0" "$_result"
# 2. arm64 macOS py3.14 -> tightened (future-proofed)
_result=$(run_constraint_snippet false macos arm64 14 "$TMPDIR_BASE/v2")
assert_eq "arm64+macos+py314 -> tightened" "torch>=2.6,<2.11.0" "$_result"
# 3. arm64 macOS py3.12 -> default
_result=$(run_constraint_snippet false macos arm64 12 "$TMPDIR_BASE/v3")
assert_eq "arm64+macos+py312 -> default" "torch>=2.4,<2.11.0" "$_result"
# 4. arm64 macOS py3.11 -> default
_result=$(run_constraint_snippet false macos arm64 11 "$TMPDIR_BASE/v4")
assert_eq "arm64+macos+py311 -> default" "torch>=2.4,<2.11.0" "$_result"
# 5. Linux x86_64 py3.13 -> default (Linux unaffected)
_result=$(run_constraint_snippet false linux x86_64 13 "$TMPDIR_BASE/v5")
assert_eq "linux+x86_64+py313 -> default" "torch>=2.4,<2.11.0" "$_result"
# 6. Linux aarch64 py3.13 -> default (guard checks OS=macos)
_result=$(run_constraint_snippet false linux aarch64 13 "$TMPDIR_BASE/v6")
assert_eq "linux+aarch64+py313 -> default" "torch>=2.4,<2.11.0" "$_result"
# 7. Intel Mac x86_64 py3.12 -> default (arch mismatch)
_result=$(run_constraint_snippet false macos x86_64 12 "$TMPDIR_BASE/v7")
assert_eq "macos+x86_64+py312 -> default" "torch>=2.4,<2.11.0" "$_result"
# 8. SKIP_TORCH=true arm64 macOS py3.13 -> block skipped, default
_result=$(run_constraint_snippet true macos arm64 13 "$TMPDIR_BASE/v8")
assert_eq "SKIP_TORCH=true -> default" "torch>=2.4,<2.11.0" "$_result"
# 9. WSL py3.13 -> default
_result=$(run_constraint_snippet false wsl x86_64 13 "$TMPDIR_BASE/v9")
assert_eq "wsl+py313 -> default" "torch>=2.4,<2.11.0" "$_result"
# 10. py_minor=0 (failed query fallback) -> default
_result=$(run_constraint_snippet false macos arm64 0 "$TMPDIR_BASE/v10")
assert_eq "py_minor=0 fallback -> default" "torch>=2.4,<2.11.0" "$_result"
# 11. Boundary: py_minor=12 -> NOT tightened
_result=$(run_constraint_snippet false macos arm64 12 "$TMPDIR_BASE/v11")
assert_eq "boundary py_minor=12 -> default" "torch>=2.4,<2.11.0" "$_result"
# 12. Boundary: py_minor=13 -> tightened
_result=$(run_constraint_snippet false macos arm64 13 "$TMPDIR_BASE/v12")
assert_eq "boundary py_minor=13 -> tightened" "torch>=2.6,<2.11.0" "$_result"
# 13. Intel Mac py3.13 -> default (arch=x86_64, not arm64)
_result=$(run_constraint_snippet false macos x86_64 13 "$TMPDIR_BASE/v13")
assert_eq "macos+x86_64+py313 -> default" "torch>=2.4,<2.11.0" "$_result"
# ======================================================================
# Mock uv integration
# ======================================================================
echo ""
echo "=== Mock uv: verify constraint passed to uv ==="
# arm64 + py313 -> uv receives torch>=2.6
_UV_LOG="$TMPDIR_BASE/uv_log_tight.txt"
make_mock_python 13 "$TMPDIR_BASE/uv_venv1"
cat > "$TMPDIR_BASE/mock_uv_tight" <<UVEOF
#!/bin/bash
echo "\$@" >> $_UV_LOG
UVEOF
chmod +x "$TMPDIR_BASE/mock_uv_tight"
bash -c "
SKIP_TORCH=false
OS=\"macos\"
_ARCH=\"arm64\"
VENV_DIR=\"$TMPDIR_BASE/uv_venv1\"
TORCH_CONSTRAINT=\"torch>=2.4,<2.11.0\"
if [ \"\$SKIP_TORCH\" = false ] && [ \"\$OS\" = \"macos\" ] && [ \"\$_ARCH\" = \"arm64\" ]; then
_PY_MINOR=\$(\"\$VENV_DIR/bin/python\" -c \"import sys; print(sys.version_info.minor)\" 2>/dev/null || echo \"0\")
if [ \"\$_PY_MINOR\" -ge 13 ] 2>/dev/null; then
TORCH_CONSTRAINT=\"torch>=2.6,<2.11.0\"
fi
fi
\"$TMPDIR_BASE/mock_uv_tight\" pip install --python \"\$VENV_DIR/bin/python\" \"\$TORCH_CONSTRAINT\" torchvision torchaudio
" 2>/dev/null
_uv_got=$(cat "$_UV_LOG" 2>/dev/null || echo "")
assert_contains "mock uv arm64+py313 receives torch>=2.6" "$_uv_got" "torch>=2.6,<2.11.0"
# arm64 + py312 -> uv receives torch>=2.4
_UV_LOG2="$TMPDIR_BASE/uv_log_default.txt"
make_mock_python 12 "$TMPDIR_BASE/uv_venv2"
cat > "$TMPDIR_BASE/mock_uv_default" <<UVEOF
#!/bin/bash
echo "\$@" >> $_UV_LOG2
UVEOF
chmod +x "$TMPDIR_BASE/mock_uv_default"
bash -c "
SKIP_TORCH=false
OS=\"macos\"
_ARCH=\"arm64\"
VENV_DIR=\"$TMPDIR_BASE/uv_venv2\"
TORCH_CONSTRAINT=\"torch>=2.4,<2.11.0\"
if [ \"\$SKIP_TORCH\" = false ] && [ \"\$OS\" = \"macos\" ] && [ \"\$_ARCH\" = \"arm64\" ]; then
_PY_MINOR=\$(\"\$VENV_DIR/bin/python\" -c \"import sys; print(sys.version_info.minor)\" 2>/dev/null || echo \"0\")
if [ \"\$_PY_MINOR\" -ge 13 ] 2>/dev/null; then
TORCH_CONSTRAINT=\"torch>=2.6,<2.11.0\"
fi
fi
\"$TMPDIR_BASE/mock_uv_default\" pip install --python \"\$VENV_DIR/bin/python\" \"\$TORCH_CONSTRAINT\" torchvision torchaudio
" 2>/dev/null
_uv_got2=$(cat "$_UV_LOG2" 2>/dev/null || echo "")
assert_contains "mock uv arm64+py312 receives torch>=2.4" "$_uv_got2" "torch>=2.4,<2.11.0"
# ======================================================================
# Summary
# ======================================================================
echo ""
echo "=== Results ==="
echo " PASS: $PASS"
echo " FAIL: $FAIL"
if [ "$FAIL" -gt 0 ]; then
echo "FAILED"
exit 1
fi
echo "ALL PASSED"
Reference in a new issue View git blame Copy permalink