* Installer: make UV_OVERRIDE space-safe on Apple Silicon (#6503) On Apple Silicon, install.sh exports UV_OVERRIDE pointing at the bundled overrides-darwin-arm64.txt. uv splits UV_OVERRIDE on whitespace, so a repo cloned under a path containing a space (e.g. /Users/me/Open Source/unsloth) truncates the value and every later uv call aborts with 'error: File not found: <truncated>' (the PyTorch install step in #6503). Copy the overrides file into a space-free temp dir and point uv at the copy when the path contains a space, mirroring the macOS/Linux handling already merged for the Python installer in #6534. The temp dir is removed in the exit trap, and the code falls back to the original path when no space-free temp dir is available, so the no-space and non-macOS paths are unchanged. Adds tests/sh/test_install_uv_override_space.sh, which extracts and runs the install.sh hardening block and checks the spaced, no-space, and spaced-TMPDIR fallback cases. * Installer: match all whitespace (not just spaces) in UV_OVERRIDE handling uv splits UV_OVERRIDE on any whitespace, so use the POSIX class *[[:space:]]* rather than a literal space in install.sh (catches tabs and newlines in the path too) and the matching test assertions. Use the portable awk bracket expression [$] instead of \$ in the extraction so the test runs the same under BSD awk (macOS) and GNU awk (Linux). Adds a tab-in-path case. * Installer: clear _UV_OVERRIDE_TMPDIR before the exit trap The exit trap rm -rf's _UV_OVERRIDE_TMPDIR. Initialize it to empty before registering the trap so an inherited environment value can never be removed; only a temp dir this script creates (Apple Silicon, spaced path) is cleaned. Adds a structural test asserting the init precedes the trap. * Run the install.sh UV_OVERRIDE space test in CI via a pytest wrapper The Shell installer tests job uses a fixed script list (not tests/run_all.sh), so the new shell test would not run on PRs. Add a pytest wrapper under tests/python/ that invokes it; the auto-discovered repo CPU test job collects tests/python/ and so executes the Apple Silicon spaced-path regression. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Daniel Han
GitHub
parent
a3954edd15
commit
f436d204f6
4 changed files with 167 additions and 0 deletions
23
install.sh
23
install.sh
|
|
@ -447,8 +447,12 @@ _on_install_exit() {
|
|||
if [ "$_status" -ne 0 ]; then
|
||||
_restore_studio_venv_replacement
|
||||
fi
|
||||
[ -n "${_UV_OVERRIDE_TMPDIR:-}" ] && rm -rf "$_UV_OVERRIDE_TMPDIR" 2>/dev/null || true
|
||||
exit "$_status"
|
||||
}
|
||||
# Empty so an inherited value can never reach the trap's rm; only a temp dir
|
||||
# this script creates below (Apple Silicon, spaced path) is ever removed.
|
||||
_UV_OVERRIDE_TMPDIR=""
|
||||
trap _on_install_exit EXIT
|
||||
|
||||
# ── Helper: download a URL to a file (supports curl and wget) ──
|
||||
|
|
@ -1427,6 +1431,25 @@ fi
|
|||
if [ "$OS" = "macos" ] && [ "$_ARCH" = "arm64" ]; then
|
||||
_OVERRIDES_FILE="$(cd "$(dirname "$0" 2>/dev/null || echo ".")" && pwd)/studio/backend/requirements/single-env/overrides-darwin-arm64.txt"
|
||||
if [ -f "$_OVERRIDES_FILE" ]; then
|
||||
# uv splits UV_OVERRIDE on whitespace, so a repo path with whitespace
|
||||
# truncates it and aborts every later uv call (issue #6503). Hand uv a copy.
|
||||
case "$_OVERRIDES_FILE" in
|
||||
*[[:space:]]*)
|
||||
_UV_OVERRIDE_TMPDIR=$(mktemp -d 2>/dev/null) || _UV_OVERRIDE_TMPDIR=""
|
||||
case "$_UV_OVERRIDE_TMPDIR" in
|
||||
"") ;;
|
||||
*[[:space:]]*) rm -rf "$_UV_OVERRIDE_TMPDIR" 2>/dev/null || true; _UV_OVERRIDE_TMPDIR="" ;;
|
||||
*)
|
||||
if cp "$_OVERRIDES_FILE" "$_UV_OVERRIDE_TMPDIR/overrides-darwin-arm64.txt" 2>/dev/null; then
|
||||
_OVERRIDES_FILE="$_UV_OVERRIDE_TMPDIR/overrides-darwin-arm64.txt"
|
||||
else
|
||||
rm -rf "$_UV_OVERRIDE_TMPDIR" 2>/dev/null || true
|
||||
_UV_OVERRIDE_TMPDIR=""
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
;;
|
||||
esac
|
||||
export UV_OVERRIDE="$_OVERRIDES_FILE"
|
||||
fi
|
||||
fi
|
||||
|
|
|
|||
31
tests/python/test_install_uv_override_space.py
Normal file
31
tests/python/test_install_uv_override_space.py
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
"""Run the install.sh UV_OVERRIDE space-safety shell test (issue #6503) under
|
||||
pytest, so the auto-discovered CPU test job executes it. The dedicated
|
||||
`Shell installer tests` CI job runs a fixed script list that this is not part
|
||||
of, so without this wrapper the regression would only be covered locally via
|
||||
tests/run_all.sh.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
SHELL_TEST = REPO_ROOT / "tests" / "sh" / "test_install_uv_override_space.sh"
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason = "POSIX shell installer test")
|
||||
@pytest.mark.skipif(shutil.which("bash") is None, reason = "bash not available")
|
||||
def test_install_uv_override_space_shell():
|
||||
assert SHELL_TEST.is_file(), f"missing shell test: {SHELL_TEST}"
|
||||
proc = subprocess.run(
|
||||
["bash", str(SHELL_TEST)],
|
||||
capture_output = True,
|
||||
text = True,
|
||||
)
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
assert "ALL PASSED" in proc.stdout, proc.stdout + proc.stderr
|
||||
|
|
@ -15,6 +15,7 @@ sh "$TESTS_DIR/sh/test_resolve_cuda_archs.sh"
|
|||
sh "$TESTS_DIR/sh/test_strixhalo_wsl_reroute.sh"
|
||||
sh "$TESTS_DIR/sh/test_uninstall_shared_icon.sh"
|
||||
sh "$TESTS_DIR/sh/test_torch_flavor.sh"
|
||||
sh "$TESTS_DIR/sh/test_install_uv_override_space.sh"
|
||||
|
||||
echo ""
|
||||
echo "=== Python tests ==="
|
||||
|
|
|
|||
112
tests/sh/test_install_uv_override_space.sh
Executable file
112
tests/sh/test_install_uv_override_space.sh
Executable file
|
|
@ -0,0 +1,112 @@
|
|||
#!/bin/bash
|
||||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
# uv splits UV_OVERRIDE on whitespace, so a repo cloned under a path with a space
|
||||
# truncates it and aborts every later uv call (issue #6503). install.sh must hand
|
||||
# uv a space-free copy. Exercises the real install.sh hardening block.
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
INSTALL_SH="$SCRIPT_DIR/../../install.sh"
|
||||
PASS=0
|
||||
FAIL=0
|
||||
|
||||
ok() { echo " PASS: $1"; PASS=$((PASS + 1)); }
|
||||
bad() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
|
||||
|
||||
# Extract the UV_OVERRIDE hardening block (outer case ... esac plus the export)
|
||||
# and run it directly, so the test tracks install.sh rather than a copy of it.
|
||||
BLOCK=$(awk '
|
||||
/case "[$]_OVERRIDES_FILE" in/ { grab = 1 }
|
||||
grab { print }
|
||||
grab && /export UV_OVERRIDE="[$]_OVERRIDES_FILE"/ { exit }
|
||||
' "$INSTALL_SH")
|
||||
if ! printf '%s' "$BLOCK" | grep -q 'export UV_OVERRIDE'; then
|
||||
echo " FAIL: could not extract UV_OVERRIDE block from install.sh"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
run_block() {
|
||||
_OVERRIDES_FILE="$1"
|
||||
_UV_OVERRIDE_TMPDIR=""
|
||||
unset UV_OVERRIDE
|
||||
eval "$BLOCK"
|
||||
}
|
||||
|
||||
echo "=== test_install_uv_override_space ==="
|
||||
|
||||
# 1. Spaced path -> space-free copy with identical contents, temp dir tracked.
|
||||
WORK=$(mktemp -d)
|
||||
mkdir -p "$WORK/Open Source"
|
||||
SRC="$WORK/Open Source/overrides-darwin-arm64.txt"
|
||||
printf 'transformers>=4.57.6\n' > "$SRC"
|
||||
run_block "$SRC"
|
||||
case "$UV_OVERRIDE" in
|
||||
*[[:space:]]*) bad "spaced path: UV_OVERRIDE still contains whitespace ($UV_OVERRIDE)" ;;
|
||||
*) ok "spaced path: UV_OVERRIDE is whitespace-free" ;;
|
||||
esac
|
||||
[ "$UV_OVERRIDE" != "$SRC" ] && ok "spaced path: points at a copy" || bad "spaced path: not copied"
|
||||
[ "$(cat "$UV_OVERRIDE" 2>/dev/null)" = "transformers>=4.57.6" ] \
|
||||
&& ok "spaced path: copy contents identical" || bad "spaced path: contents differ"
|
||||
{ [ -n "$_UV_OVERRIDE_TMPDIR" ] && [ -d "$_UV_OVERRIDE_TMPDIR" ]; } \
|
||||
&& ok "spaced path: temp dir tracked for cleanup" || bad "spaced path: temp dir not tracked"
|
||||
# The exit-trap cleanup (_on_install_exit) must then remove it.
|
||||
[ -n "$_UV_OVERRIDE_TMPDIR" ] && rm -rf "$_UV_OVERRIDE_TMPDIR" 2>/dev/null || true
|
||||
[ ! -d "$_UV_OVERRIDE_TMPDIR" ] && ok "spaced path: temp dir removable" || bad "spaced path: temp dir lingers"
|
||||
rm -rf "$WORK"
|
||||
|
||||
# 2. No-space path -> passthrough, no temp dir.
|
||||
PLAIN=$(mktemp -d)
|
||||
PSRC="$PLAIN/overrides-darwin-arm64.txt"
|
||||
printf 'transformers>=4.57.6\n' > "$PSRC"
|
||||
run_block "$PSRC"
|
||||
[ "$UV_OVERRIDE" = "$PSRC" ] && ok "no-space path: UV_OVERRIDE unchanged" || bad "no-space path: changed ($UV_OVERRIDE)"
|
||||
[ -z "$_UV_OVERRIDE_TMPDIR" ] && ok "no-space path: no temp dir created" || bad "no-space path: temp dir created"
|
||||
rm -rf "$PLAIN"
|
||||
|
||||
# 3. TMPDIR itself contains a space -> fall back to the original path, no leak.
|
||||
WORK2=$(mktemp -d)
|
||||
mkdir -p "$WORK2/Open Source" "$WORK2/tmp dir"
|
||||
SRC2="$WORK2/Open Source/overrides-darwin-arm64.txt"
|
||||
printf 'transformers>=4.57.6\n' > "$SRC2"
|
||||
RES=$( TMPDIR="$WORK2/tmp dir"; export TMPDIR; run_block "$SRC2"
|
||||
printf 'UV_OVERRIDE=%s\nTMPDIR_VAR=%s\n' "$UV_OVERRIDE" "$_UV_OVERRIDE_TMPDIR" )
|
||||
echo "$RES" | grep -qx "UV_OVERRIDE=$SRC2" \
|
||||
&& ok "spaced TMPDIR: falls back to original path" || bad "spaced TMPDIR: did not fall back ($RES)"
|
||||
echo "$RES" | grep -qx "TMPDIR_VAR=" \
|
||||
&& ok "spaced TMPDIR: no temp dir tracked" || bad "spaced TMPDIR: temp dir tracked"
|
||||
# mktemp may have created a dir under the spaced TMPDIR; it must not be leaked.
|
||||
_leftover=$(find "$WORK2/tmp dir" -mindepth 1 -maxdepth 1 -type d 2>/dev/null | head -n1)
|
||||
[ -z "$_leftover" ] && ok "spaced TMPDIR: no leaked temp dir" || bad "spaced TMPDIR: leaked $_leftover"
|
||||
rm -rf "$WORK2"
|
||||
|
||||
# 4. A tab in the path is whitespace uv also splits on -> copied like a space.
|
||||
WORK3=$(mktemp -d)
|
||||
TABDIR=$(printf 'Open\tSource')
|
||||
mkdir -p "$WORK3/$TABDIR"
|
||||
SRC3="$WORK3/$TABDIR/overrides-darwin-arm64.txt"
|
||||
printf 'transformers>=4.57.6\n' > "$SRC3"
|
||||
run_block "$SRC3"
|
||||
case "$UV_OVERRIDE" in
|
||||
*[[:space:]]*) bad "tab path: UV_OVERRIDE still contains whitespace" ;;
|
||||
*) ok "tab path: UV_OVERRIDE is whitespace-free" ;;
|
||||
esac
|
||||
[ -n "$_UV_OVERRIDE_TMPDIR" ] && rm -rf "$_UV_OVERRIDE_TMPDIR" 2>/dev/null || true
|
||||
rm -rf "$WORK3"
|
||||
|
||||
# 5. install.sh must clear _UV_OVERRIDE_TMPDIR before registering the exit trap,
|
||||
# so an inherited value can never reach the trap's rm -rf.
|
||||
_init_line=$(grep -n '^_UV_OVERRIDE_TMPDIR=""' "$INSTALL_SH" | head -n1 | cut -d: -f1)
|
||||
_trap_line=$(grep -n '^trap _on_install_exit EXIT' "$INSTALL_SH" | head -n1 | cut -d: -f1)
|
||||
{ [ -n "$_init_line" ] && [ -n "$_trap_line" ] && [ "$_init_line" -lt "$_trap_line" ]; } \
|
||||
&& ok "init: _UV_OVERRIDE_TMPDIR cleared before exit trap" \
|
||||
|| bad "init: _UV_OVERRIDE_TMPDIR not cleared before exit trap (init=$_init_line trap=$_trap_line)"
|
||||
|
||||
echo ""
|
||||
echo " PASS: $PASS"
|
||||
echo " FAIL: $FAIL"
|
||||
if [ "$FAIL" -gt 0 ]; then
|
||||
echo "FAILED"
|
||||
exit 1
|
||||
fi
|
||||
echo "ALL PASSED"
|
||||
Loading…
Add table
Add a link
Reference in a new issue