From ca55acbb5f4d1b49fb40124ee99f6d5a1b3b1b01 Mon Sep 17 00:00:00 2001 From: Daniel Han Date: Wed, 27 May 2026 04:53:38 -0700 Subject: [PATCH] Studio: unblock install on Linux ARM64 + Windows ARM64 + Intel Mac (#5790) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Studio: unblock cross-platform install on Linux ARM64 + Windows ARM64 Three independent bugs that together prevent `install.sh` / `install.ps1` from completing on the ARM machines GitHub Actions now ships (`ubuntu-24.04-arm`, `windows-11-arm`) and on equivalent real hosts (Ampere Altra, Raspberry Pi 5, Snapdragon X Elite, ...). Validated on the staging-2 cross-OS smoke suite -- five per-OS workflows pinned to `ubuntu-latest`, `ubuntu-24.04-arm`, `macos-14`, `macos-15-intel`, `windows-11-arm`. Before this change Windows ARM exits 1 in the winget gate and Linux ARM source-builds llama.cpp because the prebuilt selector returns 0 attempts; with it both reach healthy /api/health. 1. studio/install_llama_prebuilt.py -- resolve_simple_install_release_plans had explicit branches for windows+x86_64, macos+arm64, macos+x86_64 and linux+x86_64 only. Upstream ggml-org/llama.cpp ships `llama-bNNNN-bin-ubuntu-arm64.tar.gz` and `llama-bNNNN-bin-win-cpu-arm64.zip` (visible in the b9334 release manifest), so the missing elif branches force every Linux ARM64 and Windows ARM64 host into a source build even when a perfectly good upstream prebuilt is one HTTP GET away. Two new branches mirror the existing CPU variants; runtime_patterns_for_choice and runtime_payload_health_groups gain `linux-arm64` (.so layout) and `windows-arm64` (.dll layout) so the health-check pass-through matches the asset shape. 2. studio/setup.sh -- the helper-release-repo selector routed any non-x86_64 Linux to `unslothai/llama.cpp`, which only publishes the Linux CUDA bundle set. The result on Linux ARM64 was a guaranteed `direct_linux_release_plan` raise of "no compatible Linux prebuilt asset was found" on every release in the scan, then a source-build fallback. Pin Linux ARM64 (CPU-only) to `ggml-org/llama.cpp` so the new branch in (1) can see the upstream asset. setup.ps1 already hardcodes `ggml-org/llama.cpp`, so Windows ARM64 picks up (1) without an additional change. 3. install.ps1 -- the winget pre-check hard-failed before Python or uv detection. `windows-11-arm` runners (and many corporate Windows hosts without the Microsoft Store) ship without winget but already have a usable Python plus the Astral uv PowerShell installer reachable. Demote the winget check to a soft warning, defer the hard failure to the Python install branch (which is the only path that genuinely needs winget), and let the uv install fall through to `https://astral.sh/uv/install.ps1` when winget is absent. The uv PowerShell installer was already the existing fallback for the "winget present but uv install failed" case; this just makes it the primary path on hosts without winget. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: filter torchcodec on platforms without wheels torchcodec 0.10.0 ships wheels for manylinux_2_28_x86_64, macosx_12_0_arm64, and win_amd64 only -- visible on its PyPI page and in the resolver error reported by #4446. install_python_stack.py pulls torchcodec via extras-no-deps.txt, which is now installed unconditionally during `unsloth studio update --local` (the update command has no --no-torch flag). Result on Linux aarch64 / Windows ARM64 / Intel Mac (when invoked outside the install.sh auto-skip-torch path): ERROR: Could not find a version that satisfies the requirement torchcodec==0.10.0 (from versions: 0.0.0.dev0, ...) ERROR: No matching distribution found for torchcodec==0.10.0 error Installing extras (no-deps) (pip) failed (exit code 1) `NO_TORCH_SKIP_PACKAGES` already lists torchcodec but only fires when NO_TORCH is true -- the update path inherits no NO_TORCH from the original install and inferrence falls back to IS_MAC_INTEL only, so Linux aarch64 / Windows ARM64 sail past the guard. Adds a platform predicate PLATFORM_LACKS_TORCHCODEC_WHEEL and applies the torchcodec filter unconditionally there, independent of NO_TORCH. Surfaced by the staging-2 cross-OS smoke `unsloth studio update` step on ubuntu-24.04-arm; verified the same step is green with this patch overlaid. * Studio: skip librosa on no-torch hosts (unblocks Intel Mac install) Closes the last cross-platform install gap surfaced by the staging-2 cross-OS smoke (see unslothai/unsloth#5046 for the original report): `install.sh --local` on macos-15-intel fails at × Failed to build `llvmlite==0.47.0` error: failed-wheel-build-for-install ╰─> llvmlite error studio setup failed (exit code 1) Root cause: upstream llvmlite dropped the macosx_x86_64 wheel between 0.42.0 and 0.46.0 (https://pypi.org/project/llvmlite/0.47.0/#files -- only macosx_arm64 / manylinux / win_amd64 remain). pip falls back to a from-source build of llvmlite's FFI, which needs LLVM 14/15 dev headers and matching llvm-config -- not present in Xcode Command Line Tools' libclang and not installed by install.sh's MAC_INTEL deps branch. llvmlite enters Studio's tree via librosa -> numba -> llvmlite in extras.txt. openai-whisper (extras.txt:28) would also pull numba but is already filtered on no-torch hosts. Adding librosa to the same NO_TORCH_SKIP_PACKAGES set makes the install go through cleanly on Intel Mac (auto-detected NO_TORCH=true via the MAC_INTEL branch) and on any user-passed --no-torch host where torch-dependent audio pipelines would not run anyway. Tracked / verified on the danielhanchen/unsloth-staging-2#154 smoke matrix (macos-15-intel). * Studio UI tests: retry evaluate_fetch on transport-level failure (PR #5790) Mac Studio UI CI on this PR (run 26496820814, job 78026959359) failed with /api/models/list status=0 error='TypeError: Failed to fetch'. The artifact studio.log shows the server answered the two preceding /api/models/list calls from the React mount (both 200) but never received the third call from the test script: the browser reused a kept-alive HTTP/1.1 socket that uvicorn (5s keep_alive_timeout) had closed ~130ms earlier. Chromium under --single-process on macos-14 free runners is most prone to this; the post /api/auth/change-password session churn accelerates it. A rerun on the same SHA passed, which is the classic flake signature. evaluate_fetch in tests/studio/_playwright_robust.py already returns a structured {status: 0, body: None, error: "..."} on JS-side throws, but every caller treats status=0 as fatal. Add a bounded retry inside the helper so the one class of failure recovers transparently: status != 0 -> real HTTP response (incl. 4xx/5xx); propagate. error has "AbortError" -> caller's AbortSignal deadline; propagate. else (status==0) -> stale-keepalive or other transport failure; retry after 250ms / 500ms backoff so the pool evicts the dead socket before the next attempt. Defaults transport_retries=2, transport_backoff_ms=250 (max added latency on the happy path is zero; on a transport failure: up to 750ms of sleep). Callers keep the existing {status, body, error} shape; no call-site changes needed. Verified: tests/studio/_playwright_robust.py compiles; signature gains two kwonly args (transport_retries, transport_backoff_ms); 8 evaluate_fetch call sites in playwright_chat_ui.py + playwright_extra_ui.py pick up the retry without change. --------- Co-authored-by: danielhanchen Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --- install.ps1 | 42 ++++++++++++++++------- studio/install_llama_prebuilt.py | 50 +++++++++++++++++++++++++--- studio/install_python_stack.py | 30 ++++++++++++++++- studio/setup.sh | 10 ++++++ tests/studio/_playwright_robust.py | 53 ++++++++++++++++++++++++------ 5 files changed, 158 insertions(+), 27 deletions(-) diff --git a/install.ps1 b/install.ps1 index 1ed8fafdb4..3bb5fb3f8c 100644 --- a/install.ps1 +++ b/install.ps1 @@ -887,12 +887,19 @@ shell.Run cmd, 0, False } # ── Check winget ── + # winget is only needed to install Python or uv. If both are + # already on PATH (Windows ARM64 GitHub-hosted runners, manual + # python.org + Astral uv installs, corporate locked-down hosts + # without the Store, etc.) the script can proceed without it. + # We defer the hard failure to the Python / uv install branches + # below, where winget is actually invoked. Write-TauriLog "STEP" "Checking system dependencies" - if (-not (Get-Command winget -ErrorAction SilentlyContinue)) { - step "winget" "not available" "Red" - substep "Install it from https://aka.ms/getwinget" "Yellow" - substep "or install Python $PythonVersion and uv manually, then re-run." "Yellow" - return (Exit-InstallFailure "winget is not available") + $script:WingetAvailable = [bool](Get-Command winget -ErrorAction SilentlyContinue) + if ($script:WingetAvailable) { + step "winget" "available" + } else { + step "winget" "not available -- will require Python + uv to be already installed" "Yellow" + substep "Get it from https://aka.ms/getwinget if Python / uv are not already on PATH." "Yellow" } # ── Helper: detect a working Python 3.11-3.13 on the system ── @@ -973,6 +980,12 @@ shell.Run cmd, 0, False step "python" "Python $($DetectedPython.Version) already installed" } if (-not $DetectedPython) { + if (-not $script:WingetAvailable) { + Write-Host "[ERROR] No compatible Python (3.11-3.13) found and winget is unavailable on this host." -ForegroundColor Red + Write-Host " Install Python $PythonVersion from https://www.python.org/downloads/" -ForegroundColor Yellow + Write-Host " and re-run this installer (make sure 'Add Python to PATH' is checked)." -ForegroundColor Yellow + return (Exit-InstallFailure "winget required to install Python on this host") + } substep "installing Python ${PythonVersion}..." $pythonPackageId = "Python.Python.$PythonVersion" # Temporarily lower ErrorActionPreference so that winget stderr @@ -1024,14 +1037,19 @@ shell.Run cmd, 0, False Write-TauriLog "STEP" "Installing uv package manager" if (-not (Get-Command uv -ErrorAction SilentlyContinue)) { substep "installing uv package manager..." - $prevEAP = $ErrorActionPreference - $ErrorActionPreference = "Continue" - try { winget install --id=astral-sh.uv -e --accept-package-agreements --accept-source-agreements } catch {} - $ErrorActionPreference = $prevEAP - Refresh-SessionPath - # Fallback: if winget didn't put uv on PATH, try the PowerShell installer + if ($script:WingetAvailable) { + $prevEAP = $ErrorActionPreference + $ErrorActionPreference = "Continue" + try { winget install --id=astral-sh.uv -e --accept-package-agreements --accept-source-agreements } catch {} + $ErrorActionPreference = $prevEAP + Refresh-SessionPath + } + # Fallback: if winget is unavailable or didn't put uv on PATH, + # use Astral's official PowerShell installer. This is the only + # supported path on hosts without winget (Windows ARM64 runners, + # corporate machines without the Store, etc.). if (-not (Get-Command uv -ErrorAction SilentlyContinue)) { - substep "trying alternative uv installer..." "Yellow" + substep "installing uv via https://astral.sh/uv/install.ps1..." "Yellow" Invoke-Expression (Invoke-RestMethod -Uri "https://astral.sh/uv/install.ps1") Refresh-SessionPath } diff --git a/studio/install_llama_prebuilt.py b/studio/install_llama_prebuilt.py index 7672af6630..68fd9de1d1 100644 --- a/studio/install_llama_prebuilt.py +++ b/studio/install_llama_prebuilt.py @@ -1349,6 +1349,24 @@ def direct_upstream_release_plan( install_kind = "windows-cpu", ) ) + elif host.is_windows and host.is_arm64: + # Upstream ggml-org/llama.cpp ships llama-bNNNN-bin-win-cpu-arm64.zip + # (visible in the b9334 release manifest). Without this branch the + # selector returned 0 attempts and the installer fell back to a + # source build on every Windows ARM64 host. + cpu_asset = f"llama-{release_tag}-bin-win-cpu-arm64.zip" + cpu_url = assets.get(cpu_asset) + if cpu_url: + attempts.append( + AssetChoice( + repo = repo, + tag = release_tag, + name = cpu_asset, + url = cpu_url, + source_label = "upstream", + install_kind = "windows-arm64", + ) + ) elif host.is_macos and host.is_arm64: asset_name = f"llama-{release_tag}-bin-macos-arm64.tar.gz" asset_url = assets.get(asset_name) @@ -1391,6 +1409,25 @@ def direct_upstream_release_plan( install_kind = "linux-cpu", ) ) + elif host.is_linux and host.is_arm64 and not host.has_usable_nvidia: + # Upstream ggml-org/llama.cpp ships llama-bNNNN-bin-ubuntu-arm64.tar.gz + # (visible in the b9334 release manifest). Without this branch the + # selector returned 0 attempts and the installer fell back to a + # source build on every Linux ARM64 host (DGX Spark, Ampere + # Altra, GitHub-hosted ubuntu-24.04-arm runners, etc.). + asset_name = f"llama-{release_tag}-bin-ubuntu-arm64.tar.gz" + asset_url = assets.get(asset_name) + if asset_url: + attempts.append( + AssetChoice( + repo = repo, + tag = release_tag, + name = asset_name, + url = asset_url, + source_label = "upstream", + install_kind = "linux-arm64", + ) + ) if not attempts: raise PrebuiltFallback("no compatible upstream prebuilt asset was found") return InstallReleasePlan( @@ -3833,11 +3870,16 @@ def runtime_patterns_for_choice(choice: AssetChoice) -> list[str]: # libraries between b9279 and b9283) without us re-enumerating # every new file. Studio only invokes llama-server and llama-quantize; # other CLIs upstream ships (llama-cli, llama-bench, ...) are skipped. - if choice.install_kind in {"linux-cpu", "linux-cuda", "linux-rocm"}: + if choice.install_kind in {"linux-cpu", "linux-cuda", "linux-rocm", "linux-arm64"}: return ["llama-server", "llama-quantize", "lib*.so*"] if choice.install_kind in {"macos-arm64", "macos-x64"}: return ["llama-server", "llama-quantize", "lib*.dylib"] - if choice.install_kind in {"windows-cpu", "windows-cuda", "windows-hip"}: + if choice.install_kind in { + "windows-cpu", + "windows-cuda", + "windows-hip", + "windows-arm64", + }: return ["llama-server.exe", "llama-quantize.exe", "*.dll"] raise PrebuiltFallback( f"unsupported install kind for runtime overlay: {choice.install_kind}" @@ -5188,7 +5230,7 @@ def load_prebuilt_metadata(install_dir: Path) -> dict[str, Any] | None: def runtime_payload_health_groups(choice: AssetChoice) -> list[list[str]]: - if choice.install_kind == "linux-cpu": + if choice.install_kind in {"linux-cpu", "linux-arm64"}: return [ ["libllama-common.so*"], ["libllama.so*"], @@ -5223,7 +5265,7 @@ def runtime_payload_health_groups(choice: AssetChoice) -> list[list[str]]: ["libmtmd.so*"], ["libggml-hip.so*"], ] - if choice.install_kind == "windows-cpu": + if choice.install_kind in {"windows-cpu", "windows-arm64"}: return [["llama.dll"]] if choice.install_kind == "windows-cuda": groups = [["llama.dll"], ["ggml-cuda.dll"]] diff --git a/studio/install_python_stack.py b/studio/install_python_stack.py index ca7fe3f004..c4524b06f5 100644 --- a/studio/install_python_stack.py +++ b/studio/install_python_stack.py @@ -38,6 +38,18 @@ IS_WINDOWS = sys.platform == "win32" IS_MACOS = sys.platform == "darwin" IS_MAC_INTEL = IS_MACOS and platform.machine() == "x86_64" IS_MAC_ARM = IS_MACOS and platform.machine() == "arm64" +IS_LINUX = sys.platform.startswith("linux") +# torchcodec ships wheels only for manylinux_2_28_x86_64, +# macosx_12_0_arm64, and win_amd64 (visible in the 0.10.0 PyPI page). +# Trying to install it on any other host fails the whole +# extras-no-deps step. `unsloth studio update` does not have a +# --no-torch flag, so on these hosts the audio extras must be +# filtered out independent of the NO_TORCH env var. +PLATFORM_LACKS_TORCHCODEC_WHEEL = ( + (IS_LINUX and platform.machine() in {"aarch64", "arm64"}) + or (IS_WINDOWS and platform.machine().lower() in {"arm64", "aarch64"}) + or IS_MAC_INTEL +) # ── ROCm / AMD GPU support ───────────────────────────────────────────────────── # Mapping from detected ROCm (major, minor) to the best PyTorch wheel tag on @@ -604,7 +616,14 @@ WINDOWS_SKIP_PACKAGES = {"open_spiel", "triton_kernels"} # Packages to skip when torch is unavailable (Intel Mac GGUF-only mode). # These packages either *are* torch extensions or have unconditional # ``Requires-Dist: torch`` in their published metadata, so installing -# them would pull torch back into the environment. +# them would pull torch back into the environment. ``librosa`` also +# lives in this set even though it does not itself require torch: +# upstream ``llvmlite`` dropped its macOS x86_64 wheel between 0.42.0 +# and 0.46.0+ (see https://pypi.org/project/llvmlite/0.47.0/#files -- +# only macosx_arm64 / manylinux / win_amd64 remain), so on Intel Mac +# the librosa -> numba -> llvmlite chain triggers a from-source build +# that fails inside CI and on the host without LLVM 14/15 headers. +# Tracked separately in unslothai/unsloth#5046. NO_TORCH_SKIP_PACKAGES = { "torch-stoi", "timm", @@ -612,6 +631,7 @@ NO_TORCH_SKIP_PACKAGES = { "torch-c-dlpack-ext", "openai-whisper", "transformers-cfg", + "librosa", } @@ -839,6 +859,14 @@ def pip_install( if actual_req is not None and NO_TORCH and NO_TORCH_SKIP_PACKAGES: actual_req = _filter_requirements(actual_req, NO_TORCH_SKIP_PACKAGES) temp_reqs.append(actual_req) + if actual_req is not None and PLATFORM_LACKS_TORCHCODEC_WHEEL: + # Linux aarch64 / Windows ARM64 / Intel Mac have no torchcodec + # wheel. `unsloth studio update --local` does not pass + # --no-torch, so the NO_TORCH filter above does not fire; do + # the targeted skip independently so the audio extras step + # does not take down the whole update. + actual_req = _filter_requirements(actual_req, {"torchcodec"}) + temp_reqs.append(actual_req) req_args_pip: list[str] = [] req_args_uv: list[str] = [] if actual_req is not None: diff --git a/studio/setup.sh b/studio/setup.sh index c5beb7ebd3..e8fc8f6f13 100755 --- a/studio/setup.sh +++ b/studio/setup.sh @@ -671,6 +671,16 @@ elif [ "$_HOST_SYSTEM" = "Linux" ] \ && [ "$_HOST_MACHINE" = "x86_64" ] \ && [ "$_LINUX_HAS_GPU" = false ]; then _HELPER_RELEASE_REPO="ggml-org/llama.cpp" +elif [ "$_HOST_SYSTEM" = "Linux" ] \ + && { [ "$_HOST_MACHINE" = "aarch64" ] || [ "$_HOST_MACHINE" = "arm64" ]; } \ + && [ "$_LINUX_HAS_GPU" = false ]; then + # Linux ARM64 (Ampere Altra, Raspberry Pi 5, GitHub `ubuntu-24.04-arm`, + # CPU-only Jetson rescue mode, ...). unslothai/llama.cpp only ships + # the Linux CUDA bundles, so without this branch the prebuilt + # resolver returns 0 attempts on every release and the installer + # falls all the way back to a source build. Upstream ggml-org ships + # llama-bNNNN-bin-ubuntu-arm64.tar.gz from at least b9072 onward. + _HELPER_RELEASE_REPO="ggml-org/llama.cpp" else _HELPER_RELEASE_REPO="unslothai/llama.cpp" fi diff --git a/tests/studio/_playwright_robust.py b/tests/studio/_playwright_robust.py index 3deeb38cda..b190b2b3e1 100644 --- a/tests/studio/_playwright_robust.py +++ b/tests/studio/_playwright_robust.py @@ -436,6 +436,8 @@ def evaluate_fetch( headers: dict[str, str] | None = None, body: Any = None, timeout_ms: int = 20_000, + transport_retries: int = 2, + transport_backoff_ms: int = 250, ) -> dict[str, Any]: """Run `fetch(url, opts)` inside the page with an AbortSignal deadline. @@ -482,16 +484,47 @@ def evaluate_fetch( } } """ - return page.evaluate( - js, - { - "url": url, - "method": method, - "headers": headers or {}, - "body": body_arg, - "timeoutMs": int(timeout_ms), - }, - ) + payload = { + "url": url, + "method": method, + "headers": headers or {}, + "body": body_arg, + "timeoutMs": int(timeout_ms), + } + # Bounded retry on transport failures only. + # status != 0 -> real HTTP response (incl. 4xx/5xx); propagate. + # AbortError -> caller's deadline; propagate. + # else (==0) -> stale-keepalive / "TypeError: Failed to fetch" + # on macos-14 right after auth rotations close + # existing sessions. Retry after backoff so the + # browser pool evicts the dead socket. + last: dict[str, Any] | None = None + attempts = max(1, int(transport_retries) + 1) + for attempt in range(attempts): + result = page.evaluate(js, payload) + last = result + try: + status = int(result.get("status") or 0) + except (TypeError, ValueError): + status = 0 + if status != 0: + return result + err = str(result.get("error") or "") + if "AbortError" in err: + return result + if attempt < attempts - 1: + wait_ms = transport_backoff_ms * (2**attempt) + try: + sys.stderr.write( + f"[evaluate_fetch] {method} {url}: transport failure " + f"({attempt + 1}/{attempts}, err={err!r}); " + f"retrying in {wait_ms}ms\n" + ) + sys.stderr.flush() + except Exception: + pass + time.sleep(wait_ms / 1000.0) + return last or {"status": 0, "body": None, "error": "no attempt made"} # ─────────────────────────────────────────────────────────────────────