ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Studio: unblock install on Linux ARM64 + Windows ARM64 + Intel Mac (#5790)

Browse source 
* Studio: unblock cross-platform install on Linux ARM64 + Windows ARM64

Three independent bugs that together prevent `install.sh` /
`install.ps1` from completing on the ARM machines GitHub Actions now
ships (`ubuntu-24.04-arm`, `windows-11-arm`) and on equivalent real
hosts (Ampere Altra, Raspberry Pi 5, Snapdragon X Elite, ...).

Validated on the staging-2 cross-OS smoke suite -- five per-OS
workflows pinned to `ubuntu-latest`, `ubuntu-24.04-arm`, `macos-14`,
`macos-15-intel`, `windows-11-arm`. Before this change Windows ARM
exits 1 in the winget gate and Linux ARM source-builds llama.cpp
because the prebuilt selector returns 0 attempts; with it both reach
healthy /api/health.

1. studio/install_llama_prebuilt.py -- resolve_simple_install_release_plans
   had explicit branches for windows+x86_64, macos+arm64, macos+x86_64
   and linux+x86_64 only. Upstream ggml-org/llama.cpp ships
   `llama-bNNNN-bin-ubuntu-arm64.tar.gz` and
   `llama-bNNNN-bin-win-cpu-arm64.zip` (visible in the b9334 release
   manifest), so the missing elif branches force every Linux ARM64 and
   Windows ARM64 host into a source build even when a perfectly good
   upstream prebuilt is one HTTP GET away. Two new branches mirror the
   existing CPU variants; runtime_patterns_for_choice and
   runtime_payload_health_groups gain `linux-arm64` (.so layout) and
   `windows-arm64` (.dll layout) so the health-check pass-through
   matches the asset shape.

2. studio/setup.sh -- the helper-release-repo selector routed any
   non-x86_64 Linux to `unslothai/llama.cpp`, which only publishes the
   Linux CUDA bundle set. The result on Linux ARM64 was a guaranteed
   `direct_linux_release_plan` raise of "no compatible Linux prebuilt
   asset was found" on every release in the scan, then a source-build
   fallback. Pin Linux ARM64 (CPU-only) to `ggml-org/llama.cpp` so the
   new branch in (1) can see the upstream asset. setup.ps1 already
   hardcodes `ggml-org/llama.cpp`, so Windows ARM64 picks up (1)
   without an additional change.

3. install.ps1 -- the winget pre-check hard-failed before Python or uv
   detection. `windows-11-arm` runners (and many corporate Windows
   hosts without the Microsoft Store) ship without winget but already
   have a usable Python plus the Astral uv PowerShell installer
   reachable. Demote the winget check to a soft warning, defer the
   hard failure to the Python install branch (which is the only path
   that genuinely needs winget), and let the uv install fall through
   to `https://astral.sh/uv/install.ps1` when winget is absent. The
   uv PowerShell installer was already the existing fallback for the
   "winget present but uv install failed" case; this just makes it
   the primary path on hosts without winget.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: filter torchcodec on platforms without wheels

torchcodec 0.10.0 ships wheels for manylinux_2_28_x86_64,
macosx_12_0_arm64, and win_amd64 only -- visible on its PyPI page and
in the resolver error reported by #4446. install_python_stack.py
pulls torchcodec via extras-no-deps.txt, which is now installed
unconditionally during `unsloth studio update --local` (the update
command has no --no-torch flag). Result on Linux aarch64 /
Windows ARM64 / Intel Mac (when invoked outside the install.sh
auto-skip-torch path):

  ERROR: Could not find a version that satisfies the requirement
  torchcodec==0.10.0 (from versions: 0.0.0.dev0, ...)
  ERROR: No matching distribution found for torchcodec==0.10.0
  error          Installing extras (no-deps) (pip) failed (exit code 1)

`NO_TORCH_SKIP_PACKAGES` already lists torchcodec but only fires
when NO_TORCH is true -- the update path inherits no NO_TORCH from
the original install and inferrence falls back to IS_MAC_INTEL only,
so Linux aarch64 / Windows ARM64 sail past the guard. Adds a
platform predicate PLATFORM_LACKS_TORCHCODEC_WHEEL and applies the
torchcodec filter unconditionally there, independent of NO_TORCH.

Surfaced by the staging-2 cross-OS smoke `unsloth studio update`
step on ubuntu-24.04-arm; verified the same step is green with this
patch overlaid.

* Studio: skip librosa on no-torch hosts (unblocks Intel Mac install)

Closes the last cross-platform install gap surfaced by the staging-2
cross-OS smoke (see unslothai/unsloth#5046 for the original report):
`install.sh --local` on macos-15-intel fails at

  × Failed to build `llvmlite==0.47.0`
  error: failed-wheel-build-for-install
  ╰─> llvmlite
  error          studio setup failed (exit code 1)

Root cause: upstream llvmlite dropped the macosx_x86_64 wheel between
0.42.0 and 0.46.0 (https://pypi.org/project/llvmlite/0.47.0/#files --
only macosx_arm64 / manylinux / win_amd64 remain). pip falls back to
a from-source build of llvmlite's FFI, which needs LLVM 14/15 dev
headers and matching llvm-config -- not present in Xcode Command
Line Tools' libclang and not installed by install.sh's MAC_INTEL
deps branch.

llvmlite enters Studio's tree via librosa -> numba -> llvmlite in
extras.txt. openai-whisper (extras.txt:28) would also pull numba but
is already filtered on no-torch hosts. Adding librosa to the same
NO_TORCH_SKIP_PACKAGES set makes the install go through cleanly on
Intel Mac (auto-detected NO_TORCH=true via the MAC_INTEL branch) and
on any user-passed --no-torch host where torch-dependent audio
pipelines would not run anyway.

Tracked / verified on the danielhanchen/unsloth-staging-2#154 smoke
matrix (macos-15-intel).

* Studio UI tests: retry evaluate_fetch on transport-level failure (PR #5790)

Mac Studio UI CI on this PR (run 26496820814, job 78026959359) failed
with /api/models/list status=0 error='TypeError: Failed to fetch'.
The artifact studio.log shows the server answered the two preceding
/api/models/list calls from the React mount (both 200) but never
received the third call from the test script: the browser reused a
kept-alive HTTP/1.1 socket that uvicorn (5s keep_alive_timeout) had
closed ~130ms earlier. Chromium under --single-process on macos-14
free runners is most prone to this; the post /api/auth/change-password
session churn accelerates it. A rerun on the same SHA passed, which is
the classic flake signature.

evaluate_fetch in tests/studio/_playwright_robust.py already returns a
structured {status: 0, body: None, error: "..."} on JS-side throws, but
every caller treats status=0 as fatal. Add a bounded retry inside the
helper so the one class of failure recovers transparently:

  status != 0       -> real HTTP response (incl. 4xx/5xx); propagate.
  error has "AbortError" -> caller's AbortSignal deadline; propagate.
  else (status==0)  -> stale-keepalive or other transport failure;
                       retry after 250ms / 500ms backoff so the pool
                       evicts the dead socket before the next attempt.

Defaults transport_retries=2, transport_backoff_ms=250 (max added
latency on the happy path is zero; on a transport failure: up to
750ms of sleep). Callers keep the existing {status, body, error} shape;
no call-site changes needed.

Verified: tests/studio/_playwright_robust.py compiles; signature
gains two kwonly args (transport_retries, transport_backoff_ms);
8 evaluate_fetch call sites in playwright_chat_ui.py +
playwright_extra_ui.py pick up the retry without change.

---------

Co-authored-by: danielhanchen <info@unsloth.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Daniel Han 2026-05-27 04:53:38 -07:00 • committed by GitHub
commit ca55acbb5f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 158 additions and 27 deletions
Download patch file Download diff file Expand all files Collapse all files

42
install.ps1
View file

@ -887,12 +887,19 @@ shell.Run cmd, 0, False
}
# ── Check winget ──
# winget is only needed to install Python or uv. If both are
# already on PATH (Windows ARM64 GitHub-hosted runners, manual
# python.org + Astral uv installs, corporate locked-down hosts
# without the Store, etc.) the script can proceed without it.
# We defer the hard failure to the Python / uv install branches
# below, where winget is actually invoked.
Write-TauriLog "STEP" "Checking system dependencies"
if (-not (Get-Command winget -ErrorAction SilentlyContinue)) {
step "winget" "not available" "Red"
substep "Install it from https://aka.ms/getwinget" "Yellow"
substep "or install Python $PythonVersion and uv manually, then re-run." "Yellow"
return (Exit-InstallFailure "winget is not available")
$script:WingetAvailable = [bool](Get-Command winget -ErrorAction SilentlyContinue)
if ($script:WingetAvailable) {
step "winget" "available"
} else {
step "winget" "not available -- will require Python + uv to be already installed" "Yellow"
substep "Get it from https://aka.ms/getwinget if Python / uv are not already on PATH." "Yellow"
}
# ── Helper: detect a working Python 3.11-3.13 on the system ──
@ -973,6 +980,12 @@ shell.Run cmd, 0, False
step "python" "Python $($DetectedPython.Version) already installed"
}
if (-not $DetectedPython) {
if (-not $script:WingetAvailable) {
Write-Host "[ERROR] No compatible Python (3.11-3.13) found and winget is unavailable on this host." -ForegroundColor Red
Write-Host " Install Python $PythonVersion from https://www.python.org/downloads/" -ForegroundColor Yellow
Write-Host " and re-run this installer (make sure 'Add Python to PATH' is checked)." -ForegroundColor Yellow
return (Exit-InstallFailure "winget required to install Python on this host")
}
substep "installing Python ${PythonVersion}..."
$pythonPackageId = "Python.Python.$PythonVersion"
# Temporarily lower ErrorActionPreference so that winget stderr
@ -1024,14 +1037,19 @@ shell.Run cmd, 0, False
Write-TauriLog "STEP" "Installing uv package manager"
if (-not (Get-Command uv -ErrorAction SilentlyContinue)) {
substep "installing uv package manager..."
$prevEAP = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try { winget install --id=astral-sh.uv -e --accept-package-agreements --accept-source-agreements } catch {}
$ErrorActionPreference = $prevEAP
Refresh-SessionPath
# Fallback: if winget didn't put uv on PATH, try the PowerShell installer
if ($script:WingetAvailable) {
$prevEAP = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try { winget install --id=astral-sh.uv -e --accept-package-agreements --accept-source-agreements } catch {}
$ErrorActionPreference = $prevEAP
Refresh-SessionPath
}
# Fallback: if winget is unavailable or didn't put uv on PATH,
# use Astral's official PowerShell installer. This is the only
# supported path on hosts without winget (Windows ARM64 runners,
# corporate machines without the Store, etc.).
if (-not (Get-Command uv -ErrorAction SilentlyContinue)) {
substep "trying alternative uv installer..." "Yellow"
substep "installing uv via https://astral.sh/uv/install.ps1..." "Yellow"
Invoke-Expression (Invoke-RestMethod -Uri "https://astral.sh/uv/install.ps1")
Refresh-SessionPath
}

50
studio/install_llama_prebuilt.py
View file

@ -1349,6 +1349,24 @@ def direct_upstream_release_plan(
install_kind = "windows-cpu",
)
)
elif host.is_windows and host.is_arm64:
# Upstream ggml-org/llama.cpp ships llama-bNNNN-bin-win-cpu-arm64.zip
# (visible in the b9334 release manifest). Without this branch the
# selector returned 0 attempts and the installer fell back to a
# source build on every Windows ARM64 host.
cpu_asset = f"llama-{release_tag}-bin-win-cpu-arm64.zip"
cpu_url = assets.get(cpu_asset)
if cpu_url:
attempts.append(
AssetChoice(
repo = repo,
tag = release_tag,
name = cpu_asset,
url = cpu_url,
source_label = "upstream",
install_kind = "windows-arm64",
)
)
elif host.is_macos and host.is_arm64:
asset_name = f"llama-{release_tag}-bin-macos-arm64.tar.gz"
asset_url = assets.get(asset_name)
@ -1391,6 +1409,25 @@ def direct_upstream_release_plan(
install_kind = "linux-cpu",
)
)
elif host.is_linux and host.is_arm64 and not host.has_usable_nvidia:
# Upstream ggml-org/llama.cpp ships llama-bNNNN-bin-ubuntu-arm64.tar.gz
# (visible in the b9334 release manifest). Without this branch the
# selector returned 0 attempts and the installer fell back to a
# source build on every Linux ARM64 host (DGX Spark, Ampere
# Altra, GitHub-hosted ubuntu-24.04-arm runners, etc.).
asset_name = f"llama-{release_tag}-bin-ubuntu-arm64.tar.gz"
asset_url = assets.get(asset_name)
if asset_url:
attempts.append(
AssetChoice(
repo = repo,
tag = release_tag,
name = asset_name,
url = asset_url,
source_label = "upstream",
install_kind = "linux-arm64",
)
)
if not attempts:
raise PrebuiltFallback("no compatible upstream prebuilt asset was found")
return InstallReleasePlan(
@ -3833,11 +3870,16 @@ def runtime_patterns_for_choice(choice: AssetChoice) -> list[str]:
# libraries between b9279 and b9283) without us re-enumerating
# every new file. Studio only invokes llama-server and llama-quantize;
# other CLIs upstream ships (llama-cli, llama-bench, ...) are skipped.
if choice.install_kind in {"linux-cpu", "linux-cuda", "linux-rocm"}:
if choice.install_kind in {"linux-cpu", "linux-cuda", "linux-rocm", "linux-arm64"}:
return ["llama-server", "llama-quantize", "lib*.so*"]
if choice.install_kind in {"macos-arm64", "macos-x64"}:
return ["llama-server", "llama-quantize", "lib*.dylib"]
if choice.install_kind in {"windows-cpu", "windows-cuda", "windows-hip"}:
if choice.install_kind in {
"windows-cpu",
"windows-cuda",
"windows-hip",
"windows-arm64",
}:
return ["llama-server.exe", "llama-quantize.exe", "*.dll"]
raise PrebuiltFallback(
f"unsupported install kind for runtime overlay: {choice.install_kind}"
@ -5188,7 +5230,7 @@ def load_prebuilt_metadata(install_dir: Path) -> dict[str, Any] | None:
def runtime_payload_health_groups(choice: AssetChoice) -> list[list[str]]:
if choice.install_kind == "linux-cpu":
if choice.install_kind in {"linux-cpu", "linux-arm64"}:
return [
["libllama-common.so*"],
["libllama.so*"],
@ -5223,7 +5265,7 @@ def runtime_payload_health_groups(choice: AssetChoice) -> list[list[str]]:
["libmtmd.so*"],
["libggml-hip.so*"],
]
if choice.install_kind == "windows-cpu":
if choice.install_kind in {"windows-cpu", "windows-arm64"}:
return [["llama.dll"]]
if choice.install_kind == "windows-cuda":
groups = [["llama.dll"], ["ggml-cuda.dll"]]

30
studio/install_python_stack.py
View file

@ -38,6 +38,18 @@ IS_WINDOWS = sys.platform == "win32"
IS_MACOS = sys.platform == "darwin"
IS_MAC_INTEL = IS_MACOS and platform.machine() == "x86_64"
IS_MAC_ARM = IS_MACOS and platform.machine() == "arm64"
IS_LINUX = sys.platform.startswith("linux")
# torchcodec ships wheels only for manylinux_2_28_x86_64,
# macosx_12_0_arm64, and win_amd64 (visible in the 0.10.0 PyPI page).
# Trying to install it on any other host fails the whole
# extras-no-deps step. `unsloth studio update` does not have a
# --no-torch flag, so on these hosts the audio extras must be
# filtered out independent of the NO_TORCH env var.
PLATFORM_LACKS_TORCHCODEC_WHEEL = (
(IS_LINUX and platform.machine() in {"aarch64", "arm64"})
or (IS_WINDOWS and platform.machine().lower() in {"arm64", "aarch64"})
or IS_MAC_INTEL
)
# ── ROCm / AMD GPU support ─────────────────────────────────────────────────────
# Mapping from detected ROCm (major, minor) to the best PyTorch wheel tag on
@ -604,7 +616,14 @@ WINDOWS_SKIP_PACKAGES = {"open_spiel", "triton_kernels"}
# Packages to skip when torch is unavailable (Intel Mac GGUF-only mode).
# These packages either *are* torch extensions or have unconditional
# ``Requires-Dist: torch`` in their published metadata, so installing
# them would pull torch back into the environment.
# them would pull torch back into the environment. ``librosa`` also
# lives in this set even though it does not itself require torch:
# upstream ``llvmlite`` dropped its macOS x86_64 wheel between 0.42.0
# and 0.46.0+ (see https://pypi.org/project/llvmlite/0.47.0/#files --
# only macosx_arm64 / manylinux / win_amd64 remain), so on Intel Mac
# the librosa -> numba -> llvmlite chain triggers a from-source build
# that fails inside CI and on the host without LLVM 14/15 headers.
# Tracked separately in unslothai/unsloth#5046.
NO_TORCH_SKIP_PACKAGES = {
"torch-stoi",
"timm",
@ -612,6 +631,7 @@ NO_TORCH_SKIP_PACKAGES = {
"torch-c-dlpack-ext",
"openai-whisper",
"transformers-cfg",
"librosa",
}
@ -839,6 +859,14 @@ def pip_install(
if actual_req is not None and NO_TORCH and NO_TORCH_SKIP_PACKAGES:
actual_req = _filter_requirements(actual_req, NO_TORCH_SKIP_PACKAGES)
temp_reqs.append(actual_req)
if actual_req is not None and PLATFORM_LACKS_TORCHCODEC_WHEEL:
# Linux aarch64 / Windows ARM64 / Intel Mac have no torchcodec
# wheel. `unsloth studio update --local` does not pass
# --no-torch, so the NO_TORCH filter above does not fire; do
# the targeted skip independently so the audio extras step
# does not take down the whole update.
actual_req = _filter_requirements(actual_req, {"torchcodec"})
temp_reqs.append(actual_req)
req_args_pip: list[str] = []
req_args_uv: list[str] = []
if actual_req is not None:

10
studio/setup.sh
View file

@ -671,6 +671,16 @@ elif [ "$_HOST_SYSTEM" = "Linux" ] \
&& [ "$_HOST_MACHINE" = "x86_64" ] \
&& [ "$_LINUX_HAS_GPU" = false ]; then
_HELPER_RELEASE_REPO="ggml-org/llama.cpp"
elif [ "$_HOST_SYSTEM" = "Linux" ] \
&& { [ "$_HOST_MACHINE" = "aarch64" ] || [ "$_HOST_MACHINE" = "arm64" ]; } \
&& [ "$_LINUX_HAS_GPU" = false ]; then
# Linux ARM64 (Ampere Altra, Raspberry Pi 5, GitHub `ubuntu-24.04-arm`,
# CPU-only Jetson rescue mode, ...). unslothai/llama.cpp only ships
# the Linux CUDA bundles, so without this branch the prebuilt
# resolver returns 0 attempts on every release and the installer
# falls all the way back to a source build. Upstream ggml-org ships
# llama-bNNNN-bin-ubuntu-arm64.tar.gz from at least b9072 onward.
_HELPER_RELEASE_REPO="ggml-org/llama.cpp"
else
_HELPER_RELEASE_REPO="unslothai/llama.cpp"
fi

53
tests/studio/_playwright_robust.py
View file

@ -436,6 +436,8 @@ def evaluate_fetch(
headers: dict[str, str] | None = None,
body: Any = None,
timeout_ms: int = 20_000,
transport_retries: int = 2,
transport_backoff_ms: int = 250,
) -> dict[str, Any]:
"""Run `fetch(url, opts)` inside the page with an AbortSignal deadline.
@ -482,16 +484,47 @@ def evaluate_fetch(
}
}
"""
return page.evaluate(
js,
{
"url": url,
"method": method,
"headers": headers or {},
"body": body_arg,
"timeoutMs": int(timeout_ms),
},
)
payload = {
"url": url,
"method": method,
"headers": headers or {},
"body": body_arg,
"timeoutMs": int(timeout_ms),
}
# Bounded retry on transport failures only.
# status != 0 -> real HTTP response (incl. 4xx/5xx); propagate.
# AbortError -> caller's deadline; propagate.
# else (==0) -> stale-keepalive / "TypeError: Failed to fetch"
# on macos-14 right after auth rotations close
# existing sessions. Retry after backoff so the
# browser pool evicts the dead socket.
last: dict[str, Any] | None = None
attempts = max(1, int(transport_retries) + 1)
for attempt in range(attempts):
result = page.evaluate(js, payload)
last = result
try:
status = int(result.get("status") or 0)
except (TypeError, ValueError):
status = 0
if status != 0:
return result
err = str(result.get("error") or "")
if "AbortError" in err:
return result
if attempt < attempts - 1:
wait_ms = transport_backoff_ms * (2**attempt)
try:
sys.stderr.write(
f"[evaluate_fetch] {method} {url}: transport failure "
f"({attempt + 1}/{attempts}, err={err!r}); "
f"retrying in {wait_ms}ms\n"
)
sys.stderr.flush()
except Exception:
pass
time.sleep(wait_ms / 1000.0)
return last or {"status": 0, "body": None, "error": "no attempt made"}
# ─────────────────────────────────────────────────────────────────────