ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Studio: source-build arm64 Linux GPU hosts, with a CPU prebuilt fallback (#5924)

Browse source 
* Studio: fall back to source build for arm64 Linux GPU hosts

setup.sh routes a Linux host with any GPU tool to the unslothai/llama.cpp fork,
which publishes only linux-x64 bundles. On an arm64 host with a GPU (GH200,
GB200, DGX Spark) the resolver then selected an x64 CUDA bundle, which cannot run
on aarch64. Routing those hosts to ggml-org instead would install a CPU-only
arm64 build, silently losing the GPU.

Guard resolve_simple_install_release_plans so an arm64 Linux host on the fork
raises PrebuiltFallback before any release is fetched, letting setup.sh do a
source build that actually targets the GPU. x86_64 hosts and arm64 CPU hosts
(which route to ggml-org) are unaffected.

Add tests covering the arm64 fork raise, the x86_64 pass-through, and the arm64
CPU ggml-org path.

* Studio: install ggml-org arm64 CPU prebuilt when the arm64 GPU source build fails

Per review of #5924: arm64 Linux GPU hosts have no CUDA prebuilt anywhere (the
unslothai fork is x64 only, ggml-org ships no Linux CUDA build), so they source
build for the GPU. If that build produces no binary, the host was left without
llama.cpp.

Add a --cpu-fallback flag to install_llama_prebuilt.py that drops the host GPU
attributes so the CPU prebuilt for the host arch is selected (a GPU host cannot
otherwise pick the CPU bundle). setup.sh calls it against ggml-org as a last
resort for arm64 Linux when the source build degraded, installing the
ubuntu-arm64 CPU build instead of leaving the host with no llama.cpp.

Add tests: force_cpu drops GPU attrs before planning, a CPU-forced arm64 host
selects the ggml-org ubuntu-arm64 bundle, and setup.sh wires the fallback.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Daniel Han 2026-06-01 06:35:43 -07:00 • committed by GitHub
commit 172d9d1c8e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 220 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

26
studio/setup.sh
View file

@ -1363,6 +1363,32 @@ else
}
fi # end _SKIP_GGUF_BUILD check
# ── arm64 Linux GPU: CPU prebuilt as a last resort ──
# arm64 Linux with a GPU has no CUDA prebuilt anywhere (the unslothai fork is
# x64 only; ggml-org ships no Linux CUDA build), so it source-builds for the
# GPU above. If that produced no binary, install ggml-org's arm64 CPU prebuilt
# instead of leaving the host without llama.cpp.
if [ "$_LLAMA_CPP_DEGRADED" = true ] \
&& [ "$_HOST_SYSTEM" = "Linux" ] \
&& { [ "$_HOST_MACHINE" = "aarch64" ] || [ "$_HOST_MACHINE" = "arm64" ]; }; then
substep "GPU source build unavailable; trying ggml-org arm64 CPU prebuilt..."
_ARM64_CPU_CMD=(
python "$SCRIPT_DIR/install_llama_prebuilt.py"
--install-dir "$LLAMA_CPP_DIR"
--llama-tag "$_REQUESTED_LLAMA_TAG"
--published-repo "ggml-org/llama.cpp"
--simple-policy
--cpu-fallback
)
# Trust the installer's exit code: it validates the server before exiting 0,
# the same signal the primary prebuilt path above relies on.
if run_quiet_no_exit "arm64 CPU prebuilt" "${_ARM64_CPU_CMD[@]}"; then
step "llama.cpp" "arm64 CPU prebuilt installed (GPU build unavailable)" "$C_WARN"
_LLAMA_CPP_DEGRADED=false
print_installed_llama_prebuilt_release "$LLAMA_CPP_DIR"
fi
fi
# ── Footer ──
if [ "$_LLAMA_ONLY" = "1" ]; then
echo ""