From 2c4bf9ae358a80cd2e4020d33134d3816a9b6498 Mon Sep 17 00:00:00 2001
From: Daniel Han <danielhanchen@gmail.com>
Date: Tue, 5 May 2026 23:22:22 -0700
Subject: [PATCH] Route CPU-only Linux x86_64 to ggml-org/llama.cpp prebuilts
 (#5302)

* Route CPU-only Linux x86_64 to ggml-org/llama.cpp prebuilts

setup.sh hard-coded _HELPER_RELEASE_REPO=unslothai/llama.cpp for every
non-Darwin host. unslothai/llama.cpp only publishes Linux CUDA bundles
(app-*-linux-x64-cuda*.tar.gz), so a CPU-only Linux host walked ~30
releases looking for a non-existent app-*-linux-x64-cpu asset, exited
the prebuilt planner with "no compatible Linux prebuilt asset was
found", and fell through to a source build. Free CI runners
(ubuntu-latest with no GPU) hit this on every install, and anyone
running Studio on a Linux laptop without an NVIDIA GPU paid the
~3 minute cmake+make cost on first install.

ggml-org publishes llama-<tag>-bin-ubuntu-x64.tar.gz on every release
and install_llama_prebuilt.py already knows how to fetch it: when
called with --published-repo ggml-org/llama.cpp, the Linux x86_64 +
not has_usable_nvidia branch in direct_upstream_release_plan picks up
that asset directly. The fix is purely on the routing side.

Tighten the gate so a Linux host routes to ggml-org only when it is
x86_64 and has no GPU detection tool installed (nvidia-smi, rocminfo,
amd-smi, hipconfig, hipinfo). Everything else stays on the current
path:

  - macOS: already on ggml-org, unchanged
  - Windows: already on ggml-org via setup.ps1, unchanged
  - Linux CUDA: nvidia-smi present -> unslothai/llama.cpp, unchanged
  - Linux ROCm: rocminfo / amd-smi / hipconfig / hipinfo present
                -> unslothai/llama.cpp -> source build with HIP,
                unchanged
  - Linux Intel / Vulkan / SYCL: no NVIDIA / AMD tools, hits the new
                ggml-org route, gets upstream CPU asset (same as
                today's source-build CPU output, ~3 min faster)
  - Linux arm64 / s390x: not x86_64 -> unslothai/llama.cpp ->
                source build, unchanged

* Tighten routing comment in studio/setup.sh
---
 studio/setup.sh | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/studio/setup.sh b/studio/setup.sh
index 3e875eed30..1b6a3db343 100755
--- a/studio/setup.sh
+++ b/studio/setup.sh
@@ -582,11 +582,30 @@ _LLAMA_CPP_DEGRADED=false
 _LLAMA_FORCE_COMPILE="${UNSLOTH_LLAMA_FORCE_COMPILE:-0}"
 _REQUESTED_LLAMA_TAG="${UNSLOTH_LLAMA_TAG:-${_DEFAULT_LLAMA_TAG}}"
 _HOST_SYSTEM="$(uname -s 2>/dev/null || true)"
+_HOST_MACHINE="$(uname -m 2>/dev/null || true)"
+
+# Pick the release repo install_llama_prebuilt.py plans against.
+# unslothai/llama.cpp ships only Linux CUDA bundles, so CPU-only Linux
+# x86_64 routes to ggml-org for bin-ubuntu-x64.tar.gz. Anything with a
+# GPU tool installed stays on unslothai (CUDA bundle / ROCm source build).
+_LINUX_HAS_GPU=false
+for _GPU_TOOL in nvidia-smi rocminfo amd-smi hipconfig hipinfo; do
+    if command -v "$_GPU_TOOL" >/dev/null 2>&1; then
+        _LINUX_HAS_GPU=true
+        break
+    fi
+done
+
 if [ "$_HOST_SYSTEM" = "Darwin" ]; then
     _HELPER_RELEASE_REPO="ggml-org/llama.cpp"
+elif [ "$_HOST_SYSTEM" = "Linux" ] \
+        && [ "$_HOST_MACHINE" = "x86_64" ] \
+        && [ "$_LINUX_HAS_GPU" = false ]; then
+    _HELPER_RELEASE_REPO="ggml-org/llama.cpp"
 else
     _HELPER_RELEASE_REPO="unslothai/llama.cpp"
 fi
+unset _GPU_TOOL
 _LLAMA_PR="${UNSLOTH_LLAMA_PR:-}"
 _SKIP_PREBUILT_INSTALL=false
 _LLAMA_PR_FORCE="${UNSLOTH_LLAMA_PR_FORCE:-${_DEFAULT_LLAMA_PR_FORCE}}"