ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Route CPU-only Linux x86_64 to ggml-org/llama.cpp prebuilts (#5302)

Browse source 
* Route CPU-only Linux x86_64 to ggml-org/llama.cpp prebuilts

setup.sh hard-coded _HELPER_RELEASE_REPO=unslothai/llama.cpp for every
non-Darwin host. unslothai/llama.cpp only publishes Linux CUDA bundles
(app-*-linux-x64-cuda*.tar.gz), so a CPU-only Linux host walked ~30
releases looking for a non-existent app-*-linux-x64-cpu asset, exited
the prebuilt planner with "no compatible Linux prebuilt asset was
found", and fell through to a source build. Free CI runners
(ubuntu-latest with no GPU) hit this on every install, and anyone
running Studio on a Linux laptop without an NVIDIA GPU paid the
~3 minute cmake+make cost on first install.

ggml-org publishes llama-<tag>-bin-ubuntu-x64.tar.gz on every release
and install_llama_prebuilt.py already knows how to fetch it: when
called with --published-repo ggml-org/llama.cpp, the Linux x86_64 +
not has_usable_nvidia branch in direct_upstream_release_plan picks up
that asset directly. The fix is purely on the routing side.

Tighten the gate so a Linux host routes to ggml-org only when it is
x86_64 and has no GPU detection tool installed (nvidia-smi, rocminfo,
amd-smi, hipconfig, hipinfo). Everything else stays on the current
path:

  - macOS: already on ggml-org, unchanged
  - Windows: already on ggml-org via setup.ps1, unchanged
  - Linux CUDA: nvidia-smi present -> unslothai/llama.cpp, unchanged
  - Linux ROCm: rocminfo / amd-smi / hipconfig / hipinfo present
                -> unslothai/llama.cpp -> source build with HIP,
                unchanged
  - Linux Intel / Vulkan / SYCL: no NVIDIA / AMD tools, hits the new
                ggml-org route, gets upstream CPU asset (same as
                today's source-build CPU output, ~3 min faster)
  - Linux arm64 / s390x: not x86_64 -> unslothai/llama.cpp ->
                source build, unchanged

* Tighten routing comment in studio/setup.sh
This commit is contained in:
Daniel Han 2026-05-05 23:22:22 -07:00
commit 2c4bf9ae35
1 changed files with 19 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

19
studio/setup.sh
View file

@ -582,11 +582,30 @@ _LLAMA_CPP_DEGRADED=false
_LLAMA_FORCE_COMPILE="${UNSLOTH_LLAMA_FORCE_COMPILE:-0}"
_REQUESTED_LLAMA_TAG="${UNSLOTH_LLAMA_TAG:-${_DEFAULT_LLAMA_TAG}}"
_HOST_SYSTEM="$(uname -s 2>/dev/null || true)"
_HOST_MACHINE="$(uname -m 2>/dev/null || true)"
# Pick the release repo install_llama_prebuilt.py plans against.
# unslothai/llama.cpp ships only Linux CUDA bundles, so CPU-only Linux
# x86_64 routes to ggml-org for bin-ubuntu-x64.tar.gz. Anything with a
# GPU tool installed stays on unslothai (CUDA bundle / ROCm source build).
_LINUX_HAS_GPU=false
for _GPU_TOOL in nvidia-smi rocminfo amd-smi hipconfig hipinfo; do
if command -v "$_GPU_TOOL" >/dev/null 2>&1; then
_LINUX_HAS_GPU=true
break
fi
done
if [ "$_HOST_SYSTEM" = "Darwin" ]; then
_HELPER_RELEASE_REPO="ggml-org/llama.cpp"
elif [ "$_HOST_SYSTEM" = "Linux" ] \
&& [ "$_HOST_MACHINE" = "x86_64" ] \
&& [ "$_LINUX_HAS_GPU" = false ]; then
_HELPER_RELEASE_REPO="ggml-org/llama.cpp"
else
_HELPER_RELEASE_REPO="unslothai/llama.cpp"
fi
unset _GPU_TOOL
_LLAMA_PR="${UNSLOTH_LLAMA_PR:-}"
_SKIP_PREBUILT_INSTALL=false
_LLAMA_PR_FORCE="${UNSLOTH_LLAMA_PR_FORCE:-${_DEFAULT_LLAMA_PR_FORCE}}"