bb3676d2c8
Current llama.cpp ships the CUDA backend as a dynamically-loaded plugin (libggml-cuda.so* next to the binary), NOT a load-time dependency, so ldd llama-server | grep libggml-cuda is a false negative: it reports no CUDA on a perfectly good CUDA build. That made both is_cuda_server() (provision_llama_cuda.sh) and _have_cuda_llama_server() (setup.sh) force a needless full rebuild every run. Fix both: keep the ldd check (old monolithic builds) and additionally treat the presence of libggml-cuda.so* beside the binary as the CUDA signal. A CPU-only build has no such backend, so this stays correct for the CPU case. Verified on an N1X/sm_121 WSL build: llama-server --list-devices shows CUDA0 JMJWOA-Generic-GPU and serves on the GPU, while ldd lists no libggml-cuda; the new check correctly returns CUDA-present. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| scripts | ||
| src-tauri | ||
| __init__.py | ||
| install_llama_prebuilt.py | ||
| install_python_stack.py | ||
| LICENSE.AGPL-3.0 | ||
| package-lock.json | ||
| package.json | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||