install_python_stack.py:
- Print uv error output on failure for debuggability
- Refactor pip_install() to use early return after uv success,
removing duplicated pip command path
setup.sh:
- Guard nvidia-smi command substitution with || true so it does
not abort the script under set -euo pipefail when nvidia-smi
fails (e.g., containerized environments, driver quirks)
- Read all GPU compute capabilities and deduplicate, so
mixed-GPU hosts get kernels built for all present architectures
instead of only the first GPU
Restore separate cmake --build calls for llama-server and
llama-quantize on both setup.sh and setup.ps1. The combined
approach made llama-quantize failure fatal, but it was originally
best-effort (|| true on Linux, [WARN] on Windows). The timing
savings from combining was only ~2.7s, not worth the semantic
change.
The Ninja + arch detection speedups are preserved (55s vs 1m 37s).
Three improvements to the llama.cpp build step in setup.sh:
1. Detect GPU compute capability via nvidia-smi and limit
CMAKE_CUDA_ARCHITECTURES to the current GPU. Without this, cmake
builds for all default CUDA architectures which is very slow.
2. Use Ninja build generator when available. Ninja has better
parallelism than Make for CUDA compilation.
3. Build both llama-server and llama-quantize targets in a single
cmake --build invocation for better parallelism.
4. Add --threads=0 to CMAKE_CUDA_FLAGS for multi-threaded nvcc
compilation.
Measured on 192-core machine with B200 (sm_100):
Make (all archs): very slow (minutes for each arch)
Make (single arch): 1m 37s
Ninja (single arch): 55s
Speedup: ~1.7x
Combined with the uv change, total setup goes from ~4m 35s to ~1m 40s.
