GitHub macOS runners (and any headless or virtualized Mac) have no usable
Metal, so the macos-arm64 prebuilt loads the validation model on CPU. The new
offload check rejected it and forced a source build that also runs on CPU and
then failed to launch, breaking 'Install + load (macos-*)' and 'Studio API &
Auth Tests'.
A CPU-only Metal load is an unfixable environment limitation, not a fixable
binary fault like a missing cudart/cublas DLL or a PTX-only CUDA build, so a
rebuild gives the same result. Split the GPU kinds into _GPU_INSTALL_KINDS
(launched with --n-gpu-layers, so a broken libggml-metal.dylib still surfaces)
and _GPU_OFFLOAD_REQUIRED_KINDS (CUDA/ROCm/HIP only, where a CPU-only load is
rejected). validate_server, the existing-install re-validation, the fast-path
reuse gate, the smoke-test require-signal, and the success message all key on
the offload-required set, so macOS accepts its prebuilt while CUDA/ROCm
rejection is unchanged.
run_smoke_spoof drives the rejection contract with a CUDA kind on every runner
(install_kind is explicit, so it is OS-independent) and adds a macOS-only case
asserting a CPU-only Metal load is accepted. New unit tests cover the split.
- direct_upstream_release_plan: Linux x86_64 and arm64 CPU branches now also
require not host.has_rocm, so a ROCm host whose HIP prebuilt is missing or
rejected never silently takes a CPU upstream tarball as a 'success'.
- setup.ps1: a Windows ROCm host with no usable HIP prebuilt and no HIP source
build path is now marked LlamaCppDegraded so the CPU last-resort installs a
clearly labelled CPU build instead of a 'built' install that runs on CPU.
- validate_server gains require_gpu_signal; smoke_test_server_binary sets it for
GPU install kinds so the '0 = offload confirmed' CLI contract treats a
no-GPU-signal log as inconclusive (EXIT_ERROR) rather than a silent pass.
- run_smoke_spoof + unit test cover the new no-signal-is-inconclusive contract.
A metadata match used to short-circuit straight to reuse, so a previously
installed CPU-only 'GPU' binary survived every rerun and restart -- the exact
'picks CPU forever' report. install_prebuilt now smoke-tests a matching GPU
install and reinstalls if it loads on CPU; setup.sh/setup.ps1 smoke-test a reused
source build on a GPU host and rebuild if it ran on CPU. Non-GPU installs keep
the fast path.
Reviewers found a real CPU-only log shape that passed: an 'offloading 0
repeating layers to GPU' planning line, or a GPU KV/compute buffer, could read
as GPU before the definitive 'offloaded 0/33' was seen. Check the explicit
counted offload first (any N>0 wins, all zero is CPU-only), restrict the
buffer-size signal to GPU model buffers (KV/compute on GPU with weights on CPU
is still CPU inference), and add HIP/MUSA/CANN to the model-buffer markers so an
older log naming those backends is not misread as CPU. Same in both classifiers.
Exclude host-pinned buffers (CUDA_Host etc.) from the GPU buffer-size signal so a
binary that pins host memory but loads weights on CPU is not misread as GPU
offload. Scan every 'offloaded N/M layers to GPU' line and accept if any N>0 so a
speculative draft model logging 0/k before the main model's 33/33 is not flagged
CPU-only. Same fix in the installer and runtime classifiers.
A llama-server whose GPU backend fails to initialize still serves HTTP 200 from
CPU, so install validation accepted it and Studio ran 'GPU' inference on CPU
(#5807/#5106/#5830). Add a log classifier (buffer-size, offloaded-layers, and
device_info signals) and reject a GPU-intended binary that loaded on CPU, so the
resolver advances. Crucially, a GPU host no longer gets a CPU prebuilt appended
to its simple-policy attempts, so when no GPU bundle offloads the installer falls
through to a source build for the native arch instead of silently installing CPU.
Add a --smoke-test CLI for setup scripts and align the runtime classifier with
the same signals so its CPU-only warning fires on current llama.cpp.
