Studio: default llama-server --threads to -1 (auto)

Previously we passed --threads only when the caller set an explicit
value, which meant llama-server fell back to its internal default.
That default has varied across llama.cpp builds (some versions use
hardware concurrency including hyperthreads, which hurts throughput on
CPU-heavy inference). Always passing --threads -1 pins the behaviour
to llama.cpp's auto-detect (physical cores).

Caller-supplied n_threads still wins when non-None.
This commit is contained in:
Daniel Han 2026-04-24 16:17:35 +00:00
commit 4c86ee4bff

View file

@ -1528,8 +1528,10 @@ class LlamaCppBackend:
# Model fits on selected GPU(s) -- offload all layers
cmd.extend(["-ngl", "-1"])
if n_threads is not None:
cmd.extend(["--threads", str(n_threads)])
# -1 = llama.cpp auto-detect (physical cores). Pass explicitly so we
# do not inherit llama-server's internal default, which has historically
# varied (hardware concurrency incl. hyperthreads on some builds).
cmd.extend(["--threads", str(n_threads if n_threads is not None else -1)])
# Always enable Jinja chat template rendering for proper template support
cmd.extend(["--jinja"])