4c86ee4bff
Previously we passed --threads only when the caller set an explicit value, which meant llama-server fell back to its internal default. That default has varied across llama.cpp builds (some versions use hardware concurrency including hyperthreads, which hurts throughput on CPU-heavy inference). Always passing --threads -1 pins the behaviour to llama.cpp's auto-detect (physical cores). Caller-supplied n_threads still wins when non-None. |
||
|---|---|---|
| .. | ||
| data_recipe | ||
| export | ||
| inference | ||
| training | ||
| __init__.py | ||