4c86ee4bff
Previously we passed --threads only when the caller set an explicit value, which meant llama-server fell back to its internal default. That default has varied across llama.cpp builds (some versions use hardware concurrency including hyperthreads, which hurts throughput on CPU-heavy inference). Always passing --threads -1 pins the behaviour to llama.cpp's auto-detect (physical cores). Caller-supplied n_threads still wins when non-None. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| _html_to_md.py | ||
| anthropic_compat.py | ||
| audio_codecs.py | ||
| defaults.py | ||
| inference.py | ||
| llama_cpp.py | ||
| orchestrator.py | ||
| tools.py | ||
| worker.py | ||