5a38447b25* Studio: omit --threads when unset so llama.cpp picks physical cores Studio passed --threads -1 when no thread count was set. The intent was physical cores, but an explicit --threads -1 makes llama.cpp's arg parser resolve it to hardware_concurrency() (every hyperthread), which contends on the memory bus and slows CPU and hybrid decode (a user saw about 60-75 fall to about 20-30 tok/s under CPU offload). Leaving --threads unset keeps n_threads at -1, which llama.cpp resolves to physical cores via common_cpu_get_num_math(). Omit the flag when unset; still pin it for an explicit override and the Windows full-offload OpenMP cap. * Studio: drop inherited LLAMA_ARG_THREADS when omitting --threads Omitting --threads relies on llama.cpp resolving physical cores via common_cpu_get_num_math. But the child inherits os.environ and llama.cpp also reads --threads from LLAMA_ARG_THREADS, which routes through the arg handler and maps <=0 to hardware_concurrency. So an ambient LLAMA_ARG_THREADS would silently override the physical-core default. Scrub it from the child env only when we omit the flag. |
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| src-tauri | ||
| __init__.py | ||
| install_llama_prebuilt.py | ||
| install_python_stack.py | ||
| LICENSE.AGPL-3.0 | ||
| package-lock.json | ||
| package.json | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||