aa9847d337
Follow-up review on the CPU-only / NUMA hardening: - Restrict the NUMA topology to the cpuset-allowed nodes (/proc/self/status Mems_allowed_list). numactl --interleave=all only spans those, so under a Docker/systemd cpuset that exposes the host's nodes but allows one, the decision no longer sums RAM the child cannot use and wrap a launch that OOMs. - Honor env-forced CPU offload: _extra_args_forces_cpu_offload now also reads the inherited LLAMA_ARG_N_GPU_LAYERS / LLAMA_ARG_DEVICE the child would apply, so a GPU host running with LLAMA_ARG_N_GPU_LAYERS=0 or LLAMA_ARG_DEVICE=none gets the CPU-only safe defaults. CLI extras still win over the env. Unit tests and the offline simulations updated and passing. |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| core | ||
| hub | ||
| loggers | ||
| models | ||
| plugins | ||
| requirements | ||
| routes | ||
| state | ||
| storage | ||
| tests | ||
| utils | ||
| __init__.py | ||
| _platform_compat.py | ||
| cloudflare_tunnel.py | ||
| colab.py | ||
| main.py | ||
| run.py | ||
| startup_banner.py | ||