aa9847d337
Follow-up review on the CPU-only / NUMA hardening: - Restrict the NUMA topology to the cpuset-allowed nodes (/proc/self/status Mems_allowed_list). numactl --interleave=all only spans those, so under a Docker/systemd cpuset that exposes the host's nodes but allows one, the decision no longer sums RAM the child cannot use and wrap a launch that OOMs. - Honor env-forced CPU offload: _extra_args_forces_cpu_offload now also reads the inherited LLAMA_ARG_N_GPU_LAYERS / LLAMA_ARG_DEVICE the child would apply, so a GPU host running with LLAMA_ARG_N_GPU_LAYERS=0 or LLAMA_ARG_DEVICE=none gets the CPU-only safe defaults. CLI extras still win over the env. Unit tests and the offline simulations updated and passing. |
||
|---|---|---|
| .. | ||
| data_recipe | ||
| export | ||
| inference | ||
| rag | ||
| training | ||
| __init__.py | ||
| _torchao_stub.py | ||
| import_guards.py | ||
| tool_healing.py | ||