* report a complete load once llama-server is healthy
load_progress() derived its fraction purely from the llama-server's VmRSS over the GGUF shard total. With layers offloaded to VRAM (-ngl) the process releases the mmap'd weight pages after upload, so VmRSS sinks back well below the shard total: the fraction climbs toward ~1.0 during mmap, then collapses to a small value (~8%) once the weights are on the GPU. A fraction-driven progress bar therefore restarts and sticks there indefinitely even though the model is loaded and serving, which reads as a hang at "Starting model...".
Once the server is healthy the load is complete by definition, so report
fraction 1.0 (and bytes_loaded == bytes_total) in the ready phase regardless of resident set size. The VmRSS read is factored into _read_rss_bytes() with its original semantics preserved (0 on a missing VmRSS line, None when /proc is unavailable) so it can be unit-tested off Linux.
Fixes#5740
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* stub heavy deps in the load-progress test and guard a valueless VmRSS
Two review fixes:
1. The new test imported core.inference.llama_cpp at module top, which pulls in
loggers/structlog/httpx and fails collection with ModuleNotFoundError in the
lightweight backend test env when the file is run on its own. Stub loggers,
structlog and httpx via sys.modules.setdefault before the import, mirroring
test_llama_cpp_load_progress_matrix.py; setdefault keeps the real modules when installed. Verified the file now collects and passes with only pytest present.
2. Catch IndexError in _read_rss_bytes: a "VmRSS:" line with no value column
would make line.split()[1] raise and crash a load-progress poll. Return None
instead, with a test for the valueless line.
* Hold load-progress high-water mark and explain a never-healthy load (#5740)
load_progress() now holds a per-process VmRSS high-water mark, so the bar
no longer regresses to ~8% when -ngl offloads the weights and frees the
mmap pages mid-load.
A live server that never returns 200 on /health now gets a specific error
(context/VRAM too large, or a local proxy/VPN intercepting the loopback
probe) instead of the generic invalid-GGUF/out-of-memory message.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Hakan Baysal <hakan.baysal@trmix.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
