* Studio: kill in-flight llama-server before spawning a new one
Two rapid Apply clicks in the chat settings panel can race two
load_model calls. Both pass the Phase 1 _kill_process (because neither
has stored its Popen handle yet), both download / read metadata, and
both reach Phase 3 and spawn a server. Only the last reference is
tracked in self._process. The first server becomes an orphan that
holds the model in RAM until the kernel OOM kicks in. Addresses #5161.
Two complementary changes in studio/backend/core/inference/llama_cpp.py:
1. At load_model entry, set the existing _cancel_event so any in-flight
load aborts at its next checkpoint, then bind a fresh Event for
the new load. Subsequent _kill_process and download phases pick up
the new event.
2. Inside the Phase 3 lock, immediately before subprocess.Popen, run a
defensive _kill_process that removes any orphan handle a racing
load might have stored after the first kill ran.
Speculative decoding cleanup (also touched while in this code path):
The chat UI used to send "ngram-mod" as the wire value when the
speculative dropdown was On, and the backend mapped that to the
4-flag combo --spec-type ngram-mod --spec-ngram-size-n 24 --draft-min
48 --draft-max 64. Switch the wire value to "default" and have the
backend pass the single llama-server flag --spec-default. That flag
expands to the exact same params (see common/arg.cpp:3905-3914 in
llama.cpp). Default-on for non-vision models is preserved.
Verified:
- Unit test: planted Popen orphan handle is terminated before
self._process is overwritten.
- All ten spec-cmd mappings produce the expected llama-server args
("default" -> --spec-default, "off" / null -> no flag, vision ->
always disabled, manual "ngram-mod" / "ngram-simple" still work).
* Address review feedback on PR #5171
The previous attempt at cancelling in-flight loads via
``self._cancel_event.set()`` followed by
``self._cancel_event = threading.Event()`` was broken in two ways
(flagged independently by Gemini and Codex):
1. Sub-methods like _download_gguf consult ``self._cancel_event``
on every check. After the rebind, the in-flight thread reads the
FRESH unset Event, not the one we just set, so cancellation never
propagates.
2. Worse, if unload_model() lands between ``set()`` and the rebind,
unload's signal hits the OLD event and is then immediately
discarded when load_model swaps in a fresh Event. The user's
stop-request silently no-ops.
Revert to the original ``self._cancel_event.clear()``. The Phase 3
defensive ``_kill_process()`` introduced in this branch still closes
the orphan-process race that #5161 reports: even if two concurrent
loads both pass Phase 1 with self._process == None, the loser's
Phase 3 kill terminates the winner's Popen handle before overwriting
it, so we end up with exactly one llama-server process.
* Studio: coerce legacy speculative-type values for the simplified dropdown
The Speculative Decoding control was simplified to On (default) / Off,
but the backend still accepts and reports the older manual modes
(ngram-mod, ngram-simple). When a load response or status refresh comes
back with one of those values -- whether from an external API caller, a
model loaded before this PR landed, or a not-yet-upgraded backend -- the
controlled Select renders with an empty trigger because the value is not
in the SelectItem list.
Add a tiny normaliser at both entry points (status refresh + post-load)
so legacy manual modes coerce to "default". The user sees "On" instead
of a blank dropdown, and reapplying lets llama.cpp pick its own preferred
strategy via --spec-default.
Reviewer-flagged finding on PR #5171.
c2dc2eb1b1