Builds on the 5-mode Speculative Decoding dropdown (PR #5582) by adding
two upstream-driven knobs that landed in llama.cpp #23269 (MTP
clean-up):
1. spec_draft_p_min: minimum draft probability threshold for MTP
speculative decoding (--spec-draft-p-min). Drafts below this
probability are rejected. Was non-functional pre-#23269; now
defaults to 0.0 upstream. Studio exposes it as a "Draft p-min"
numeric input below "Draft Tokens", visible only when the dropdown
is MTP or MTP+Ngram (the only modes where MTP actually engages and
the knob has effect).
2. ngram-map-k / ngram-map-k4v: new spec types added alongside
ngram-mod. Each carries its own knob triplet
(--spec-{variant}-size-n/m/min-hits). They are NOT in the
dropdown -- power-user-only -- but the load API accepts them and
the resolver emits the correct flag set when probed support is
present.
Backend
- _canonicalize_spec_mode recognises ngram-map-k / ngram-map-k4v.
- New helper _build_ngram_map_k_flags(caps, variant=...) emits the
knob triplet only when the binary advertises the knobs as real
flags (not removal stubs).
- _build_speculative_flags grows two branches and an inline
_maybe_emit_p_min helper that flows p_min through the MTP path
only. Auto on an MTP GGUF still gets p_min applied because the
resolved emission is MTP.
- LoadRequest.spec_draft_p_min (Optional[float], 0..1). Threaded
through routes/inference.py at the four wire sites and the
_request_matches_loaded_settings comparator.
- _already_in_target_state takes spec_draft_p_min so a changed p_min
bounces a reload even on the Auto-promoted path.
- probe_server_capabilities now reports spec_draft_p_min_flag,
supports_ngram_map_k, and supports_ngram_map_k4v.
Frontend
- chat-runtime-store: specDraftPMin / loadedSpecDraftPMin / setter.
- use-chat-model-runtime: hydrate p_min from /api/inference/status
and the load response. Reset p_min alongside spec mode and n_max
when the user switches to a different model.
- chat-settings-sheet: new "Draft p-min" number input
(min 0, max 1, step 0.05), visible when speculativeType is mtp
or mtp+ngram. Wired into the Reset and dirty-state machinery.
Tests
- 12 new assertions in test_llama_cpp_mtp_detection.py: p_min emission
matrix (MTP modes only; never for auto/ngram/off; auto-promoted
draft-mtp still gets p_min; graceful degrade when binary lacks
--spec-draft-p-min), ngram-map-k / ngram-map-k4v emission with the
right knob triplet, no-emit-when-unsupported, canonicalize
recognition. 373 total backend tests pass (was 361 before).
9e8b63c826