9e8b63c826
Builds on the 5-mode Speculative Decoding dropdown (PR #5582) by adding two upstream-driven knobs that landed in llama.cpp #23269 (MTP clean-up): 1. spec_draft_p_min: minimum draft probability threshold for MTP speculative decoding (--spec-draft-p-min). Drafts below this probability are rejected. Was non-functional pre-#23269; now defaults to 0.0 upstream. Studio exposes it as a "Draft p-min" numeric input below "Draft Tokens", visible only when the dropdown is MTP or MTP+Ngram (the only modes where MTP actually engages and the knob has effect). 2. ngram-map-k / ngram-map-k4v: new spec types added alongside ngram-mod. Each carries its own knob triplet (--spec-{variant}-size-n/m/min-hits). They are NOT in the dropdown -- power-user-only -- but the load API accepts them and the resolver emits the correct flag set when probed support is present. Backend - _canonicalize_spec_mode recognises ngram-map-k / ngram-map-k4v. - New helper _build_ngram_map_k_flags(caps, variant=...) emits the knob triplet only when the binary advertises the knobs as real flags (not removal stubs). - _build_speculative_flags grows two branches and an inline _maybe_emit_p_min helper that flows p_min through the MTP path only. Auto on an MTP GGUF still gets p_min applied because the resolved emission is MTP. - LoadRequest.spec_draft_p_min (Optional[float], 0..1). Threaded through routes/inference.py at the four wire sites and the _request_matches_loaded_settings comparator. - _already_in_target_state takes spec_draft_p_min so a changed p_min bounces a reload even on the Auto-promoted path. - probe_server_capabilities now reports spec_draft_p_min_flag, supports_ngram_map_k, and supports_ngram_map_k4v. Frontend - chat-runtime-store: specDraftPMin / loadedSpecDraftPMin / setter. - use-chat-model-runtime: hydrate p_min from /api/inference/status and the load response. Reset p_min alongside spec mode and n_max when the user switches to a different model. - chat-settings-sheet: new "Draft p-min" number input (min 0, max 1, step 0.05), visible when speculativeType is mtp or mtp+ngram. Wired into the Reset and dirty-state machinery. Tests - 12 new assertions in test_llama_cpp_mtp_detection.py: p_min emission matrix (MTP modes only; never for auto/ngram/off; auto-promoted draft-mtp still gets p_min; graceful degrade when binary lacks --spec-draft-p-min), ngram-map-k / ngram-map-k4v emission with the right knob triplet, no-emit-when-unsupported, canonicalize recognition. 373 total backend tests pass (was 361 before). |
||
|---|---|---|
| .. | ||
| public | ||
| src | ||
| .gitignore | ||
| .gitkeep | ||
| .npmrc | ||
| biome.json | ||
| components.json | ||
| data-designer.openapi (1).yaml | ||
| eslint.config.js | ||
| index.html | ||
| package-lock.json | ||
| package.json | ||
| tsconfig.app.json | ||
| tsconfig.json | ||
| tsconfig.node.json | ||
| vite.config.ts | ||