* fix(studio/mmproj): block cross-family projectors in flat local GGUF dirs (#5347)
When a flat local GGUF directory holds several unrelated models with their
own mmproj siblings, detect_mmproj_file() returned the first projector it
walked into. For the layout reported in #5347 (Qwen weights + a Gemma
mmproj in the same dir) that meant llama-server was launched with
--mmproj pointing at the Gemma projector, which fails to load and surfaces
as a confusing crash.
Disambiguation rules:
- Drop candidates whose family token (qwen/gemma/llama/mistral/phi/...)
disagrees with the model's family. Candidates with no recognised
family token (e.g. the HF-convention 'mmproj-F16.gguf') are kept.
- Among same-family candidates, prefer the one whose stem shares the
longest prefix with the model (Qwen3.5-9B mmproj beats Qwen3.5-35B
mmproj for a Qwen3.5-9B model).
- If every candidate is dropped, return None — better than attaching
a wrong projector and getting a server-launch failure.
Tests cover the cross-family block, multi-candidate prefix tie-break,
HF-convention 'mmproj-F16.gguf', unrecognised families, and the
existing search_root walk.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/mmproj: word-bounded family match, expanded token list, launcher guard
Tighten the family-token detector to match only on word boundaries so
substring collisions stop tagging false families: phi no longer matches
sapphire, yi no longer matches yip, mimo no longer matches mimosa, and
mistral does not bleed into ministral/magistral/devstral. Pick the token
whose first occurrence is leftmost in the filename rather than the first
hit in tuple order, so merge models disambiguate predictably (llama-phi
tags llama; phi-llama tags phi).
Expand _MODEL_FAMILY_TOKENS with the families an audit of the unsloth
HF org turned up that the previous list missed: devstral, ministral,
magistral (Mistral-derivative naming), nemotron, kimi, nanonets, cosmos,
mimo, apriel, lfm. Without these, a flat local GGUF directory containing
one of these weights plus an unrelated renamed projector still hit the
original #5347 failure.
Add mmproj_matches_model_family() and call it at the llama-server launch
site in core/inference/llama_cpp.py. detect_mmproj_file already drops
cross-family candidates at discovery time, but mmproj_path can also reach
the launcher via config injection or future overrides; this guard keeps
those paths from silently loading a known-wrong projector.
Tests: 12 new cases covering substring rejection, leftmost-position
selection, new family tokens, a new flat-dir Nemotron + Gemma rejection
case, and the launcher-level guard. All 21 detect_mmproj_file tests and
the existing 106 llama_cpp tests pass.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/mmproj: pair via GGUF general.* metadata, not just filenames
Real Unsloth vision GGUFs carry rich identity metadata that has been
ignored by the discovery path. Every projector under the unsloth org
has general.type='mmproj' plus general.base_model.0.repo_url pointing
at the same upstream HF repo as its weight, and the equivalent
basename, base_model.0.name, and base_model.0.organization fields. A
flat-dir mismatch is therefore decidable from the headers alone, no
matter how the user has renamed the files.
Add utils/models/gguf_metadata.py with read_gguf_general_metadata():
a fast (~30 ms) header walk that pulls only the general.* string
fields and skips everything else, cached by (resolved path, mtime_ns,
size). Mirrors the parser shape already used by
LlamaCppBackend._read_gguf_metadata so the format handling is
consistent.
is_mmproj_by_metadata() returns True/False/None from general.type,
and pairing_score() returns 100 for an exact base_model URL match,
80 for basename plus organization match, 60 for basename only, -1
for definitive metadata disagreement, and 0 when neither side has
enough metadata to decide.
Rewire detect_mmproj_file() to a two-stage selector:
1. Detect projectors via metadata (general.type) when present, else
fall back to the filename substring heuristic. This recovers
headerless projectors AND projectors whose name does not contain
'mmproj' but whose header advertises one.
2. Score each candidate against the weight via pairing_score. Drop
candidates with score -1 (definitive metadata disagreement). For
candidates with score 0 (no usable metadata) fall back to the
existing filename family-token check, dropping recognised-family
mismatches. Pick the survivor with the highest (score,
longest_prefix, -len(stem)) tuple, so a metadata URL match
always wins over a filename-prefix match.
Tests: 16 new cases. tests/test_gguf_metadata.py covers the parser
(missing file, non-GGUF, string extraction, walking past arrays and
uint32s, cache invalidation by mtime/size) and the score helpers.
tests/test_detect_mmproj_file.py adds end-to-end cases that synthesise
real on-disk GGUF headers: URL match wins over a longer-prefix
sibling, URL mismatch returns None even when filenames match, a
projector named 'vision-projector.gguf' is still discovered via
general.type, and a 100-score header match outranks a near-perfect
filename prefix on a headerless candidate.
All 75 tests across detect_mmproj_file, gguf_metadata, llama_cpp
load progress, cached gguf routes, trained model scan, and vision
cache pass.
* studio/mmproj: shorten comments and docstrings across the #5347 changes
Trim verbose explanations to one-line statements of intent. The
behaviour is unchanged: 161 tests across detect_mmproj_file,
gguf_metadata, llama_cpp_load_progress (+ matrix), llama_server_args,
llama_cpp_cache_aware_disk_check, trained_model_scan, and vision_cache
all pass.
* studio/mmproj: shorten remaining detect_mmproj_file body comments
Trim the docstring and the dir-walking block comments inside
detect_mmproj_file to one-liners. Behaviour unchanged; 44 mmproj +
gguf_metadata + llama_cpp_load_progress tests pass.
* studio/mmproj: cap gguf_metadata cache below ceiling on every insert
The eviction branch popped exactly one entry when len >= max, so the
cache size could only converge to the cap when entries were added
slowly enough for natural growth. After a sandbox sim that reduced
the cap mid-run, len stayed above the cap because each insert popped
one and added one. Switch to a while loop so we evict until len is
strictly below the cap before inserting. Steady-state behaviour at
the default 4096 ceiling is unchanged.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
63c6750532