unsloth/studio/backend/utils/models
Daniel Han bbd0ba0c25
studio/mmproj: skip unwanted GGUF values via seek instead of read (#5431)
The previous _skip_gguf_value walked past discarded values with
f.read(n), which allocates and immediately drops a Python bytes
object. For weight GGUFs that carry tokenizer.ggml.tokens (~150K
unicode strings) this wasted ~10 MB of allocation per cold call.

Switch the discard path to f.seek(n, 1). The kernel never has to
copy the bytes into userspace and Python never allocates. Truncation
is now detected on the next read attempt rather than inline (an
out-of-range seek on a regular file is legal and the next read
returns short).

Measured on real downloaded GGUFs (Qwen3.5-4B IQ2_XXS 1.52 GB,
bartowski Qwen3.5-4B IQ2_M 1.70 GB, Qwen3.5-4B-MTP IQ2_M 1.94 GB):

  before:  142 ms cold per weight, ~11 MB read
  after:    90 ms cold per weight, ~4 MB read

Mmproj reads are unaffected (no tokenizer to skip). Cached re-reads
remain ~50 microseconds. All 161 in-tree backend tests + 85 isolated
sandbox tests pass.
2026-05-14 21:57:04 -07:00
..
__init__.py [Studio] Show non exported models in chat UI (#4892) 2026-04-14 15:03:58 +04:00
checkpoints.py feat: support GGUF export for non-PEFT models + fix venv_t5 switching for local checkpoints (#4455) 2026-03-20 12:13:18 +04:00
gguf_metadata.py studio/mmproj: skip unwanted GGUF values via seek instead of read (#5431) 2026-05-14 21:57:04 -07:00
model_config.py fix(studio/mmproj): block cross-family projectors in flat local GGUF dirs (#5347) (#5350) 2026-05-14 20:31:20 -07:00