unsloth/studio/backend/core
Daniel Han bac2cfac7c feat(studio): architecture-aware KV cache VRAM estimation
Replace the single legacy formula (2 * n_kv_heads * head_dim * n_layers
* n_ctx * bpe) with 5-path estimation that reads 8 additional GGUF
metadata fields:

  1. MLA (DeepSeek-V2/V3, GLM-4.7, GLM-5, Kimi-K2.5) -- K-only cache
     using compressed KV latent + RoPE; no separate V allocation
  2. Hybrid Mamba (Qwen3.5-27B, Qwen3.5-35B-A3B) -- only attention
     layers (1 in N) carry KV; Mamba layers have none
  3. Sliding Window (Gemma-3, gpt-oss) -- SWA layers cache
     min(ctx, window) tokens instead of the full context
  4. Standard GQA -- uses explicit key_length/value_length from GGUF
     instead of embed // n_heads (which is wrong for many models)
  5. Legacy fallback -- identical to old formula for old GGUFs

New GGUF fields parsed: attention.key_length, attention.value_length,
attention.sliding_window, full_attention_interval,
attention.kv_lora_rank, attention.key_length_mla, ssm.inner_size,
ssm.state_size.

Validated against 9 real GGUF files (72/72 field checks pass).
The legacy formula was off by +682% for Gemma-3 and -81% for
DeepSeek-V3.1.
2026-04-01 12:29:20 +00:00
..
data_recipe build(deps): bump oxc-parser (#4571) 2026-03-25 02:44:38 -07:00
export studio: add HF/local model selection UI for GGUF export (#4365) 2026-03-28 22:18:25 +04:00
inference feat(studio): architecture-aware KV cache VRAM estimation 2026-04-01 12:29:20 +00:00
training fix(studio): correct default weight_decay and learning rate (#4695) 2026-03-31 13:50:25 +04:00
__init__.py Combine studio setup fixes: frontend caching, venv isolation, Windows CPU support (#4413) 2026-03-18 03:52:25 -07:00