unsloth/studio/backend
Daniel Han bac2cfac7c feat(studio): architecture-aware KV cache VRAM estimation
Replace the single legacy formula (2 * n_kv_heads * head_dim * n_layers
* n_ctx * bpe) with 5-path estimation that reads 8 additional GGUF
metadata fields:

  1. MLA (DeepSeek-V2/V3, GLM-4.7, GLM-5, Kimi-K2.5) -- K-only cache
     using compressed KV latent + RoPE; no separate V allocation
  2. Hybrid Mamba (Qwen3.5-27B, Qwen3.5-35B-A3B) -- only attention
     layers (1 in N) carry KV; Mamba layers have none
  3. Sliding Window (Gemma-3, gpt-oss) -- SWA layers cache
     min(ctx, window) tokens instead of the full context
  4. Standard GQA -- uses explicit key_length/value_length from GGUF
     instead of embed // n_heads (which is wrong for many models)
  5. Legacy fallback -- identical to old formula for old GGUFs

New GGUF fields parsed: attention.key_length, attention.value_length,
attention.sliding_window, full_attention_interval,
attention.kv_lora_rank, attention.key_length_mla, ssm.inner_size,
ssm.state_size.

Validated against 9 real GGUF files (72/72 field checks pass).
The legacy formula was off by +682% for Gemma-3 and -81% for
DeepSeek-V3.1.
2026-04-01 12:29:20 +00:00
..
assets fix(studio): correct default weight_decay and learning rate (#4695) 2026-03-31 13:50:25 +04:00
auth fix: remove old comments (#4292) 2026-03-14 16:50:13 +04:00
core feat(studio): architecture-aware KV cache VRAM estimation 2026-04-01 12:29:20 +00:00
loggers Final cleanup 2026-03-12 18:28:04 +00:00
models feat: custom scan folders for GGUF model discovery (#4723) 2026-03-31 06:40:31 -07:00
plugins Bump Data Designer to 0.5.4 (removes litellm dependency) (#4569) 2026-03-25 02:01:43 -07:00
requirements fix: no-torch install deps without pulling torch transitively (#4650) 2026-03-27 05:19:26 -07:00
routes feat: custom scan folders for GGUF model discovery (#4723) 2026-03-31 06:40:31 -07:00
state Final cleanup 2026-03-12 18:28:04 +00:00
storage feat: custom scan folders for GGUF model discovery (#4723) 2026-03-31 06:40:31 -07:00
tests Studio: simplify tool-call dedup and replace html2text with builtin converter (#4722) 2026-03-31 06:15:18 -07:00
utils Fix Windows local GGUF model loading crash (#4730) 2026-04-01 01:38:09 -07:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_platform_compat.py Fix Studio crash on Anaconda/conda-forge Python (#4484) 2026-03-22 05:36:55 -07:00
colab.py Allow install_python_stack to run on Colab (#4633) 2026-03-27 00:29:27 +04:00
main.py studio: Polish Windows installer/setup logs (#4736) 2026-03-31 23:12:42 -07:00
run.py fix(studio): avoid UnicodeEncodeError on Windows cp1252 consoles (#4699) 2026-03-30 06:40:47 -07:00
startup_banner.py studio: unify Windows installer/setup logging style, verbosity controls, and startup messaging (#4651) 2026-03-30 00:53:23 -07:00