unsloth/studio/backend/core
oobabooga 346d96d7f2
Studio: cap GGUF context to unified memory on Apple Silicon (#6622)
* Studio: cap GGUF context to unified memory on Apple Silicon

* Studio: tighten Apple ctx-cap comments and drop the overstated MLX-sync claim

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: reserve flat MTP fraction and floor sparse-KV ctx in the Apple unified-memory cap

The Apple Silicon GGUF context cap mirrored the discrete-GPU auto-fit branch but
missed two protections the discrete path already applies:

- It passed the full unified-memory budget with budget_frac=1.0 without first
  reserving the flat MTP fraction the discrete path takes off via _pin_fraction.
  With an MTP draft whose KV cannot be byte-sized (e.g. Qwen3.6-MTP, #6529), the
  cap filled the whole budget and left nothing for the draft, so unified memory
  could still over-commit. Reserve _flat_mtp_reserve up front; this is a no-op
  when MTP is not engaged.

- It required _can_estimate_kv(), so a GGUF with sparse KV metadata skipped the
  cap entirely and launched at full native context. Mirror the discrete
  file-size-only fallback and floor the auto context to 4096 when the cache
  cannot be sized.

Adds regression tests for both paths.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: tighten comments in the Apple unified-memory context cap

Condense the verbose comment blocks in the Apple budget helper, the no-GPU
Metal branch, and the context-fit tests. Comments only, no code change
(verified with ast-based comment_tools check); suite still green.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-06-24 04:39:33 -07:00
..
data_recipe Studio: use an isolated Node.js for the frontend build instead of replacing the system Node/npm (#6533) 2026-06-21 21:17:29 -07:00
export Generalize transformers tier selection by probing AutoConfig (#6550) 2026-06-22 08:20:06 -07:00
inference Studio: cap GGUF context to unified memory on Apple Silicon (#6622) 2026-06-24 04:39:33 -07:00
rag studio: tighten torchao Windows-ROCm comments and test docstrings (#6610) 2026-06-23 05:49:25 -07:00
training Studio: lazy-import matplotlib so the server starts when the wheel is blocked (#6596) 2026-06-23 06:22:20 -07:00
__init__.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
_torchao_stub.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
import_guards.py Studio: self-heal unsloth namespace shadows; clearer failed-load messages (#6532) 2026-06-21 22:43:31 -07:00
tool_healing.py Fix Gemma 4 GGUF OpenAI API streams (#6476) 2026-06-23 06:13:56 -07:00