ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Studio: decide diffusion routing before the SWA resolver (#6299)

Browse source 
* Studio: decide diffusion routing before the SWA resolver

Loading a DiffusionGemma GGUF could fail with "llama-server failed to start. Check that the GGUF file is valid" while llama-diffusion-cli ran the same model fine.

_read_gguf_metadata set self._is_diffusion after calling _resolve_swa_pattern, both inside one try/except. The resolver reaches into transformers/HF, which can raise for an architecture transformers does not know (diffusion-gemma); the shared except then swallowed it and left _is_diffusion False, so the model was routed to plain llama-server instead of the diffusion runner. It only reproduced where the SWA pattern was not already cached/inline.

Set _is_diffusion right after the KV parse loop (before the resolver) so a resolver error can no longer drop the routing, and skip the resolver for diffusion models, which do not use Studio's SWA pattern.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Daniel Han 2026-06-13 04:22:50 -07:00 • committed by GitHub
commit a9a38da454
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 19 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

33
studio/backend/core/inference/llama_cpp.py
View file

@ -2380,6 +2380,17 @@ class LlamaCppBackend:
# what we have.
break
# Decide diffusion routing before the SWA resolver below: it can raise on an arch transformers
# does not know, which would otherwise drop a DiffusionGemma model to plain llama-server.
self._is_diffusion = bool(
(arch and arch.lower().startswith("diffusion")) or canvas_seen
)
if self._is_diffusion:
logger.info(
f"GGUF metadata: diffusion model detected (architecture={arch}); "
"will serve via the diffusion runner"
)
# Expand a scalar period straight from the GGUF first.
if (
self._sliding_window_pattern is None
@ -2390,9 +2401,14 @@ class LlamaCppBackend:
(i + 1) % sliding_window_pattern_period != 0 for i in range(self._n_layers)
]
# Otherwise hand off to the resolver (cache / bootstrap /
# transformers / HF); see `_resolve_swa_pattern`.
if self._sliding_window_pattern is None and self._sliding_window and self._n_layers:
# Otherwise hand off to the resolver (cache / bootstrap / transformers / HF). Diffusion models
# skip it: they do not use Studio's SWA pattern and the resolver can raise for them.
if (
self._sliding_window_pattern is None
and self._sliding_window
and self._n_layers
and not self._is_diffusion
):
hf_repo_candidates = (
general.get("general.source.huggingface.repository"),
_hf_repo_from_url(general.get("general.source.url")),
@ -2419,17 +2435,6 @@ class LlamaCppBackend:
hf_repo_candidates,
)
# Block-diffusion models (DiffusionGemma) report a diffusion arch
# and/or a diffusion.canvas_length KV; they need the diffusion runner.
self._is_diffusion = bool(
(arch and arch.lower().startswith("diffusion")) or canvas_seen
)
if self._is_diffusion:
logger.info(
f"GGUF metadata: diffusion model detected (architecture={arch}); "
"will serve via the diffusion runner"
)
if self._context_length:
logger.info(f"GGUF metadata: context_length={self._context_length}")
if self._chat_template: