When a swap load fails after the previous pipeline is released,
status() previously reported is_loaded=false on top of the OLD
repo/family/base_repo metadata, which the frontend then rendered
as a misleading 'still loaded: X' label. Clear all metadata
atomically with the pipe drop so a failed swap reports a clean
empty status plus last_error. Add regression test.
- detect_family adds _FAMILY_EXCLUDE so 'stable-diffusion-3.5' no
longer matches the SD3 Medium family and 'qwen-image-edit' no
longer matches Qwen-Image. Both were misleading silent loads.
- from_single_file now forwards config=<effective_base>,
subfolder='transformer', and the HF token. Diffusers-format GGUFs
(FLUX.2 klein, Qwen-Image, SD3) need the matching base config or
the transformer load picks the wrong shapes; gated GGUFs need the
token both for download and config read.
- Move _release_chat_backend_for_diffusion + new
_release_other_gpu_owners_for_diffusion to AFTER the GGUF download
and pipeline class lookup so a typo or transient Hub error does
not kill the user's currently-loaded chat model. Peak VRAM still
stays at one model's worth because the releases run right before
from_pretrained.
- _release_other_gpu_owners_for_diffusion: shut down the export
subprocess and any active training subprocess before a diffusion
load. Symmetric with the export load path.
- routes/training.py: unload diffusion before starting training so
the new subprocess does not race FLUX/Qwen for VRAM.
- routes/export.py: also unload the GGUF llama-server before export
load (the existing inference-backend unload only covered the
safetensors path).
_release_chat_backend_for_diffusion was importing
get_inference_backend from core.inference.inference (the in-subprocess
class) and calling unload_model() without the required model_name
argument. The TypeError was swallowed and the active chat model
stayed resident, defeating the chat-to-diffusion lifecycle handoff.
Switch to the orchestrator's accessor at core.inference and pass
active_model_name through, mirroring the GGUF chat-load path. Add a
regression test that stubs both backends and verifies unload_model
is called with the active model name.
- _smart_base_repo: pick 9B base for unsloth/FLUX.2-klein-9B-GGUF
and -base- variants per the repo id, instead of always falling
back to the 4B family default.
- pipe_kwargs use_safetensors=True so diffusers refuses pickle .bin
weights at load time (defends against compromised base_repo).
- Release the previous pipeline BEFORE allocating the new one so
peak VRAM stays at one model's worth instead of two on swap.
- Reject empty gguf_filename when repo_id ends with -GGUF; the prior
behavior tried from_pretrained on a GGUF-only repo and 500'd deep
in diffusers with a confusing model-index error.
- Status returns gguf_filename (basename) instead of gguf_path so
the local cache path / username does not leak to authenticated
Studio sessions.
- requirements/no-torch-runtime.txt: pin diffusers>=0.37.0 so older
installs cannot resolve a version without Flux2KleinPipeline.
- Frontend curated distilled klein entries now point at the
matching non-base diffusers repos (FLUX.2-klein-4B / -9B) per
the published model cards. Update api.ts to mirror the renamed
status field.
Backend
- Fix FLUX.2 klein family default base_repo: black-forest-labs/FLUX.2-klein
does not exist on the Hub. Point at the Apache 2.0 4B Base instead so
the from_pretrained call works out of the box for ungated users.
- Serialise concurrent load_model calls with a dedicated _load_lock so
two /images/load requests cannot both reach pipeline_cls.from_pretrained
at the same time (would double-spend VRAM and corrupt _pipe).
- When the caller passes a full diffusers repo (no gguf_filename),
use repo_id directly instead of silently substituting the family
default. Closes the load-the-wrong-model regression flagged by review.
- Drop negative_prompt from the pipeline call when the loaded pipeline
does not accept it (FLUX.2 / FLUX.2 klein). Inspect __call__ via
inspect.signature so we do not maintain a manual class list.
- Best-effort unload the chat backend (llama-server) before a diffusion
load so a 24 GB consumer GPU can swap between chat and diffusion
without manual unload steps.
Frontend
- Replace the four curated entries with the actual filenames published
on the Hub (lowercase flux-2-klein-Nb-Q4_K_S.gguf and flux2-dev*).
- Add an explicit base_repo per curated entry so the backend never
falls back to the family default for the curated picker.
- Add the Apache 2.0 FLUX.2 klein base 4B entry so first-time users
have an ungated, no-token-required default.
- Hide the negative prompt field for FLUX.2 / FLUX.2 klein and show a
small explanatory note instead.
Tests
- Add 6 new backend tests: base_repo override, full-repo (no GGUF)
no-substitution, concurrent serialise race, signature-based kwarg
filter, negative_prompt strip on FLUX.2, negative_prompt preserved
on supporting pipelines. 33 tests passing.
Backend
- core/inference/diffusion.py: DiffusionBackend singleton that loads
diffusion GGUFs from Hugging Face via diffusers.GGUFQuantizationConfig
and runs them on the active CUDA / MPS / CPU device. Supports FLUX.2,
FLUX.2 klein, FLUX.1, Qwen-Image, Stable Diffusion 3, and SDXL.
- routes/inference.py: POST /api/inference/images/load,
POST /api/inference/images/generate, POST /api/inference/images/unload,
GET /api/inference/images/status mirroring the llama-server lifecycle.
- models/inference.py: DiffusionLoadRequest, DiffusionGenerateRequest,
DiffusionGenerateResponse pydantic schemas with prompt / step / size
validation up front so callers get clear 422s rather than VAE crashes.
- requirements/no-torch-runtime.txt: pin gguf alongside the existing
diffusers entry so GGUFQuantizationConfig works out of the box.
- tests/test_diffusion_backend.py + tests/test_diffusion_routes.py:
27 unit tests covering family detection, validation, lifecycle, and
the full FastAPI round trip with the backend stubbed. No torch /
diffusers / GPU required to run.
Frontend
- features/images/: standalone images-page.tsx with curated model picker
(FLUX.2 klein 4B / 9B, FLUX.2 dev, FLUX.1 dev), HF token field,
prompt + negative prompt, resolution presets, steps + guidance
sliders, seed input, and a result gallery that renders base64 PNGs
inline.
- app/routes/images.tsx: lazy /images route wired into router.tsx.
- components/app-sidebar.tsx: PaintBrush02Icon nav item between
Recipes and Export, hidden in chat-only mode.
