- Free reserved VRAM in the diffusion load worker's failure path: a load-time OOM
never commits _state and the next load's _unload_locked early-returns, so nothing
else reclaimed the half-built pipeline's memory
- Use a monotonic clock for the denoise ETA rate
- Sync _GENERATION_DEFAULTS with the UI table: kontext, flux.2-dev, sdxl-turbo and
SDXL base rows so /v1/images/generations stops falling back to 9 steps / CFG 0
- 400 (not sanitized 500) when /v1/images/generations hits an edit-only model
- Fail fast on pre-Ampere CUDA in the DiT trainer instead of dying in model load
- Run the trainer trust gate in the diffusion training route before freeing GPU
residents so an untrusted base cannot tear down loaded chat/Images models
- Protect native sd.cpp companion VAE/text-encoder repos from cache deletion while
a load is downloading them
- Exempt the task-scoped Images picker from the chat-only GGUF/MLX format gate so
local diffusers pipelines stay selectable on no-GPU hosts
The generic Studio config dict path can deliver these flags as strings, and a
non-empty string like "false" is truthy, so an opt-out silently no-ops (the
latent cache still builds, TF32 stays on). Coerce them the same way
gradient_checkpointing already is.
Measured on the fresh linux x64 prebuilt (z-image Q8_0, sd-cli, 192 CPU
threads, 512x512, 9 steps, steady state): sampling 56.1s vs 51.3s (about
9 percent faster), VAE decode unchanged, peak RSS identical. The sd.cpp
engine only serves the no-GPU tier, so the default profile now matches
max: --diffusion-fa plus --diffusion-conv-direct.
An unset transformer_quant used to mean off (run the GGUF as-is), so the
hardware ladder only engaged when auto was explicitly chosen and the panel
showed Off as the default. Unset (or auto) now hands the decision to the
ladder: a dense-capable GPU gets at least int8, data-center silicon fp8,
falling back to the GGUF when the device, VRAM, family deny table or disk
cannot take it. An explicit none/off pins GGUF-as-is and is now
expressible in the API (previously only omission meant off, so pinned-off
and unset were indistinguishable); an explicit scheme pins that scheme.
The dense candidate also gains a free-disk gate: with auto as the default
the bf16 base download (up to ~40 GB) must never wedge a nearly-full
model-cache disk, so the candidate is dropped (GGUF build kept) when free
space cannot hold it plus a 10 GiB margin. Unprobeable disk passes.
Frontend: the Dtype select defaults to Auto (fastest for GPU), keeps Off
as an explicit choice, and sends none through instead of omitting it.
Suite: 622 diffusion tests green (default-load test rewritten to the new
contract, explicit-off short-circuit covered), CI-sim green.
resolve_dense_quant_candidate now passes fam.name to
select_transformer_quant_scheme so the policy's proposed scheme honors the
family deny table (qwen-image lands on int8 instead of proposing fp8 that
the execution path would refuse). Test stub updated for the new keyword.
A 28-pair accuracy gate on a B200 (same-seed vs the dense bf16 reference)
found per-row fp8 dynamic quant renders EVERY qwen-image frame black
(mean luma 0.0000, SSIM 0.016), reproduced identically with on-the-fly
quantize_ on the dense transformer, so it is the model's activation range,
not a checkpoint artifact. mxfp8 shows real semantic damage at 1024px
(CLIP delta mean 0.0146, worst cases 0.064/0.102) and nvfp4 measures
LPIPS mean 0.51. int8 dynamic (per-token scales) is excellent on Qwen:
LPIPS mean 0.069, SSIM 0.958.
The per-scheme smoke probe only proves the GEMM kernel runs, so it cannot
catch model-level breakage. Add _FAMILY_SCHEME_DENY consulted by
select_transformer_quant_scheme: auto skips denied schemes (Qwen lands on
int8) and an explicit denied request returns None, the same GGUF-fallback
contract as an unsupported scheme. Family is threaded from the three
diffusion.py call sites; existing behavior is unchanged for every other
family. 4 new tests; 529 diffusion tests green; CI-sim green.
train_precision_modes gates int8/fp8/mxfp8 on has_functional_torchao, and the
Backend CI runner does not install torchao, so the three capability-gating
tests collapsed to nf4/bf16/auto and failed. They exercise the CAPABILITY
gate, not torchao presence: stub the probe functional alongside the CUDA
capability patch. Validated with a torchao-blocked run (22 passed).
spec.forward imports Krea2Pipeline for prepare_position_ids, so the test needs a
real diffusers install; the backend CI matrix runs without one and failed on the
import. Same importorskip guard the sigmas gather test already uses.
The loader used to plan memory from the GGUF file size and only offer the dense
transformer-quant fast path when that plan was already resident, so on a card where
the GGUF forced offload the int8/fp8 build (roughly half the bf16 bytes, or exactly
the quantised size when a pre-quantized checkpoint exists) was never attempted.
diffusion_auto_policy.py is a pure decision layer: a bf16-resident component table
per family (transformer / text encoders / VAE, with base-repo overrides for the
multi-size families), per-scheme size factors with separate steady and transient
(build peak) numbers, and resolve_dense_quant_candidate which the loader now uses to
re-plan memory against the candidate artifact before settling for offload. The
engaged plan is adopted only when the dense build succeeds; the GGUF fallback keeps
its own plan.
Status now carries a resolved provenance record per Advanced control (value, source
auto or explicit, reason) so the UI can label backend decisions.
