The wheel-only pip install for an optional attention kernel ran inside
load_pipeline under _lock and _generate_lock, so a slow or hanging install
blocked unload and cancellation for up to the 600s timeout. Resolve and install
the kernel before taking the locks (only an explicit backend ever pulls a
package, and its resolution ignores the speed tier); the in-lock apply call is
then a fast no-op. Also decode and log pip's stderr on a failed install so the
fallback to native is diagnosable instead of showing only the exit code.
Measured on the fresh linux x64 prebuilt (z-image Q8_0, sd-cli, 192 CPU
threads, 512x512, 9 steps, steady state): sampling 56.1s vs 51.3s (about
9 percent faster), VAE decode unchanged, peak RSS identical. The sd.cpp
engine only serves the no-GPU tier, so the default profile now matches
max: --diffusion-fa plus --diffusion-conv-direct.
An unset transformer_quant used to mean off (run the GGUF as-is), so the
hardware ladder only engaged when auto was explicitly chosen and the panel
showed Off as the default. Unset (or auto) now hands the decision to the
ladder: a dense-capable GPU gets at least int8, data-center silicon fp8,
falling back to the GGUF when the device, VRAM, family deny table or disk
cannot take it. An explicit none/off pins GGUF-as-is and is now
expressible in the API (previously only omission meant off, so pinned-off
and unset were indistinguishable); an explicit scheme pins that scheme.
The dense candidate also gains a free-disk gate: with auto as the default
the bf16 base download (up to ~40 GB) must never wedge a nearly-full
model-cache disk, so the candidate is dropped (GGUF build kept) when free
space cannot hold it plus a 10 GiB margin. Unprobeable disk passes.
Frontend: the Dtype select defaults to Auto (fastest for GPU), keeps Off
as an explicit choice, and sends none through instead of omitting it.
Suite: 622 diffusion tests green (default-load test rewritten to the new
contract, explicit-off short-circuit covered), CI-sim green.
resolve_dense_quant_candidate now passes fam.name to
select_transformer_quant_scheme so the policy's proposed scheme honors the
family deny table (qwen-image lands on int8 instead of proposing fp8 that
the execution path would refuse). Test stub updated for the new keyword.
A 28-pair accuracy gate on a B200 (same-seed vs the dense bf16 reference)
found per-row fp8 dynamic quant renders EVERY qwen-image frame black
(mean luma 0.0000, SSIM 0.016), reproduced identically with on-the-fly
quantize_ on the dense transformer, so it is the model's activation range,
not a checkpoint artifact. mxfp8 shows real semantic damage at 1024px
(CLIP delta mean 0.0146, worst cases 0.064/0.102) and nvfp4 measures
LPIPS mean 0.51. int8 dynamic (per-token scales) is excellent on Qwen:
LPIPS mean 0.069, SSIM 0.958.
The per-scheme smoke probe only proves the GEMM kernel runs, so it cannot
catch model-level breakage. Add _FAMILY_SCHEME_DENY consulted by
select_transformer_quant_scheme: auto skips denied schemes (Qwen lands on
int8) and an explicit denied request returns None, the same GGUF-fallback
contract as an unsupported scheme. Family is threaded from the three
diffusion.py call sites; existing behavior is unchanged for every other
family. 4 new tests; 529 diffusion tests green; CI-sim green.
train_precision_modes gates int8/fp8/mxfp8 on has_functional_torchao, and the
Backend CI runner does not install torchao, so the three capability-gating
tests collapsed to nf4/bf16/auto and failed. They exercise the CAPABILITY
gate, not torchao presence: stub the probe functional alongside the CUDA
capability patch. Validated with a torchao-blocked run (22 passed).
threads = None let sd.cpp default to the logical-core count. The diffusion CPU
path is compute-bound GGML matmuls, where oversubscribing hyperthreads adds
scheduling contention without extra throughput, so both the persistent server
and the one-shot sd-cli now pass cpu_count // 2 (min 1, fallback 8).
Attention: apply_attention_backend now best-effort installs the package an
explicitly requested optional backend needs (sage -> sageattention, flash ->
flash-attn, flash3/flash4 -> kernels, xformers), wheel-only via pip
--only-binary=:all: so a host without a CUDA toolchain never starts a source
build. Gated by UNSLOTH_DIFFUSION_ATTENTION_INSTALL (auto|0), mirroring the
sd.cpp prebuilt installer gate, and only reached after the arch gating in
select_attention_backend, so no install is attempted for a kernel this card
cannot run. Any failure keeps today's native fallback.
Step cache: transformer_cache gains a real auto state (unset or "auto"). At
load the policy engages FBCache when the model's default schedule reaches
FBCACHE_MIN_STEPS = 20 (dev-style 28-step models win ~1.4x; 4-9-step distilled
models never engage, a skipped step costs too much there). generate() then
re-checks the ACTUAL step count and toggles the cache idempotently across the
bar, so one resident load serves both a 28-step and a 4-step request with the
right cache state, and status/resolved provenance follow the toggle. An explicit
off or fbcache request is pinned and never toggled. Compile drops fullgraph when
an auto cache could still engage on a cache-capable transformer, since enabling
FBCache under a fullgraph-compiled transformer would crash.
Verified on GPU: flux.1-schnell load starts uncached (4-step default), engages
fbcache at 24 steps, disengages at 4, re-engages at 28, with images at each
step and the provenance record tracking each transition.
The A/B harness measured two regimes. bf16 loads (the Studio default on Ampere+)
are bit-identical with the flag on across all six families, 36/36 same-seed cases,
because the flag only changes fp16 GEMM accumulation. fp16 loads (the pre-Ampere
fallback dtype) show real same-seed drift on the families that genuinely run fp16
GEMMs: SDXL up to 0.050 mean abs diff, FLUX.1 0.028, FLUX.2-klein 0.045, all
finite, no new black frames. qwen-image renders black in fp16 with the flag off
too and z-image fp16 fails in attention, so both are dtype limitations, not
accumulation ones.
So the gate now takes the compute dtype and the speed tier: bf16 engages on any
active tier (provably output-neutral), fp16 engages only under max, the tier that
already trades exactness for measured speed. The deny-list stays empty by
measurement.
