train_precision_modes gates int8/fp8/mxfp8 on has_functional_torchao, and the
Backend CI runner does not install torchao, so the three capability-gating
tests collapsed to nf4/bf16/auto and failed. They exercise the CAPABILITY
gate, not torchao presence: stub the probe functional alongside the CUDA
capability patch. Validated with a torchao-blocked run (22 passed).
threads = None let sd.cpp default to the logical-core count. The diffusion CPU
path is compute-bound GGML matmuls, where oversubscribing hyperthreads adds
scheduling contention without extra throughput, so both the persistent server
and the one-shot sd-cli now pass cpu_count // 2 (min 1, fallback 8).
Attention: apply_attention_backend now best-effort installs the package an
explicitly requested optional backend needs (sage -> sageattention, flash ->
flash-attn, flash3/flash4 -> kernels, xformers), wheel-only via pip
--only-binary=:all: so a host without a CUDA toolchain never starts a source
build. Gated by UNSLOTH_DIFFUSION_ATTENTION_INSTALL (auto|0), mirroring the
sd.cpp prebuilt installer gate, and only reached after the arch gating in
select_attention_backend, so no install is attempted for a kernel this card
cannot run. Any failure keeps today's native fallback.
Step cache: transformer_cache gains a real auto state (unset or "auto"). At
load the policy engages FBCache when the model's default schedule reaches
FBCACHE_MIN_STEPS = 20 (dev-style 28-step models win ~1.4x; 4-9-step distilled
models never engage, a skipped step costs too much there). generate() then
re-checks the ACTUAL step count and toggles the cache idempotently across the
bar, so one resident load serves both a 28-step and a 4-step request with the
right cache state, and status/resolved provenance follow the toggle. An explicit
off or fbcache request is pinned and never toggled. Compile drops fullgraph when
an auto cache could still engage on a cache-capable transformer, since enabling
FBCache under a fullgraph-compiled transformer would crash.
Verified on GPU: flux.1-schnell load starts uncached (4-step default), engages
fbcache at 24 steps, disengages at 4, re-engages at 28, with images at each
step and the provenance record tracking each transition.
The A/B harness measured two regimes. bf16 loads (the Studio default on Ampere+)
are bit-identical with the flag on across all six families, 36/36 same-seed cases,
because the flag only changes fp16 GEMM accumulation. fp16 loads (the pre-Ampere
fallback dtype) show real same-seed drift on the families that genuinely run fp16
GEMMs: SDXL up to 0.050 mean abs diff, FLUX.1 0.028, FLUX.2-klein 0.045, all
finite, no new black frames. qwen-image renders black in fp16 with the flag off
too and z-image fp16 fails in attention, so both are dtype limitations, not
accumulation ones.
So the gate now takes the compute dtype and the speed tier: bf16 engages on any
active tier (provably output-neutral), fp16 engages only under max, the tier that
already trades exactness for measured speed. The deny-list stays empty by
measurement.
spec.forward imports Krea2Pipeline for prepare_position_ids, so the test needs a
real diffusers install; the backend CI matrix runs without one and failed on the
import. Same importorskip guard the sigmas gather test already uses.
fp16 accumulation (torch.backends.cuda.matmul.allow_fp16_accumulation) roughly
doubles fp16 GEMM throughput on consumer tensor cores by keeping the accumulator
in fp16. The flag only affects fp16 GEMMs: bf16-compute DiT families are untouched
by construction, while SDXL's fp16 UNet and any fp16 text encoder or VAE path get
the speedup.
Gate in apply_speed_optims: CUDA target, consumer GPU (datacenter parts keep fp32
accumulation), torch exposes the flag, family not in _FP16_ACCUM_DENY, and the
UNSLOTH_DISABLE_FP16_ACCUM kill switch is unset. The flag is captured in
snapshot_backend_flags and restored on unload like the other process-wide knobs.
_FP16_ACCUM_DENY starts empty: a same-seed A/B harness (off vs on per family at
512 and 1024 with long-prompt and high-guidance stress cases, non-finite, black
frame and drift checks) backs the empty list and populates it if a family ever
overflows.
The loader used to plan memory from the GGUF file size and only offer the dense
transformer-quant fast path when that plan was already resident, so on a card where
the GGUF forced offload the int8/fp8 build (roughly half the bf16 bytes, or exactly
the quantised size when a pre-quantized checkpoint exists) was never attempted.
diffusion_auto_policy.py is a pure decision layer: a bf16-resident component table
per family (transformer / text encoders / VAE, with base-repo overrides for the
multi-size families), per-scheme size factors with separate steady and transient
(build peak) numbers, and resolve_dense_quant_candidate which the loader now uses to
re-plan memory against the candidate artifact before settling for offload. The
engaged plan is adopted only when the dense build succeeds; the GGUF fallback keeps
its own plan.
Status now carries a resolved provenance record per Advanced control (value, source
auto or explicit, reason) so the UI can label backend decisions.
The always-visible description under the select is gone (the hint tooltip keeps the
full detail) and the no-model gallery placeholder now reads 'Select a diffusion model
to load'.
The fp16-on-bf16-family refusal in run_dit_lora_training now fires before the heavy
imports, so a host without diffusers gets the real validation error instead of
ModuleNotFoundError. test_in_progress_returns_409_after_validation_passes pins the
resolved device to cuda because the load route only takes the GPU arbiter for non-CPU
loads, which made the ownership assert host-dependent.
Falling through to hf_hub_download with a filesystem path as the repo id
raised an opaque HFValidationError; a local dir without the file now
raises FileNotFoundError naming the directory
num_epochs was only int-coerced for the range check, so a string value
from a dict-built config would reach resolve_train_steps' arithmetic;
normalized() now stores the coerced int. Run record reads/writes pass
encoding utf-8 explicitly so non-ASCII prompts survive on Windows
- Trainers emit the pre-clip gradient norm; the service keeps a bounded
grad_norm history and the Train tab renders a Grad Norm chart next to
Loss and LR
- Completed runs show 'Training complete' with a celebratory marker in
the success color instead of a plain status word
- metadata.jsonl caption keys now match on Windows (as_posix relative
paths) in both the trainer discovery and the dataset image records
- RMSNorm eager patch skips installation on torch builds without
F.rms_norm instead of failing at forward time
- GGUF compute description no longer says the GGUF is dequantised: the
INT8/FP8/FP4 modes load the base model's bf16 transformer and quantise
that directly; label no longer wraps in the Advanced panel
