threads = None let sd.cpp default to the logical-core count. The diffusion CPU
path is compute-bound GGML matmuls, where oversubscribing hyperthreads adds
scheduling contention without extra throughput, so both the persistent server
and the one-shot sd-cli now pass cpu_count // 2 (min 1, fallback 8).
Attention: apply_attention_backend now best-effort installs the package an
explicitly requested optional backend needs (sage -> sageattention, flash ->
flash-attn, flash3/flash4 -> kernels, xformers), wheel-only via pip
--only-binary=:all: so a host without a CUDA toolchain never starts a source
build. Gated by UNSLOTH_DIFFUSION_ATTENTION_INSTALL (auto|0), mirroring the
sd.cpp prebuilt installer gate, and only reached after the arch gating in
select_attention_backend, so no install is attempted for a kernel this card
cannot run. Any failure keeps today's native fallback.
Step cache: transformer_cache gains a real auto state (unset or "auto"). At
load the policy engages FBCache when the model's default schedule reaches
FBCACHE_MIN_STEPS = 20 (dev-style 28-step models win ~1.4x; 4-9-step distilled
models never engage, a skipped step costs too much there). generate() then
re-checks the ACTUAL step count and toggles the cache idempotently across the
bar, so one resident load serves both a 28-step and a 4-step request with the
right cache state, and status/resolved provenance follow the toggle. An explicit
off or fbcache request is pinned and never toggled. Compile drops fullgraph when
an auto cache could still engage on a cache-capable transformer, since enabling
FBCache under a fullgraph-compiled transformer would crash.
Verified on GPU: flux.1-schnell load starts uncached (4-step default), engages
fbcache at 24 steps, disengages at 4, re-engages at 28, with images at each
step and the provenance record tracking each transition.
The A/B harness measured two regimes. bf16 loads (the Studio default on Ampere+)
are bit-identical with the flag on across all six families, 36/36 same-seed cases,
because the flag only changes fp16 GEMM accumulation. fp16 loads (the pre-Ampere
fallback dtype) show real same-seed drift on the families that genuinely run fp16
GEMMs: SDXL up to 0.050 mean abs diff, FLUX.1 0.028, FLUX.2-klein 0.045, all
finite, no new black frames. qwen-image renders black in fp16 with the flag off
too and z-image fp16 fails in attention, so both are dtype limitations, not
accumulation ones.
So the gate now takes the compute dtype and the speed tier: bf16 engages on any
active tier (provably output-neutral), fp16 engages only under max, the tier that
already trades exactness for measured speed. The deny-list stays empty by
measurement.
fp16 accumulation (torch.backends.cuda.matmul.allow_fp16_accumulation) roughly
doubles fp16 GEMM throughput on consumer tensor cores by keeping the accumulator
in fp16. The flag only affects fp16 GEMMs: bf16-compute DiT families are untouched
by construction, while SDXL's fp16 UNet and any fp16 text encoder or VAE path get
the speedup.
Gate in apply_speed_optims: CUDA target, consumer GPU (datacenter parts keep fp32
accumulation), torch exposes the flag, family not in _FP16_ACCUM_DENY, and the
UNSLOTH_DISABLE_FP16_ACCUM kill switch is unset. The flag is captured in
snapshot_backend_flags and restored on unload like the other process-wide knobs.
_FP16_ACCUM_DENY starts empty: a same-seed A/B harness (off vs on per family at
512 and 1024 with long-prompt and high-guidance stress cases, non-finite, black
frame and drift checks) backs the empty list and populates it if a family ever
overflows.
The loader used to plan memory from the GGUF file size and only offer the dense
transformer-quant fast path when that plan was already resident, so on a card where
the GGUF forced offload the int8/fp8 build (roughly half the bf16 bytes, or exactly
the quantised size when a pre-quantized checkpoint exists) was never attempted.
diffusion_auto_policy.py is a pure decision layer: a bf16-resident component table
per family (transformer / text encoders / VAE, with base-repo overrides for the
multi-size families), per-scheme size factors with separate steady and transient
(build peak) numbers, and resolve_dense_quant_candidate which the loader now uses to
re-plan memory against the candidate artifact before settling for offload. The
engaged plan is adopted only when the dense build succeeds; the GGUF fallback keeps
its own plan.
Status now carries a resolved provenance record per Advanced control (value, source
auto or explicit, reason) so the UI can label backend decisions.
The fp16-on-bf16-family refusal in run_dit_lora_training now fires before the heavy
imports, so a host without diffusers gets the real validation error instead of
ModuleNotFoundError. test_in_progress_returns_409_after_validation_passes pins the
resolved device to cuda because the load route only takes the GPU arbiter for non-CPU
loads, which made the ownership assert host-dependent.
Falling through to hf_hub_download with a filesystem path as the repo id
raised an opaque HFValidationError; a local dir without the file now
raises FileNotFoundError naming the directory
num_epochs was only int-coerced for the range check, so a string value
from a dict-built config would reach resolve_train_steps' arithmetic;
normalized() now stores the coerced int. Run record reads/writes pass
encoding utf-8 explicitly so non-ASCII prompts survive on Windows
- Trainers emit the pre-clip gradient norm; the service keeps a bounded
grad_norm history and the Train tab renders a Grad Norm chart next to
Loss and LR
- Completed runs show 'Training complete' with a celebratory marker in
the success color instead of a plain status word
- metadata.jsonl caption keys now match on Windows (as_posix relative
paths) in both the trainer discovery and the dataset image records
- RMSNorm eager patch skips installation on torch builds without
F.rms_norm instead of failing at forward time
- GGUF compute description no longer says the GGUF is dequantised: the
INT8/FP8/FP4 modes load the base model's bf16 transformer and quantise
that directly; label no longer wraps in the Advanced panel
torchao 0.17 removed MXLinearConfig from prototype.mx_formats in favour of
MXFP8TrainingOpConfig.from_recipe shared with MoE training. _mxfp8_training_config
tries the 0.16 API first and falls back to the 0.17 one; both feed quantize_.
mxfp8 still degrades to bf16 with a warning when neither import resolves
Krea's release guidance is to train on Krea-2-Raw and run adapters on
Turbo. Raw now leads the krea-2 training bases (Turbo stays available),
both vendor repos are trust-listed, and load_krea2_pipeline fails fast
with an upgrade hint on diffusers older than 0.39 instead of a bare
AttributeError mid-load
- list_diffusion_runs skips wrong-shape records and the runs route tolerates
per-record ValidationError so one bad file never breaks the panel
- max_steps: 0 epoch-mode sentinel no longer trips train_steps validation
before epochs are resolved
- numberField keeps an explicit 0 (Seed, LR warmup) instead of falling back
- previous-runs list refetches once more shortly after a terminal status so
the just-finished run appears even if the record write races the fetch
The Windows ROCm torchao import stub satisfies find_spec and even lets
from torchao.quantization import quantize_ succeed, but its quantize_ is a
no-op: auto would pick int8, leave the transformer dense, and disable
compile as if it were quantized. has_functional_torchao imports the exact
symbols the int8 path uses and rejects the stub via its sentinel; both the
auto picker and the /info advertised modes now use it
- Unload the ACTIVE image engine (sd_cpp or diffusers) before diffusion training starts, not just the diffusers singleton
- Count metadata.jsonl captions in dataset summaries so metadata-captioned datasets are not reported as uncaptioned
- Sidecar captions now override metadata rows everywhere (grid edits win); trainer and dataset API agree
- Tag local diffusers image checkpoints with text-to-image so they appear in the Images picker
- Family LoRA targets (_FLUX_TARGETS etc) apply when the config carries the generic defaults; explicit overrides still win
