train_precision_modes gates int8/fp8/mxfp8 on has_functional_torchao, and the
Backend CI runner does not install torchao, so the three capability-gating
tests collapsed to nf4/bf16/auto and failed. They exercise the CAPABILITY
gate, not torchao presence: stub the probe functional alongside the CUDA
capability patch. Validated with a torchao-blocked run (22 passed).
spec.forward imports Krea2Pipeline for prepare_position_ids, so the test needs a
real diffusers install; the backend CI matrix runs without one and failed on the
import. Same importorskip guard the sigmas gather test already uses.
The fp16-on-bf16-family refusal in run_dit_lora_training now fires before the heavy
imports, so a host without diffusers gets the real validation error instead of
ModuleNotFoundError. test_in_progress_returns_409_after_validation_passes pins the
resolved device to cuda because the load route only takes the GPU arbiter for non-CPU
loads, which made the ownership assert host-dependent.
torchao 0.17 removed MXLinearConfig from prototype.mx_formats in favour of
MXFP8TrainingOpConfig.from_recipe shared with MoE training. _mxfp8_training_config
tries the 0.16 API first and falls back to the 0.17 one; both feed quantize_.
mxfp8 still degrades to bf16 with a warning when neither import resolves
Krea's release guidance is to train on Krea-2-Raw and run adapters on
Turbo. Raw now leads the krea-2 training bases (Turbo stays available),
both vendor repos are trust-listed, and load_krea2_pipeline fails fast
with an upgrade hint on diffusers older than 0.39 instead of a bare
AttributeError mid-load
- list_diffusion_runs skips wrong-shape records and the runs route tolerates
per-record ValidationError so one bad file never breaks the panel
- max_steps: 0 epoch-mode sentinel no longer trips train_steps validation
before epochs are resolved
- numberField keeps an explicit 0 (Seed, LR warmup) instead of falling back
- previous-runs list refetches once more shortly after a terminal status so
the just-finished run appears even if the record write races the fetch
The Windows ROCm torchao import stub satisfies find_spec and even lets
from torchao.quantization import quantize_ succeed, but its quantize_ is a
no-op: auto would pick int8, leave the transformer dense, and disable
compile as if it were quantized. has_functional_torchao imports the exact
symbols the int8 path uses and rejects the stub via its sentinel; both the
auto picker and the /info advertised modes now use it
- Unload the ACTIVE image engine (sd_cpp or diffusers) before diffusion training starts, not just the diffusers singleton
- Count metadata.jsonl captions in dataset summaries so metadata-captioned datasets are not reported as uncaptioned
- Sidecar captions now override metadata rows everywhere (grid edits win); trainer and dataset API agree
- Tag local diffusers image checkpoints with text-to-image so they appear in the Images picker
- Family LoRA targets (_FLUX_TARGETS etc) apply when the config carries the generic defaults; explicit overrides still win
The SDXL trainer now regionally compiles its transformer blocks, so /info
advertises supports_compile for every family; the precision selector stays
DiT-only.
- base_precision="auto" only picks int8 when torchao is importable (the int8
quantize has no runtime fallback, unlike fp8); otherwise the middle band falls
back to nf4. Threaded as a parameter so the policy stays pure.
- The dense-mode validation (prequant base / bf16 compute) now applies only to
DiT families: sdxl ignores base_precision entirely, so a leftover value can no
longer fail an SDXL run. The mode-name validity check still runs everywhere.
- Latent caches (DiT + SDXL) now hold the posterior mean/std in fp32 and draw the
per-step sample in fp32, casting only the result to the training dtype. This
matches the in-loop path (encode fp32 -> sample fp32 -> cast) exactly instead of
sampling in bf16; the cache is tiny so the doubled RAM is negligible.
- The training service nulls non-finite floats (NaN/Inf loss, avg_loss,
learning_rate) at its single ingestion point so status snapshots and persisted
run records stay strict-JSON serializable; the metric history skips non-finite
loss points. Test covers NaN/Inf progress followed by a finite point.
