- diffusion_attention: arch-gate FlashAttention 2 to Ampere (SM80)+ in both the
primary selector and the heterogeneous-replica guard (it crashed on pre-Ampere).
- diffusion_cfg_parallel: convert boolean attn masks to additive bias before the direct
cuDNN op so partial masks match F.scaled_dot_product_attention; make proxy disable_cache
transactional (clean both branches, mark broken, surface a reload-required error).
- diffusion_cache: fail closed when a magcache step-count resize or below-threshold
disable cannot remove the old cache; surface a failed enable+cleanup instead of a false
uncached None.
- video: roll back earlier experts when a later expert raises in the all-or-none step-cache
loop; fail the load when the primary-only cache cannot be re-engaged through the
CFG-parallel proxy; validate transformer_cache_quality and cfg_parallel before the worker.
- scripts: place the fp8 ablation pipeline on CUDA; fail closed on a failed magcache resize
in the speedmem bench; label OOM distinctly in the SDPA mask probe.
- tests: regressions for the FA2 arch gate, transactional proxy disable, all-or-none
exception rollback, magcache fail-closed transitions, and enable+cleanup failure.
- diffusion_attention: clear the HunyuanVideo-1.5 null-mask flag with an always_call
post-hook so it is scoped to one hooked forward and never latches across an
exception; add attention_backend_supported_on_device to arch-gate an
already-resolved backend on a specific (heterogeneous) CUDA device.
- video: make the explicit MagCache resize transactional via _step_cache_all_or_none
(refuse to stack a fresh cache over one that could not be disabled; roll a mixed
resize back and report the true state); raise on a failed all-or-none rollback
instead of falsely reporting an uncached pipeline.
- diffusion_cfg_parallel: re-validate the attention backend on the replica device
and pin native there when unsupported; mirror the primary's max tier on the
replica (max-autotune compile + direct QKV fusion) via a new speed_mode arg;
prefer a viable heterogeneous secondary GPU over an unusable identical one; clear
the const cache at each plan_generation.
- diffusion_vae_quant / diffusion_precision: detect a partial diffusers
layerwise-fp8 mutation (leftover casting hooks the torchao detector cannot see)
and fail the load closed, while a clean failure still falls back to dense.
- video_speedmem_bench: engage the dual-expert cache all-or-none like the loader.
- frontend video api: add text_encoder_quant / vae_quant and the auto/off literals
to VideoLoadRequest so typed callers match the backend contract.
- Wan2.2-A14B step cache: pin the balanced FBCache threshold to 0.08 even when
quant is active (per-family override in diffusion_cache.py). Auto-fp8 made the
generic quant promotion (0.12) the family's effective default at pairwise LPIPS
0.128, over the 0.08 quality gate the balanced preset is held to. Measured
operating point with fp8 actually engaged (1280x720/81f/50 steps, B200):
fb@0.08 = 1.08x at 0.129 vs the old fb@0.12 = 2.58x at 0.181; documented in
the preset table. Explicit thresholds and the fast preset are unaffected.
- MagCache curves: validated the shipped 33-frame calibrations at the production
121-frame default for hunyuanvideo-1.5-720p, hunyuanvideo-1.5 (480p) and
wan2.2-ti2v-5b. Fresh 121-frame calibrations differ by <= 0.024 max abs entry
and produce byte-identical frames at the auto presets (hv720 quality 1.69x at
LPIPS 0.042, hv480 quality 1.66x at 0.018, wan5b balanced 1.74x at 0.026, all
pairwise vs the same-load uncached stack), so the curves ship unchanged with
the frame-count transfer documented next to them.
- Dual-GPU CFG parallelism: the secondary-device pick now prefers a device whose
name and compute capability match the primary, and the gate declines a
mismatched pair in auto mode (eager kernel selection is arch-dependent, so the
advertised bit-identity cannot hold across different GPU models); an explicit
cfg_parallel=on proceeds but is downgraded to lossless=False with a warning.
- A14B expert step cache is now all-or-none, mirroring the transactional quant
loop: a mixed outcome (cache engaged on one expert but not the other) is
rolled back and reported uncached with the failure reason, on both the load
path and the generation-time auto toggle.
- Partial torchao quantization is no longer reported as dense: after an
in-place quantize_/caster failure, the DiT / text encoder / VAE is scanned
for leftover torchao tensor-subclass parameters and the load fails with a
clear error when any are found (a half-quantized module cannot run as dense,
and offload's Module.to() crashes on torchao tensors). Failures that swapped
nothing keep the best-effort dense fallback.
- Cleanup: apply_attention_backend / apply_speed_optims / the attention trim
are called once on the pipe (they already fan out over every DiT internally),
so the second A14B expert no longer passes through them twice; the stale
dual-DiT helper comment is rewritten to match the two helper shapes.
Tests: device-identity picker/gate/lossy-plan coverage, per-family threshold
pin scoping, all-or-none rollback in both failure directions, and partial-quant
detection for all three quant modules.
Speed=off is the reference contract: the loaders pin every auto speed
lever (transformer/TE/VAE quant tri-states) to off, but the cfg_parallel
auto path never consulted speed_active, so a resident two GPU
HunyuanVideo-1.5 load with Speed=off could still reserve a second GPU
and install the CFG-parallel proxy. Auto now returns off when
speed_active is false; an explicit cfg_parallel=on stays honored as a
deliberate override (the install-failure test now exercises exactly
that override path).
Only settle the CFG-parallel dispatch key after a run that actually routed
the replica: a guidance-near-1 generation disables the overlap without
warming the replica, so its completed key must not unlock thread dispatch
for the next CFG-enabled run at the same shape (that first compile has to
stay serialized).
Restore the process-global thread-safe cuDNN attention patch when the
CFG-parallel install fails after the patch landed: no proxy is committed on
that path, so teardown would never reach it and later single-device
generations would keep running the direct aten replacement.
Tear down a CFG-parallel proxy installed by a load that is cancelled or
fails before the _VideoLoadState commit: the proxy owns a daemon worker,
the DiT replica's VRAM, and possibly the cuDNN patch. The load stashes the
proxy pre-commit and _run_load's error handler rolls it back, token-scoped
exactly like the speed-globals rollback.
Re-engage an EXPLICIT magcache choice when the actual step count differs
from the configured one, so the ratio curve, retention window, and skip
budget are re-interpolated over the real schedule (the on/off choice never
changes); auto already re-engaged via maybe_toggle_step_cache. The step
marker comparison uses endswith so #s5 cannot match inside #s50.
Bench fidelity: the e2e auto row quantizes companions before CUDA placement
(mirroring the loader, so load_peak_gb records the measured configuration),
the video bench clears step-cache residuals before every generation exactly
like VideoBackend.generate, and the image-interface dit/e2e modes reject
video families with a pointer to video_speedmem_bench.py.
Cuts the shipped default's LPIPS vs the bit-exact reference from 0.224 to 0.139
while going faster (24.9 s to 21.2 s at 720p/33f/30 steps, 22.7x vs reference),
and makes the remaining speed/accuracy trade a user knob.
- inductor precision parity: set emulate_precision_casts=True for the regional
compile (fused pointwise kernels kept fp32 intermediates where eager rounds to
bf16 between ops); full-clip LPIPS vs bit-exact 0.221 to 0.052 at zero speed
cost. Snapshot/restored with the other process-wide backend flags.
- cache x compile composition fix: diffusers cache hooks are
torch.compiler.disable'd, so every COMPUTED step ran eager (1.69 vs 1.09
s/step) under MagCache/FBCache in both enable orders. Re-point each hook's
fn_ref.original_forward at a torch.compile'd wrapper of the same bound method
(armed only where the speed layer compiled the block; restored before every
disable_cache so the uncached path stays pristine). Balanced MagCache at 50
steps: 1.48x to 2.17x, identical skip counts, bit-identical uncached rerun
after enable/disable cycles.
- transformer_cache_quality knob (quality|balanced|fast; API + UI + bench)
mapping to (threshold, max_skip_steps, retention_ratio). Auto resolves to the
near-lossless quality preset (0.06, 2, 0.3; 1.63-1.64x at pairwise LPIPS
0.05-0.09) for the HunyuanVideo-1.5 families and to balanced (the pre-knob
values, byte-identical behaviour) everywhere else.
- TE auto-quant resolves dense for HunyuanVideo-1.5: TE fp8_dynamic alone moves
the clip to LPIPS 0.236 vs bit-exact for zero speed win (the quantised encoder
perturbs the conditioning and the trajectory amplifies it chaotically); VAE
fp8 stays in auto (0.053, at the compile floor). Explicit schemes honored.
- dual-GPU CFG branch parallelism (new diffusion_cfg_parallel.py): transformer
proxy + DiT replica on the most-free second CUDA device + worker thread,
branch-routed off the pipeline's own cache_context names. Auto engages only
where measured bit-identical (eager tier: max abs diff 0.0, 1.66x); the
compiled stack is explicit cfg_parallel=on (1.52x over the sequential
default; per-device compiled artifacts differ by 1 bf16 ulp/step, documented
in the resolved record). Fail-soft gates: family allowlist, guider CFG,
pipeline kind, dense DiT, no offload, free-VRAM check; single-GPU loads are
untouched and the memory plan stays single-device.
- video API: the transformer_cache literal now accepts auto/magcache (an
explicit magcache request was rejected at the pydantic layer); the mxfp8
family deny records the round-2 measurement (block-32 MX scaling fixes the
zero-row collapse, no black frames, but is latency-neutral at LPIPS 0.37:
fails both ship bars).
Measured on B200 via the production lever path (video_speedmem_bench.py, which
gained a --cache-quality lever and companion-quant isolation configs). Tests:
441 passing across the video inference suite (32 new for cfg-parallel, 20 for
presets/arming, 3 for the inductor flag, 2 for TE auto-dense); ruff clean.
