Only settle the CFG-parallel dispatch key after a run that actually routed
the replica: a guidance-near-1 generation disables the overlap without
warming the replica, so its completed key must not unlock thread dispatch
for the next CFG-enabled run at the same shape (that first compile has to
stay serialized).
Restore the process-global thread-safe cuDNN attention patch when the
CFG-parallel install fails after the patch landed: no proxy is committed on
that path, so teardown would never reach it and later single-device
generations would keep running the direct aten replacement.
Tear down a CFG-parallel proxy installed by a load that is cancelled or
fails before the _VideoLoadState commit: the proxy owns a daemon worker,
the DiT replica's VRAM, and possibly the cuDNN patch. The load stashes the
proxy pre-commit and _run_load's error handler rolls it back, token-scoped
exactly like the speed-globals rollback.
Re-engage an EXPLICIT magcache choice when the actual step count differs
from the configured one, so the ratio curve, retention window, and skip
budget are re-interpolated over the real schedule (the on/off choice never
changes); auto already re-engaged via maybe_toggle_step_cache. The step
marker comparison uses endswith so #s5 cannot match inside #s50.
Bench fidelity: the e2e auto row quantizes companions before CUDA placement
(mirroring the loader, so load_peak_gb records the measured configuration),
the video bench clears step-cache residuals before every generation exactly
like VideoBackend.generate, and the image-interface dit/e2e modes reject
video families with a pointer to video_speedmem_bench.py.
Cuts the shipped default's LPIPS vs the bit-exact reference from 0.224 to 0.139
while going faster (24.9 s to 21.2 s at 720p/33f/30 steps, 22.7x vs reference),
and makes the remaining speed/accuracy trade a user knob.
- inductor precision parity: set emulate_precision_casts=True for the regional
compile (fused pointwise kernels kept fp32 intermediates where eager rounds to
bf16 between ops); full-clip LPIPS vs bit-exact 0.221 to 0.052 at zero speed
cost. Snapshot/restored with the other process-wide backend flags.
- cache x compile composition fix: diffusers cache hooks are
torch.compiler.disable'd, so every COMPUTED step ran eager (1.69 vs 1.09
s/step) under MagCache/FBCache in both enable orders. Re-point each hook's
fn_ref.original_forward at a torch.compile'd wrapper of the same bound method
(armed only where the speed layer compiled the block; restored before every
disable_cache so the uncached path stays pristine). Balanced MagCache at 50
steps: 1.48x to 2.17x, identical skip counts, bit-identical uncached rerun
after enable/disable cycles.
- transformer_cache_quality knob (quality|balanced|fast; API + UI + bench)
mapping to (threshold, max_skip_steps, retention_ratio). Auto resolves to the
near-lossless quality preset (0.06, 2, 0.3; 1.63-1.64x at pairwise LPIPS
0.05-0.09) for the HunyuanVideo-1.5 families and to balanced (the pre-knob
values, byte-identical behaviour) everywhere else.
- TE auto-quant resolves dense for HunyuanVideo-1.5: TE fp8_dynamic alone moves
the clip to LPIPS 0.236 vs bit-exact for zero speed win (the quantised encoder
perturbs the conditioning and the trajectory amplifies it chaotically); VAE
fp8 stays in auto (0.053, at the compile floor). Explicit schemes honored.
- dual-GPU CFG branch parallelism (new diffusion_cfg_parallel.py): transformer
proxy + DiT replica on the most-free second CUDA device + worker thread,
branch-routed off the pipeline's own cache_context names. Auto engages only
where measured bit-identical (eager tier: max abs diff 0.0, 1.66x); the
compiled stack is explicit cfg_parallel=on (1.52x over the sequential
default; per-device compiled artifacts differ by 1 bf16 ulp/step, documented
in the resolved record). Fail-soft gates: family allowlist, guider CFG,
pipeline kind, dense DiT, no offload, free-VRAM check; single-GPU loads are
untouched and the memory plan stays single-device.
- video API: the transformer_cache literal now accepts auto/magcache (an
explicit magcache request was rejected at the pydantic layer); the mxfp8
family deny records the round-2 measurement (block-32 MX scaling fixes the
zero-row collapse, no black frames, but is latency-neutral at LPIPS 0.37:
fails both ship bars).
Measured on B200 via the production lever path (video_speedmem_bench.py, which
gained a --cache-quality lever and companion-quant isolation configs). Tests:
441 passing across the video inference suite (32 new for cfg-parallel, 20 for
presets/arming, 3 for the inductor flag, 2 for TE auto-dense); ruff clean.
