perf(video): accuracy-first round 2 for HunyuanVideo-1.5: compile parity, cache quality presets, dual-GPU CFG
Cuts the shipped default's LPIPS vs the bit-exact reference from 0.224 to 0.139 while going faster (24.9 s to 21.2 s at 720p/33f/30 steps, 22.7x vs reference), and makes the remaining speed/accuracy trade a user knob. - inductor precision parity: set emulate_precision_casts=True for the regional compile (fused pointwise kernels kept fp32 intermediates where eager rounds to bf16 between ops); full-clip LPIPS vs bit-exact 0.221 to 0.052 at zero speed cost. Snapshot/restored with the other process-wide backend flags. - cache x compile composition fix: diffusers cache hooks are torch.compiler.disable'd, so every COMPUTED step ran eager (1.69 vs 1.09 s/step) under MagCache/FBCache in both enable orders. Re-point each hook's fn_ref.original_forward at a torch.compile'd wrapper of the same bound method (armed only where the speed layer compiled the block; restored before every disable_cache so the uncached path stays pristine). Balanced MagCache at 50 steps: 1.48x to 2.17x, identical skip counts, bit-identical uncached rerun after enable/disable cycles. - transformer_cache_quality knob (quality|balanced|fast; API + UI + bench) mapping to (threshold, max_skip_steps, retention_ratio). Auto resolves to the near-lossless quality preset (0.06, 2, 0.3; 1.63-1.64x at pairwise LPIPS 0.05-0.09) for the HunyuanVideo-1.5 families and to balanced (the pre-knob values, byte-identical behaviour) everywhere else. - TE auto-quant resolves dense for HunyuanVideo-1.5: TE fp8_dynamic alone moves the clip to LPIPS 0.236 vs bit-exact for zero speed win (the quantised encoder perturbs the conditioning and the trajectory amplifies it chaotically); VAE fp8 stays in auto (0.053, at the compile floor). Explicit schemes honored. - dual-GPU CFG branch parallelism (new diffusion_cfg_parallel.py): transformer proxy + DiT replica on the most-free second CUDA device + worker thread, branch-routed off the pipeline's own cache_context names. Auto engages only where measured bit-identical (eager tier: max abs diff 0.0, 1.66x); the compiled stack is explicit cfg_parallel=on (1.52x over the sequential default; per-device compiled artifacts differ by 1 bf16 ulp/step, documented in the resolved record). Fail-soft gates: family allowlist, guider CFG, pipeline kind, dense DiT, no offload, free-VRAM check; single-GPU loads are untouched and the memory plan stays single-device. - video API: the transformer_cache literal now accepts auto/magcache (an explicit magcache request was rejected at the pydantic layer); the mxfp8 family deny records the round-2 measurement (block-32 MX scaling fixes the zero-row collapse, no black frames, but is latency-neutral at LPIPS 0.37: fails both ship bars). Measured on B200 via the production lever path (video_speedmem_bench.py, which gained a --cache-quality lever and companion-quant isolation configs). Tests: 441 passing across the video inference suite (32 new for cfg-parallel, 20 for presets/arming, 3 for the inductor flag, 2 for TE auto-dense); ruff clean.
This commit is contained in:
parent
f7824b9db1
commit
7dbdd28161
15 changed files with 1929 additions and 22 deletions
|
|
@ -2336,19 +2336,43 @@ class VideoLoadRequest(BaseModel):
|
|||
"attention; xformers/aiter are memory-efficient (NVIDIA) / AMD ROCm. An unavailable "
|
||||
"kernel falls back to the default.",
|
||||
)
|
||||
transformer_cache: Optional[Literal["off", "fbcache"]] = Field(
|
||||
transformer_cache: Optional[Literal["off", "auto", "fbcache", "magcache"]] = Field(
|
||||
None,
|
||||
description = "Opt-in step caching (off by default). fbcache = First-Block-Cache: "
|
||||
"reuse the transformer tail across denoise steps when the first block's residual "
|
||||
"barely changes. Engages on many-step schedules only; incompatible models run "
|
||||
"uncached.",
|
||||
description = "Step caching (null/auto: the family's measured mode engages on "
|
||||
"many-step schedules, re-checked per generation). fbcache = First-Block-Cache: reuse "
|
||||
"the transformer tail across denoise steps when the first block's residual barely "
|
||||
"changes. magcache = MagCache: skip whole steps from a per-family calibrated "
|
||||
"magnitude curve with a bounded error budget (the auto mode for HunyuanVideo-1.5, "
|
||||
"where FBCache derails the trajectory; needs a calibrated family curve, else runs "
|
||||
"uncached). Incompatible models run uncached.",
|
||||
)
|
||||
transformer_cache_threshold: Optional[float] = Field(
|
||||
None,
|
||||
ge = 0.0,
|
||||
le = 1.0,
|
||||
description = "FBCache residual threshold (higher = skips more steps = faster, lower "
|
||||
"quality). null auto-picks the family default.",
|
||||
description = "Step-cache residual threshold (higher = skips more steps = faster, "
|
||||
"lower quality). null auto-picks the engaged mode's family default.",
|
||||
)
|
||||
transformer_cache_quality: Optional[Literal["auto", "quality", "balanced", "fast"]] = Field(
|
||||
None,
|
||||
description = "Step-cache speed/accuracy preset. quality = near-lossless (lower "
|
||||
"threshold + tighter skip budget, smaller speedup); balanced = the measured family "
|
||||
"defaults; fast = more skipping for more speed at a visible quality cost. null/auto "
|
||||
"picks the family's measured default: quality for HunyuanVideo-1.5 (1.6x at half the "
|
||||
"drift of balanced), balanced elsewhere. An explicit transformer_cache_threshold "
|
||||
"overrides the preset's threshold; the preset still sets the MagCache skip cap / "
|
||||
"retention window.",
|
||||
)
|
||||
cfg_parallel: Optional[Literal["off", "auto", "on"]] = Field(
|
||||
None,
|
||||
description = "Dual-GPU CFG branch parallelism: run the two guidance branches "
|
||||
"concurrently, one on a DiT replica on a second CUDA device (~1.7x end-to-end on "
|
||||
"HunyuanVideo-1.5, replica ~20 GB VRAM). null/auto engages only where the output is "
|
||||
"bit-identical to single-GPU: the measured families on an EAGER speed tier (each "
|
||||
"compiled stack's per-device inductor artifacts drift ~1 ulp/step, which a clip "
|
||||
"trajectory amplifies). on = engage wherever mechanically possible, including the "
|
||||
"compiled stack, accepting that fp-noise divergence (composition/brightness "
|
||||
"preserved). off = never.",
|
||||
)
|
||||
transformer_quant: Optional[Literal["auto", "none", "off", "int8", "fp8", "nvfp4", "mxfp8"]] = (
|
||||
Field(
|
||||
|
|
@ -2545,7 +2569,15 @@ class VideoStatusResponse(BaseModel):
|
|||
description = "Attention backend engaged via the diffusers dispatcher (e.g. "
|
||||
"_native_cudnn), or null for the default SDPA",
|
||||
)
|
||||
transformer_cache: Optional[str] = Field(None, description = "Step cache engaged: fbcache | null")
|
||||
transformer_cache: Optional[str] = Field(
|
||||
None, description = "Step cache engaged: fbcache | magcache | null"
|
||||
)
|
||||
cfg_parallel: Optional[str] = Field(
|
||||
None,
|
||||
description = "Dual-GPU CFG branch parallelism engaged: 'on' (DiT replica on a second "
|
||||
"CUDA device runs one guidance branch) | null (single-device). The resolved record "
|
||||
"carries the gate reason.",
|
||||
)
|
||||
transformer_quant: Optional[str] = Field(
|
||||
None,
|
||||
description = "Dense transformer quant engaged on a pipeline load: int8 | fp8 | nvfp4 | "
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue