Harden video diffusion cache, CFG-parallel replica, and layerwise-fp8 rollback

- diffusion_attention: clear the HunyuanVideo-1.5 null-mask flag with an always_call
  post-hook so it is scoped to one hooked forward and never latches across an
  exception; add attention_backend_supported_on_device to arch-gate an
  already-resolved backend on a specific (heterogeneous) CUDA device.
- video: make the explicit MagCache resize transactional via _step_cache_all_or_none
  (refuse to stack a fresh cache over one that could not be disabled; roll a mixed
  resize back and report the true state); raise on a failed all-or-none rollback
  instead of falsely reporting an uncached pipeline.
- diffusion_cfg_parallel: re-validate the attention backend on the replica device
  and pin native there when unsupported; mirror the primary's max tier on the
  replica (max-autotune compile + direct QKV fusion) via a new speed_mode arg;
  prefer a viable heterogeneous secondary GPU over an unusable identical one; clear
  the const cache at each plan_generation.
- diffusion_vae_quant / diffusion_precision: detect a partial diffusers
  layerwise-fp8 mutation (leftover casting hooks the torchao detector cannot see)
  and fail the load closed, while a clean failure still falls back to dense.
- video_speedmem_bench: engage the dual-expert cache all-or-none like the loader.
- frontend video api: add text_encoder_quant / vae_quant and the auto/off literals
  to VideoLoadRequest so typed callers match the backend contract.
This commit is contained in:
Daniel Han 2026-07-13 01:30:22 +00:00
commit 6e2e8c846c
13 changed files with 638 additions and 46 deletions

View file

@ -402,6 +402,7 @@ def _apply_levers(
normalize_cache_quality,
FBCACHE_MIN_STEPS,
)
from core.inference.video import _step_cache_all_or_none
tgt = _target()
engaged = {
@ -488,11 +489,14 @@ def _apply_levers(
cache_request = cfg["cache"]
if cache_request is not None:
# Quality preset like the loader: an unset request takes the family's auto default.
# Expert names zip with the views so a dual-expert MoE resolves per-expert curves.
quality = normalize_cache_quality(cache_quality) or auto_cache_quality(fam_name)
experts = ("transformer", "transformer_2")
for v, expert in zip(views, experts):
engaged["cache"] = apply_step_cache(
# All-or-none across MoE experts, exactly like the loader: overwriting engaged["cache"]
# per expert would leave one expert cached and one dense on a partial engage while the
# row reports the cache off -- a config that never runs in production. The shared helper
# rolls the engaged expert(s) back so the row measures a real configuration.
def _engage_cache(v: Any, expert: str) -> Optional[str]:
return apply_step_cache(
v,
mode = cache_request,
threshold = cache_threshold,
@ -503,6 +507,12 @@ def _apply_levers(
expert = expert,
logger = logger,
)
engaged["cache"], cache_partial_reason = _step_cache_all_or_none(
pipe, fam_obj, _engage_cache, logger = logger
)
if cache_partial_reason and logger is not None:
logger.warning("benchmark cache disabled: %s", cache_partial_reason)
cache_active = engaged["cache"] not in (None, "off")
# HunyuanVideo-1.5 joint-attention trim (per expert), BEFORE the backend set like the loader.