Harden video diffusion cache, CFG-parallel replica, and layerwise-fp8 rollback
- diffusion_attention: clear the HunyuanVideo-1.5 null-mask flag with an always_call post-hook so it is scoped to one hooked forward and never latches across an exception; add attention_backend_supported_on_device to arch-gate an already-resolved backend on a specific (heterogeneous) CUDA device. - video: make the explicit MagCache resize transactional via _step_cache_all_or_none (refuse to stack a fresh cache over one that could not be disabled; roll a mixed resize back and report the true state); raise on a failed all-or-none rollback instead of falsely reporting an uncached pipeline. - diffusion_cfg_parallel: re-validate the attention backend on the replica device and pin native there when unsupported; mirror the primary's max tier on the replica (max-autotune compile + direct QKV fusion) via a new speed_mode arg; prefer a viable heterogeneous secondary GPU over an unusable identical one; clear the const cache at each plan_generation. - diffusion_vae_quant / diffusion_precision: detect a partial diffusers layerwise-fp8 mutation (leftover casting hooks the torchao detector cannot see) and fail the load closed, while a clean failure still falls back to dense. - video_speedmem_bench: engage the dual-expert cache all-or-none like the loader. - frontend video api: add text_encoder_quant / vae_quant and the auto/off literals to VideoLoadRequest so typed callers match the backend contract.
This commit is contained in:
parent
e727b9e82f
commit
6e2e8c846c
13 changed files with 638 additions and 46 deletions
|
|
@ -402,6 +402,7 @@ def _apply_levers(
|
|||
normalize_cache_quality,
|
||||
FBCACHE_MIN_STEPS,
|
||||
)
|
||||
from core.inference.video import _step_cache_all_or_none
|
||||
|
||||
tgt = _target()
|
||||
engaged = {
|
||||
|
|
@ -488,11 +489,14 @@ def _apply_levers(
|
|||
cache_request = cfg["cache"]
|
||||
if cache_request is not None:
|
||||
# Quality preset like the loader: an unset request takes the family's auto default.
|
||||
# Expert names zip with the views so a dual-expert MoE resolves per-expert curves.
|
||||
quality = normalize_cache_quality(cache_quality) or auto_cache_quality(fam_name)
|
||||
experts = ("transformer", "transformer_2")
|
||||
for v, expert in zip(views, experts):
|
||||
engaged["cache"] = apply_step_cache(
|
||||
|
||||
# All-or-none across MoE experts, exactly like the loader: overwriting engaged["cache"]
|
||||
# per expert would leave one expert cached and one dense on a partial engage while the
|
||||
# row reports the cache off -- a config that never runs in production. The shared helper
|
||||
# rolls the engaged expert(s) back so the row measures a real configuration.
|
||||
def _engage_cache(v: Any, expert: str) -> Optional[str]:
|
||||
return apply_step_cache(
|
||||
v,
|
||||
mode = cache_request,
|
||||
threshold = cache_threshold,
|
||||
|
|
@ -503,6 +507,12 @@ def _apply_levers(
|
|||
expert = expert,
|
||||
logger = logger,
|
||||
)
|
||||
|
||||
engaged["cache"], cache_partial_reason = _step_cache_all_or_none(
|
||||
pipe, fam_obj, _engage_cache, logger = logger
|
||||
)
|
||||
if cache_partial_reason and logger is not None:
|
||||
logger.warning("benchmark cache disabled: %s", cache_partial_reason)
|
||||
cache_active = engaged["cache"] not in (None, "off")
|
||||
|
||||
# HunyuanVideo-1.5 joint-attention trim (per expert), BEFORE the backend set like the loader.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue