Fix black frames on Wan video: deny fp8 DiT auto-quant, fall to int8

The dense video default engages transformer auto-quant, and on Blackwell the
auto ladder leads with fp8. On the Wan DiT the production per-row fp8 path
(torch._scaled_mm) renders every frame black (mean luma 0.0 at 512x320 and
704x480, LPIPS ~0.80 vs bf16): Wan's activation outliers exceed per-row fp8's
range, the same failure already denied for qwen-image. First-Block-Cache then
over-caches the degenerate activations (per-step collapses to ~10ms),
compounding it.

Add the Wan families (wan2.2-ti2v-5b, wan2.2-t2v-a14b, same WanTransformer3DModel)
to _FAMILY_SCHEME_DENY for fp8/mxfp8/nvfp4 so auto falls through to int8, which is
clean on Wan (per-token, outlier-robust), saves the same weight memory on the DiT,
and lets First-Block-Cache engage normally instead of over-caching. mxfp8/nvfp4 are
denied alongside fp8 conservatively so auto lands on the battle-tested int8; they
can be re-enabled per family once validated in-bar, like the nvfp4 auto-ladder TODO.

Validated on B200: the shipped video default now selects int8 for the Wan DiT and
renders clean frames (mean 172.6) at 15.6 GB resident (down from 24.2 GB dense),
with First-Block-Cache engaged. Adds two deny tests; 48/48 transformer-quant tests pass.
This commit is contained in:
Daniel Han 2026-07-08 14:35:48 +00:00
commit cbf24dd847
2 changed files with 39 additions and 4 deletions

View file

@ -133,13 +133,27 @@ _AUTO_LADDER: tuple[tuple[tuple[int, int], tuple[str, ...]], ...] = (
# qwen-image + mxfp8 -> real semantic damage at 1024px (CLIP delta mean 0.0146, worst
# cases 0.064 / 0.102 -- 2x the per-case bound).
# qwen-image + nvfp4 -> LPIPS mean 0.51 vs bf16: unusable.
# int8 dynamic (per-token) is excellent on Qwen (LPIPS mean 0.069 / SSIM 0.958), so the
# auto ladder falls through to it. The deny also applies to an EXPLICIT request: a
# scheme that renders black frames has no legitimate use, and returning None gives the
# caller the same fallback contract as an unsupported scheme (GGUF build).
# wan2.2 + fp8 -> every frame black (mean luma 0.0000, LPIPS ~0.80 vs bf16),
# reproduced on B200 at 512x320 and 704x480 with the production
# torch._scaled_mm per-row fp8 path (no MSLK): the Wan DiT's
# activation outliers exceed per-row fp8's range, the same failure
# mode as qwen-image. int8 dynamic (per-token) is clean on Wan
# (non-black, correct contrast; First-Block-Cache engages normally
# instead of over-caching the degenerate black activations).
# int8 dynamic (per-token) is excellent on Qwen (LPIPS mean 0.069 / SSIM 0.958) and clean on
# Wan, so the auto ladder falls through to it. mxfp8 / nvfp4 are denied alongside fp8 on the
# Wan families conservatively (the same per-block scaled_mm family as the confirmed-black fp8,
# and mxfp8 is a prototype) so auto lands on the battle-tested int8; they can be re-enabled per
# family once separately validated in-bar, like the nvfp4 auto-ladder TODO. The deny also
# applies to an EXPLICIT request: a scheme that renders black frames has no legitimate use, and
# returning None gives the caller the same fallback contract as an unsupported scheme (GGUF).
_FAMILY_SCHEME_DENY: dict[str, frozenset[str]] = {
"qwen-image": frozenset({TQ_FP8, TQ_MXFP8, TQ_NVFP4}),
"qwen-image-edit": frozenset({TQ_FP8, TQ_MXFP8, TQ_NVFP4}), # same DiT + activations
# Wan2.2 video DiTs (WanTransformer3DModel): fp8 renders black frames (measured); both the
# 5B TI2V and the A14B MoE share the DiT class + activation profile, so both deny -> int8.
"wan2.2-ti2v-5b": frozenset({TQ_FP8, TQ_MXFP8, TQ_NVFP4}),
"wan2.2-t2v-a14b": frozenset({TQ_FP8, TQ_MXFP8, TQ_NVFP4}),
}

View file

@ -554,6 +554,27 @@ def test_family_deny_refuses_explicit_fp8_for_qwen(monkeypatch):
assert select_transformer_quant_scheme(_target(), "fp8", family = "z-image") == TQ_FP8
def test_family_deny_auto_skips_fp8_for_wan(monkeypatch):
# B200 with every scheme available: auto must NOT pick fp8 / nvfp4 / mxfp8 for the Wan
# video DiT (per-row fp8 renders black frames on it, measured; see _FAMILY_SCHEME_DENY)
# and falls through the ladder to int8, which is clean on Wan. Both the 5B TI2V and the
# A14B MoE share the WanTransformer3DModel activation profile, so both deny to int8.
_stub_torch(monkeypatch, cc = (10, 0))
_allow(monkeypatch, {TQ_FP8, TQ_NVFP4, TQ_MXFP8, TQ_INT8})
assert select_transformer_quant_scheme(_target(), "auto", family = "wan2.2-ti2v-5b") == TQ_INT8
assert select_transformer_quant_scheme(_target(), "auto", family = "wan2.2-t2v-a14b") == TQ_INT8
def test_family_deny_refuses_explicit_fp8_for_wan(monkeypatch):
# An explicit fp8 request on a Wan family returns None (same GGUF-fallback contract as
# qwen); int8 stays honored on Wan, and fp8 stays honored on video families outside the
# deny table (e.g. an untested family keeps the default ladder until validated).
_stub_torch(monkeypatch, cc = (10, 0))
_allow(monkeypatch, {TQ_FP8, TQ_INT8})
assert select_transformer_quant_scheme(_target(), "fp8", family = "wan2.2-ti2v-5b") is None
assert select_transformer_quant_scheme(_target(), "int8", family = "wan2.2-ti2v-5b") == TQ_INT8
def test_family_deny_no_family_keeps_ladder(monkeypatch):
# Without a family (or an unknown one) the ladder is unchanged: fp8 first on B200.
_stub_torch(monkeypatch, cc = (10, 0))