HunyuanVideo-1.5's DiT runs a joint [video; text] self-attention and, on every
block and step, builds a dense [B,1,N,N] boolean mask so the video never attends
to the padded text. A dense bool attn_mask disables every fused SDPA kernel
(flash rejects it; cuDNN and memory-efficient fall back), so the attention runs
the slow math-style path: at the production shape (121 frames, 480p, N about 50k)
one attention call is ~421ms with the mask vs ~19ms with attn_mask=None. The text
is ~99.5% padding (a t2v prompt fills ~9 of ~1985 slots), so nearly all of that
cost is spent masking padding.
install_hunyuan_attention_trim installs an eager forward pre-hook that drops the
all-zero image stream (t2v) and trims the mllm/byt5 text streams to their
globally-valid columns, plus a null-mask attention processor that runs
attn_mask=None once no partially-padded column remains (the batch-1 /
per-guidance-branch case) and otherwise delegates to the stock dense-mask
processor. The model already zeroes and masks the padded text and discards its
attention output (only the video split feeds proj_out), so removing it is exact
for the video; the only numeric change is the SDPA kernel (masked fallback to
fused). Measured on a B200: 23.3s to 1.3s per DiT forward at 121 frames (~18x with
regional compile, 0 graph breaks); per-forward cosine 0.99998 vs stock; equal
distance to an fp32 reference (LPIPS fp32-vs-stock 0.292, fp32-vs-trim 0.307), so
it is not less accurate than the current bf16 default.
Wired auto-on for HunyuanVideo-1.5 in the video loader, before the attention
backend set so the requested kernel pins onto the new processors; a no-op for
every other family and reversible (stock dense-mask path on any anomaly). Adds
hermetic tests and the diagnostic/validation scripts.
a5928064a0