video/image: honor explicit Speed=off for companions + trim, probe explicit TE kernels, bench fidelity

Address the Codex review round on the video/quant work:

- Companion auto-quant now honors an explicit Speed=off. Both loaders already pin the DiT dense
  under an explicit off (bit-exact reference), but the unset text-encoder / VAE quant still promoted
  to auto and silently fp8/int8'd the companions, breaking the bit-exact request. An UNSET speed
  still auto-quantises; an explicit companion scheme still forces it.
- The HunyuanVideo joint-attention trim is a speed lever (it swaps to the fused SDPA kernel), so gate
  it on a non-off speed tier exactly like the adjacent attention-backend selection -- the off path
  keeps the stock dense-mask attention.
- Explicit torchao text-encoder modes (int8 / fp8_dynamic / nvfp4) now run the same kernel smoke
  test the auto ladder uses. They could clear the capability gate yet fail the real GEMM on a build
  where quantize_ wraps the encoder but the kernel is broken; the caster's try/except only covers the
  cast, not the first forward, so the load would report engaged then crash at generation. Now it
  falls back to dense. Layerwise fp8 has no torchao GEMM, so the probe is a no-op for it.
- The trim pre-hook's fallback restores the caller's original kwargs (it may have emptied the image
  stream / trimmed a text stream before failing), so the stock dense-mask path runs on exactly what
  it expects, matching the empty-prompt guard.
- video_speedmem_bench mirrors the loader: installs the Hunyuan trim before the backend set (gated on
  an active tier) and skips the auto int8 quant when it is the fp8-denied memory fallback and dense
  fits resident, so the shipped/auto rows measure what the loader actually runs.

Tests: TE explicit-mode kernel probe (+ layerwise-fp8 bypass), trim mid-trim restore, and loader-level
speed=off companion suppression + trim skip for both backends. 262 backend tests pass; ruff clean.
This commit is contained in:
Daniel Han 2026-07-09 09:24:28 +00:00
commit be04ba00f4
9 changed files with 201 additions and 15 deletions

View file

@ -1217,17 +1217,23 @@ class DiffusionBackend:
normalize_transformer_cache(transformer_cache)
normalize_te_quant(text_encoder_quant)
normalize_vae_quant(vae_quant)
# An explicit Speed="off" (bit-exact reference) load pins the companions dense too, mirroring
# the transformer_quant default below (load_pipeline): promoting an UNSET TE/VAE to auto-quant
# here would silently fp8/int8 the text encoder + VAE and break the bit-exact request -- an
# auto DEFAULT overriding the EXPLICIT off control. Only an EXPLICIT off suppresses; an unset
# speed still auto-quantises, and an explicit companion scheme still forces it.
speed_off = speed_mode is not None and str(speed_mode).strip().lower() == SPEED_OFF
# text_encoder_quant tri-state, mirroring transformer_quant: UNSET (None / "") -> auto,
# which picks the best accurate TE scheme for this GPU + family (fp8_dynamic / int8 /
# layerwise fp8) or stays dense when none qualifies. An explicit "none"/"off" pins the
# encoder dense; an explicit scheme forces it. So the shipped default is auto.
# encoder dense; an explicit scheme forces it. So the shipped default is auto (dense under off).
if text_encoder_quant is None or str(text_encoder_quant).strip() == "":
text_encoder_quant = TE_QUANT_AUTO
text_encoder_quant = "off" if speed_off else TE_QUANT_AUTO
# vae_quant tri-state, same contract: UNSET -> auto (fp8_dynamic conv compute on resident
# fp8-GEMM silicon that passes the conv probe, else layerwise fp8, else dense); none/off ->
# dense; an explicit scheme forces it.
# dense; an explicit scheme forces it. Also pinned dense under an explicit Speed="off".
if vae_quant is None or str(vae_quant).strip() == "":
vae_quant = VAE_QUANT_AUTO
vae_quant = "off" if speed_off else VAE_QUANT_AUTO
# For a full pipeline the repo itself supplies every component, so it is its
# own base; the single-file kinds resolve the companion base diffusers repo.
base = (