perf(image): compile numeric parity, cache-hook compile arming, FBCache toggle crash fix, TE fp8 zero-row guard

Applies the video round-2 accuracy findings to the image diffusion stack and fixes
two real image-path bugs found while measuring. All numbers B200, production
settings (family default steps/guidance, 1024px, seed 42, 4 fixed prompts), LPIPS
(AlexNet) via the new scripts/image_speedmem_bench.py, which drives the production
lever functions in the loader's own order.

- inductor precision parity: emulate_precision_casts=True on the regional-compile
  path (fused pointwise kernels keep fp32 intermediates where eager rounds to bf16
  between ops). Pairwise LPIPS of the compiled tier vs the same-stack eager tier:
  Qwen-Image 0.019 to 0.006 at identical speed (72.4 vs 72.5 ms/step), FLUX.1-dev
  0.046 to 0.029 at +2% step time (69.8 vs 68.3, reproduced), FLUX.2-klein-4B
  0.018 to 0.017 at identical speed. Snapshot/restored with the other process-wide
  backend flags so an off load never inherits it.
- cache x compile composition: re-point each cache hook's fn_ref.original_forward
  at a torch.compile'd wrapper of the same bound method (armed only where the
  speed layer compiled the block; restored before every disable_cache and before
  the partial-hook cleanup). Qwen-Image FBCache computed steps 91.8 to 71.2 ms
  (back at the uncached compiled rate), 1.21x end to end (7.36 to 6.06 s per 4
  images); FLUX.1-dev already traced through its FBCache hook and is measured
  neutral (same-process armed vs unarmed latents bit-identical). Skip counts
  within noise (13 vs 11 of 76; pairwise LPIPS 0.005).
- FBCache mid-session toggle crash: diffusers 0.39 caches the HookRegistry child
  list on first cache_context use, so an uncached generation followed by a
  20+-step generation (the auto toggle path) enabled hooks the context never
  reached and crashed with "No context is set" (reproduced live on FLUX.1-dev).
  Invalidate the stale child cache after every enable_cache.
- TE fp8_dynamic zero-row guard: torchao per-row fp8 derives a per-output-channel
  scale from the row amax, so an all-zero weight row is 0/0 = NaN. SDXL's
  text_encoder_2 (OpenCLIP bigG) ships exactly such a row, and every explicit
  fp8_dynamic SDXL render came out black; keep zero-row Linears dense (LPIPS
  0.976 black to 0.096 working). Other families' encoders have no such rows and
  are byte-identical.
- No AUTO TE quant exists on the image branch (text_encoder_quant defaults dense,
  explicit-only), so the video round's auto-dense retune has no image analogue;
  the explicit lever's cost is now measured (TE fp8_dynamic alone, LPIPS vs
  bit-exact: Qwen-Image 0.038, FLUX.1-dev 0.084, SDXL 0.096; no speed win, VRAM
  -6.5 GB on Qwen-Image) for the docs.

Tests: 96 passing across the cache/speed/precision suites (11 new arming, 2
child-registry, 2 zero-row, 4 inductor-flag); ruff clean.
This commit is contained in:
Daniel Han 2026-07-10 16:07:46 +00:00
commit de2f22df2b
7 changed files with 997 additions and 2 deletions

View file

@ -393,3 +393,81 @@ def test_nvfp4_filter_keeps_vision_tower_dense(monkeypatch):
assert ff(object(), "lm_head") is False
assert ff(object(), "model.decoder.wo") is False
assert ff(object(), "model.layers.5.self_attn.q_proj") is True
# ── zero-output-row guard (per-row fp8 NaN protection) ───────────────────────────
class _FakeAmaxVec:
def __init__(self, vals):
self._vals = vals
def __eq__(self, other): # noqa: PLW0642 -- tensor-style elementwise compare
return _FakeAmaxVec([v == other for v in self._vals])
def any(self):
return _FakeScalar(any(self._vals))
class _FakeScalar:
def __init__(self, v):
self._v = v
def item(self):
return self._v
class _FakeWeight:
"""Tensor-shaped stand-in supporting the exact chain the guard runs:
``weight.abs().amax(dim = -1) == 0 -> .any().item()``."""
ndim = 2
def __init__(self, rows):
self._rows = rows
def abs(self):
return _FakeWeight([[abs(v) for v in r] for r in self._rows])
def amax(self, dim = -1):
return _FakeAmaxVec([max(r) for r in self._rows])
def test_weight_zero_output_row_detection():
# A dead output row NaNs torchao's per-row fp8 (scale 0 -> 0/0); SDXL's
# text_encoder_2 (OpenCLIP bigG) really ships one in layers.2.self_attn.out_proj --
# measured: every fp8_dynamic SDXL render was black until the row is kept dense.
zero_row = types.SimpleNamespace(weight = _FakeWeight([[0.1, 0.2], [0.0, 0.0]]))
dense = types.SimpleNamespace(weight = _FakeWeight([[0.1, 0.2], [0.3, 0.0]]))
assert dp._weight_has_zero_output_row(zero_row) is True
assert dp._weight_has_zero_output_row(dense) is False
# Non-2D / absent weights are not the per-row scheme's input: never flagged.
w3 = _FakeWeight([[1.0]])
w3.ndim = 3
assert dp._weight_has_zero_output_row(types.SimpleNamespace(weight = w3)) is False
assert dp._weight_has_zero_output_row(types.SimpleNamespace()) is False
# An unreadable weight falls through to quantize_'s own handling.
class _Boom:
@property
def weight(self):
raise RuntimeError("meta tensor")
assert dp._weight_has_zero_output_row(_Boom()) is False
def test_fp8_dynamic_filter_skips_zero_row_linear(monkeypatch):
# The fp8_dynamic caster must leave a zero-output-row Linear dense while the rest
# of the encoder still quantises (a family-wide deny would forfeit the whole win).
_stub_torch(monkeypatch)
captured: dict = {}
_stub_transformer_quant(monkeypatch, captured)
enc = types.SimpleNamespace(_keep_in_fp32_modules = [])
dp._cast_fp8_dynamic(enc, _target())
ff = captured["filter_fn"]
dead = types.SimpleNamespace(weight = _FakeWeight([[0.5, 0.5], [0.0, 0.0]]))
live = types.SimpleNamespace(weight = _FakeWeight([[0.5, 0.5], [0.5, 0.5]]))
assert ff(dead, "text_model.encoder.layers.2.self_attn.out_proj") is False
assert ff(live, "text_model.encoder.layers.2.mlp.fc1") is True