Studio diffusion (Phase 15): build int8 pre-quantized checkpoints (skip M=1 modulation linears)

The prequant-checkpoint builder applied the dense quant filter without the int8-only
M=1 modulation / conditioning-embedder exclusion the runtime path uses, so a built int8
checkpoint baked those projections as int8 and crashed (torch._int_mm needs M>16) at the
first denoise step on Flux / Qwen. Factor the scheme->exclusion decision into a shared
exclude_tokens_for_scheme() used by both the runtime quantise path and the offline builder
so they can never drift, and apply it in build_prequant_checkpoint.py. int8 prequant now
produces a working checkpoint on every supported model, giving int8 (the consumer-preferred
scheme) the same ~2x load-VRAM and download reduction fp8 already had.
This commit is contained in:
Daniel Han 2026-06-27 07:32:18 +00:00
commit 5ef000f912
3 changed files with 37 additions and 2 deletions

View file

@ -57,6 +57,7 @@ def main(argv = None) -> int:
from core.inference.diffusion_transformer_quant import (
TQ_SCHEMES,
_make_quant_config,
exclude_tokens_for_scheme,
make_filter_fn,
)
from torchao.quantization import quantize_
@ -78,7 +79,15 @@ def main(argv = None) -> int:
args.base, subfolder = "transformer", torch_dtype = torch.bfloat16, token = args.hf_token
).to("cuda")
print(f" quantising in place ({scheme}) ...", flush = True)
quantize_(transformer, _make_quant_config(scheme), filter_fn = make_filter_fn(args.min_features))
# Mirror the runtime path EXACTLY (the offline == runtime, LPIPS-0 invariant): for int8 also
# skip the M=1 AdaLN-modulation / conditioning-embedder projections, else the saved checkpoint
# bakes them as int8 and crashes (torch._int_mm needs M>16) at the first denoise step on
# Flux / Qwen. fp8 / fp4 / mx use scaled_mm (no M limit) -> exclude_tokens_for_scheme returns ().
quantize_(
transformer,
_make_quant_config(scheme),
filter_fn = make_filter_fn(args.min_features, exclude_name_tokens = exclude_tokens_for_scheme(scheme)),
)
# Move the state dict to CPU for a portable, GPU-free artifact.
state_dict = {