Review pass over the merged diffusion phases: seven correctness fixes
Re-reviewed each merged phase PR against this branch's tip and fixed what is still real: - A superseded background load no longer cancels the current model's in-flight generation: the load-token check now runs BEFORE the cancel signal, with a re-check under the generate lock (Phase 1 review). - enable_model_cpu_offload / enable_sequential_cpu_offload now forward the resolved target device; diffusers defaults to CUDA, which broke offloaded loads on non-CUDA accelerators such as Intel XPU (Phase 2 review). - build_sd_cpp_command rejects a None prompt (str(None) previously slipped into argv as the literal "None") and a mask without an init image, which is an invalid sd-cli inpaint invocation (Phase 4/6 review). - The dense-quant OOM fallback drops the caught exception before clear_gpu_cache(): the traceback pinned the partially built dense transformer, so the VRAM this cleanup exists to reclaim stayed allocated through the GGUF rebuild (Phase 8 review). - Pre-quantized transformers (built via from_config) are eval()'d to match the from_pretrained paths, so train-mode layers cannot make prequant inference nondeterministic (Phase 9 review). - FBCache state is reset before each generation when a step cache is engaged: diffusers never clears the stateful first-block residuals on the resident transformer, so a resolution or batch change on the next request hit a shape mismatch, and an unchanged request could reuse stale residuals (Phase 12 review). Each fix carries a regression test; the full diffusion battery passes.
This commit is contained in:
parent
49ea887312
commit
7f59cd6c1e
8 changed files with 176 additions and 12 deletions
|
|
@ -193,6 +193,15 @@ def load_prequantized_transformer(
|
|||
transformer.load_state_dict(state_dict, strict = True, assign = True)
|
||||
|
||||
transformer = transformer.to(device)
|
||||
# Built via from_config (not from_pretrained), so it starts in TRAIN mode; the
|
||||
# dense and GGUF paths load through from_pretrained, which diffusers documents as
|
||||
# returning an eval()'d module. Match that here so any train/eval-sensitive layer
|
||||
# (e.g. dropout) can't make prequant inference nondeterministic or diverge from
|
||||
# the other load paths.
|
||||
try:
|
||||
transformer.eval()
|
||||
except Exception: # noqa: BLE001 — eval() is best-effort
|
||||
pass
|
||||
try: # diagnostic marker, mirrors the runtime-quant path
|
||||
transformer._unsloth_runtime_quant = scheme
|
||||
except Exception: # noqa: BLE001 — marker is best-effort
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue