bb2b14db97
- The FP8 Qwen3-VL text encoder was constructed at the process fp32 default before the dequantized bf16 weights are copied in. That ~8B-param fp32 scaffold peaks ~2x on host RAM (loading FIRST, before the DiTs), so a 64 GB host can OOM. Build it at the target dtype under set_default_dtype, mirroring the DiT loader; rotary inv_freq is still computed in explicit fp32. - The auto-policy memory table listed the text encoder at 8.8 GB, its FP8 on-disk size, while the DiTs were doubled to their bf16-resident sizes. The loader dequantizes the encoder to bf16 too (~16.3 GB), so the entry understated the resident footprint by ~7.5 GB and could let the planner pick a resident placement that OOMs. Size it as bf16-resident. - Speed (regional compile, QKV fuse) and the attention backend only touched pipe.transformer, so ideogram-4's second denoiser (unconditional_transformer, run every step for dual-branch CFG) stayed eager/native while status reported the optimization as engaged. Iterate every denoiser DiT (mirroring the offload path) so both experts are optimized. Guarded on attr presence, so single-DiT families are unchanged. |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| core | ||
| hub | ||
| loggers | ||
| models | ||
| plugins | ||
| requirements | ||
| routes | ||
| state | ||
| storage | ||
| tests | ||
| utils | ||
| __init__.py | ||
| _platform_compat.py | ||
| cloudflare_tunnel.py | ||
| colab.py | ||
| main.py | ||
| run.py | ||
| startup_banner.py | ||