70af9fa75a
The fp8 loader builds Ideogram4Transformer2DModel via from_config, which materializes the full ~9B-parameter module at the process default dtype (fp32) before the dequantized bf16 weights are copied in and cast at the end. That fp32 scaffold is ~2x the bf16 model (~37 GB vs ~18 GB) on host RAM, and the second (unconditional) DiT builds while the first DiT and the text encoder are already resident, so it can OOM smaller hosts. Wrap from_config in set_default_dtype(dtype) so the module is built at the target dtype directly. rotary_emb.inv_freq (the only __init__ state absent from the checkpoint) is computed in explicit fp32, so a bf16 default leaves it correct. |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| core | ||
| hub | ||
| loggers | ||
| models | ||
| plugins | ||
| requirements | ||
| routes | ||
| state | ||
| storage | ||
| tests | ||
| utils | ||
| __init__.py | ||
| _platform_compat.py | ||
| cloudflare_tunnel.py | ||
| colab.py | ||
| main.py | ||
| run.py | ||
| startup_banner.py | ||