70af9fa75a
The fp8 loader builds Ideogram4Transformer2DModel via from_config, which materializes the full ~9B-parameter module at the process default dtype (fp32) before the dequantized bf16 weights are copied in and cast at the end. That fp32 scaffold is ~2x the bf16 model (~37 GB vs ~18 GB) on host RAM, and the second (unconditional) DiT builds while the first DiT and the text encoder are already resident, so it can OOM smaller hosts. Wrap from_config in set_default_dtype(dtype) so the module is built at the target dtype directly. rotary_emb.inv_freq (the only __init__ state absent from the checkpoint) is computed in explicit fp32, so a bf16 default leaves it correct. |
||
|---|---|---|
| .. | ||
| data_recipe | ||
| export | ||
| inference | ||
| rag | ||
| training | ||
| __init__.py | ||
| _torchao_stub.py | ||
| import_guards.py | ||
| tool_healing.py | ||