70af9fa75a
The fp8 loader builds Ideogram4Transformer2DModel via from_config, which materializes the full ~9B-parameter module at the process default dtype (fp32) before the dequantized bf16 weights are copied in and cast at the end. That fp32 scaffold is ~2x the bf16 model (~37 GB vs ~18 GB) on host RAM, and the second (unconditional) DiT builds while the first DiT and the text encoder are already resident, so it can OOM smaller hosts. Wrap from_config in set_default_dtype(dtype) so the module is built at the target dtype directly. rotary_emb.inv_freq (the only __init__ state absent from the checkpoint) is computed in explicit fp32, so a bf16 default leaves it correct. |
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| src-tauri | ||
| __init__.py | ||
| install_llama_prebuilt.py | ||
| install_node_prebuilt.py | ||
| install_python_stack.py | ||
| install_sd_cpp_prebuilt.py | ||
| LICENSE.AGPL-3.0 | ||
| node_prebuilt_pins.json | ||
| package-lock.json | ||
| package.json | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||