092fc91e08
The dense-quant re-plan passes transformer_resident_override_mib (the bf16 build peak) AND computes companions via _companion_cache_bytes(base), which sums every flat blob in the HF cache. Because the dense path prefetches the base transformer/ shards into that same cache before load_pipeline runs, the transformer is counted twice, inflating the footprint (~44 GB instead of ~20 GB in the reproduction) and wrongly forcing offload for models that fit resident -- the case this path exists to enable. Add companion_override_mib and pass the auto-policy's own text-encoder plus VAE estimate on the re-plan so the cache (with its prefetched transformer) is not read for this artifact. |
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| src-tauri | ||
| __init__.py | ||
| install_llama_prebuilt.py | ||
| install_node_prebuilt.py | ||
| install_python_stack.py | ||
| install_sd_cpp_prebuilt.py | ||
| LICENSE.AGPL-3.0 | ||
| node_prebuilt_pins.json | ||
| package-lock.json | ||
| package.json | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||