40b285aa7c
The transformer already defaults to auto-quant (fp8/int8); the companion text encoder was opt-in and stayed dense bf16 unless a scheme was named, even though it is often the largest resident component. Add an auto policy mirroring the transformer's ladder: select_te_quant_scheme walks a per-capability ladder (data-center fp8-GEMM: fp8_dynamic -> int8 -> layerwise fp8; Ampere: int8 -> fp8), reorders int8 first on consumer GDDR parts, falls to layerwise fp8 under group offload (the only offload-safe cast), only picks int8 for a family with a measured keep-bf16 schedule, honors a per-family deny list, and smoke-probes the torchao kernel so a missing build degrades gracefully. The image + video loaders now map an unset text_encoder_quant to auto (explicit none/off stays dense; a named scheme is forced), so the shipped default quantizes the encoder to the fastest accurate scheme for the GPU. Records the engaged scheme in the image resolved-record too. Verified on a B200: auto -> fp8_dynamic, offload -> layerwise fp8. |
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| src-tauri | ||
| __init__.py | ||
| install_llama_prebuilt.py | ||
| install_node_prebuilt.py | ||
| install_python_stack.py | ||
| install_sd_cpp_prebuilt.py | ||
| LICENSE.AGPL-3.0 | ||
| node_prebuilt_pins.json | ||
| package-lock.json | ||
| package.json | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||