dbb0292561
Generalise the text-encoder precision knob from a fp8 bool to text_encoder_quant (fp8 | nvfp4). nvfp4 quantises the companion text encoder to 4-bit via torchao NVFP4 weight-only (two-level microscaling) on Blackwell's FP4 tensor cores; fp8 stays the broader-hardware path (cc>=8.9). Both are gated, best-effort, and run before placement; status reports the mode actually engaged. This is the lean realisation of GGUF-native text-encoder quant: 4-bit on the encoder without the 3045-line port. Verified on Z-Image (B200, balanced/group where the encoder stays resident), vs the bf16 encoder: nvfp4 cut generation peak VRAM 48% (10840 -> 5593 MB, the lowest TE option, below whole-model offload) at near-fp8 quality (16.4 vs 17.1 dB PSNR), and both quants ran faster than bf16. A memory-vs-quality tradeoff (off by default); size it per model with the Phase 5 quality harness. diffusion_bench gains --text-encoder-quant. 129 CPU tests pass. |
||
|---|---|---|
| .. | ||
| data_recipe | ||
| .gitkeep | ||
| __init__.py | ||
| auth.py | ||
| chat_history.py | ||
| datasets.py | ||
| export.py | ||
| inference.py | ||
| llama.py | ||
| mcp_servers.py | ||
| models.py | ||
| preview.py | ||
| prompts.py | ||
| providers.py | ||
| rag.py | ||
| settings.py | ||
| training.py | ||
| training_history.py | ||
| training_vram.py | ||