7bf470f8b9
A B200 speed/memory sweep (new scripts/quant_speedmem_bench.py) shows the VAE quant win is a video story. Image AutoencoderKLs are ~0.15-0.26 GB, so fp8 saves ~0.1 GB and only slows their tiny decode (+6-16%); the video Conv3d VAEs are ~2.5 GB and halve to ~1.2 GB at ~2% decode cost. So VAE auto now only engages above a ~1 GB size floor: small image VAEs stay dense (faster decode, no quant quality risk), video VAEs still quantize. An explicit fp8 / fp8_dynamic request skips the gate (opted in). The same sweep confirmed the text-encoder default is already right: fp8_dynamic is E2E-neutral (denoise per-step unchanged; +2% one-time encode) and, by hidden-state cosine vs bf16, marginally more accurate than layerwise fp8 -- so that default is left as is. Tests cover the gate (small skipped, large quantized, explicit bypasses). |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| core | ||
| hub | ||
| loggers | ||
| models | ||
| plugins | ||
| requirements | ||
| routes | ||
| state | ||
| storage | ||
| tests | ||
| utils | ||
| __init__.py | ||
| _platform_compat.py | ||
| cloudflare_tunnel.py | ||
| colab.py | ||
| main.py | ||
| run.py | ||
| startup_banner.py | ||