ac16073923
fp16 accumulation (torch.backends.cuda.matmul.allow_fp16_accumulation) roughly doubles fp16 GEMM throughput on consumer tensor cores by keeping the accumulator in fp16. The flag only affects fp16 GEMMs: bf16-compute DiT families are untouched by construction, while SDXL's fp16 UNet and any fp16 text encoder or VAE path get the speedup. Gate in apply_speed_optims: CUDA target, consumer GPU (datacenter parts keep fp32 accumulation), torch exposes the flag, family not in _FP16_ACCUM_DENY, and the UNSLOTH_DISABLE_FP16_ACCUM kill switch is unset. The flag is captured in snapshot_backend_flags and restored on unload like the other process-wide knobs. _FP16_ACCUM_DENY starts empty: a same-seed A/B harness (off vs on per family at 512 and 1024 with long-prompt and high-guidance stress cases, non-finite, black frame and drift checks) backs the empty list and populates it if a family ever overflows. |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| core | ||
| hub | ||
| loggers | ||
| models | ||
| plugins | ||
| requirements | ||
| routes | ||
| state | ||
| storage | ||
| tests | ||
| utils | ||
| __init__.py | ||
| _platform_compat.py | ||
| cloudflare_tunnel.py | ||
| colab.py | ||
| main.py | ||
| run.py | ||
| startup_banner.py | ||