unsloth/scripts
Daniel Han 983f2c14a0 Studio diffusion (Phase 8): opt-in fast transformer (torchao int8/fp8/fp4 on a dense source)
Add an opt-in transformer_quant mode that loads the dense bf16 transformer and
torchao-quantises it onto the low-precision tensor cores, instead of the GGUF
transformer (which dequantises to bf16 per matmul and so runs at bf16 rate). On a
B200 (Z-Image-Turbo, 1024px/8 steps): auto picks fp8 at 0.614s vs GGUF+compile's
0.823s (1.34x), int8 0.626s (1.32x), both at lower LPIPS than GGUF's own 4-bit floor.

GGUF+compile stays the low-memory default and the fallback. The mode is gated on
CUDA + bf16 + resident VRAM headroom (the dense load peaks ~21GB vs GGUF's 13GB);
any unsupported arch/scheme, OOM, or quant failure falls back to GGUF with a logged
reason. auto picks the best scheme per GPU via a real quantise+matmul smoke probe
(Blackwell nvfp4/fp8/mxfp8, Ada/Hopper fp8, Ampere int8); a min-features filter skips
the tiny projections that crash int8's torch._int_mm. New module mirrors
diffusion_precision.py; quant runs before compile before placement.

184 -> tests pass; new test_diffusion_transformer_quant.py plus backend/route
coverage. scripts/diffusion_bench.py gains --transformer-quant; scripts/quant_probe.py
is the standalone torchao lever probe.
2026-06-26 06:15:18 +00:00
..
data CI: scope GITHUB_TOKEN permissions, add MLX CI, unblock ~60 skipped tests (#5312) 2026-05-11 03:19:13 -07:00
check_frontend_dep_removal.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
check_new_install_scripts.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
compare_engines.py Studio diffusion (Phase 7): max tier uses max-autotune-no-cudagraphs + engine/lever benchmarks 2026-06-26 05:04:10 +00:00
compile_probe.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-06-26 03:20:02 +00:00
diffusion_bench.py Studio diffusion (Phase 8): opt-in fast transformer (torchao int8/fp8/fp4 on a dense source) 2026-06-26 06:15:18 +00:00
diffusion_quality.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-06-25 13:56:37 +00:00
enforce_kwargs_spacing.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
install_gemma4_mlx.sh Update Install Scripts (#5968) 2026-06-03 05:39:42 -07:00
install_qwen3_6_mlx.sh Update Install Scripts (#5968) 2026-06-03 05:39:42 -07:00
install_rocm_wsl_strixhalo.sh Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) 2026-06-10 04:24:49 -07:00
leverage_probe.py Studio diffusion (Phase 7): max tier uses max-autotune-no-cudagraphs + engine/lever benchmarks 2026-06-26 05:04:10 +00:00
lint_workflow_triggers.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
lockfile_supply_chain_audit.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
notebook_to_python.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
notebook_validator.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
perf_verify.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-06-26 03:20:02 +00:00
quant_probe.py Studio diffusion (Phase 8): opt-in fast transformer (torchao int8/fp8/fp4 on a dense source) 2026-06-26 06:15:18 +00:00
run_ruff_format.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
scan_npm_packages.py Package scanners: close fail-open gaps in the sdist fallback and hidden-payload paths (#6359) 2026-06-18 06:50:16 -07:00
scan_npm_packages_baseline.json Package scanners: close fail-open gaps in the sdist fallback and hidden-payload paths (#6359) 2026-06-18 06:50:16 -07:00
scan_packages.py Package scanners: close fail-open gaps in the sdist fallback and hidden-payload paths (#6359) 2026-06-18 06:50:16 -07:00
scan_packages_baseline.json Studio: remove OpenEnv and other unused packages (#6585) 2026-06-23 07:20:47 -07:00
sd_cpp_smoke.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-06-25 16:08:55 +00:00
stamp_studio_release.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
sync_allow_scripts_pins.py Studio: auto-sync allowScripts pins after dependency bumps (#6136) 2026-06-10 02:35:37 -07:00
uninstall.ps1 Windows installer: fix DiskPart UAC mid-install, drive-root cache, and spurious unsloth.exe rename warning (#6296) 2026-06-22 03:09:08 -07:00
uninstall.sh Windows installer: fix DiskPart UAC mid-install, drive-root cache, and spurious unsloth.exe rename warning (#6296) 2026-06-22 03:09:08 -07:00
verify_comment_only_diff.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
verify_import_hoist.py CI: fix import-hoist false positive, vision-cache test cwd, llama.cpp CLI smoke (#6598) 2026-06-23 01:16:47 -07:00