unsloth/scripts
Daniel Han e58f30be5f Run video DiT dense+compile when it fits instead of a slower int8 fallback
Root-caused "HunyuanVideo-1.5 int8 is slower than dense" with a per-forward profiler
(scripts/hunyuan_int8_profile.py, dynamo-reset, back-to-back on a clean B200): int8 compiles
cleanly (0 recompiles, 0 graph breaks, steady 268.3 ms/forward) and is only ~7% slower than dense
+ regional compile (250.5 ms/forward), not the 38% a contended-GPU bench run suggested. int8 is
also less accurate (LPIPS 0.085 vs dense+compile 0.037). So for a family where fp8 is denied
(Hunyuan black-frames on per-row fp8), int8 is a MEMORY lever, not a speed win, yet the auto-quant
default quantised it even when the dense DiT already fit resident.

Fix: is_int8_memory_fallback(target, family) is True only when AUTO quant lands on int8 as a
denied/black-frame fallback on a data-center, fp8-capable GPU (fp8 would be the arch pick but is
denied for the family). The video loader now skips the auto-quant and runs dense+compile when that
holds AND the bf16 memory plan already fits resident (offload_policy == none), so there is no new
OOM risk. Scoped tightly: only an AUTO request (explicit int8/fp8 honored), only int8-fallback
families (Wan / LTX resolve to fp8 -> keep quantising), only data-center fp8-capable parts (consumer
GPUs and pre-Ada, where int8 is a genuine accelerator, keep int8), and only when dense provably
fits; a memory-constrained plan still quantises. Result: Hunyuan on a resident-fit B200 now runs
faster AND more accurate, quantising only when memory is the constraint.

Also resets dynamo per config in the video bench (so compiled graphs cannot leak across configs in
one process) and adds the per-forward profiler used for the diagnosis.
2026-07-09 02:31:22 +00:00
..
data CI: scope GITHUB_TOKEN permissions, add MLX CI, unblock ~60 skipped tests (#5312) 2026-05-11 03:19:13 -07:00
build_prequant_checkpoint.py Restore fp8 DiT quant for Wan video via a per-family embedder exclude 2026-07-08 23:22:34 +00:00
check_frontend_dep_removal.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
check_new_install_scripts.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
compare_engines.py Studio diffusion (Phase 7): accuracy-preserving speed pass (2.2x via GGUF compile) (#6690) 2026-07-01 15:31:50 -03:00
compile_probe.py Studio diffusion (Phase 7): accuracy-preserving speed pass (2.2x via GGUF compile) (#6690) 2026-07-01 15:31:50 -03:00
diffusion_bench.py Studio diffusion (Phase 8): opt-in fast transformer (torchao int8/fp8/fp4 on a dense source) (#6694) 2026-07-01 15:36:43 -03:00
diffusion_quality.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-07-02 03:30:16 +00:00
enforce_kwargs_spacing.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
fbcache_flux_probe.py Studio diffusion (Phase 12): First-Block-Cache step caching for many-step DiT (#6703) 2026-07-01 15:40:52 -03:00
fp8_layer_ablation.py Restore fp8 DiT quant for Wan video via a per-family embedder exclude 2026-07-08 23:22:34 +00:00
fp8_overflow_check.py Studio diffusion (Phase 8): opt-in fast transformer (torchao int8/fp8/fp4 on a dense source) (#6694) 2026-07-01 15:36:43 -03:00
hunyuan_int8_profile.py Run video DiT dense+compile when it fits instead of a slower int8 fallback 2026-07-09 02:31:22 +00:00
install_gemma4_mlx.sh Update Install Scripts (#5968) 2026-06-03 05:39:42 -07:00
install_qwen3_6_mlx.sh Update Install Scripts (#5968) 2026-06-03 05:39:42 -07:00
install_rocm_wsl_strixhalo.sh ROCm-on-WSL: support discrete Radeon (RDNA 3/4) in WSL, not just Strix Halo (#6915) 2026-07-07 02:29:37 -07:00
int8_linear_probe.py Studio diffusion (Phase 14): fix int8 dense quant on Flux / Qwen (skip M=1 modulation linears) (#6716) 2026-07-01 15:41:49 -03:00
leverage_probe.py Studio diffusion (Phase 7): accuracy-preserving speed pass (2.2x via GGUF compile) (#6690) 2026-07-01 15:31:50 -03:00
lint_workflow_triggers.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
lockfile_supply_chain_audit.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
notebook_to_python.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
notebook_validator.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
nvfp4_probe.py Studio diffusion (Phase 8): opt-in fast transformer (torchao int8/fp8/fp4 on a dense source) (#6694) 2026-07-01 15:36:43 -03:00
nvfp4_t211_probe.py Fix diffusion training validation, dataset upload atomicity, and LoRA error mapping 2026-07-04 03:17:12 -03:00
perf_levers_probe.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-07-02 03:30:16 +00:00
perf_verify.py Studio diffusion (Phase 7): accuracy-preserving speed pass (2.2x via GGUF compile) (#6690) 2026-07-01 15:31:50 -03:00
prequant_probe.py Studio diffusion (Phase 9): pre-quantized transformer loading (#6700) 2026-07-01 15:37:53 -03:00
quant_accuracy_sweep.py Gate VAE fp8 quant by decoded-image accuracy (auto layerwise fp8, fp8_dynamic opt-in) 2026-07-08 10:45:27 +00:00
quant_probe.py Studio diffusion (Phase 8): opt-in fast transformer (torchao int8/fp8/fp4 on a dense source) (#6694) 2026-07-01 15:36:43 -03:00
quant_speedmem_bench.py Add video speed/memory lever benchmark; extend quant bench with DiT mode 2026-07-08 14:35:58 +00:00
run_ruff_format.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
scan_npm_packages.py scan_packages: key baseline on matched-code hash so payloads in baselined files are not auto-suppressed (#6552) 2026-07-01 04:03:59 -07:00
scan_npm_packages_baseline.json scan_packages: key baseline on matched-code hash so payloads in baselined files are not auto-suppressed (#6552) 2026-07-01 04:03:59 -07:00
scan_packages.py CI: baseline the fastapi/gguf scan findings for the studio dependency set 2026-07-02 02:04:28 +00:00
scan_packages_baseline.json Merge remote-tracking branch 'origin/main' into fold-integration 2026-07-07 05:52:16 +00:00
sd_cpp_smoke.py Studio diffusion (Phase 6): img2img / inpaint / edit / LoRA / upscale on the native engine (#6680) 2026-07-01 15:18:38 -03:00
sparse_accum_probe.py Fix diffusion training validation, dataset upload atomicity, and LoRA error mapping 2026-07-04 03:17:12 -03:00
stamp_studio_release.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
sync_allow_scripts_pins.py Studio: auto-sync allowScripts pins after dependency bumps (#6136) 2026-06-10 02:35:37 -07:00
uninstall.ps1 Merge image-generation bug fixes (#6872) 2026-07-07 16:14:06 +00:00
uninstall.sh Keep an unowned default-mode sd.cpp checkout on uninstall 2026-07-07 10:22:40 +00:00
verify_comment_only_diff.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
verify_import_hoist.py studio: tool calling for DeepSeek (R1/V3/V3.1), GLM 4.x, Kimi K2 on safetensors + MLX (#5624) 2026-07-06 15:40:46 -07:00
verify_prequant_backend.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-07-02 03:30:16 +00:00
video_quality.py Fail video quality gate on candidate frame-count mismatch 2026-07-05 07:46:42 +00:00
video_speedmem_bench.py Run video DiT dense+compile when it fits instead of a slower int8 fallback 2026-07-09 02:31:22 +00:00