unsloth/studio
Daniel Han e58f30be5f Run video DiT dense+compile when it fits instead of a slower int8 fallback
Root-caused "HunyuanVideo-1.5 int8 is slower than dense" with a per-forward profiler
(scripts/hunyuan_int8_profile.py, dynamo-reset, back-to-back on a clean B200): int8 compiles
cleanly (0 recompiles, 0 graph breaks, steady 268.3 ms/forward) and is only ~7% slower than dense
+ regional compile (250.5 ms/forward), not the 38% a contended-GPU bench run suggested. int8 is
also less accurate (LPIPS 0.085 vs dense+compile 0.037). So for a family where fp8 is denied
(Hunyuan black-frames on per-row fp8), int8 is a MEMORY lever, not a speed win, yet the auto-quant
default quantised it even when the dense DiT already fit resident.

Fix: is_int8_memory_fallback(target, family) is True only when AUTO quant lands on int8 as a
denied/black-frame fallback on a data-center, fp8-capable GPU (fp8 would be the arch pick but is
denied for the family). The video loader now skips the auto-quant and runs dense+compile when that
holds AND the bf16 memory plan already fits resident (offload_policy == none), so there is no new
OOM risk. Scoped tightly: only an AUTO request (explicit int8/fp8 honored), only int8-fallback
families (Wan / LTX resolve to fp8 -> keep quantising), only data-center fp8-capable parts (consumer
GPUs and pre-Ada, where int8 is a genuine accelerator, keep int8), and only when dense provably
fits; a memory-constrained plan still quantises. Result: Hunyuan on a resident-fit B200 now runs
faster AND more accurate, quantising only when memory is the constraint.

Also resets dynamo per config in the video bench (so compiled graphs cannot leak across configs in
one process) and adds the per-forward profiler used for the diagnosis.
2026-07-09 02:31:22 +00:00
..
backend Run video DiT dense+compile when it fits instead of a slower int8 fallback 2026-07-09 02:31:22 +00:00
frontend Key image-gen LoRA rows by index so typing a repo id keeps input focus 2026-07-08 07:19:13 +00:00
src-tauri Speed up Studio startup path (#6899) 2026-07-07 18:08:07 -07:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
install_llama_prebuilt.py Studio: stop handing CI/user secrets to downloaded llama.cpp binaries (#6696) 2026-06-27 05:21:05 -07:00
install_node_prebuilt.py Pin isolated Node.js installer to committed sha256 digests (#6625) 2026-06-24 05:47:58 -07:00
install_python_stack.py Studio: exclude mlx-lm 0.31.3 (broke gemma4/qwen3_5 QK-norm load on Apple Silicon) (#6803) 2026-07-06 19:40:06 -07:00
install_sd_cpp_prebuilt.py Studio: classify local video pipelines, harden video preflight, and gate custom-root sd.cpp removal on ownership 2026-07-07 03:15:07 +00:00
LICENSE.AGPL-3.0 Add AGPL-3.0 license to studio folder 2026-03-09 19:36:25 +00:00
node_prebuilt_pins.json Pin isolated Node.js installer to committed sha256 digests (#6625) 2026-06-24 05:47:58 -07:00
package-lock.json ci: advisory lockfile supply-chain audit (no install-script changes) (#5604) 2026-05-19 05:56:56 -07:00
package.json ci: advisory lockfile supply-chain audit (no install-script changes) (#5604) 2026-05-19 05:56:56 -07:00
setup.bat Final cleanup 2026-03-12 18:28:04 +00:00
setup.ps1 [Studio] Add --with-llama-cpp-dir installer flag to reuse a local llama.cpp (#6472) 2026-07-02 22:11:20 +01:00
setup.sh [Studio] Add --with-llama-cpp-dir installer flag to reuse a local llama.cpp (#6472) 2026-07-02 22:11:20 +01:00
Unsloth_Studio_Colab.ipynb Studio Colab: opt-in shareable Cloudflare tunnel link (#6684) 2026-06-26 00:56:23 -07:00