unsloth/studio/backend
Daniel Han ff853c3977 Restore fp8 DiT quant for Wan video via a per-family embedder exclude
The Wan fp8 black frame was root-caused (scripts/fp8_layer_ablation.py,
measured on B200 with the production torch._scaled_mm path): per-row fp8
scales each activation row by row_amax/448, and the text prompt is padded to
512 tokens (~all padding for a short prompt), so condition_embedder's text
embedder divides a zero padding row by a zero scale, which infs and renders
every frame black. That embedder's bias makes every downstream row non-zero,
so the whole 30-block attn1/attn2/ffn stack is fp8-clean (fp8-except-
condition_embedder measured cosine 0.9998 vs bf16, 0 non-finite; fp8-
everywhere is 100% non-finite).

So the blanket fp8 deny was heavier than needed for Wan. Remove fp8 from the
Wan deny and keep only condition_embedder in bf16 via a new
_FP8_FAMILY_EXCLUDE_NAME_TOKENS; auto now restores fp8 (the Blackwell ladder
head) for Wan2.2-TI2V-5B and -T2V-A14B (shared DiT class and padded-text
conditioning). Full-generation check (512x320, 25 frames, 30 steps, cache on
and off): mixed-fp8 is non-black (mean luma 182.6 vs dense 181.2), more
accurate than int8 (LPIPS 0.129 vs 0.180 no-cache, 0.224 vs 0.251 with
FBCache), faster (49.9 vs 64.6 ms/step; int8 was a per-step regression vs the
59.8 ms/step dense), at the same memory (19.34 GB, both -20% vs dense).

HunyuanVideo-1.5 keeps the fp8 deny: its MMDiT masks the padding text tokens
to zero inside every block, so the per-block context stream (add_*_proj /
to_add_out / ff_context) regenerates zero rows layer after layer (fp8 on only
the main blocks is 100% non-finite) so no small exclude set exists and int8
stays. mxfp8 / nvfp4 remain denied for Wan (same per-row scaled_mm family, not
separately validated).

exclude_tokens_for_scheme now takes an optional family, threaded through the
runtime quantiser and the offline prequant builder + validator so offline ==
runtime (a stale Wan fp8 checkpoint baked without the exclude is rejected and
re-quantised rather than loaded). Adds scripts/fp8_layer_ablation.py (the
per-layer ablation probe) and a mean-luma black-frame metric plus mixed-fp8
vs int8 configs to the video bench.
2026-07-08 23:22:34 +00:00
..
assets Add DeepSeek-V4-Flash-GGUF to Studio with none/high/max reasoning (#6908) 2026-07-07 06:13:43 -07:00
auth Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm (#6392) 2026-07-01 06:42:23 -07:00
core Restore fp8 DiT quant for Wan video via a per-family embedder exclude 2026-07-08 23:22:34 +00:00
hub Fix Studio custom folders on Linux external drives (#6799) 2026-07-03 19:10:04 +01:00
loggers Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
models Auto-quantize the VAE (image/video decoder) to fp8 (default on, gated) 2026-07-08 10:04:52 +00:00
plugins Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
requirements Merge remote-tracking branch 'origin/main' into fold-integration 2026-07-07 05:52:16 +00:00
routes Auto-quantize the VAE (image/video decoder) to fp8 (default on, gated) 2026-07-08 10:04:52 +00:00
state Studio: shareable per-checkpoint preview links (#6486) 2026-06-24 06:31:53 -07:00
storage Sort chat recents by last activity (#6844) 2026-07-07 17:54:32 +01:00
tests Restore fp8 DiT quant for Wan video via a per-family embedder exclude 2026-07-08 23:22:34 +00:00
utils fix: match qwen3-thinking double-newline in train_on_responses_only response pattern (#6926) 2026-07-07 21:28:18 +03:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_platform_compat.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
cloudflare_tunnel.py Reap Studio child processes when the parent dies abnormally (#6425) 2026-06-18 05:51:22 -07:00
colab.py Studio Colab: opt-in shareable Cloudflare tunnel link (#6684) 2026-06-26 00:56:23 -07:00
main.py Scope the diffusion dataset body-cap passthrough to the exact upload route 2026-07-07 09:51:44 +00:00
run.py Studio: fix torch CUDA undefined-symbol errors from a conflicting LD_LIBRARY_PATH (#6905) 2026-07-06 23:48:23 -07:00
startup_banner.py Fix Windows Studio UTF-8 startup handling (#6614) 2026-07-01 13:47:33 +01:00