unsloth/studio/backend
Daniel Han b58098b219 LTX-2.3 checkpoint support: full pipeline assembly
diffusers 0.39 ships every LTX-2.3 model class but its single-file loader
maps all LTX-2 checkpoints to the 2.0 config, so 2.3 checkpoints (9-row
modulation tables, gated attention, per-modality connectors) fail a shape
check at load. The community transformer-only GGUFs also lack the text
projections, VAEs, and vocoder that 2.3 moved out of the transformer.

New core/inference/video_ltx2.py detects a 2.3 checkpoint from its header
(6 vs 9 modulation rows, no weight data read) and assembles the full
pipeline: the DiT through from_single_file with the 2.3 config overrides
and the prompt_adaln key renames the stock converter lacks, the 8-layer
per-modality connectors from the same checkpoint plus the text projection
companion file, and the 2.3 video VAE, audio VAE, and BWE vocoder from
the companion files in unsloth/LTX-2.3-GGUF. Configs and rename tables
mirror diffusers' own scripts/convert_ltx2_to_diffusers.py, which the
library loader has not absorbed yet. Assembled through the constructor
because the base repo pins LTX2Vocoder while 2.3 needs LTX2VocoderWithBWE
and the from_pretrained type gate rejects the substitution.

Verified on a B200: distilled-1.1 Q4_K_M GGUF loads in 37s, generates a
49-frame 768x512 clip with synchronized audio in 18s (8 steps), frames
on-prompt and non-black, container decodes fully. Meta-tensor validation
confirms exact key and shape match for all five converted components.
Unit tests cover 2.3 detection (gguf + safetensors headers), combined
checkpoint partitioning, and companion-set choice.
2026-07-04 13:50:44 +00:00
..
assets Studio: require signed capability tokens for /p preview links (#6666) 2026-06-25 21:40:48 -07:00
auth Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm (#6392) 2026-07-01 06:42:23 -07:00
core LTX-2.3 checkpoint support: full pipeline assembly 2026-07-04 13:50:44 +00:00
hub Fix Studio custom folders on Linux external drives (#6799) 2026-07-03 19:10:04 +01:00
loggers Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
models Video HTTP surface: /api/inference/video routes + request/response models 2026-07-04 13:17:01 +00:00
plugins Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
requirements Video inference engine: LTX-2 family registry, VideoBackend, MP4 gallery 2026-07-04 13:08:43 +00:00
routes Video HTTP surface: /api/inference/video routes + request/response models 2026-07-04 13:17:01 +00:00
state Studio: shareable per-checkpoint preview links (#6486) 2026-06-24 06:31:53 -07:00
storage Fix Studio custom folders on Linux external drives (#6799) 2026-07-03 19:10:04 +01:00
tests LTX-2.3 checkpoint support: full pipeline assembly 2026-07-04 13:50:44 +00:00
utils Fix Studio custom folders on Linux external drives (#6799) 2026-07-03 19:10:04 +01:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_platform_compat.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
cloudflare_tunnel.py Reap Studio child processes when the parent dies abnormally (#6425) 2026-06-18 05:51:22 -07:00
colab.py Studio Colab: opt-in shareable Cloudflare tunnel link (#6684) 2026-06-26 00:56:23 -07:00
main.py Video HTTP surface: /api/inference/video routes + request/response models 2026-07-04 13:17:01 +00:00
run.py studio: explicit Cloudflare tunnel notice and public-exposure warning at startup (#6515) 2026-06-30 17:47:48 +02:00
startup_banner.py Fix Windows Studio UTF-8 startup handling (#6614) 2026-07-01 13:47:33 +01:00