ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Correct the ltx-2 resident TE estimate to the bf16 cast size

Browse source 
The memory plan's bf16_components_gb held 50.4 GB for the LTX text
encoder, which is the fp32 hub store of Gemma3-12B (~49 GB download),
not what sits on device: the pipeline loads it torch_dtype=bf16, ~24.4
GB resident. The 26 GB over-estimate pushed the auto plan toward offload
on cards that fit the real footprint. Comments and the size-table test
now pin the resident semantics.
This commit is contained in:
Daniel Han 2026-07-18 08:00:34 +00:00
commit 855275a139
2 changed files with 13 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

11
studio/backend/core/inference/video_families.py
View file

@ -99,8 +99,8 @@ class VideoFamily:
_FAMILIES: tuple[VideoFamily, ...] = (
# LTX-2 (diffusers >= 0.39): ~19B single-stream video DiT generating synchronized audio +
# video in one pass. The Gemma3-27B text encoder is the memory heavyweight (~50 GB bf16,
# more than the DiT). Base repo carries the dev config (40 steps, CFG 4); distilled runs few-step.
# video in one pass. The Gemma3-12B text encoder is stored fp32 on the hub (~49 GB
# download; ~24 GB resident once cast to bf16). Base repo carries the dev config (40 steps, CFG 4); distilled runs few-step.
VideoFamily(
name = "ltx-2",
pipeline_class = "LTX2Pipeline",
@ -116,8 +116,11 @@ _FAMILIES: tuple[VideoFamily, ...] = (
resolution_multiple = 32,
# 768x512 native default; 1216x704 the card's quality target; 704x1216 vertical.
resolution_presets = ((768, 512), (1216, 704), (704, 1216), (512, 768)),
# transformer 37.8 bf16; Gemma3-27B TE ~50.4; VAE 2.4 + connectors 2.9 + audio 0.2.
bf16_components_gb = (37.8, 50.4, 5.5),
# transformer 37.8 bf16; Gemma3-12B TE ~24.4 bf16 RESIDENT (the hub stores it fp32,
# ~49 GB download, but the pipeline loads torch_dtype=bf16); VAE 2.4 + connectors 2.9
# + audio 0.2. The old 50.4 figure double-counted the fp32 store and pushed the auto
# memory plan toward offload on cards that fit the real footprint.
bf16_components_gb = (37.8, 24.4, 5.5),
gguf_repo = "unsloth/LTX-2.3-GGUF",
# Pre-cast Gemma3-12B TE (hub store is fp32 ~49 GB, pre-cast ~13.2 GB): the biggest
# download win of the hosted TE set.

9
studio/backend/tests/test_video_families.py
View file

@ -309,9 +309,12 @@ def test_family_size_table_present():
fam = detect_video_family("unsloth/LTX-2.3-GGUF")
assert fam.bf16_components_gb is not None
transformer_gb, text_encoder_gb, companions_gb = fam.bf16_components_gb
# The Gemma3-27B text encoder outweighs the DiT itself; a table that lost
# that would let auto planning under-reserve by ~50 GB.
assert text_encoder_gb > transformer_gb > 20.0
# RESIDENT bf16 figures: the 37.8 GB DiT and the Gemma3-12B TE at ~24.4 GB once cast
# to bf16 (the fp32 hub store is ~49 GB but never sits on device). A table that
# regressed to the fp32 download size would push auto planning to offload on cards
# that fit the real footprint.
assert transformer_gb > text_encoder_gb > 20.0
assert text_encoder_gb < 30.0
assert companions_gb > 0.0