Comment-only pass over the Python this PR touches: drop what the code already
says, collapse multi-line explanations that still read on one line, and keep
the reasoning that is not recoverable from the code. No code, docstring
semantics or behaviour changes; verified with an AST comparison against the
previous revision, and the backend suite is unchanged (same 37 environment
failures as before: the API integration tests that need a live keyed server,
the flash-attn install hooks, and the GPU memory fields).
- start_training forwards resume_source_run_id to _start_training_impl, which
reads it. Without it every start raised NameError.
- Restore main's anchor in the load-marker order test: the file now has an
earlier `if config.is_gguf:`, so indexing the first one compared the wrong
branch.
- The two diffusion tests that reach diffusers now skip when it is absent,
matching the CPU repo-test env.
- The UI smoke finds nav rows that live in the sidebar's More flyout.
Round 2 of the hosted TE set, each bit-identical to dense-load-then-cast
and gated through the real backend (marker + status fp8 + same-seed LPIPS
vs dense TEs):
- FLUX.1 T5-XXL (text_encoder_2): 9.52 -> 5.90 GB, one artifact for
schnell/dev/Krea-dev (T5 shards byte-identical across all three,
verified sha256). 220 tensors, 144 fp8, LPIPS 0.109.
- Lumina Gemma2-2B: fp32 hub store 10.46 -> 3.20 GB (3.3x download cut).
288 tensors, 182 fp8, LPIPS 0.041.
- Z-Image Qwen3-4B: 8.04 -> 4.41 GB. 399 tensors, 252 fp8, LPIPS 0.112.
NOT shared with flux.2-klein-4B: klein retrained layer 35's MLP
(verified tensor diff, maxdiff 0.86), so klein hosts no entry.
- Krea-2 Qwen3-VL-4B: 8.88 -> 4.83 GB. 713 tensors, 460 fp8, LPIPS 0.082.
The constructor-assembled krea pipeline takes the encoder directly
(load_krea2_pipeline text_encoder kwarg); the loader remaps 5.x
rope_parameters and re-ties weights after assign so the rebuilt encoder
matches the builder's structure.
HunyuanImage 2.1 reuses the Qwen-Image artifact outright: its Qwen2.5-VL
text encoder is byte-identical (every shard sha256, 16,584,414,544 bytes),
recorded in the new component-level base-equivalence table the checkpoint
validator consults. The injection loop now covers text_encoder.._3 so a
family can host several components. Live check: LPIPS 0.123 vs dense.
The generic quantize_text_encoders pass only covers text_encoder.._3, so
HiDream's HEAVIEST encoder (Llama-3.1-8B TE4, 16.1 GB bf16) always stayed
dense. TE4 is assembled separately (hidream_te4_kwargs), so the fp8 path
now lives there: when the requested TE quant is layerwise fp8 and the
device/family qualify, TE4 prefers the hosted pre-cast checkpoint
(unsloth/HiDream-I1-Full-FP8, 8.6 GB) and falls back to dense-load-then-
cast; a mid-pass cast failure reloads a fresh dense encoder instead of
shipping partial state. The pre-cast loader and builder gain
config_subfolder/config_overrides for standalone encoder repos whose
config sits at the root and whose pipeline needs forward flags
(output_hidden_states/attentions).
Verified on B200: bit-identity 291 tensors (225 fp8, 0 mismatches),
hosted checkpoint engages through the real backend (marker + status fp8),
load 24.3 s vs 48.0 s dense, LPIPS 0.133 mean over 3 same-seed pairs vs
the dense-TE render (gate 0.25), non-black frames.
Two more findings from the hosted-TE GPU smokes:
- Module.dtype reports the first floating parameter, which after the
layerwise fp8 cast is the fp8 STORAGE dtype. Flux2 derives its prompt
embed and latent dtypes from encoder.dtype and feeds them to
randn_tensor, which has no fp8 kernel, so ANY flux.2 load with
text_encoder_quant=fp8 crashed at generation (pre-existing, runtime
cast included). The cast now swaps in a subclass whose dtype property
reports the compute dtype; forward behaviour is unchanged.
- The dense transformer_quant fast path assembles companions through
_assemble_pipe, which never received the pre-cast TE injection, so the
hosted encoder engaged on full-pipeline and GGUF builds but not on the
fast path. Threaded through like the other two branches.
Verified live on B200: qwen-image (full pipeline), flux.2-dev (GGUF picker
with int8 DiT prequant), ltx-2 (video backend) all engage the hosted TE,
render non-black, and report text_encoder_quant=fp8 truthfully.
qwen-image and flux.2-dev (diffusion) and ltx-2 (video) now resolve a
hosted pre-cast fp8 text encoder from their unsloth -FP8 repos:
- unsloth/Qwen-Image-FP8: Qwen2.5-VL-7B, 16.6 GB dense -> 8.8 GB
- unsloth/FLUX.2-dev-FP8: Mistral-Small-24B, 48.0 GB dense -> 24.7 GB
- unsloth/LTX-2-FP8: Gemma3-12B, 48.7 GB fp32 store -> 13.2 GB
Every checkpoint verified bit-identical to dense-load-then-cast
(729 / 585 / 1066 tensors, zero mismatches) and smoke-tested through the
real backends with the repo engagement marker. Tests cover the wired
entries, the resolver filenames, builder metadata weights_only survival,
and the idempotent re-cast.
Hermetic CPU coverage for diffusion_te_prequant: the checkpoint
filename convention, family-table resolution by scheme and component
with malformed entries skipped, resolution priority (path override,
hosted repo, none) and the fp8-only scheme gate, the checkpoint
validation matrix (wrong format, missing state_dict, wrong scheme,
wrong component, wrong or missing base_model_id) with base case
folding, the local-path allowlist refusal and missing-file fallback,
and the assembly injection gating (mode, hosted entry, device support,
family deny, load failure, successful injection). Also pins the
te_prequant_repos field on both family dataclasses and that no family
ships a hosted TE checkpoint until the campaign wires one.
