Ideogram 4 assembles two DiTs per-component (a conditional transformer plus a
separate unconditional_transformer), so there is no transformer-only single-file
or GGUF artifact that could supply both. Add a pipeline_only family flag and
reject the gguf/single_file kinds in validate_load_request, before a load evicts
the current model, instead of assembling a pipeline missing its second DiT.
Extend the fp8 bf16-resident size override to a LOCAL directory mirror of the
ideogram-4-fp8 base: such a path never string-matches base_repo, so detect the
fp8 layout from the transformer shard headers (a *.weight_scale marker) and
reserve the bf16 footprint, matching the remote-base behaviour. A local nf4
mirror has no fp8 scales and correctly stays planned against its compressed bytes.
After the upstream cascade generalised _apply_group_offload to stream every
DiT it finds on the pipe (transformer AND transformer_2), apply_memory_plan's
group tier already block-streams both A14B experts, and model/sequential
offload hook every top-level module. The explicit per-expert pass this branch
added is therefore redundant: on a real group-tier load it re-registers the
group-offload hooks transformer_2 already carries, which diffusers rejects with
a duplicate-hook ValueError. That was swallowed by the helper's try/except and
surfaced as a misleading "second expert stays resident" warning even though the
expert was in fact streamed. Remove the pass and document why MoE needs none.
A successful on-demand wheel install writes into site-packages after the
import system already cached that directory's listing, so the very next
find_spec / import in the same process can miss the freshly installed
package when the install lands within the directory mtime's resolution.
That silently falls set_attention_backend back to native on first use.
Call importlib.invalidate_caches() after a successful install so the
kernel is picked up immediately, and cover both the success and failure
paths in the attention installer tests.
Review follow-ups on the image-generation PR:
- ControlNet: resolve_controlnet accepts a bare owner/name repo without the
non-GGUF base trust gate, and _controlnet_pipe hands it straight to
from_pretrained. A malicious pickle .bin would deserialize on load, so run
the same Hugging Face malware preflight (evaluate_file_security) the chat and
export loaders use before any remote ControlNet load; local dirs are exempt.
- Dataset thumbnails: key the cache on the full filename instead of the stem so
sample.png and sample.jpg no longer collide on one .thumbs file (which could
serve or delete the wrong image); the delete cleanup globs the same key.
- Diffusion training start: mirror start_training's API-key guard so an API
client cannot start training (which frees VRAM by unloading chat) while an
inference request is streaming; it now returns 409 before any GPU is freed.
- Model picker: include the curated safetensors row keys in the recommended
roving key list so arrow-key navigation reaches those rows instead of hitting
the duplicate option-missing id.
Tests: ControlNet malware gate (remote blocked before from_pretrained, local
skipped), thumbnail same-stem cache separation, API-key diffusion-start 409
before GPU free. Full diffusion suites green.
