Comment-only pass over the Python this PR touches: drop what the code already
says, collapse multi-line explanations that still read on one line, and keep
the reasoning that is not recoverable from the code. No code, docstring
semantics or behaviour changes; verified with an AST comparison against the
previous revision, and the backend suite is unchanged (same 37 environment
failures as before: the API integration tests that need a live keyed server,
the flash-attn install hooks, and the GPU memory fields).
Four fixes from the latest review round:
- The GPU arbiter's chat evictor only cancelled the llama.cpp side. The
orchestrator publishes active_model_name once its worker reports success, so
an in-flight safetensors load was visible only as an entry in loading_models
and finished onto the GPU after ownership had transferred. Cancel every
pending load, and give the safetensors branch the post-load ownership recheck
the GGUF branch already had.
- A manual gpu_layers=0 GGUF load runs on the CPU with the GPUs hidden from the
child, yet it took the arbiter unconditionally: it cancelled a running image
or video generation for a model needing no VRAM, then held CHAT ownership so
the next GPU workload unloaded it for nothing. Gate the acquire on the same
predicate the launch-time CPU-only mask uses, as the image and video loaders
gate on their resolved device.
- The staged-download hook subscribes per repo, not per job, so another job on
the same repo advanced the staged queue (starting a load whose scoped files
were still downloading) or wiped a queue that was still running. Compare the
variant each callback carries, like the chat page's auto-load does.
- The video gallery fetched every record of a page into an object URL that
lives until the page closes: 50 clips at tens to hundreds of MB each, for
cards the user may never scroll to. Fetch a clip as its card nears the strip's
edge, plus the selected one the player needs.
Six review findings, three of them evict-then-fail orderings:
- The chat load reclaimed the GPU without telling the arbiter it existed. A
chat load holds no llama-server process until its GGUF has downloaded,
which is minutes, so a competing Images/Video acquire in that window
found nothing to cancel, took the GPU, and the chat load then spawned
onto the same device. It now registers an in-flight marker through
acquire_for's register hook (under the arbiter lock, as the image and
video loads do), the evictor cancels a marked load, and the route undoes
itself if ownership moved while it loaded.
- The Hub-download conflict check ran after that handoff, so a GGUF the
download manager already owns destroyed the resident Images/Video
pipeline and then 409'd, having loaded nothing. It moves above the
handoff, together with the marker it handshakes with.
- The image load released the engine router's transition lock before
registering the load, so a second load choosing the other engine could
unload the still-idle engine this one captured; the load then landed on a
deactivated engine, where generate, status, unload and the arbiter's
evictor can no longer reach it. Registration now happens under that lock
and refuses if the engine changed.
- Training a DiT family on a host with no GPU was accepted: nf4 is not a
CPU fallback, its 4-bit load goes through bitsandbytes, which requires
CUDA, XPU or MPS. The start unloaded the working Images pipeline, pulled
the text encoders, and only then died in the child. Rejected before the
teardown now, and /info stops advertising a precision that always 400s.
SDXL keeps its documented fp32-on-CPU path.
- Both diffusion pages kept the routed-pick marker forever, so re-picking
the same checkpoint (after chat evicted it) neither loaded nor cleared
the query string. The marker is released once the query is gone. The
Images key also carried a stray NUL byte, which made the file read as
binary to grep and other tooling.
- diffusers dropped Python 3.9 in 0.38, so the unconditional >=0.39.0 pin
left pip no candidate at all on 3.9 and made every install that composes
the huggingface extras unresolvable there. The floor is conditional now.
Also fixes tests that were already red on the branch: two hand-built
request fakes had gone stale against fields this branch added, and the
handoff-ordering test only failed on a host with fewer than two GPUs.
Gallery clear/delete now scope to Studio-owned files: image_gallery and
video_gallery skip PNGs / MP4s without a readable recipe (a hand-dropped or
orphan file the listing already hides), so clear() and a guessed-id delete no
longer destroy files the gallery never surfaced.
Remote ControlNets now force use_safetensors: a bare owner/name reaches
from_pretrained without the base trust gate, and the Hub scan fails open when
unavailable, so requiring safetensors closes the pickle deserialization vector.
POSIX uninstall now stops resident sd-server / sd-cli under an owned sd.cpp root
before removing the tree (marker-gated), mirroring the Windows stop-before-delete
scan; a live native server no longer survives unlinking its binary.
Diffusion dataset containment: the training-start read path and the discovery
picker route bare names through the protected resolver, so a symlinked dataset
is rejected / not advertised like the caption/delete routes already do. Uploads
gain the inference decode guard (oversized real images 400 before OOMing the
trainer) and dataset upload/caption/delete/import are blocked with 409 while a
diffusion run is active.
JSONL readers (trainer + routes) tolerate non-object JSON and invalid UTF-8
instead of raising AttributeError / 500.
LoRA family compatibility is enforced in the shared resolver, not only the
picker, so a direct API client cannot apply a mismatched-family adapter.
GPU arbiter gains release_if so the image/video unload idle-check and release
are atomic against a concurrent same-owner load's registration. Native batch
recipes persist the base batch_seed and restore replays from it, so a native
batch_index>0 image no longer advances its seed twice.
FLUX.2-klein selects its sd.cpp text encoder by variant (4B -> Qwen3-4B,
9B -> Qwen3-8B) instead of the single family default.
Publish native sd.cpp generate progress (_gen) before LoRA resolution so a reload probe reads active during setup, matching the diffusers path.
Register the diffusion/video GPU load under the arbiter lock (acquire_for now takes a register callback) so a competing acquire cannot evict an owner before its load is marked in-flight and let two loaders allocate VRAM at once.
Admit local diffusers pipeline folders (root model_index.json, weights in component subdirs) in the local model scan so they reach task tagging and the On Device picker.
Text-to-video lands as a SIBLING of the image diffusion backend, not a mode of
it: video pipelines take frame/fps arguments, return frame stacks plus, for
LTX-2, synchronized audio, and persist MP4s -- none of the image module's
img2img/inpaint/ControlNet/LoRA surface applies. The image backend's hardware
and optimisation layers are imported unchanged (device/dtype resolution, memory
planning + offload tiers, attention backends, speed profiles, FBCache), and the
load-token/cancel-event concurrency skeleton is copied verbatim so lifecycle
behaviour cannot diverge.
core/inference/video_families.py: a pure VideoFamily registry (no torch) with
the ltx-2 entry -- LTX2Pipeline + LTX2VideoTransformer3DModel, base
Lightricks/LTX-2, unsloth/LTX-2.3-GGUF as the curated GGUF source, audio on,
frame lattice k*8+1, /32 resolutions with a vertical preset, and measured bf16
component sizes (the Gemma3-27B text encoder outweighs the 19B DiT itself).
MoE fields (transformer_2, guidance_scale_2) are declared now so the Wan2.2
A14B family lands later without churning the schema.
core/inference/video.py: VideoBackend with async begin_load + cache-scan
download progress, GGUF / single-file / full-pipeline loads (the GGUF DiT
assembles onto the base repo exactly like the image path), generation with
frame/size snapping BEFORE latents allocate, per-step progress + ETA and
cooperative cancel via the standard diffusers callback, and MP4 (H.264) export
through diffusers' PyAV encoder with the audio track muxed when the family
produces one. VAE tiling is always on: decoding a 100+ frame clip is the
memory peak, and the frames-aware estimate_video_runtime_mib (new, in
diffusion_memory) feeds the planner where the pixel-area image estimate would
badly undershoot. Loads are gated to unsloth/*, the official Lightricks base
repos, or local paths; PyAV availability is checked at load time so a missing
encoder cannot fail a clip after a multi-minute denoise.
core/inference/video_gallery.py: {id}.mp4 + {id}.json recipe sidecar pairs
under studio_root()/videos (an MP4 has no PNG text chunk to embed the recipe
in), with the image gallery's id/containment guards, newest-first listing that
skips orphans, delete/clear.
gpu_arbiter gains the VIDEO owner: ownership is exclusive, so the existing
evict-the-current-owner already generalises to chat/image/video all evicting
each other. The av (PyAV) dependency joins requirements/studio.txt.
Tests: video family detection/snapping/defaults, backend lifecycle on a faked
torch/diffusers runtime (GGUF assembly, shape snapping, distilled defaults,
cancel/progress, sentinel), gallery roundtrip/containment/orphans. 52 new
tests green plus the arbiter suite.
When no CUDA/ROCm/XPU GPU is available, route diffusion load/generate to the
native stable-diffusion.cpp engine instead of diffusers, with diffusers as the
guaranteed fallback. On CPU sd.cpp is 1.4-2.8x faster and uses 1.5-2.2x less RAM.
- diffusion_engine_router: centralised engine selection (built on the existing
select_diffusion_engine), env opt-outs, MPS gating, recorded fallback reason.
- sd_cpp_backend (SdCppDiffusionBackend): the diffusers backend method surface
backed by sd-cli, with lazy binary install, registry-driven asset fetch,
step-progress parsing, and cancellation.
- diffusion_families: per-family single-file VAE + text-encoder asset mapping.
- sd_cpp_engine: cancellation support (process-group kill + SdCppCancelled).
- routes/inference + gpu_arbiter: drive the active engine via the router; the
API now reports the active engine and any fallback reason.
- tests for the backend, router, route selection, and cancellation.
