Five more items from the review round.
The conditioning-cache revision marker read huggingface_hub's import-time HF_HUB_CACHE
constant. Studio can move its cache during a session and loading follows the live setting, so
after a move the marker went unresolved (or pointed into the previous root) and pulling a new
revision of the same checkpoint no longer invalidated the cache: a warm run could reuse the
old encoder's embeddings and the old VAE's latents. It now looks in the active Studio cache
first and keeps the environment and the library constant as fallbacks, which the trainer
subprocess still needs.
The dataset interlock counts mutations rather than excluding them, so two imports of different
examples into the same empty name both got past the emptiness check. The winner promoted its
staging directory atomically; the loser found the folder non-empty, fell back to a per-file
move, and merged its images and captions into the winner's dataset. Imports now take a
per-folder lock, a second one is refused with 409, and the emptiness check is repeated under
the lock.
On Windows the sd.cpp asset resolver filtered only by accelerator token, so a Windows arm64
host matched an x64 zip, downloaded and installed it, and failed later when the binary would
not run. It now filters by architecture the way the Darwin and Linux branches do.
Every gallery page fetched every PNG up front and kept the object URL for the session, so
scrolling a large gallery grew memory without bound for tiles the user may never look at. The
Images strip now fetches a tile as it nears view, like the Video strip, and keeps the eager
path only where IntersectionObserver is unavailable.
A 503 carrying a JSON body comes from the application, not a proxy, so it is surfaced as the
error it is instead of entering lost-response settlement and being reported as a request that
never reached the server.
- source_revision() scanned the checkpoint root plus text_encoder/tokenizer but
not vae, so swapping or fine-tuning the VAE in place left the conditioning
cache namespace unchanged and a warm run trained against latents from the old
checkpoint. Include the vae directory, like any other component the cached
tensors come from.
- /images/generate answers only when the images are saved, and secure mode's
tunnel caps an origin response near 100 seconds, which a native CPU or a
high-step run passes routinely. The page reported failure while the work kept
running, and a retry would duplicate it. A lost response (fetch rejection or
a gateway status the origin never answered) is now told apart from a refusal:
the page waits out generate-progress and reloads the gallery, so the run it
started still lands.
- The trainer emits the EMA adapter's path with the terminal event, but the
state update dropped it, so neither the run history nor either response
schema carried it and an enabled EMA left nothing discoverable. Keep it, and
show it next to the primary adapter.
- weighting_scheme advertised a choice of timestep sampling; sampling is always
logit-normal and the flag only selects the bell loss weights. Describe what
it does.
- The trainer keyed its persistent conditioning cache on family and
resolution only, while the keys themselves carry just the caption or
image content and crop variant. One cache directory reused for two
checkpoints, or for the same repo at a new revision, let a warm run
skip loading its encoders and train on the other model's embeddings
and latent statistics. Namespace on the base checkpoint and its
resolved revision as well. The revision helper now lives beside the
cache in diffusion_train_extras and the inference wrapper delegates to
it, so the two cannot disagree about what counts as the same source.
- The diffusion learning rate only checked positivity, but 1e309 floats
to inf and satisfies gt, so the route evicted the resident models and
started AdamW with an infinite rate: the first step destroys the
adapter while progress looks normal and the result is saved. Bound it
below 1.0, matching the LLM schema, which rejects inf for the same
reason.
diffusion_train_extras hosts the opt-in training extras: LoRAEMA shadows only
the trainable adapter params (warmup-ramped decay, default 0.99, exported as a
second adapter under output_dir/ema), PersistentConditioningCache stores latent
posterior stats and caption embeddings as safetensors keyed by content hash +
family + resolution, and the aspect-ratio bucketing helpers group mixed-aspect
datasets into same-area divisor-snapped shapes.
The DiT trainer wires the first two behind config flags that default to the
current behavior: ema_decay (0 disables) and cond_cache_dir (None disables).
A fully warm cache skips loading the VAE and text encoders entirely; a cache
hit is bit-identical to a fresh encode, including the per-channel qwen latent
normalization. Also fixes the stale _gather_sigmas call in the perf test that
still passed the scheduler instead of the sigma table.
