The start-route preflight caught the bf16-GPU and int8-torchao requirements but not the dense
precisions' CUDA requirement: on a GPU-less host bf16_unsupported_reason exempts CPU-only, so a
bf16/fp8 (or int8-with-torchao) DiT request passed the preflight, evicted resident workloads, then
raised only in the trainer child. Add the dense-mode CUDA gate mirroring _resolve_base_precision so
the doomed run is rejected up front. Also pin bf16_unsupported_reason in the two positive-path
family-info tests so they are deterministic across GPU types (a non-bf16 CUDA box would otherwise
empty every DiT family's advertised modes).
The auto FBCache policy keyed on the full step count whenever strength was omitted, but the
loader only passes the strength kwarg when it is set, so an img2img/inpaint pipe then runs its
OWN signature default (< 1, e.g. FluxImg2ImgPipeline's 0.6). FBCache would engage on the full
28 steps while the pipe actually denoises ~16, degrading the image on exactly the short
trajectory the policy exists to keep uncached. Thread the pipe's signature default into the
policy via a new effective_request_strength helper (unit-tested), so the effective denoise count
matches what the pipe runs.
_dense_quant_prefetch_needed widened the prefetch to pull the base repo's transformer/
shards whenever a dense-quant candidate resolved, but balanced/low_vram (and the legacy
cpu_offload flag) force load_pipeline onto offload unconditionally in plan_diffusion_memory,
so its re-plan never flips to OFFLOAD_NONE and the dense build never runs. The offloaded GGUF
path then never touches those shards, so the widened prefetch only wastes a multi-GB download,
and a disk-full on that begin_load pull has no GGUF fallback (unlike the in-load_pipeline dense
failure). Mirror plan_diffusion_memory's definite-offload gates so the prefetch stays scoped.
Krea-2-Raw is in _TRUSTED_NON_GGUF_REPOS, so it is inference-loadable, but the generic
"krea" generation-defaults key matched it too and applied Turbo's distilled 8-step / no-CFG
recipe, producing degraded output on the undistilled base. Add a more specific krea-2-raw key
(52 steps, guidance 3.5 per the model card) ahead of the generic one, in both the backend
table and the frontend MODEL_DEFAULTS so the Studio UI and the OpenAI images route agree.
The start route preflight only rejected non-bf16 GPUs; an explicit int8 request on
a host with a missing or stub torchao passed the preflight, evicted resident GPU
workloads, then died in the trainer child (its int8 base quantizer has no fallback).
Fold both gates into training_precision_preflight_error so int8-without-torchao fails
fast before eviction. Also empty the advertised DiT precision_modes (and surface the
reason in vram_note, drop compile) whenever the bf16 preflight would reject the family,
so /info never offers an nf4 DiT option the route always 400s.
- _dense_quant_prefetch_needed widened the transformer/ prefetch to pull the base
repo's full dense bf16 shards even when a prequant checkpoint is configured
(candidate.prequant), contradicting its own docstring. That both defeats the
prequant download savings and can hard-fail begin_load on a disk-full (no GGUF
fallback there). Only widen for a real dense build (candidate is not None and
not candidate.prequant).
- DiffusionStatusResponse declared no 'resolved' field, so Pydantic's default
extra='ignore' silently dropped the per-control auto-policy provenance the
backend records (build_resolved_record / state.resolved) -- the plumbing never
reached any client. Declare the field so it round-trips.
- DiffusionFamily gains deploy_base_repo (krea/Krea-2-Turbo): deploying a LoRA
trained on Raw now previews it on Turbo, not the non-distilled Raw checkpoint.
Scoped to a same-precision override so it never turns an nf4 train base into a
larger bf16 deploy load; exposed through family_train_infos -> the Train UI's
onDeployClick / historical-run deploy resolve the deploy base.
- _GENERATION_DEFAULTS gains a Krea entry (8 steps, 0 CFG) so the OpenAI
/v1/images/generations route matches the Create UI's documented distilled recipe
instead of falling through to the generic (9, 0.0).
- normalized() + family_train_infos() mirror the inference fp8 deny for
Qwen-Image (activation outliers exceed fp8's range and corrupt the trained
result); int8 stays allowed and the UI no longer advertises fp8 for it.
- _resolve_base_precision() gates an explicit int8 on a FUNCTIONAL torchao, the
same gate auto and /info already apply, so a missing/stub torchao fails fast
instead of silently loading dense with compile disabled.
- train_precision_modes() gates the dense modes (bf16/int8/fp8/auto) on
torch.cuda.is_bf16_supported(), so a non-bf16 CUDA GPU (T4/V100/RTX 20xx) is
offered only nf4 instead of a start that evicts resident models and then fails.
- start_diffusion_training preflights bf16 support for the DiT families BEFORE
_free_gpu_for_diffusion_training(), so any DiT start (nf4 included, since the
trainer requires bf16 unconditionally on CUDA) fails fast without eviction.
- _dense_quant_prefetch_needed widened the prefetch to pull the base repo's bf16 transformer/ shards
whenever a dense-quant scheme could resolve, with no disk check. On the offload path that can fill
the cache volume mid-download and hard-fail the load in a spot unload/cancel cannot preempt, instead
of the disk guard falling back to running the GGUF as-is (the Dtype hint's documented disk fallback).
Defer to resolve_dense_quant_candidate, the same disk-aware resolver load_pipeline re-plans against,
so the prefetch widens only when the dense build would really run.
- An explicit Speed=off (bit-exact) load with an unset dtype was promoted to auto-quant by the Dtype
default, silently engaging int8/fp8 + compile and breaking the bit-exact request (an auto DEFAULT
overriding an EXPLICIT control). Suppress the auto-dtype default when speed is explicitly off, in both
load_pipeline and the prefetch.
- The torchao 0.17 MX training path swaps a matched frozen Linear's weight for a wrapper tensor
whose linear override computes input @ weight_t and drops the bias, so mxfp8'ing a biased frozen
linear silently loses its bias and corrupts the base output the LoRA regresses against (verified
on Blackwell: the bias term is fully dropped). Skip biased linears in _mx_module_filter.
- _resolve_base_precision re-checked explicit dense modes against the live device but only rejected
CPU, so an explicit mxfp8 request on a non-Blackwell CUDA GPU passed and then crashed at the first
MX GEMM after a full dense-transformer load. /info only advertises mxfp8 on sm100+; mirror that
gate here and fail fast for a stale or direct client below Blackwell.
UNSLOTH_DISABLE_FP16_ACCUM is the documented safety escape hatch for fp16-accumulation
numerical drift, but it was matched as .strip() in (1, true, yes) with no lowercasing, so
UNSLOTH_DISABLE_FP16_ACCUM=TRUE (or YES / On) was silently ignored and fp16 accumulation
stayed on. Lowercase before matching (the family-name check on the next line already does)
and accept on. Existing 1/true/yes still match.
effective_denoise_steps computed ceil(steps * strength) (steps - int(steps - steps*strength)),
but diffusers get_timesteps denoises init_timestep = min(int(num_inference_steps * strength),
num_inference_steps), i.e. the floored product. The two differ by one whenever the product is
fractional, and that flips the auto FBCache decision in the (19, 20) band: a strength-0.7
28-step img2img denoises int(19.6) = 19 real steps (below FBCACHE_MIN_STEPS = 20) but the old
formula returned 20 and engaged FBCache on that short trajectory, exactly the quality hit the
auto policy exists to avoid. Return min(int(steps * strength), steps) to match diffusers, and
fix the two tests that replayed the old formula.
Also honor _default_threads' documented fallback: (os.cpu_count() or 8) // 2 yields 4 when the
count is unknown, contradicting the docstring's 'falls back to 8'. Return 8 in that case.
The dense-quant re-plan passes transformer_resident_override_mib (the bf16 build
peak) AND computes companions via _companion_cache_bytes(base), which sums every
flat blob in the HF cache. Because the dense path prefetches the base transformer/
shards into that same cache before load_pipeline runs, the transformer is counted
twice, inflating the footprint (~44 GB instead of ~20 GB in the reproduction) and
wrongly forcing offload for models that fit resident -- the case this path exists
to enable. Add companion_override_mib and pass the auto-policy's own text-encoder
plus VAE estimate on the re-plan so the cache (with its prefetched transformer) is
not read for this artifact.
- Step cache 'Off' is preserved: the frontend defaulted to 'off' and mapped it
to an omitted transformer_cache, which the backend now reads as 'auto', so
leaving the control at Off silently enabled FBCache on 20+ step families.
Default the control to Auto, add an explicit Auto option, and send
auto -> omitted so Off maps to an explicit cache-off.
- Kernel auto-install adds --no-deps: 'pip install --only-binary :all: xformers'
resolves xformers' pinned torch and replaces the running torch/triton. --no-deps
installs only the best-effort kernel wheel; an ABI mismatch just fails to import
and falls back to native, never clobbering core deps.
- Do not retry a failed kernel install under the load lock: the pre-install runs
outside the locks, then the in-lock resolve re-attempts pip (up to 600s) while
holding _generate_lock/_lock and blocking unload/cancel/new loads. Record the
attempt in a process-level set so the in-lock call short-circuits to native.
- Cache auto-toggle keys on effective denoise steps: an image-conditioned run with
strength < 1 (upscale default 0.35) denoises a fraction of the requested steps,
so a 28-step request runs ~10 steps. Compute the effective count the way diffusers
get_timesteps does and gate FBCache on it, only when strength is actually applied.
A successful on-demand wheel install writes into site-packages after the
import system already cached that directory's listing, so the very next
find_spec / import in the same process can miss the freshly installed
package when the install lands within the directory mtime's resolution.
That silently falls set_attention_backend back to native on first use.
Call importlib.invalidate_caches() after a successful install so the
kernel is picked up immediately, and cover both the success and failure
paths in the attention installer tests.
Review follow-ups on the image-generation PR:
- ControlNet: resolve_controlnet accepts a bare owner/name repo without the
non-GGUF base trust gate, and _controlnet_pipe hands it straight to
from_pretrained. A malicious pickle .bin would deserialize on load, so run
the same Hugging Face malware preflight (evaluate_file_security) the chat and
export loaders use before any remote ControlNet load; local dirs are exempt.
- Dataset thumbnails: key the cache on the full filename instead of the stem so
sample.png and sample.jpg no longer collide on one .thumbs file (which could
serve or delete the wrong image); the delete cleanup globs the same key.
- Diffusion training start: mirror start_training's API-key guard so an API
client cannot start training (which frees VRAM by unloading chat) while an
inference request is streaming; it now returns 409 before any GPU is freed.
- Model picker: include the curated safetensors row keys in the recommended
roving key list so arrow-key navigation reaches those rows instead of hitting
the duplicate option-missing id.
Tests: ControlNet malware gate (remote blocked before from_pretrained, local
skipped), thumbnail same-stem cache separation, API-key diffusion-start 409
before GPU free. Full diffusion suites green.
