Commit graph

7,196 commits

Author SHA1 Message Date
Daniel Han
53912b2f99 Studio: tighten image-generation fix comments and docstrings 2026-07-13 13:32:28 +00:00
Daniel Han
be0bd00064 Studio: gate local-pipeline image tagging on a real family; validate video sidecars before delete/clear 2026-07-13 12:28:52 +00:00
pre-commit-ci[bot]
d0f00f2f5f [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-13 11:22:23 +00:00
Daniel Han
06b543b880 Studio: track the loaded GGUF filename so native companion resolution reproduces the load identity 2026-07-13 11:21:34 +00:00
pre-commit-ci[bot]
3fa9fdbc0d [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-13 10:04:07 +00:00
Daniel Han
5eef2f4003 Studio: preserve foreign gallery files, force safetensors on remote ControlNets, and close dataset/seed/GPU gaps
Gallery clear/delete now scope to Studio-owned files: image_gallery and
video_gallery skip PNGs / MP4s without a readable recipe (a hand-dropped or
orphan file the listing already hides), so clear() and a guessed-id delete no
longer destroy files the gallery never surfaced.

Remote ControlNets now force use_safetensors: a bare owner/name reaches
from_pretrained without the base trust gate, and the Hub scan fails open when
unavailable, so requiring safetensors closes the pickle deserialization vector.

POSIX uninstall now stops resident sd-server / sd-cli under an owned sd.cpp root
before removing the tree (marker-gated), mirroring the Windows stop-before-delete
scan; a live native server no longer survives unlinking its binary.

Diffusion dataset containment: the training-start read path and the discovery
picker route bare names through the protected resolver, so a symlinked dataset
is rejected / not advertised like the caption/delete routes already do. Uploads
gain the inference decode guard (oversized real images 400 before OOMing the
trainer) and dataset upload/caption/delete/import are blocked with 409 while a
diffusion run is active.

JSONL readers (trainer + routes) tolerate non-object JSON and invalid UTF-8
instead of raising AttributeError / 500.

LoRA family compatibility is enforced in the shared resolver, not only the
picker, so a direct API client cannot apply a mismatched-family adapter.

GPU arbiter gains release_if so the image/video unload idle-check and release
are atomic against a concurrent same-owner load's registration. Native batch
recipes persist the base batch_seed and restore replays from it, so a native
batch_index>0 image no longer advances its seed twice.

FLUX.2-klein selects its sd.cpp text encoder by variant (4B -> Qwen3-4B,
9B -> Qwen3-8B) instead of the single family default.
2026-07-13 10:02:42 +00:00
Daniel Han
b127256eb4 Studio: surface local pipeline scan roots, tag single-file checkpoints by filename, mark companion-only pipelines partial
- _scan_models_dir: admit a scan folder that is itself a diffusers pipeline
  (root model_index.json, weights in transformer/ vae/ subdirs). _is_model_directory
  rejects such a root, so the child scan would list the component subdirs as bogus
  models and hide the real pipeline; treat the root as one model via _local_pipeline_index.

- _local_is_diffusers / _local_model_task: include the sole checkpoint filename in the
  family-detection needles (_local_family_needles, resolved via resolve_local_single_file).
  A generically named folder holding one loadable qwen-image-*.safetensors / ltx-*.safetensors
  identifies its family only from the filename; the load route already resolves that file, so
  tag it or the task-scoped picker (which rejects task=null) hides the on-device model.

- list_cached_models: mark a companion-only base snapshot partial. A GGUF image load prefetches
  the base repo's VAE / text-encoder / model_index.json but skips the transformer (the GGUF
  supplies it); the snapshot has a pipeline manifest yet is not a loadable BF16 pipeline, and
  _cached_repo_partial misses it. _repo_pipeline_missing_denoiser flags a pipeline snapshot whose
  transformer/ or unet/ component carries no weight, so the picker drops it instead of advertising
  it as fully on-device.
2026-07-13 08:26:43 +00:00
Daniel Han
53668c9f66 Studio: include marker-owned custom sd.cpp roots in the uninstall stop scan; harden Pester install against the nuget.exe PSGallery bootstrap 2026-07-13 07:26:18 +00:00
pre-commit-ci[bot]
11330b8de5 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-13 06:31:51 +00:00
Daniel Han
899465ed80 Studio: close arbiter load-registration race and surface native progress + local pipeline folders
Publish native sd.cpp generate progress (_gen) before LoRA resolution so a reload probe reads active during setup, matching the diffusers path.

Register the diffusion/video GPU load under the arbiter lock (acquire_for now takes a register callback) so a competing acquire cannot evict an owner before its load is marked in-flight and let two loaders allocate VRAM at once.

Admit local diffusers pipeline folders (root model_index.json, weights in component subdirs) in the local model scan so they reach task tagging and the On Device picker.
2026-07-13 06:31:00 +00:00
Daniel Han
e0ef488f47 Tighten comments and docstrings added by the image-generation fixes 2026-07-13 05:29:09 +00:00
Daniel Han
dabb12e198 Studio: gate dataset uploads on the symlink check and surface local video single-file checkpoints 2026-07-13 04:21:36 +00:00
Daniel Han
da1770bb44 Refuse sd.cpp install into unowned non-empty target dir
When the install target already exists, is non-empty and lacks the
.unsloth-studio-owned marker (a user's own stable-diffusion.cpp checkout,
or unrelated files beside a custom Studio root), install() previously still
extracted the release into it. Skipping the ownership marker only stopped the
uninstaller from deleting the directory; extraction still merged binaries into
the user's working tree and could overwrite same-named files.

Fail up front with a clear message pointing the user at a fresh/empty location
before any download or extraction, leaving their directory untouched. Update
the ownership test suite to assert the refusal.
2026-07-13 03:17:49 +00:00
Daniel Han
5a17614b51 Reject native batch seeds outside the JSON-safe range 2026-07-13 02:09:15 +00:00
pre-commit-ci[bot]
499bada598 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-13 01:23:46 +00:00
Daniel Han
7ec9e77a5d Studio: fix diffusion install ownership, dataset upload atomicity, gallery pagination, and teardown races
install_sd_cpp_prebuilt: only write the .unsloth-studio-owned marker when the
install created the target directory or it was empty. Adopting a pre-existing,
unowned, non-empty directory (a user's own stable-diffusion.cpp checkout) made
it eligible for the uninstaller's recursive delete.

routes/training upload: make the multi-file promotion transactional. Back up
each displaced original and roll every destination back on any failure, so a
mid-loop rename error can no longer partially overwrite the live dataset.

routes/training _resolve_dataset_folder: reject a symlinked dataset directory
and prove the resolved folder stays under the datasets root, so image
read/caption/delete cannot escape the root through a link.

routes/training delete: escape glob metacharacters in the thumbnail filename so
deleting an image named like [ab].png removes only its own thumbnails.

image_gallery / video_gallery listing: filter records against the response
schema inside the pager via a valid callback, so offset/limit/has_more all count
over accepted records. A leading schema-invalid record no longer returns an
empty page with has_more=true and stalls infinite scroll at offset 0.

image_gallery / video_gallery save: publish via a temp file plus atomic rename
(the sidecar is the video pair's commit marker) and clean up on failure, so a
partial write never surfaces a truncated PNG or strands an orphan MP4.

diffusion_train_common discovery: treat an empty caption sidecar as a metadata
tombstone that still falls through to the dreambooth instance prompt, so
clearing every metadata caption no longer fails with no captioned images found.

diffusion backend unload: wait for an in-flight denoise to exit before tearing
down process-wide patches and state, mirroring the load path.

diffusion_engine_router: serialize the whole check/unload/publish transition so
a concurrent selection cannot return the engine being unloaded.

uninstall.ps1: gate the default sd.cpp process stop on the owner marker so a
user's own sd-server is not terminated for a directory we then keep.
2026-07-13 01:22:54 +00:00
Daniel Han
21052db120 Publish image generation active state before pre-denoise setup
generate() assigned self._gen only at the pipe() call, after deferred
compile, LoRA resolution/application, and ControlNet download/build had
run. Across that setup window generate_progress() reported inactive even
though _generate_lock was held, so a reloaded page's mount probe showed
idle and let a second generate queue behind the first.

Publish an active step-0 _GenState the moment the generation lock is
acquired, before the setup work, and clear it in the outer finally so a
setup-time error cannot leave the UI stuck active. Mirrors the video
backend's queued phase and the training start guard.
2026-07-12 13:42:24 +00:00
pre-commit-ci[bot]
7776148463 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-12 12:44:19 +00:00
Daniel Han
23a71b1c2c Close video single-file, training reservation, and image mount-resume gaps
Route on-device single-checkpoint video folders through the single_file loader:
a bare local .safetensors directory (no model_index.json) is advertised as a
pipeline with no filename, so validation rejected it before it could load.
Reinterpret the pick as a single_file load of the sole checkpoint, mirroring the
image load route.

Treat a reserved-but-not-yet-spawned LLM training start as active in
is_training_active() so /images/load, /video/load, and /diffusion/start cannot
race the reserved run for VRAM during the pre-spawn free window. Mirrors the
diffusion training service reservation.

Resume an in-flight image generation on the Images page mount: probe
generate-progress, re-enter the poll loop, and refresh the gallery on completion
so a run started elsewhere is reflected and its saved image appears without a
manual refresh. Seed resident image defaults from the resolved base_repo rather
than a possibly path-shaped repo_id so the first resident generation uses the
right recipe.
2026-07-12 12:41:21 +00:00
Daniel Han
4cc35aa87a Tighten comments in the image stack tests and scripts 2026-07-12 12:21:14 +00:00
Daniel Han
0017674664 Tighten comments in the image and video frontend 2026-07-12 12:15:36 +00:00
Daniel Han
ecae46cbfb Tighten comments in the diffusion training core and API models 2026-07-12 12:06:44 +00:00
Daniel Han
3a6a007038 Tighten comments in the image stack routes 2026-07-12 12:00:44 +00:00
Daniel Han
1762dab12f Tighten comments across the remaining image stack files 2026-07-12 11:46:23 +00:00
Daniel Han
9fde4b9991 Tighten comments across the remaining image stack files 2026-07-12 11:40:05 +00:00
Daniel Han
892696733e Tighten comments across the image generation stack 2026-07-12 10:55:39 +00:00
Daniel Han
dcd6666f8d Thread the task-scoped GGUF fit budget through every picker expander
The previous commit introduced the single-device budget but wired it
into only the downloaded-group and Hub search sites. The LM Studio,
custom-folder, local-dir, live-search and exported-GGUF expanders
reachable from the Images/Video pickers still measured against the
summed multi-GPU total, as did the size-based GGUF row badge, so those
paths could still recommend a quant that OOMs on a single device. All
GgufVariantExpander call sites in HubModelPicker now share
expanderGpuGb, and the row badge derives the same task-scoped budget.
2026-07-11 18:36:31 +00:00
Daniel Han
cece544fd9 Fix stale LoRA closure, variant fit budget, and hidden dataset remove button
The generate callback omitted loraCapable from its dependencies, so when
an auto-compile flips supports_lora off mid-session the memoized handler
still sent the previously selected adapters and the next generation
failed with the backend's LoRA-not-supported error instead of omitting
adapters the UI had already hidden.

HubModelPicker's GGUF variant expanders, format lists and Hub row fit
hints measured against the summed multi-GPU total. When the picker is
task-scoped (Images/Video) the loaders place the whole pipeline on one
device, so a variant could be recommended as fitting and then OOM at
load; those sites now share the single-device budget the group fit gate
already uses, while chat pickers keep the summed total since llama.cpp
splits layers across devices.

The dataset labeling grid's Remove button relied on group-hover with no
group parent, leaving it permanently invisible to mouse users; the image
wrapper now carries the group class.
2026-07-11 17:32:05 +00:00
pre-commit-ci[bot]
668088436e [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-11 16:51:12 +00:00
Daniel Han
1a2dc57c4a scripts: baseline the gguf 0.19.0 HF download helper for the studio scan shard
The studio dependency spec resolves gguf 0.19.0, whose gguf/utility.py
legitimately sends an HF_TOKEN Authorization header from the authenticated
Hugging Face download helper used by convert_hf_to_gguf; main's baseline
entry covers a different gguf version so the evidence hash differs.
Verified locally: the full studio shard scan exits 0 with the updated
baseline and zero unsuppressed CRITICAL or HIGH findings.
2026-07-11 16:50:19 +00:00
Daniel Han
e1fa4fec04 Studio diffusion: fix static compile shape registration and prequant path validation
Register the dims the forward actually compiled with: image-conditioned
workflows (img2img, inpaint, upscale, edit) run at the input image's size,
not the slider's, so recording the slider values marked never-compiled
shapes as covered and warm restarts kept paying compile for the real one.

Validate a request-supplied transformer_prequant_path (existence plus the
UNSLOTH_ALLOW_LOCAL_PREQUANT_PATH allowlist) before treating prequant as
available at the resident-fit re-check: an unusable path skipped the dense
fit check up front and then fell back to materializing dense bf16 after
the previous pipeline was evicted, recreating the post-eviction OOM path.
Shared as usable_prequant_source, also used by the auto-policy planner.
2026-07-11 16:50:19 +00:00
Daniel Han
316c8b5a81 Studio video: resume a background generation on page mount
A generation runs on a backend daemon thread and survives a page reload,
but the mount effect only probed the load progress, so reloading during a
generate showed an idle page that never picked up the finished clip until
a manual refresh. Hoist the generate poll loop out of handleGenerate and
re-enter it on mount when generate-progress reports an active job; merge a
terminal completed record into the gallery to cover the race with the
mount gallery fetch.
2026-07-11 16:50:19 +00:00
Daniel Han
57f08ebfb2 Merge remote-tracking branch 'origin/main' into ig_merge 2026-07-11 15:15:54 +00:00
Daniel Han
9fa6fd40e1
scripts: refresh scan_packages allowlist baseline (#7078)
New releases of huggingface-hub (1.23.0) and openai (2.45.0) shifted or
added polling loops that the C2 polling/beaconing check flags, failing
all three pip scan-packages shards (studio 1, hf-stack 1, extras 3 new
CRITICAL findings) org-wide including on main.

Regenerated with scan_packages.py --write-baseline per CI shard (same
shard-to-requirements mapping and --with-deps as security-audit.yml)
and merged. All entries were manually reviewed at the resolved versions:

- huggingface-hub hf_api.py: create_repo 409-concurrency retry loop
  body changed in 1.23.0; refreshed evidence hash. The loop POSTs to
  the canonical Hub endpoint and retries only on a specific conflict
  error. Benign client retry.
- openai beta/threads/runs/runs.py: create_and_poll run-status helper
  refactored in 2.45.0 (Assistants deprecation annotations); refreshed
  evidence hash. Documented polling helper against api.openai.com.
- openai beta/responses/responses.py: new beta websocket client whose
  __aiter__ yields server events until the connection closes. New
  entry; standard event-stream iterator, not beaconing.
- openai resources/responses/responses.py: evidence line number
  refreshed only, hash unchanged.

The two dropped entries are the pre-refactor hashes of the same two
loops above; they no longer occur at the resolved versions. Verified
locally: all three shards exit 0 with 0 unsuppressed CRITICAL/HIGH
(hf-stack 120, studio 151, extras 99 suppressed).
2026-07-11 08:15:43 -07:00
Daniel Han
6412efd7d9
Studio: auto-detect completion masking markers, stop silent full-sequence training (#7054)
* Auto-detect completion masking markers with template table fallback

Studio's train_on_completions previously relied only on the hardcoded
MODEL_TO_TEMPLATE_MAPPER / TEMPLATE_TO_RESPONSES_MAPPER tables and
silently disabled masking when a model was not in the table, so unmapped
models (LFM2-8B-A1B, DeepSeek, and others) trained on full sequences
without telling the user. Several mapped templates (glm, mistral, llama,
starling, zephyr, qwen3-thinking) also carried markers that mask every
assistant token, which made every row drop in the post-masking filter.

Both training callsites (CUDA trainer.py and MLX worker.py) now share
utils.datasets.completion_masking.apply_completion_masking:

- Try unsloth_zoo chat template auto-detection first; it raises loudly
  when the template cannot be parsed and never masks the EOS token.
- gpt-oss models keep their manual markers so non-final assistant
  <|end|> tokens stay trained, matching current behavior.
- If auto-detection raises, fall back to the template table exactly as
  before.
- If the table also misses, emit an explicit user-visible warning that
  completion masking could not be applied and full-sequence training
  will occur, instead of a quiet log line.

The >30 percent dropped-rows safety net in trainer.py now guards the
auto path as well. Table consumers for inference and chat templates are
unchanged. Validated against one representative tokenizer for every
template in TEMPLATE_TO_RESPONSES_MAPPER plus the unmapped models:
no template regresses; unit tests cover the four decision paths.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Restrict masking fallback to marker detection failures

The auto branch wrapped the whole train_on_responses_only call, so a real
failure while applying the masking (dataset map, tokenization) was treated
as a detection miss and training silently proceeded on full sequences.
Detect markers separately via get_chat_template_parts (test seam via
detect_fn), then apply them with errors propagating, matching the manual
path. Tokenizers with preset unsloth marker attrs skip detection and call
bare so zoo reuses the stored parts.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fail the run when applying completion masking raises

The helper already falls back internally on detection failures and returns
applied=False on a double miss, so an exception reaching the callsites is a
real failure applying the masking. Remove the callsite catches that
downgraded it to full-sequence training; the run now fails visibly instead.

Also use the explicit re-export alias form in utils/datasets/__init__.py for
the two new names, satisfying the import-hoist source lint.

* Import completion masking from its submodule

The import-hoist source lint counts only real name loads, so package-level
re-exports of the two new names cannot satisfy it. Import
apply_completion_masking from utils.datasets.completion_masking directly at
both callsites and leave utils/datasets/__init__.py untouched.

* Completion masking: gpt-oss renames and MLX raw/alpaca parity

Renamed or private gpt-oss checkpoints are name-detected as gpt-oss but miss
the exact-name table; default them to the gpt-oss template markers instead of
falling through to full-sequence training.

Gate the MLX masking call on not raw_text_mode and format_type != alpaca,
mirroring the CUDA path: raw/CPT text has no chat turns to mask and
Alpaca-rendered text lacks the tokenizer's chat markers.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Define raw_text_mode outside the MLX feature-detect block

With an older zoo lacking the append_eos config field, the masking
gate referenced raw_text_mode before assignment. Hoist the assignment
above the feature detection so both consumers see it.

* Gate MLX masking on the formatter's resolved format

format_type auto can resolve to alpaca or raw text; the masking skip
checked only the requested value, so auto-detected Alpaca data got
chat-template markers applied to rendered prompt text. Track the
final_format returned by format_and_template_dataset and gate on it,
matching the CUDA path.

* Unwrap the mlx-lm TokenizerWrapper before marker checks

The wrapper delegates plain reads to the wrapped HF tokenizer but hides
underscore attrs, so preset unsloth markers were invisible and detection
relied on the loader's call patch. Unwrap to the real tokenizer first,
as the zoo MLX resolver does.

* Tighten masking comments

* gpt-oss: auto-detect markers first like every other template

The quantized and BF16 gpt-oss checkpoints ship a chat template without
the channel final header, so the pinned manual markers match nothing
there and masking trained zero tokens. Auto-detection derives markers
from whichever template the checkpoint ships and keeps the final
terminator trained; the manual gpt-oss markers remain the detection
failure fallback, including for renamed checkpoints.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-11 05:13:45 -07:00
Daniel Han
97161c89d6
Studio: route models by CONFIG_MAPPING_NAMES instead of hardcoded tables (#7043)
* Studio: route models by CONFIG_MAPPING_NAMES instead of hardcoded tables

A model whose model_type is absent from an overlay's transformers cannot load
there, so a new MoE arch not yet in the tier tables gets routed to default and
fails (e.g. lfm2_moe, deepseek_v4). Add a static resolver that parses each
overlay's CONFIG_MAPPING_NAMES straight from source (AST only, no import, no
network, no trust_remote_code) and picks the lowest tier that ships the
model_type. Runs after the existing checks and only ever upgrades default, so
no existing routing changes and new archs no longer need a table edit.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio router: harden the CONFIG_MAPPING_NAMES resolver

- Resolve the default tier map from the base install, skipping any .venv_t5_*
  sidecar on sys.path, so an in-process 5.x activation cannot make a 5.x-only
  model look loadable by 4.x.
- Do not cache an overlay whose sidecar dir is absent, so a later call re-reads
  it once provisioned instead of serving a stale empty map.
- Also collect model types added via CONFIG_MAPPING_NAMES.update({...}) and
  **{...} unpacking, not just the literal assignment (5.10 uses both).
- Wrap the AST walk in the try/except so a malformed source can never crash tier
  resolution.
- Feed the mapping fallback from _load_config_json so a config served from the
  hub cache during a transient outage still routes new architectures.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-11 05:12:37 -07:00
Daniel Han
c3feac6160
Studio: route lfm2_moe (LFM2-8B-A1B) to transformers 5.3.0 (#7040)
LFM2-8B-A1B and any other lfm2_moe checkpoint were missing from the
transformers tier tables, so they fell through to the default 4.57.x
sidecar, which does not register lfm2_moe and errors with
"not supported yet in transformers==4.57.6". Only lfm2_vl was listed.

Add Lfm2MoeForCausalLM / lfm2_moe to the 5.3.0 tier (lfm2_moe is
registered in transformers 5.3.0). get_transformers_tier now returns
530 for LFM2-8B-A1B and the model loads and trains as expected.
2026-07-11 05:08:07 -07:00
Daniel Han
d439508853 Reset the step cache before every bench generation to match production
The image bench drove pipe() directly with no cache reset between the
warmup and measured prompts, while the production backend clears the
FBCache residuals before each generation. diffusers keys those residuals
on the long-lived transformer and never resets them itself, so step 0 of
each measured prompt compared its first-block residual against the
previous prompt's final one, a state production never runs. Mirror the
backend's _reset_step_cache (best-effort, no-op for uncached configs and
for SDXL's unet) inside _generate so both warmup and measured passes
start clean, and note that pre-fix FBCache rows may overstate results.
2026-07-11 07:55:01 +00:00
Daniel Han
352fb40089 Warm-save the compile cache by default, compile U-Net denoisers whole-module
diffusion_compile_cache: auto mode now saves the Mega-cache bundle after the
first compiled generation (UNSLOTH_DIFFUSION_COMPILE_CACHE_SAVE=0 opts out), so
users get warm restarts without the distributor env; a bundle hit starts clean
(no pointless rewrite of the just-loaded artifacts) and explicit mode 1/on keeps
the distributor-style re-save. New register_shape + manifest shape coverage: a
STATIC compile produces new artifacts per (width, height, batch), so the
generate path registers each generation's shape and an uncovered shape
re-dirties the context, growing the bundle to cover every shape the session
used. Measured (B200, real backend): Qwen-Image deferred gen-3 hitch 29.1 ->
22.2 s warm with bit-identical output (7.9 MB bundle, ~0.5 s save); SDXL gen-3
115.7 -> 24.7 s and a mid-session 768px recompile 65.8 -> 12.6 s (bundle 63.6 ->
98.7 MB after the 768 re-save).

diffusion_speed: U-Net denoisers (UNet2DConditionModel; no _repeated_blocks, so
the regional compile never reached them) now get a whole-module STATIC
torch.compile on the default tier, plus fused QKV projections and a compiled VAE
decode. Measured on SDXL (30 steps / 7.0 / 1024px, 4 prompts, LPIPS vs the
bit-exact reference): 6.16 -> 3.14 s end to end (1.96x) at LPIPS 0.035, steady
state 0.70-0.88 s/image through the real backend. Rejected on measurement:
dynamic=True whole-module (366 s compile for 39.3 ms/step vs static's 73 s for
26.9), regional BasicTransformerBlock only (45.0 ms/step; ResNet convs stay
eager), max-autotune + inductor flags (25.9 ms/step for a 445 s warmup),
channels-last UNet alone (neutral). DiT tiers unchanged: fused QKV measured
exactly neutral under the regional compile (Qwen-Image 6.53 vs 6.52 s), so it
stays max-only there, and the DiT VAE decode stays eager (a few % of a DiT
generation). compiled_shapes_are_static tells the cache layer which loads are
per-shape (max tier, U-Net whole-module).

diffusion: register each generation's shape with the compile cache before the
save, pass pipe.unet to the cache fingerprint when the pipe has no transformer,
and correct the transformer_quant resolved reason on dense loads (it claimed a
GGUF transformer was loaded on every non-quantized pipeline load).

Tests: 333 passing across the related suites (speed 42, compile_cache 27, cache
40, precision 20, backend, base_precision, transformer_quant, memory); ruff
clean. Full measurement record: outputs/image_optim_round2_audit.md.
2026-07-11 06:18:29 +00:00
oobabooga
d105bd7b42
Studio: detect Windows Intel GPUs via the registry before WMI (#7064) 2026-07-10 17:59:04 -03:00
oobabooga
7bfa209623
Studio: hint at Model auto-switch in the OpenAI "No model loaded" 400 (#7006) 2026-07-10 17:48:27 -03:00
Apoze
fef37cb25b
Studio: queue local GGUF OpenAI-compatible requests before llama-server (#7047)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-10 17:05:48 -03:00
Daniel Han
6cb44270fc Reject extension-case sidecar collisions, gate untrainable families, hide dead Reapply
Reject an image whose name differs from an existing one only by extension
case (cat.PNG vs cat.png): the stems are exactly equal, so on a
case-sensitive filesystem both files land and both resolve to one cat.txt
caption sidecar, silently sharing and corrupting the caption. Stem case
variants (Pic.png vs pic.png) stay exempt: they are one file on
case-insensitive filesystems and write separate sidecars on Linux.

Treat an empty precision_modes list on a DiT family as the backend's
deliberate cannot-train signal (a non-bf16 CUDA GPU fails the trainer's
preflight for every mode) instead of falling back to the full mode list:
the precision selector disables and the start button reads not supported,
so the form no longer offers a run that always 400s. An absent field still
means an older backend and keeps the fallback.

Hide the Images page Reapply button when no reload target is known: a
resident GGUF or single_file model discovered by refresh carries no
checkpoint filename in status, so clicking was a silent no-op. A resident
full pipeline keeps the button (it reloads by repo id alone).
2026-07-10 19:03:17 +00:00
Vineeth Sai
33119c9bf7
fix: guard remove_special_tokens against tokenizers without a BOS token (#7048) 2026-07-10 14:55:11 -03:00
Daniel Han
04f2cad5ab Restore the Reapply target when a model load fails to start
A load request that is rejected up front (validation error, gated repo,
training guard) leaves the previously loaded model resident, but both the
Images and Video pages had already pointed lastLoad at the failed pick, so
Reapply and the resident-default seeding retried the wrong model. Snapshot
the prior target before the optimistic assignment and restore it (plus the
video page's canReapply flag) when the start POST rejects; successful loads
and failures after the background load starts behave as before.
2026-07-10 17:49:28 +00:00
Daniel Han
39b256f8c9 Fix root-only pipeline index detection, seed range overflow, local safetensors loads
Require the model_index.json to sit at the snapshot ROOT before flagging a
cached repo as pipeline-loadable: CachedFileInfo.file_name is the basename,
so the previous name match also claimed nested copies (subdir/model_index.json)
and the picker then sent a from_pretrained load that fails only after the GPU
handoff. Scope by file_path against the revision's snapshot_path.

Validate the maximum derived seed before the multi-run image loop: an explicit
seed near 2**53-1 plus the per-run offset (base + i*batchSize) exceeded the
backend cap and 422'd a later run after earlier images had already generated.

Route local single-file .safetensors picks on the Images and Video pages
through the single_file load path (parent dir + basename), matching the local
GGUF branch: the pipeline route rejects a bare file with no model_index.json,
and only after evicting the resident model.
2026-07-10 17:01:24 +00:00
Daniel Han
de2f22df2b perf(image): compile numeric parity, cache-hook compile arming, FBCache toggle crash fix, TE fp8 zero-row guard
Applies the video round-2 accuracy findings to the image diffusion stack and fixes
two real image-path bugs found while measuring. All numbers B200, production
settings (family default steps/guidance, 1024px, seed 42, 4 fixed prompts), LPIPS
(AlexNet) via the new scripts/image_speedmem_bench.py, which drives the production
lever functions in the loader's own order.

- inductor precision parity: emulate_precision_casts=True on the regional-compile
  path (fused pointwise kernels keep fp32 intermediates where eager rounds to bf16
  between ops). Pairwise LPIPS of the compiled tier vs the same-stack eager tier:
  Qwen-Image 0.019 to 0.006 at identical speed (72.4 vs 72.5 ms/step), FLUX.1-dev
  0.046 to 0.029 at +2% step time (69.8 vs 68.3, reproduced), FLUX.2-klein-4B
  0.018 to 0.017 at identical speed. Snapshot/restored with the other process-wide
  backend flags so an off load never inherits it.
- cache x compile composition: re-point each cache hook's fn_ref.original_forward
  at a torch.compile'd wrapper of the same bound method (armed only where the
  speed layer compiled the block; restored before every disable_cache and before
  the partial-hook cleanup). Qwen-Image FBCache computed steps 91.8 to 71.2 ms
  (back at the uncached compiled rate), 1.21x end to end (7.36 to 6.06 s per 4
  images); FLUX.1-dev already traced through its FBCache hook and is measured
  neutral (same-process armed vs unarmed latents bit-identical). Skip counts
  within noise (13 vs 11 of 76; pairwise LPIPS 0.005).
- FBCache mid-session toggle crash: diffusers 0.39 caches the HookRegistry child
  list on first cache_context use, so an uncached generation followed by a
  20+-step generation (the auto toggle path) enabled hooks the context never
  reached and crashed with "No context is set" (reproduced live on FLUX.1-dev).
  Invalidate the stale child cache after every enable_cache.
- TE fp8_dynamic zero-row guard: torchao per-row fp8 derives a per-output-channel
  scale from the row amax, so an all-zero weight row is 0/0 = NaN. SDXL's
  text_encoder_2 (OpenCLIP bigG) ships exactly such a row, and every explicit
  fp8_dynamic SDXL render came out black; keep zero-row Linears dense (LPIPS
  0.976 black to 0.096 working). Other families' encoders have no such rows and
  are byte-identical.
- No AUTO TE quant exists on the image branch (text_encoder_quant defaults dense,
  explicit-only), so the video round's auto-dense retune has no image analogue;
  the explicit lever's cost is now measured (TE fp8_dynamic alone, LPIPS vs
  bit-exact: Qwen-Image 0.038, FLUX.1-dev 0.084, SDXL 0.096; no speed win, VRAM
  -6.5 GB on Qwen-Image) for the docs.

Tests: 96 passing across the cache/speed/precision suites (11 new arming, 2
child-registry, 2 zero-row, 4 inductor-flag); ruff clean.
2026-07-10 16:07:46 +00:00
Daniel Han
ec90b8658d Show a preparing label before the first denoise step and poll immediately on tab return
Step 0 now reads "Preparing (text encoding + warmup)..." on the video and
images pages: text encoding and warmup run before the first scheduler tick,
so the bar otherwise sits on "step 0/N" for up to a minute at 720p.

Generation progress polls are also wired to a visibilitychange listener for
their lifetime: background tabs clamp setInterval to one second and can
suspend it entirely after a few minutes, so returning to the tab now fires
one immediate poll (overlap-guarded) instead of showing a stale label until
the next throttled tick. The listener is removed with the interval on the
terminal phase, generation end, and unmount.
2026-07-10 10:21:32 +00:00
Daniel Han
fbcd3fa511
CI: retry transient HTTP timeouts in Studio smoke probes (#7052)
* CI: retry transient HTTP timeouts in Studio smoke probes

The post() helper in the Studio inference smoke workflows does a single
urlopen with a 240s timeout against the local Studio server. On shared
runners this sporadically hits TimeoutError while the server is stalled,
failing the whole job for a transport hiccup; the same flake has recurred
across unrelated PRs on Linux and Windows (JSON/images and tool-calling
jobs) and passes on rerun.

Retry the probe up to 3 times on transport-level failures only
(TimeoutError, ConnectionError, non-HTTP URLError), 15s apart. HTTP
status errors still surface immediately, so genuine server failures are
unaffected. post_sse() is left unchanged: it has a 600s budget and has
not flaked.

* CI: retry only short probes so worst case fits the job budget

Some json-images calls pass timeout=600; three attempts there could spend
30 minutes in one step and hit the job's timeout-minutes instead of failing
with the Python error. Retry (3 attempts) only when timeout <= 300s, which
covers the observed flaky 180-240s probes; longer probes keep the pre-PR
single attempt.

* CI: give long smoke probes one capped retry

Round two of bounding the retries: timeout>300s probes previously got a
single attempt, so a transient stall in the 600s JSON-mode probes still
failed on first occurrence. Give them one retry with the attempt timeout
capped at 300s. Worst cases stay inside timeout-minutes: 240s probes
12.5 min, one 600s probe 15.25 min, the Windows JSON job's two long
probes 30.5 min against its 35 minute budget.
2026-07-10 03:07:39 -07:00
pre-commit-ci[bot]
cee2bf6ed2 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-10 09:20:11 +00:00