Commit graph

6,189 commits

Author SHA1 Message Date
oobabooga
1437f48e08 Merge remote-tracking branch 'origin/diffusion-sdxl' into diffusion-lora-ux 2026-07-01 13:56:05 -03:00
oobabooga
0813849aed Merge remote-tracking branch 'origin/diffusion-controlnet' into diffusion-sdxl 2026-07-01 13:50:07 -03:00
oobabooga
1d4b551a61 Merge remote-tracking branch 'origin/diffusion-lora' into diffusion-controlnet 2026-07-01 13:37:13 -03:00
oobabooga
9ffb28b923 Merge remote-tracking branch 'origin/diffusion-image-workflows' into diffusion-lora 2026-07-01 13:30:26 -03:00
oobabooga
30f40ed44c Merge remote-tracking branch 'origin/diffusion-phase16-native-engine-routing' into diffusion-image-workflows 2026-07-01 13:15:13 -03:00
oobabooga
7d9a59c722 Merge remote-tracking branch 'origin/diffusion-phase15-int8-prequant' into diffusion-phase16-native-engine-routing 2026-07-01 12:54:08 -03:00
oobabooga
707983642a Merge remote-tracking branch 'origin/diffusion-phase14-int8-modulation' into diffusion-phase15-int8-prequant 2026-07-01 12:44:49 -03:00
oobabooga
761cc21157 Merge remote-tracking branch 'origin/diffusion-phase12-fbcache' into diffusion-phase14-int8-modulation 2026-07-01 12:34:16 -03:00
oobabooga
ff9dcf847c Merge remote-tracking branch 'origin/diffusion-phase11-consumer-int8' into diffusion-phase12-fbcache 2026-07-01 12:28:36 -03:00
oobabooga
ed4336dbf7 Merge remote-tracking branch 'origin/diffusion-phase10-attention' into diffusion-phase11-consumer-int8 2026-07-01 12:19:17 -03:00
oobabooga
6ebaf64dc6 Merge remote-tracking branch 'origin/diffusion-phase9-prequant' into diffusion-phase10-attention 2026-07-01 12:13:04 -03:00
oobabooga
731fb20bde Merge remote-tracking branch 'origin/diffusion-phase8-quant' into diffusion-phase9-prequant 2026-07-01 12:07:42 -03:00
oobabooga
b02eacd64f Merge remote-tracking branch 'origin/diffusion-phase7-perf' into diffusion-phase8-quant 2026-07-01 12:02:11 -03:00
oobabooga
6adbb45eb9 Merge remote-tracking branch 'origin/diffusion-phase6-features' into diffusion-phase7-perf 2026-07-01 11:56:04 -03:00
oobabooga
1fa8e307ea Merge remote-tracking branch 'origin/diffusion-phase4-native' into diffusion-phase6-features 2026-07-01 11:50:02 -03:00
oobabooga
d285a4d250 Merge remote-tracking branch 'origin/image-generation' into diffusion-phase4-native 2026-07-01 11:36:04 -03:00
Daniel Han
3d187f1bab Images: LoRA free-text Hugging Face entry + recipe round-trip
The backend has always accepted a bare Hugging Face repo id (owner/name, or
owner/name:weight-file.safetensors) as a LoRA, downloading and applying it. But the
picker only rendered when the curated catalog had entries, and the catalog is empty,
so there was no UI path to apply any LoRA. Show the LoRA section whenever the loaded
model supports LoRA, and replace the curated-only dropdown with a text input: type a
Hub repo id, or pick a discovered adapter from a datalist of suggestions when the
catalog is populated.

Also restore LoRAs when loading a recipe. restoreSettings now parses the recipe's
"id:weight" strings (splitting on the last colon, since the id itself may contain one
for a specific weight file) back into the selection, so replaying a saved image
reproduces its adapters. The generate payload trims hand-typed ids and drops empty /
zero-weight rows, and a model swap clears the selection (a LoRA is family-specific)
without discarding a free-text pick that is not in the curated list.
2026-07-01 14:05:16 +00:00
pre-commit-ci[bot]
a927f4dc1e [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 13:58:13 +00:00
Daniel Han
977c9b7dd0 Add SDXL diffusion family (U-Net pipeline support)
SDXL is the first U-Net family in the diffusion backend: its denoiser is
pipe.unet (UNet2DConditionModel), not a DiT pipe.transformer, and a single-file
.safetensors is the whole pipeline rather than a transformer-only file. The
backend previously assumed a DiT transformer everywhere, so add the two hooks a
U-Net family needs and register SDXL.

DiffusionFamily gains denoiser_attr ("transformer" for DiT, "unet" for SDXL) and
single_file_is_pipeline (SDXL loads a single file via pipeline_class.from_single_file
with the base repo as config, instead of transformer_class.from_single_file plus a
companion assembly). _align_vae_dtype now reads the denoiser generically so img2img
and inpaint keep the VAE and U-Net dtypes aligned.

The non-GGUF trust gate is extended with a short, exact-match, safetensors-only
allowlist of official base repos (the SDXL base/refiner and sdxl-turbo), because
SDXL ships only as a full pipeline and has no unsloth-hosted GGUF. Local paths stay
trusted as before; a random repo, even one that detects as SDXL, is still rejected.

The image-conditioned and ControlNet workflows are the standard SDXL pipelines,
built around the resident modules via from_pipe like every other family, so SDXL
gets txt2img, img2img, inpaint, outpaint, upscale, LoRA and ControlNet. There is no
native sd.cpp mapping yet, so the no-GPU route falls back to diffusers.

Frontend catalog gains SDXL Base 1.0 and SDXL Turbo entries with SDXL step/guidance
defaults (Turbo: few steps, no CFG; base: ~30 steps, real CFG).

Tests: new test_diffusion_sdxl.py (family shape, detection, trust allowlist, model
kind, U-Net VAE-dtype alignment, LoRA gate) plus loader-branch tests in
test_diffusion_backend.py (pipeline-kind from_pretrained, single-file whole-pipeline
from_single_file, allowlist accept/reject). Verified live on GPU: sdxl-turbo loads
both as a pipeline and as a single file and generates coherent txt2img + img2img.
2026-07-01 13:57:07 +00:00
Daniel Han
355f54f6e4 Merge remote-tracking branch 'origin/diffusion-lora' into diffusion-controlnet 2026-07-01 11:57:56 +00:00
pre-commit-ci[bot]
5728670f6e [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 11:57:22 +00:00
Daniel Han
9637e85311 Merge remote-tracking branch 'origin/diffusion-image-workflows' into diffusion-lora 2026-07-01 11:56:47 +00:00
Daniel Han
38ed3ce5b5 Merge remote-tracking branch 'origin/diffusion-phase16-native-engine-routing' into diffusion-image-workflows
# Conflicts:
#	studio/backend/core/inference/diffusion.py
#	studio/backend/core/inference/diffusion_families.py
#	studio/backend/tests/test_sd_cpp_install.py
#	studio/frontend/src/components/assistant-ui/model-selector/pickers.tsx
#	studio/frontend/src/features/images/api.ts
#	studio/frontend/src/features/images/images-page.tsx
#	studio/install_sd_cpp_prebuilt.py
2026-07-01 11:48:33 +00:00
pre-commit-ci[bot]
44beb54df5 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 11:36:08 +00:00
pre-commit-ci[bot]
db081d67de [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 11:34:22 +00:00
pre-commit-ci[bot]
ffe5b69773 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 11:33:51 +00:00
pre-commit-ci[bot]
a8e87ac77b [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 11:33:20 +00:00
pre-commit-ci[bot]
c1b4ed1233 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 11:32:45 +00:00
pre-commit-ci[bot]
1ab9db02ff [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 11:32:13 +00:00
Daniel Han
91edf06748 Merge branch 'diffusion-phase15-int8-prequant' into diffusion-phase16-native-engine-routing
# Conflicts:
#	studio/backend/routes/inference.py
2026-07-01 11:31:54 +00:00
pre-commit-ci[bot]
ab27ea8520 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 11:31:18 +00:00
pre-commit-ci[bot]
394f7985df [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 11:30:40 +00:00
Daniel Han
f899927652 Merge branch 'diffusion-phase14-int8-modulation' into diffusion-phase15-int8-prequant 2026-07-01 11:30:29 +00:00
Daniel Han
5951dbe145 Merge branch 'diffusion-phase12-fbcache' into diffusion-phase14-int8-modulation 2026-07-01 11:30:28 +00:00
Daniel Han
2a5713aff5 Merge branch 'diffusion-phase11-consumer-int8' into diffusion-phase12-fbcache 2026-07-01 11:30:26 +00:00
Daniel Han
2f26a785b1 Merge branch 'diffusion-phase10-attention' into diffusion-phase11-consumer-int8 2026-07-01 11:30:24 +00:00
Daniel Han
b84d7ea7f3 Merge branch 'diffusion-phase9-prequant' into diffusion-phase10-attention
# Conflicts:
#	studio/backend/tests/test_diffusion_routes.py
2026-07-01 11:30:23 +00:00
Daniel Han
4fec0bb79c Merge branch 'diffusion-phase8-quant' into diffusion-phase9-prequant
# Conflicts:
#	scripts/compare_engines.py
2026-07-01 11:29:36 +00:00
Daniel Han
5d86a7685a Merge branch 'diffusion-phase7-perf' into diffusion-phase8-quant
# Conflicts:
#	studio/backend/core/inference/diffusion.py
2026-07-01 11:29:00 +00:00
pre-commit-ci[bot]
7410f8fa1c [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 11:28:58 +00:00
Daniel Han
114baa6607 Merge branch 'diffusion-phase6-features' into diffusion-phase7-perf
# Conflicts:
#	studio/backend/core/inference/diffusion_memory.py
#	studio/backend/core/inference/diffusion_speed.py
2026-07-01 11:27:02 +00:00
pre-commit-ci[bot]
91780a250e [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 11:26:13 +00:00
Daniel Han
172a4eb642 Merge branch 'diffusion-phase4-native' into diffusion-phase6-features
# Conflicts:
#	studio/backend/core/inference/sd_cpp_engine.py
#	studio/backend/tests/test_sd_cpp_engine.py
2026-07-01 11:24:14 +00:00
Daniel Han
48628252bd Merge remote-tracking branch 'origin/image-generation' into diffusion-phase4-native
# Conflicts:
#	scripts/diffusion_bench.py
#	scripts/diffusion_quality.py
#	studio/backend/core/inference/diffusion.py
#	studio/backend/core/inference/diffusion_device.py
#	studio/backend/core/inference/diffusion_families.py
#	studio/backend/core/inference/diffusion_memory.py
#	studio/backend/core/inference/diffusion_precision.py
#	studio/backend/core/inference/diffusion_speed.py
#	studio/backend/models/inference.py
#	studio/backend/routes/inference.py
#	studio/backend/tests/test_diffusion_backend.py
#	studio/backend/tests/test_diffusion_device.py
#	studio/backend/tests/test_diffusion_memory.py
#	studio/backend/tests/test_diffusion_precision.py
#	studio/backend/tests/test_diffusion_speed.py
2026-07-01 11:19:15 +00:00
pre-commit-ci[bot]
387cfd268b [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 10:47:53 +00:00
Daniel Han
2cae1c1e7c Merge remote-tracking branch 'origin/main' into image-generation
# Conflicts:
#	studio/backend/routes/models.py
#	studio/frontend/src/components/assistant-ui/model-selector/pickers.tsx
2026-07-01 10:47:21 +00:00
Daniel Han
237d07b035 Studio Images: clearer error for an unsupported diffusion model
When a repo id resolves to no diffusion family the load raised 'Could not infer a
diffusion family... Pass family_override (z-image)', which points at an unrelated
family and doesn't say what is supported. Replace it with a message that lists the
supported families (from a new supported_family_names helper) and notes that video
models and image models whose diffusers transformer has no single-file loader are
not supported. Applies to both the diffusers and native sd.cpp load paths. Also
refreshes two stale family-registry comments that still called FLUX.2-dev omitted.
2026-07-01 10:17:35 +00:00
pre-commit-ci[bot]
4f8295e829 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 09:58:52 +00:00
Daniel Han
dfa9777fe3 Studio Images: add the FLUX.2-dev model family
Loading unsloth/FLUX.2-dev-GGUF failed because detect_family knew only the
Qwen3-based FLUX.2-klein, so FLUX.2-dev (the full, Mistral-based Flux2Pipeline)
resolved to nothing and the load errored. Add a flux.2-dev family: Flux2Pipeline
+ Flux2Transformer2DModel over the black-forest-labs/FLUX.2-dev base repo (gated,
reachable with an HF token), with its FLUX.2 32-channel VAE and Mistral text
encoder wired for the sd-cli path from the open Comfy-Org/flux2-dev mirror.
text-to-image only: diffusers 0.38 ships no Flux2 img2img / inpaint pipeline for
dev. Frontend gets sensible dev defaults (28 steps, guidance 4), distinct from
klein's turbo defaults. Verified live: GGUF load resolves the family + gated base
repo and generates a real 1024x1024 image on GPU.
2026-07-01 09:58:09 +00:00
pre-commit-ci[bot]
de729c0193 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-07-01 09:30:27 +00:00