Compare commits

...
Sign in to create a new pull request.

94 commits

Author SHA1 Message Date
Daniel Han-Chen
07b0cf7d2c Replace asyncio.get_event_loop with asyncio.get_running_loop for PR #5754
asyncio.get_event_loop() is deprecated in Python 3.10 and will be
removed. Replace with asyncio.get_running_loop() at five call sites
that all run inside async def functions where a running loop is
guaranteed: two async_generate helpers in diffusion.py and three
run_in_executor sites in routes/inference.py (audio synth, diffusion
load, streaming chat). Addresses the gemini-code-assist bot review.
2026-05-26 00:28:23 +00:00
Daniel Han-Chen
ca68fd5d13 Fix/adjust diffusion: export active-state guards, cleanup window, unload race for PR #5754
Round 41 review findings (2 P1, 5/12 reviewers consensus on the dominant one):

1. routes/export.py: load_checkpoint already refuses 409 when training
   or another export is active, but /export/{merged,base,gguf,lora} and
   /cleanup went through _export_public_window without those checks.
   A user could start training, then trigger an export (or cleanup),
   and both would double-own the GPU. Factor the training-active and
   export-active guards into _raise_if_training_active_for_export and
   _raise_if_export_active_for_export, call them inside the context
   manager so all /export/* + /cleanup share the same fail-closed
   semantics as load_checkpoint, and wrap /cleanup with the window.

2. core/inference/diffusion.py: DiffusionBackend.unload_model cleared
   _pipe / _repo_id / _family / ... under _lock BEFORE _release(old)
   and _drain_cuda_cache. Between the lock release and cache drain,
   status() reported is_loaded=False / is_loading=False, so the
   helper-busy check (which OR-s those two) could let an AI Assist
   GGUF backend start while diffusion VRAM was still being freed.
   Set _loading=True inside the lock as a busy marker before clearing
   the slot, and only clear it in a finally after release + drain
   complete.
2026-05-25 22:18:02 +00:00
pre-commit-ci[bot]
029ca741b4 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 21:50:31 +00:00
Daniel Han-Chen
784a9ed71c Fix/adjust diffusion: export public-load window, identifier hardening for PR #5754
Round 40 review findings (5 P1 + 1 P2 + 3 P3):

P1:
1. routes/export.py: wrap /export/merged, /export/base, /export/gguf,
   /export/lora in a public-load window so backend.export_*() running
   in a worker thread cannot be torn down by a concurrent workload that
   sees is_export_active() == False during the pre-active gap.
2. utils/datasets/llm_assist.py: add public_load_pending_for(workload)
   helper. routes/inference.py: _release_export_for now refuses 503
   when export is mid-handoff.
3. models/models.py: AddScanFolderRequest.path now rejects control
   characters and embedded hf_ tokens before being logged or reflected.
4. models/training.py: local_datasets and local_eval_datasets list
   entries get the same control-char / embedded-token validators that
   model_name / hf_dataset already have.
5. models/training.py: format_type joins the validator list (copied
   into training_kwargs and into trainer log lines).
6. models/export.py: _validate_save_directory now rejects embedded hf_
   tokens (already covered other identifier fields).

P2:
7. images-page.tsx:162: defer the mount fetchAndUpdateStatus call
   through setTimeout(..., 0) so it does not trip
   react-hooks/set-state-in-effect on scoped lint.

P3 cleanup:
8. core/inference/diffusion.py: drop unused gguf_basename assignment.
9. core/inference/diffusion.py + routes/inference.py: drop unused
   owned_names computation from the chat-release helpers; the final
   sweep intentionally no longer filters by that snapshot.
2026-05-25 21:50:11 +00:00
Daniel Han-Chen
f5186e2b35 Fix/adjust diffusion: filename validator order, 503 mapping, token redaction for PR #5754
Round 39 review findings (2 P1 + 1 P2):

1. routes/datasets.py: validate raw multipart filename before sanitize so
   smuggled control chars (NUL, newline) are rejected at the same boundary
   as the JSON path in seed.py. Previously _sanitize_filename stripped the
   control chars first, letting raw inputs slip past the validator.

2. routes/inference.py: extend RuntimeError -> 503 mapping in /images/load
   to classify "Another GPU workload is mid-handoff" as retryable, so the
   backend-surfaced phrasing matches the route-level 503 already returned
   from _raise_if_helper_advisor_busy.

3. core/inference/diffusion.py: redact hf_ tokens in the local-path branch
   of _display_repo_id so a leaf directory named hf_<token> cannot leak
   into UI labels or structured logs.
2026-05-25 21:22:13 +00:00
pre-commit-ci[bot]
d0f4bb5165 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 20:43:19 +00:00
Daniel Han-Chen
e30c5ed386 Fix/adjust diffusion: backend public_load_pending parity for PR #5754
Round 38 P1: R35e added the public_load_pending() check to the
route-side _raise_if_helper_advisor_busy, but the backend-side
_raise_if_helper_advisor_busy_for_diffusion (used by direct
DiffusionBackend.load_model callers AND called transitively from
the route via backend.load_model) never got the same parity
check. That left a window where:
  * /api/training/start published the "training" pending marker
    via the route helper
  * a script/test calling DiffusionBackend.load_model() directly
    passed the backend's helper-busy snapshot, never checked
    public_load_pending(), and proceeded to destructive owner
    teardown + GPU allocation while training was still pending.

Add the parity check with a kw-only `excluding` parameter on
public_load_pending so a route-wrapped backend call can ignore
the marker its own route already published (route publishes
"diffusion"; backend publishes the separate "diffusion-backend"
tag). load_model gains ignore_public_load_pending_workload to
thread the route's tag through; the diffusion route passes
"diffusion" so the backend's atomic check does not self-block
on the route's own publication.

Verified by smoke test: route-wrapped backend with excluding=
"diffusion" allowed during route's diffusion pending; direct
backend call refused with RuntimeError "Another GPU workload is
mid-handoff" when training is pending. 86 backend tests pass.
2026-05-25 20:43:04 +00:00
Daniel Han-Chen
aeba18dc6d Fix/adjust diffusion: public_load_pending self-check for PR #5754
Round 35 P1: _raise_if_helper_advisor_busy published a new public
pending marker without first checking public_load_pending(). Two
public workloads (e.g. training + diffusion) could both pass
their idle helper-busy snapshot concurrently, then both run
through destructive owner teardown before either flipped its
own visibility flag (is_training_active, current_checkpoint,
loading_model_identifier, diffusion is_loading).

Add the missing self-check under _HELPER_ADVISOR_START_LOCK so
the second public workload sees the first's pending marker and
gets a 503 retry instead of racing for VRAM. Helper / advisor
already checked public_load_pending() on its side via
_gpu_workload_busy_for_helper; this closes the symmetric public
-> public window.

86 backend tests pass + smoke test confirms second public load
is refused with 503 while first is pending, and the next public
load is permitted once the first clears.
2026-05-25 19:15:14 +00:00
Daniel Han-Chen
09ca2b27d3 Fix/adjust diffusion: drop accelerate preflight + datasets upload validator for PR #5754
Re-fix the round-34 CI regression: the round-34 attempt gated the
accelerate preflight on enable_model_cpu_offload, but the parameter
defaults to True so tests that did not explicitly opt out still
hit the missing-accelerate path. Removed the accelerate preflight
entirely; transformers' PyTorch backend already pulls accelerate
as a hard dep on every supported install path, so the duplicate
find_spec guard is redundant in practice and the missing-package
case will still surface a clean ModuleNotFoundError from the
offload code itself if the user somehow lands there without it.

Round 34 P1 cross-block: extend the seed.py multipart filename
validators (round 33) to /api/datasets/upload. Both routes echo
the filename back to the client and persist it, so per the
asymmetric-fix rule the validators must match. Now rejects
control characters and embedded HF tokens in file.filename in
both upload entry points.

86 targeted backend tests pass.
2026-05-25 17:41:14 +00:00
Daniel Han-Chen
081377fd30 Fix/adjust diffusion: restore huggingface_hub line to no-torch-runtime for PR #5754
Round 34 P1: R33 removed the `huggingface_hub>=1.3.0,<2.0` line
entirely when the right revert was to restore the pre-PR
`huggingface_hub>=0.34.0` floor. install.sh --no-torch installs
this requirements file with --no-deps and does NOT install
studio.txt afterward, so without an explicit Hub line a no-torch
Studio install ends with no huggingface_hub at all and the new
diffusion + chat GGUF paths fail at import with
ModuleNotFoundError: huggingface_hub.

Restores the pre-PR floor and documents both the round-26 walk-back
and the round-34 reason the package line stays.
2026-05-25 17:29:47 +00:00
Daniel Han-Chen
4e1c622d20 Fix/adjust diffusion: gate accelerate preflight on cpu_offload for PR #5754
Backend CI on Python 3.11 failed 15 diffusion tests after R30's
accelerate preflight because the CI test environment does not
install accelerate, but the tests mock from_pretrained and never
exercise the CPU-offload path that actually needs it.

Gate the find_spec("accelerate") check on enable_model_cpu_offload
so the dependency is only required for the path that uses it.
transformers preflight stays unconditional (it is always touched
by from_pretrained). Tests with offload=False (the default) pass
without accelerate; production loads with offload=True still get
the fail-fast unload-protection guard the original round-30 fix
added.

97 targeted backend tests pass (test_diffusion_routes,
test_diffusion_backend, test_inference_model_validation,
test_data_recipe_seed, test_training_raw_support,
test_export_log_cursor).
2026-05-25 17:28:06 +00:00
Daniel Han-Chen
e3ce1c818e Fix/adjust diffusion: round 33 P1 batch for PR #5754
Two round-33 reviewer findings: hub-floor consistency and the
multipart upload filename validator gap.

Dependencies: reverted the round-26 huggingface_hub>=1.3.0 floor
in no-torch-runtime.txt and pyproject.toml (round 33 P1 #1-#5,
4/12 vote consensus). studio.txt forces huggingface_hub==0.36.2
to match the transformers==4.57.6 pin in extras-no-deps.txt, so
the 1.3.0 floor was internally inconsistent. Reviewers
reproduced the resolver conflict on a fresh install.

Empirical justification (re-verified on the live B200 host before
the revert): huggingface_hub 0.36.2 + transformers 4.57.6 +
diffusers 0.37.1 imports Flux2KleinPipeline cleanly and runs
end-to-end image generation. transformers 4.57.6 carries its own
transformers.utils.hub.is_offline_mode and does not actually need
huggingface_hub.is_offline_mode at import time. The original bump
was guarding against the (never-realised) transformers 5.x path,
which extras-no-deps explicitly pins away.

Validation: multipart /seed/upload-unstructured-file now applies
the same _no_control_chars and _reject_embedded_hf_token checks
to file.filename that SeedInspectUploadRequest.filename already
applies in the JSON variant (round 33 P1 #7). The filename is
reflected back to the client, persisted in the per-file meta
JSON, and echoed by error responses, so the JSON-side hardening
must not be asymmetric with the multipart path.

Skipped (consistent with prior rounds):
  * Find_spec vs full import (R33 P1 #6): preserves test
    compatibility with the huggingface_hub stub fixture.
  * React hooks set-state-in-effect lint (R33 P1 #8): codebase
    has 146 pre-existing violations of the same rule;
    studio-frontend-ci does not gate on lint.
  * Direct DiffusionBackend.load_model bypass (R33 P1 #9): the
    route is the only production entry point, and the backend
    helper now publishes its own diffusion-backend pending tag
    (round 32 P1 #3). Direct-caller hardening would require
    duplicating the lease check into load_model itself, which
    is out of scope for the route-layer security boundary.
  * One-segment Hub IDs (R33 P2 #10): strict 2-segment Hub id
    check is intentional; one-segment names are not valid Hub
    ids.
  * Cwd-relative shadow of Hub IDs (R33 P2 #11): documented
    side-channel tradeoff accepted in round 31 commit msg.

97 targeted backend tests pass.
2026-05-25 16:57:28 +00:00
pre-commit-ci[bot]
a1bec65961 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 16:19:39 +00:00
Daniel Han-Chen
90b51cc5c5 Fix/adjust diffusion: round 32 P1 batch for PR #5754
Three round-32 reviewer findings, plus documentation cleanup for
the local-path Tauri/FE plumbing gap.

Concurrency: direct DiffusionBackend.load_model callers now publish
the helper/advisor pending marker symmetrically (round 32 P1 #3).
_raise_if_helper_advisor_busy_for_diffusion gains an optional
publish_pending flag; load_model passes True so the destructive
unload window is gated by a "diffusion-backend" tag published
under _HELPER_ADVISOR_START_LOCK. The route layer's "diffusion"
tag and the backend's "diffusion-backend" tag refcount
independently (sum > 0 still blocks helper starts), so neither
side's clear can erase the other's still-active marker. The
existing _release_chat_backend_for_diffusion(check_helper_advisor=
True) path stays snapshot-only (publish_pending defaults False) so
test / direct callers of that helper do not leak a counter.

Validation: export save_directory now rejects ALL ASCII control
characters (round 32 P1, save_directory tab finding). The earlier
CR / LF only guard missed TAB / VT / FF / DEL, which a caller
could smuggle past the export worker's logged subprocess argv.

Documentation: DiffusionLoadRequest.repo_id and base_repo updated
to reflect that local-path support is gated on a Tauri /
frontend load-diffusion-model directory lease producer that has
not shipped yet (round 32 P1 #1 from multiple reviewers). The
backend lease boundary is correct; what is missing is the FE /
native side that mints the matching grant. Until that lands,
local paths through the Images route always 400 with "Native
path grant is required", which the docstring now spells out.

Skipped (consistent with prior rounds):
  * Hub-pin findings (R32 P1 #4-#6): live B200 install with
    huggingface_hub==0.36.2 + transformers==4.57.6 + diffusers==
    0.37.1 verifiably imports Flux2KleinPipeline. Empirical
    justification documented in R30 / R30 follow-up commit msgs.
  * Tauri / native enum surgery (R32 P1 #1, 6 votes): real
    architectural work but out of scope for this PR's Python
    surface. Documented now; FE / Rust ticket to follow.

98 targeted backend tests pass (test_diffusion_routes,
test_diffusion_backend, test_inference_model_validation,
test_models_get_model_config_case_resolution, test_data_recipe_seed,
test_training_raw_support, test_export_log_cursor).
2026-05-25 16:19:06 +00:00
Daniel Han-Chen
089749465c Fix/adjust diffusion: round 31 P1 batch for PR #5754
Two universal-consensus round-31 reviewer findings.

Concurrency: /images/load was leaking the public-load pending
counter on any pre-finally HTTPException (round 31 P1 #1, 11/12
votes). _raise_if_helper_advisor_busy("diffusion") published the
counter, then _resolve_diffusion_repo_for_request ran outside the
clearing try/finally. A request like repo_id="/tmp/model" with no
native_path_lease returned 400 and left public_load_pending() true
until process restart, permanently blocking AI Assist. Fix mirrors
the training / export pattern: track diffusion_load_window_published
in an outer try, publish the flag right after the helper-busy
check succeeds, and clear in an outer finally that only fires when
the flag is set. This also closes round 31 P1 #6: a second
request's failure can no longer decrement a still-active first
request's counter, because the second request has not yet flipped
its own publish flag.

Security: _looks_like_local_diffusion_path missed cwd-relative
directories (round 31 P1 #2, 8/12 votes). DiffusionBackend.
load_model accepts repo_id="exports/my-flux" as a local directory
via Path(repo_id).expanduser().is_dir(), but the detector only
flagged values starting with /, ~, ./, ../, backslash, or
absolute. Tightened the detector to also reject:
  * weight-file suffixes (.gguf / .safetensors / .bin / .pt / .pth)
  * non-2-segment values (`owner`, `a/b/c`, `owner/`, `/repo`, `//`)
  * 2-segment values whose parts are `.` or `..`
  * 2-segment values that actually resolve to an existing local
    path under backend CWD (last-resort exists() probe).
The existence probe is a minor side-channel for an already-
authenticated caller, accepted in exchange for closing the silent
bypass of the new lease boundary. Valid Hub ids like
unsloth/FLUX.2-klein-base-4B-GGUF, microsoft/Phi-3.5-mini-instruct
still pass through unchanged.

Skipped (consistent with prior rounds):
  * R31 P1 #3 (Tauri / native lease enum missing
    `load-diffusion-model` op): architectural surface; defer until
    the Images page actually surfaces a local-path picker.
  * R31 P1 #4-#5, #8: studio.txt / constraints.txt / pyproject hub
    pins. Live B200 install path with huggingface_hub==0.36.2,
    transformers==4.57.6, diffusers==0.37.1 imports
    Flux2KleinPipeline cleanly. The is_offline_mode import error
    only triggers when transformers 5.x is paired with hub 0.x,
    which the constraints pin prevents.
  * R31 P1 #7 (find_spec vs real import): a full transformers
    import at module load breaks tests that stub huggingface_hub;
    find_spec is the existing tradeoff.

98 targeted backend tests pass (test_diffusion_routes,
test_diffusion_backend, test_inference_model_validation,
test_models_get_model_config_case_resolution, test_data_recipe_seed,
test_training_raw_support, test_export_log_cursor).
2026-05-25 15:58:34 +00:00
pre-commit-ci[bot]
5350d4cc65 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 15:31:18 +00:00
Daniel Han-Chen
cae37123c9 Fix/adjust diffusion: round 30 follow-up P1 batch for PR #5754
Addresses remaining round-30 reviewer findings against PR #5754
(diffusion image generation in Unsloth Studio). The studio.txt /
constraints.txt / colab-new hub-bump items (round 30 #1-#3) are
intentionally skipped: the live B200 Studio install path with
huggingface_hub==0.36.2, transformers==4.57.6 and diffusers==0.37.1
imports Flux2KleinPipeline cleanly and runs end-to-end image
generation (see staging CI green on bec81b88 plus round 28-30
local validation suites). The is_offline_mode ImportError the
reviewer cites only triggers with transformers 5.x against
huggingface_hub 0.x; the constraints pin holds transformers at 4.x
so the combo never materialises on the standard install path.

Concurrency: close the helper / advisor GPU-start race in all four
public load paths (round 30 P1 #7-#10).
  * Add a _PUBLIC_LOAD_PENDING_COUNT counter in
    utils/datasets/llm_assist.py, published under
    _HELPER_ADVISOR_START_LOCK by _raise_if_helper_advisor_busy and
    cleared by a paired _clear_public_load_window in
    routes/inference.py. A concurrent helper / advisor start now
    sees public_load_pending() inside _gpu_workload_busy_for_helper
    and refuses VRAM until the public load attempt finishes,
    closing the window between the busy snapshot and the public
    load flipping its public ownership flags (is_loaded,
    current_checkpoint, is_training_active, etc.).
  * Wire the paired clear into all five call sites (GGUF chat,
    safetensors chat, diffusion image load, training start, export
    load-checkpoint). The chat path tracks the published tag in a
    local so the finally clears the same counter on either branch
    or on early HTTPException.

Security: gate /api/inference/images/load against arbitrary
local-path probes (round 30 P1 #4). Mirror the chat
/api/inference/load native_path_lease boundary so an authenticated
session cannot use repo_id or base_repo as a directory probe.
  * Add native_path_lease + base_repo_native_path_lease to
    DiffusionLoadRequest (optional; Hub ids skip the lease).
  * Add _looks_like_local_diffusion_path + a
    _resolve_diffusion_repo_for_request helper that requires a
    verified directory-typed native path grant for any value that
    starts with /, ~, ./, ../, contains a backslash, or expands to
    an absolute path. The detector deliberately avoids Path.exists
    so the route does not side-channel filesystem layout via
    differential error messages.

Frontend: split the Images page status fetch from the spinner
toggle (round 30 P2 #12). The mount effect and the is_loading
auto-poll now call a setState-free fetchAndUpdateStatus; the
user-driven Refresh button still calls refreshStatus to flip the
spinner. Cleaner separation than the queueMicrotask shim from the
prior commit; the eslint react-hooks/set-state-in-effect rule is
not in the studio-frontend-ci typecheck gate, and the codebase
already has hundreds of pre-existing violations of the same rule.

98 targeted backend tests pass (test_diffusion_routes,
test_diffusion_backend, test_inference_model_validation,
test_models_get_model_config_case_resolution, test_data_recipe_seed,
test_training_raw_support, test_export_log_cursor). Frontend
typecheck passes.
2026-05-25 15:30:58 +00:00
pre-commit-ci[bot]
91e3a281d8 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 15:11:34 +00:00
Daniel Han-Chen
3b60d40f92 Fix/adjust diffusion: round 30 P1 + P2 batch for PR #5754
Four actionable findings from round 30. Skipped P1 #1 / #2 / #3
(huggingface-hub bump in studio.txt / single-env / colab-new) because
the live B200 Studio that successfully generated FLUX.2 klein images
runs the exact combo the reviewer flags as broken:
    huggingface_hub 0.36.2 + transformers 4.57.6 + diffusers 0.37.1
    Flux2KleinPipeline: True (imports cleanly)
The is_offline_mode ImportError only fires with transformers 5.x, and
the standard install path pins transformers==4.57.6 via constraints.
The round 26 fix bumped no-torch-runtime.txt + pyproject huggingfacenotorch
where the --no-deps install path can land on transformers 5.x; that
remains the correct surface.

1. core/inference/diffusion.py: preflight transformers + accelerate
   via importlib.util.find_spec BEFORE any destructive GPU-owner
   unload. Diffusers can expose stub pipeline classes when
   transformers / accelerate are missing, so the load used to drop
   chat first and fail later inside from_pretrained. find_spec
   keeps existing tests that stub these modules passing because no
   real module is executed (round 30 P1 #11).

2. models/export.py ExportGGUFRequest.quantization_method: extend
   the embedded HF token validator to this field too. Round 23
   added the control-char guard but not the token guard; the value
   is forwarded into worker command lines and reflected in error /
   success text (round 30 P1 #5).

3. models/data_recipe.py SeedInspectUploadRequest: add
   _no_control_chars + _reject_embedded_hf_token field_validators
   to filename and to each entry of file_names. Mirrors the sibling
   SeedInspectRequest.dataset_name hardening (round 30 P1 #6).

4. frontend/src/features/images/images-page.tsx: defer the initial
   refreshStatus() call via queueMicrotask so the synchronous
   setRefreshingStatus(true) inside it does not trip the
   react-hooks/set-state-in-effect lint on mount (round 30 P2 #12).

Deferred (need larger surgery / out of scope for this round):
   P1 #4 native_path_lease for diffusion local-path loads
   P1 #7-#10 helper/advisor + public-start window mutual lock symmetry

Tests: 98 targeted (diffusion + cached_gguf + inference_validation)
pass locally; frontend npm run typecheck passes.
2026-05-25 15:11:05 +00:00
pre-commit-ci[bot]
b8152a5cec [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 14:45:52 +00:00
Daniel Han-Chen
bec81b882d Fix/adjust diffusion: round 29 P1 + P2 batch for PR #5754
Five actionable findings from round 29 reviewer aggregate, plus an
origin/main merge that absorbs the chat_templates.py fix landed in
PR #5763. Skipped #4 / #5 (studio.txt + constraints.txt hub bump)
because CI evidence from round 26 contradicts that suggestion; the
real broken combo only happens via the --no-deps no-torch path
which is already bumped in no-torch-runtime.txt + pyproject.toml.

1. core/inference/diffusion.py: round 28 reordered
   _release_chat_backend_for_diffusion BEFORE
   _release_other_gpu_owners_for_diffusion to surface the helper /
   advisor busy check early, but that meant the chat unload inside
   _release_chat_backend_for_diffusion now fired before the
   training / export conflict check in the second helper. A direct
   backend caller (tests, scripts) or a route-precheck race with a
   newly-started training run would then unload the user's chat and
   then 409 with nothing loaded. Split the helper busy check into
   _raise_if_helper_advisor_busy_for_diffusion (cheap, no side
   effects), keep _release_chat_backend_for_diffusion as the
   actual chat unload with an opt-out flag, and reorder load_model
   to: (a) helper check, (b) training / export check + idle export
   shutdown, (c) chat unload. All raises now fire BEFORE any
   destructive unload.

2. Merge origin/main: absorbs af6504f9 (PR #5763
   chat_templates.py find() guards + the new
   tests/python/test_construct_chat_template_validation.py
   regression test). Removes the 101-line stale-rebase silent
   revert that round 29 reviewer 5 and 8 flagged.

3. frontend/src/features/images/images-page.tsx: supportsNegativePrompt
   now also honours customFamily when no model is loaded yet, so a
   Custom HF repo with family flux.2 / flux.2-klein correctly hides
   the negative prompt field instead of silently sending it.

4. routes/inference.py /images/generate: report the ACTUAL PNG
   width / height from PIL Image.size instead of echoing back the
   requested payload values. FLUX-family pipelines round to
   vae_scale_factor * 2, so a request for 520x520 lands as 512x512
   internally; metadata now matches the bytes on the wire.

Tests: 98 targeted (diffusion + cached_gguf + inference_validation)
and frontend npm run typecheck pass locally.
2026-05-25 14:45:15 +00:00
Daniel Han-Chen
1f5f13c986 Merge remote-tracking branch 'origin/main' into studio-diffusion-images 2026-05-25 14:43:31 +00:00
pre-commit-ci[bot]
760bd38dda [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 14:22:45 +00:00
Daniel Han-Chen
c4c9e2aeec Fix/adjust diffusion: round 28 P1 + P2 batch for PR #5754
Twelve actionable P1/P2 findings from round 28 reviewer aggregate.
Skipped #3 (studio.txt huggingface-hub bump) because the empirical
CI evidence in round 26 contradicts that suggestion: bumping the
pin there breaks installs that apply constraints.txt
(transformers==4.57.6 requires hub<1.0). The actual broken combo
only happens via the --no-deps no-torch path which is already
bumped in no-torch-runtime.txt and pyproject.toml huggingfacenotorch.

1. utils/datasets/llm_assist.py: split _HELPER_ADVISOR_REFCOUNT
   into CACHE vs GPU counters. helper_advisor_owns_repo (used by
   delete-cache) reads CACHE; helper_advisor_busy (used by public
   handoffs) reads GPU. precache_helper_gguf now registers with
   gpu_owner=False so a background pre-cache download does not
   503 every chat / training / export / diffusion load.

2. utils/datasets/llm_assist.py: introduce _HELPER_ADVISOR_START_LOCK
   and wrap the busy precheck + register pair in _run_with_helper
   and _run_multi_pass_advisor. Two concurrent helper / advisor
   invocations could both pass _gpu_workload_busy_for_helper before
   either registered, then OOM each other.

3. utils/datasets/llm_assist.py: _gpu_workload_busy_for_helper now
   also returns True when another helper/advisor already holds the
   private LlamaCppBackend.

4. routes/inference.py: add _raise_if_helper_advisor_busy(workload)
   that 503s when AI Assist owns the GPU. Wire it into both chat
   load branches (GGUF + safetensors) BEFORE the existing
   _release_export_for / _release_diffusion_for calls so we do not
   first tear down an idle export / diffusion just to fail on the
   helper check.

5. routes/training.py + routes/export.py + diffusion.load_model:
   call the helper-busy check FIRST before any release helper
   fires. Mirrors the chat-load ordering.

6. routes/inference.py _release_llama_for: poll
   loading_model_identifier for up to 5 s after unload_model() so a
   cancelled pending GGUF download has time to clear its
   identifier. Mirrors the same wait round 26 added to the explicit
   /api/inference/unload route.

7. core/inference/diffusion.py _release_chat_backend_for_diffusion:
   same 5 s settling wait for cancelled pending GGUF downloads.

8. models/inference.py LoadRequest: validate every llama_extra_args
   entry through _no_control_chars + _reject_embedded_hf_token.
   The list was forwarded verbatim to a logged llama-server command
   line, so a smuggled control char or hf_... token would land in
   logs and subprocess args.

9. routes/models.py /gguf-download-progress: apply
   _validate_logged_identifier to repo_id and variant, matching the
   round 24 hardening on the adjacent generic /download-progress.

10. routes/inference.py diffusion-load RuntimeError classifier:
    treat "AI Assist ..." messages as retryable 503 instead of 400
    (round 28 P2 #15). Mirrors the round 18/19 markers for chat
    unload failures.

Tests: 105 targeted + 1768 broader backend tests pass locally.
2026-05-25 14:21:37 +00:00
Daniel Han-Chen
79da5d910d Fix/adjust diffusion: round 27 follow-up P1 batch for PR #5754
Five additional P1 findings round 27 reviewer flagged on top of the
round 27 commit 6c528fb0 (Counter refcount + handoff visibility were
already covered). Three remaining studio.txt / no-torch-runtime hub
suggestions are NOT applied because they would re-break CI; the
empirical evidence (round 26 commit 65ea3a2c restored CI green) takes
precedence over the reviewer's stale-state suggestion.

1. models/training.py TrainingStartRequest: extend the embedded HF
   token validator to subset, train_split, eval_split. Round 26 only
   added the control-char guard to those three; the token guard was
   asymmetric and would accept owner/data\\nFAKE hf_abcdef...
   payloads through subset / split fields.

2. models/datasets.py CheckFormatRequest: extend both validators
   (control chars + embedded HF token) to subset and train_split.
   Same asymmetric-fix bug as #1.

3. models/data_recipe.py SeedInspectRequest: extend both validators
   to subset and split. Same pattern.

4. utils/datasets/llm_assist.py precache_helper_gguf: register the
   helper repo in the helper/advisor refcount registry around the
   hf_hub_download loop, then unregister in the finally. Without
   this, the FastAPI-startup background pre-cache could be racing
   a concurrent DELETE /api/models/delete-cached against the same
   cache directory. The runtime helper / advisor calls already
   register (round 26 P1 #13/#14) but the precache was the
   asymmetric gap.

5. routes/models.py _loaded_model_matches_deleted_path: match
   bidirectionally (active under target OR target under active) so
   deleting a child directory of a loaded local model (.../my-flux/
   text_encoder while .../my-flux is loaded) trips the guard.
   Mirrors the diffusion delete-guard symmetric path-overlap check.

Tests: 105 targeted (diffusion + cache + inference_validation) and
the broader backend suite pass locally.
2026-05-25 13:43:51 +00:00
Daniel Han-Chen
6c528fb013 Fix/adjust diffusion: round 27 P1 + P2 batch for PR #5754
Round 27 findings (Opus parallel concurrency + frontend reviews).

Backend P1 fixes:

1. utils/datasets/llm_assist.py: the round 26 helper/advisor active
   registry used a plain set, so two concurrent helper / advisor
   loads of the same DEFAULT_HELPER_MODEL_REPO would both
   set.add() (no-op the second time) and then the first finally
   set.discard() would underflow the registration while the second
   call was still mmap'ing the GGUF. Switch to a Counter with
   proper refcount increment/decrement so the repo stays registered
   until the last user releases it.

2. routes/inference.py _release_chat_for and
   core/inference/diffusion.py _release_chat_backend_for_diffusion:
   helper/advisor GGUF runs on a PRIVATE LlamaCppBackend (round 26
   P1 #1), so the global llama checks below could not see them.
   A user-driven /training/start, /export/load-checkpoint, or
   /images/load would skip the unload and allocate FLUX VRAM on top
   of the helper's resident weights, OOMing on 16-24 GB consumer
   GPUs. Both release paths now consult helper_advisor_busy() and
   fail 503 (or RuntimeError for the in-backend path) so the user
   retries instead of double-owning VRAM.

Frontend P2 fixes:

3. studio/frontend/src/features/images/images-page.tsx: handleUnload
   now calls refreshStatus() in the catch path so a partial unload
   (503 from the backend) does not leave the UI showing a stale
   "Loaded:" label. Matches the handleLoad pattern.

4. images-page.tsx: when status.is_loading is true, auto-poll
   refreshStatus every 2 s so the user sees real progress instead
   of a frozen "Loading..." label until they manually click Refresh.

5. images-page.tsx: aria-label="Inference steps" / "Guidance scale"
   on the two sliders so screen readers can announce them.

6. images-page.tsx: defensive (r.guidance_scale ?? 0).toFixed(1)
   in the results caption so a future backend that serialises
   NaN/None for guidance does not throw at render.

Tests: 105 targeted (diffusion + cached_gguf + inference_validation)
and 1768 broader backend tests pass locally. Frontend
`npm run typecheck` passes.
2026-05-25 13:39:40 +00:00
pre-commit-ci[bot]
e17aea6c81 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 13:13:42 +00:00
Daniel Han-Chen
65ea3a2c81 Fix/adjust diffusion: round 26 P1 batch for PR #5754
Twelve P1 findings from round 26 reviewer aggregate, plus the CI
revert of round 25 P1 #5 to a less invasive location.

1. requirements/studio.txt + requirements/single-env/constraints.txt:
   revert the round 25 huggingface-hub bump (broke Studio Update CI,
   Mac Studio Update CI, Mac Studio UI CI, Studio UI CI all with
   ResolutionImpossible against transformers==4.57.6 which requires
   hub<1.0). Standard install path stays on the well-tested 4.57.6 +
   0.36.2 + trl 0.23.1 trio.

2. requirements/no-torch-runtime.txt + pyproject.toml
   [huggingfacenotorch]: bump huggingface_hub floor from >=0.34.0 to
   >=1.3.0,<2.0 -- this is where the actual transformers 5.x +
   hub 0.36.2 broken combo can land because the file installs
   --no-deps. transformers 5.x calls hub.is_offline_mode which only
   exists in hub 1.x.

3. utils/datasets/llm_assist.py: revert round 25 P1 #4 (helper/advisor
   sharing the global llama backend) which introduced three
   regressions: a chat-evict load race after the busy precheck, a
   finally-block that could unload a user chat model, and an
   identifier mismatch the delete guard could not canonicalize. Go
   back to PRIVATE LlamaCppBackend instances and expose the active
   helper/advisor repos through a new thread-safe registry
   (helper_advisor_owns_repo / _register_helper_advisor_repo /
   _unregister_helper_advisor_repo) so DELETE /api/models/delete-cached
   can still block the rmtree.

4. routes/models.py delete_cached_model: check the new helper/advisor
   registry up front and 409 if a helper/advisor still owns the
   target repo. Closes round 26 P1 #13 and #14 (helper/advisor
   identifiers were prefixed and would never equal the raw repo id).

5. routes/models.py get_lora_base_model: validate lora_path with
   _validate_logged_identifier before it is reflected in 404 detail
   and error logs (round 26 P1 #12).

6. routes/inference.py /unload: round 21 P1 #3 added a "or not
   is_loaded" fallback that let an unload of owner/B cancel a pending
   llama load of owner/A. Replace it with a narrow
   llama_is_starting_without_identifier branch that only fires when
   llama-server is mid-startup with neither identifier set (round 26
   P1 #5).

7. routes/inference.py /unload: poll loading_model_identifier for up
   to 5 s after asyncio.to_thread(unload_model) so a legitimate
   pending-load cancel does not 503 because the load thread has not
   yet observed _cancel_event in its finally (round 26 P2 #15).

8. models/training.py TrainingStartRequest: extend identifier
   hardening to hf_dataset, subset, train_split, eval_split. Round 22
   only guarded model_name (round 26 P1 #10).

9. models/data_recipe.py SeedInspectRequest: add _no_control_chars +
   _reject_embedded_hf_token field_validators on dataset_name (round
   26 P1 #11).

Tests: 105 targeted (diffusion + cached_gguf + llama_cpp_cache +
inference_model_validation + models_get_model_config) and 1768
broader backend tests pass locally. Pre-existing
test_desktop_auth.py, test_studio_api.py, and
test_training_worker_flash_attn.py failures reproduce on HEAD
without these changes.
2026-05-25 13:13:19 +00:00
Daniel Han-Chen
fd7d334d10 Fix studio.txt vs constraints.txt huggingface-hub conflict (PR #5754)
Round 25 P1 #5 bumped studio.txt huggingface-hub to >=1.3.0,<2.0 but

single-env/constraints.txt still pinned ==0.36.2, which made fresh

Studio Update CI / Mac Studio Update CI / Mac Studio UI CI fail at

the studio-deps install step with ResolutionImpossible. Bump the

constraint pin to ==1.8.0 to match the setup.sh / setup.ps1 t5

sub-env pins and satisfy the new studio.txt floor.
2026-05-25 12:40:35 +00:00
pre-commit-ci[bot]
4785f76fa2 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 12:14:59 +00:00
Daniel Han-Chen
7b5fe1cf10 Fix/adjust diffusion: round 25 P1 batch for PR #5754
Five P1 findings from round 25 reviewer aggregate.

1. routes/datasets.py: /download-progress now reuses the same
   identifier hardening that round 24 added to the model route.
   Token-shaped repo_ids like owner/hf_abcdefghij0123456789 used to
   pass the cheap _is_valid_repo_id regex and end up in warning logs.

2. routes/models.py: extend the llama.cpp cache-delete guard to
   path-ownership matching. A GGUF chat model loaded via a local HF
   snapshot path under models--owner--repo/snapshots used to slip
   past the owner/repo string compare and could be rmtree'd while
   llama-server still mmap'd it. Shares one fail-closed HF cache
   scan and an _owned_cache_path_matches helper with the
   safetensors and diffusion guards (round 25 also dedupes the
   diffusion-specific rescan).

3. routes/models.py: extend the safetensors cache-delete guard the
   same way for safetensors models loaded from local snapshot paths.

4. utils/datasets/llm_assist.py: _run_with_helper and
   _run_multi_pass_advisor now acquire the global llama backend via
   routes.inference.get_llama_cpp_backend instead of instantiating
   a private LlamaCppBackend. _gpu_workload_busy_for_helper already
   ensures the global backend is idle on entry, so this is safe, and
   it makes the helper/advisor load visible to the global delete
   guards (loading_model_identifier and friends).

5. requirements/studio.txt: bump huggingface-hub from 0.36.2 pin to
   1.3.0,<2.0 floor and mirror the no-torch-runtime.txt transformers
   and tokenizers constraints. Fresh installs from studio.txt used
   to resolve transformers 5.x with hub 0.36.2, which crashed
   Flux2KleinPipeline import on missing is_offline_mode the first
   time the user hit /api/inference/images/load.

Includes merge of origin/main (PR #5753 install pin bumps and the
mlx export save_method fix from #5727) so the PR diff stops showing
silent reverts of those landed changes.

Tests: PYTHONPATH=studio/backend pytest
test_diffusion_backend.py test_diffusion_routes.py
test_cached_gguf_routes.py test_llama_cpp_cache_aware_disk_check.py
test_inference_model_validation.py
test_models_get_model_config_case_resolution.py
==> 105 passed locally. The 15 flash-attention test failures and
the test_studio_api SDK suite errors reproduce on HEAD without
these changes (pre-existing, unrelated infrastructure).
2026-05-25 12:14:40 +00:00
Daniel Han-Chen
3df9386ac7 Merge remote-tracking branch 'origin/main' into studio-diffusion-images 2026-05-25 12:13:43 +00:00
Daniel Han-Chen
48740c2664 Fix/adjust diffusion: round 24 P1 batch for PR #5754
P1 #1: ``_gpu_workload_busy_for_helper`` in
``utils/datasets/llm_assist.py`` now also gates on the GGUF chat
backend (llama-server) AND the safetensors chat backend. Round 23
extended it to training + export but missed Chat, so a helper /
advisor GGUF could still race a loaded chat model for VRAM.
Both checks fail closed when status is unverifiable.

P1 #2 / #3 / #4 / #5: re-ordered the route-level GPU-handoff
unloads so the diffusion release runs BEFORE the chat releases.
A wedged diffusion unload used to fire AFTER chat was already
gone, so the user lost both on a single failure. Drop chat last
so an earlier failure preserves it. Applied to
``/training/start`` (training.py), ``/export/load`` (export.py),
``/chat/load`` GGUF branch and ``/chat/load`` safetensors branch
(routes/inference.py).

P1 #7 + P2 #13: ``/delete-finetuned`` body now hardens
``model_path`` and ``gguf_variant`` via the shared
``_validate_logged_identifier`` helper, so control characters
and URL-form HF tokens can no longer log-line-smuggle.

P1 #8 + #10: ``/delete-cached`` body hardens ``repo_id`` and
``variant`` the same way.

P1 #9: ``/download-progress`` ``repo_id`` query parameter is
also hardened; the value flows into log lines deep inside
``_get_repo_size_cached`` on lookup failure.

P1 #11: ``CheckFormatRequest.dataset_name`` and
``AiAssistMappingRequest.{dataset_name, model_name}`` in
``models/datasets.py`` now apply the same control-char +
embedded-HF-token validators, matching every other public
request-body model.

All 115 diffusion + training-validation + cached_gguf + export
+ inference model-validation tests pass locally.

(P1 #6 native-path-lease enforcement for diffusion local paths
and P1 #12 React Compiler frontend lint deferred -- both need
focused design / frontend touchups separate from this batch.)
2026-05-25 11:45:14 +00:00
pre-commit-ci[bot]
0a7fe59a37 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 11:21:11 +00:00
Daniel Han-Chen
c6c4378f38 Fix/adjust diffusion: round 23 P1+P2 batch for PR #5754
P1 #1 + #2 + #6: extended the chat / diffusion / training
identifier hardening to every export-side request model.
ExportCommonOptions (parent of ExportMergedModelRequest /
ExportBaseModelRequest / ExportLoRAAdapterRequest) now applies
_no_control_chars and _reject_embedded_hf_token to repo_id and
base_model_id; ExportGGUFRequest gets the same on its repo_id
plus a control-char check on quantization_method; and
LoadCheckpointRequest validates checkpoint_path. Previously
"/api/export/*" accepted newline-smuggled identifiers and
URL-form ``hf_xxxxx`` tokens that flowed into log lines.

P1 #3 + #4: ``_run_with_helper`` and ``_run_multi_pass_advisor``
now use a shared ``_gpu_workload_busy_for_helper`` that gates on
diffusion (round 22 already), training, AND export. The round 22
guard only checked diffusion, so the dataset helper / advisor
could still load llama-server on top of an active training run
or a resident export checkpoint. Each step fails closed
(unverifiable status counts as busy) so the user's primary
workload is preserved.

P1 #5: PublishDatasetRequest in models/data_recipe.py also
applies the identifier hardening to repo_id; the publish path
previously accepted control characters and URL-form tokens.

P1 #7-10: added _validate_logged_identifier helper to
routes/models.py and applied it to the path / query parameter
endpoints that flow into logger.info(...) calls --
``/config/{model_name}``, ``/check-vision/{model_name}``,
``/check-embedding/{model_name}``, ``/gguf-variants``. Mapped
the validator's ValueError to HTTP 422 so the client sees the
same shape as a Pydantic validation failure.

P2 #11 + #12: ``Loading diffusion model %s`` and
``Diffusion load failed for %s`` log lines route ``repo_id`` /
``effective_base`` through ``_display_repo_id`` (collapses
absolute local paths to the leaf, still scrubs HF tokens)
instead of plain ``_redact_hf_tokens``. The error path was
already collapsed in the user-facing 400 / RuntimeError, but
the structured-log lines kept the full path.

All 97 diffusion + training-validation + related tests pass
locally.
2026-05-25 11:20:05 +00:00
Daniel Han-Chen
09c51147a9 Fix/adjust diffusion: round 22 P1+P2 batch for PR #5754
P1 #1: ``TrainingStartRequest.model_name`` now runs the same
control-character and embedded-HF-token validators that the chat
and diffusion request models gained in rounds 5 / 15 / 20 / 21.
``/api/training/start`` previously accepted newline / tab /
control characters and URL-form ``hf_xxxxx`` tokens that flowed
into structured-log sinks via "Loading model %s" lines.

P1 #2: ``_run_with_helper`` in ``utils/datasets/llm_assist.py``
now skips the helper GGUF when the diffusion image backend
reports loaded / loading. The public chat / training / export
routes already do this through ``_release_diffusion_for``, but
this dataset-side helper loaded llama-server directly with no
diffusion guard, so an Images-page allocation would race the
helper for VRAM. New ``_diffusion_image_model_busy`` helper
fails closed (treats status() failure as busy) so the resident
image model is preserved instead of being overwritten.

P1 #3: same ``_diffusion_image_model_busy`` guard added to
``_run_multi_pass_advisor`` (the dataset conversion advisor),
which has the same direct llama.cpp load shape.

P2 #4: the early "Could not infer a diffusion family" RuntimeError
now routes ``repo_id`` through ``_display_repo_id`` before
formatting. A local absolute path that did not match any known
family used to leak the operator's filesystem layout via the 400
response body, last_error, and log line.

All 97 diffusion + training-validation + related tests pass
locally.
2026-05-25 10:56:35 +00:00
pre-commit-ci[bot]
63f3faf022 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 10:33:06 +00:00
Daniel Han-Chen
04bd9b2da5 Fix/adjust diffusion: round 21 P1+P2 batch for PR #5754
P1 #1 + #2: ``LoadRequest._no_embedded_hf_tokens`` and
``ValidateModelRequest._no_embedded_hf_tokens`` now cover
``gguf_variant`` in addition to ``model_path``. A caller could
pass a variant like ``Q4_K_M-hf_xxxxxxxx`` that flowed into
structured log sinks via the GGUF resolver path; the matching
``DiffusionLoadRequest`` validator already covered every string
field, so this restores parity.

P1 #3: ``/api/inference/unload`` now also matches the llama
``loading_model_identifier`` when picking the GGUF branch. A
pending GGUF download (``is_active`` still False,
``loading_model_identifier`` populated) used to fall through to
the safetensors branch and respond ``status="unloaded"`` while
llama-server kept downloading.

P1 #4 + #5: the final safetensors-handoff sweeps (route-level
``_release_safetensors_chat_for`` and backend
``_release_chat_backend_for_diffusion``) now check ``active_model_name``
and ``loading_models`` WITHOUT the initial ``owned_names`` filter.
A concurrent ``/load`` that landed AFTER the snapshot was
previously ignored, so a chat model that began loading during the
unload window let training / export / GGUF chat / diffusion start
anyway and race the new chat for VRAM.

P2 #6: added ``_preflight_diffusers_subfolder_config`` and
invoked it for GGUF loads with a transformer class
(``effective_base``, ``"transformer"``). A custom base companion
that had ``model_index.json`` but lacked
``transformer/config.json`` previously passed the round 19
preflight, unloaded chat, then failed inside
``from_single_file``.

P2 #7: ``_scrub_validation_obj`` in main.py also scrubs string
dict KEYS. Pydantic ``string_type`` errors surface ``input``
verbatim, and a malformed payload like
``{"repo_id": {"hf_xxxxx": "owner/repo"}}`` would otherwise leak
the token through the 422 response body.

All 85 diffusion-relevant + 35 model-validation tests pass
locally. Existing fakes for ``hf_hub_download`` updated to
accept the new ``subfolder=`` kwarg the round 21 preflight uses.

(P1 #3 cross-workload GPU handoff lock from round 20 is still
deferred; round 21's P1 #4 / #5 raised the sweep-level guarantee,
which closes the most common race without the deadlock risk of
holding a process-wide lock across the entire load.)
2026-05-25 10:32:43 +00:00
Daniel Han-Chen
ff3bad37fe Fix/adjust diffusion: round 20 P1+P2 batch for PR #5754
P1 #1: ``_preflight_full_diffusers_repo(effective_base, hf_token)``
now runs for every load mode, including the GGUF-with-auto-base
path. Round 19 only preflighted the full repo or an explicit
``base_repo``, so an auto-picked companion that turned out to be
gated / private / missing still unloaded the user's chat model
before ``from_pretrained`` failed. ``effective_base`` is the same
value that feeds every downstream allocation, so preflighting it
unconditionally catches all three modes.

P1 #2: ``diffusers.GGUFQuantizationConfig`` (which imports the
``gguf`` package at construction time) is now built up front,
inside the same try block that surfaces "Re-run Studio setup".
Previously the missing-dependency exception fired AFTER
``_release_other_gpu_owners_for_diffusion`` and
``_release_chat_backend_for_diffusion`` had already taken the
chat / export models down. The downstream from_single_file call
reuses the same ``quant_config`` reference.

P1 #4: ``studio/backend/requirements/studio.txt`` now lists
``diffusers>=0.37.0`` and ``gguf>=0.10.0``. These were only in
the extras files, so fresh standard Studio installs failed on
/images/load with the round 20 P1 #2 dependency error message.

P1 #5: ``LoadRequest``, ``UnloadRequest``, and
``ValidateModelRequest`` now apply the same control-character +
embedded-HF-token validators that ``DiffusionLoadRequest``
already had. /api/inference/load, /api/inference/validate, and
/api/inference/unload used to accept newline / tab / control
characters in ``model_path`` (log-line smuggling) and URL-form
``https://hf_xxxxx@huggingface.co/...`` (credential leak through
structured log sinks).

P2 #6: ``_collapse_local`` in the diffusion load-error scrubber
now resolves relative candidates and adds the absolute form to
the substring set. A relative ``exports/my-flux`` used to leak
``/mnt/disks/.../exports/my-flux/...`` via downstream library
errors because the scrubber only matched the original literal.
Replacement is longest-first so a leaf-only context survives.

All 85 diffusion-relevant + 35 related model-validation tests
pass locally.

(P1 #3 cross-workload GPU handoff lock is deferred: deserves a
focused design pass across /images/load, /chat/load (both
branches), /training/start, and /export/load to pick a lock
boundary that does not deadlock against the backend load locks
or stall the SSE log stream.)
2026-05-25 10:07:41 +00:00
pre-commit-ci[bot]
c520a473ae [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 09:46:16 +00:00
Daniel Han-Chen
c20ed25ec6 Fix/adjust diffusion: round 19 P1+P2 batch for PR #5754
P1 #1: ``_release_safetensors_chat_for`` now re-reads
``active_model_name`` and ``loading_models`` after each unload AND
runs a final sweep against the initial owned-name set. The previous
helper trusted ``unload_model() -> True`` even though the
orchestrator can respond ``unloaded`` while still holding weights
or a concurrent ``load`` can repopulate the tracker between calls.
Per-name and global post-state mismatches now raise HTTP 503 so
the caller retries.

P1 #2: same post-state guarantee inside
``_release_chat_backend_for_diffusion`` for direct backend
callers. ``DiffusionBackend.load_model`` now raises RuntimeError
when the safetensors tracker still owns a previously-resident
name after the unload, matching the route-level helper. The route
layer's existing classifier maps the new wording to HTTP 503.

P1 #3: ``DiffusionBackend.load_model`` now preflights the full
diffusers repo (or explicit GGUF ``base_repo``) via
``hf_hub_download(filename="model_index.json")`` BEFORE the
chat / export unload runs. The GGUF path was already covered by
the existing ``hf_hub_download(gguf_filename)`` round-trip; the
full-repo path used to skip validation and let a typo / private /
gated repo only surface inside ``from_pretrained`` AFTER the
user's chat model was already dropped. Local paths are checked
structurally (must be a directory containing ``model_index.json``)
so we do not network-round-trip for an on-disk miss. Error
messages route through ``_display_repo_id`` so an absolute
filesystem path does not leak the operator's layout.

P1 #6: ``/api/inference/unload`` (the direct chat unload endpoint)
now treats ``unload_model() -> False`` AND a leftover state
(``is_loaded`` / ``is_active`` / ``loading_model_identifier`` for
GGUF, ``active_model_name`` / ``loading_models`` for safetensors)
as 503 instead of unconditionally responding
``status="unloaded"``. The UI used to show the model as gone while
the backend still owned VRAM.

P2 #7: extended the /images/load RuntimeError -> HTTPException
marker list with ``still active or loading after unload`` and
``still loading after unload``. Round 18 introduced these exact
phrasings on the backend side; without the extension a retryable
unload failure was returning HTTP 400 to the user instead of 503.

P2 #8: removed the unused ``unsloth_backend = get_inference_backend()``
eager construction in the GGUF chat-load branch. Eager
construction made the GGUF-only path needlessly fail or pay
startup cost when the safetensors backend was unavailable / lazy;
``_release_safetensors_chat_for`` already handles that case as a
no-op.

All 85 diffusion-relevant + 98 related backend tests pass locally.
2026-05-25 09:46:01 +00:00
pre-commit-ci[bot]
369573b784 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 08:44:17 +00:00
Daniel Han-Chen
da27143520 Fix/adjust diffusion: round 18 P1+P2 batch for PR #5754
P1 #1: ``_release_llama_for()`` now verifies ``llama.unload_model``
did not return False AND that ``is_loaded`` / ``is_active`` /
``loading_model_identifier`` are all cleared after the call. The
previous version only treated raised exceptions as failure, so a
subprocess refusing to terminate or an in-flight GGUF download
let the next workload allocate on top.

P1 #2: ``DiffusionBackend._release_other_gpu_owners_for_diffusion``
now raises RuntimeError when ``exp._shutdown_subprocess`` fails on
a settled checkpoint. Direct backend callers used to log at debug
level and proceed toward diffusion allocation while the export
checkpoint still owned VRAM.

P1 #3 + P1 #7: ``/images/load`` no longer drops chat + idle export
before the cheap backend validation runs. ``DiffusionBackend.load_model``
already calls the strict ``_release_other_gpu_owners_for_diffusion``
and ``_release_chat_backend_for_diffusion`` helpers AFTER family
inference and GGUF filename checks pass, so the GPU is still
freed before allocation and a malformed payload no longer
silently unloads the user's chat / chat-export pair.

P1 #4: ``_release_chat_backend_for_diffusion`` now also rejects a
post-unload state where ``loading_model_identifier`` is still set,
matching the route-level ``_release_llama_for`` strictness. A GGUF
download mid-flight before the diffusion handoff used to slip
through and end up double-owning VRAM after diffusion allocated.

P1 #5: ``_release_diffusion_for`` no longer swallows a post-unload
``status()`` failure as ``after = {}``. Training / chat / export
handoffs need proof that the diffusion pipeline released VRAM;
the helper now raises HTTP 503 when the verification status call
itself raises, so the caller retries.

P1 #6: ``DiffusionBackend._release_other_gpu_owners_for_diffusion``
raises RuntimeError when ``get_export_backend()`` itself raises.
Direct backend callers used to silently ``return`` here and
proceed to GPU allocation without being able to verify export
ownership.

P1 #8: ``/training/start`` releases settled export BEFORE chat,
matching the chat-load helpers. If idle export shutdown fails the
user's chat model is preserved instead of being dropped for a
training run that never starts.

P2 #9: GGUF load-error scrubber also collapses ``local_gguf_path``,
the resolved HF cache path passed to
``transformer_cls.from_single_file()``. Without this an exception
like ``OSError: cannot load /home/alice/.cache/huggingface/.../flux.gguf``
would leak the operator's filesystem layout through ``last_error``
and ``/images/status``.

All 85 diffusion-relevant backend tests pass locally.
2026-05-25 08:43:59 +00:00
pre-commit-ci[bot]
72ec67034c [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 08:12:32 +00:00
Daniel Han-Chen
e2f41e4069 Fix/adjust diffusion: round 17 P1+P2 batch for PR #5754
P1: route-layer chat/diffusion/export releases were still
asymmetric. Training start and export load called
``diff_backend.unload_model`` inside a best-effort try/except so a
wedged diffusion backend let the next workload allocate over the
top of the resident pipeline and OOM. Both now use the strict
``_release_diffusion_for`` helper from routes.inference, which
raises HTTPException 503 on status/unload failure or post-check
mismatch.

P2 #9: diffusion load exceptions can include the absolute local
repo / base / gguf path verbatim (FileNotFoundError, OSError from
diffusers / safetensors). The path flows into ``_last_error``,
which ``status()`` returns to every authenticated session. Collapse
the known repo_id / effective_base / gguf_filename paths to their
leaf name before storing the error, mirroring the
``_display_repo_id`` convention used for the public repo label.

P2 #10: when ``repo_id`` is an absolute local path,
``detect_family`` matched _FAMILY_EXCLUDE deny lists against the
full path, so models stored under a parent directory containing
``qwen-image-edit`` or ``3.5`` were misclassified as None. Reduce
the family-detection needle to the leaf directory when the input
looks like a filesystem path; Hub-style ``owner/repo`` ids
continue to use the original needle so existing detection rules
keep working.

P2 #12: ``gguf_filename`` was missing from the
``_reject_embedded_hf_token`` validator. A URL-form quant path
like ``https://hf_xxxxx@huggingface.co/.../flux.gguf`` would be
stored on ``DiffusionBackend._gguf_filename`` and surface in
status() / log lines. Extend the validator to gguf_filename so the
token is dropped before it can leak.

All 85 diffusion-relevant backend tests pass locally.
2026-05-25 08:11:42 +00:00
Daniel Han-Chen
6ac67571dd Fix Windows test failures from round 16 changes for PR #5754
Two diffusion tests broke on the Windows runner after round 16:

- test_display_repo_id_collapses_absolute_path used hardcoded
  POSIX absolute paths; Windows reads /home/... as drive-
  relative so Path.is_absolute() returns False. Use pytest's
  tmp_path so the path is platform-correct.
- test_load_publishes_pending_target_during_loading regressed
  because round 16 moved _release_other_gpu_owners_for_diffusion
  ahead of the chat unload. That helper imports core.training and
  core.export; on Windows CI the import resolved to a real but
  partially configured backend, which raised inside the new
  status-verification path and aborted the load before
  from_pretrained ran. Stub both modules with idle backends in
  _install_fake_diffusers.

Also updated test_public_status_does_not_leak_local_path_via
_active_fields and test_generate_image_with_metadata_redacts_
local_path to use tmp_path for the same Windows reason.
2026-05-25 07:36:26 +00:00
pre-commit-ci[bot]
e948a9601c [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 07:32:13 +00:00
Daniel Han-Chen
2ef9b0e09f Fix/adjust diffusion: round 16 P1+P2 batch for PR #5754
Round 16 reviewer aggregate (logs/review_round16_aggregate.md):

P1 fixes:
- routes/models.py /delete-cached llama guard pairs loading_id with
  loading_hf_variant so deleting a different cached quant (Q8_0)
  while another variant (Q4_K_M) is loading is no longer blocked.
- core/inference/diffusion.py load_model now calls
  _release_other_gpu_owners_for_diffusion BEFORE
  _release_chat_backend_for_diffusion. The other-owners helper
  RAISES on active training/export, so a route -> worker race or
  direct backend caller no longer drops the user's chat model
  before the diffusion load is refused.
- routes/models.py /delete-cached diffusion guard fails CLOSED
  (503) on HF cache scan failure instead of silently falling
  through to repo-id-only matching, which could miss a loaded
  local snapshot path.
- routes/inference.py _release_llama_for and
  _release_safetensors_chat_for now raise 503 on actual unload
  failure (exception or False return), so new GPU workloads do
  not start while the old chat process still owns VRAM.
- core/inference/diffusion.py status() now takes
  include_internal=False by default and only exposes the
  guard-facing active_*/pending_* paths when callers opt in. The
  public /api/inference/images/status route gets the redacted
  payload; routes/models.py delete guards pass
  include_internal=True so they still see the raw paths.
- core/inference/diffusion.py generate_image_with_metadata routes
  the response model through _display_repo_id so /images/generate
  cannot echo back an absolute local path.

P2 fixes:
- routes/inference.py /images/load now maps backend "Could not
  verify training/export status" to 503 instead of 409, matching
  the route-level pre-check.
- core/inference/diffusion.py _release_other_gpu_owners_for_diffusion
  raises "Could not verify export status" when the
  is_export_active() probe itself raises, instead of silently
  treating it as active export.
- core/inference/diffusion.py detect_family compares compact family
  spellings (Flux2Klein) against per-token compact strings so
  unsloth/Flux2Klein-GGUF matches the flux.2-klein family without
  matching the embedded substring inside flux.20.
- main.py installs a RequestValidationError handler that scrubs
  hf_xxxxx tokens out of the 422 response body so a rejected
  ``repo_id`` containing a URL-embedded HF token does not echo it
  back to the browser.

Tests:
- 3 new regression cases (Flux2Klein compact alias, public status
  redaction, generate_image_with_metadata redaction).
- All 75 diffusion backend + route tests pass.
2026-05-25 07:31:59 +00:00
pre-commit-ci[bot]
7c8f1eb40d [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 07:15:31 +00:00
Daniel Han-Chen
05184ad15a Fix llama_cpp source-inspection tests for split load_model for PR #5754
Round 15 split LlamaCppBackend.load_model into a thin wrapper that
publishes _loading_model_identifier + _loading_hf_variant under
_serial_load_lock and an inner _load_model_impl_locked body that
actually launches llama-server. The pre-existing source-inspection
regression tests inspected only load_model and broke because the
flag literals and _wait_for_vram_settle call now live in the inner
method:

- tests/test_llama_cpp_no_context_shift.py
  test_no_context_shift_is_in_load_model
  test_flag_sits_inside_the_base_cmd_list
- tests/test_llama_cpp_wait_for_vram_settle.py
  test_load_model_calls_helper_outside_lock_and_uses_last_kill_timestamp

Update both helpers to concatenate the source of load_model AND
_load_model_impl_locked so the assertions still cover the launch
path without weakening their scope to the full module.
2026-05-25 07:15:16 +00:00
pre-commit-ci[bot]
2f9bb6929e [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 07:01:32 +00:00
Daniel Han-Chen
59aa75b8ff Fix/adjust diffusion: round 15 P1+P2+P3 batch for PR #5754
Round 15 reviewer aggregate (logs/review_round15_aggregate.md):

P1 fixes:
- core/inference/llama_cpp.py publishes loading_model_identifier +
  loading_hf_variant AFTER acquiring _serial_load_lock; previously
  a queued second load could overwrite or clear the identifier
  currently in flight, breaking delete-safety and GPU handoff guards.
- routes/models.py /delete-finetuned compares the pending llama
  load against loading_hf_variant (new), not the stale hf_variant
  from the previous loaded model. Without this, a Q4-loaded
  directory loading Q8 would still accept a Q8 delete.
- core/inference/diffusion.py _release_other_gpu_owners_for_diffusion
  now also raises when training is active so direct backend callers
  cannot bypass the route layer's 409 guard. Mirrors the
  export-active check the same helper already enforces.
- routes/models.py /delete-cached diffusion guard compares owned
  diffusion paths against the HF cache root for the target repo
  via _all_hf_cache_scans + _is_path_under. Without this, loading
  from a local models--owner--model/snapshots/<sha> path let the
  cache delete proceed while the snapshot was still mmap'd.
- models/inference.py DiffusionLoadRequest refuses URL-embedded
  hf_xxxxx tokens in repo_id / base_repo at the API boundary, so
  the value never reaches self._repo_id and status() can never
  echo it back to other authenticated sessions.

P2 fixes:
- core/inference/diffusion.py status() routes UI-facing repo_id /
  base_repo through _display_repo_id, which collapses absolute
  local paths to the leaf name (delete guards still see the full
  path via active_*/pending_*).
- routes/inference.py /images/load maps backend RuntimeError that
  reports an export/training conflict to HTTP 409 instead of 400.
- core/inference/diffusion.py detect_family now uses token-boundary
  matching so owner/flux.20-model does not collide with flux.2.

P3 fixes:
- tests/test_diffusion_routes.py drops the partial routes.inference
  module from sys.modules if exec_module() raises, so the real
  ImportError surfaces instead of a misleading AttributeError on
  follow-up tests.

Tests:
- 5 new regression cases (display_repo_id, token-boundary family
  detection, training-active raise from backend helper, embedded HF
  token rejection).
- All 72 diffusion backend + route tests pass.
2026-05-25 07:00:29 +00:00
pre-commit-ci[bot]
a9b3d1a672 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 06:34:24 +00:00
Daniel Han-Chen
e03ed3dd29 Fix/adjust diffusion: round 14 P1+P2 batch for PR #5754
Round 14 reviewer aggregate (logs/review_round14_aggregate.md):

P1 fixes:
- routes/export.py /load-checkpoint now runs the active-export 409
  guard BEFORE the chat / diffusion unloads, so a rejected request
  no longer tears down unrelated GPU state.
- core/inference/llama_cpp.py wraps the WHOLE load_model body in a
  single try/finally that publishes loading_model_identifier across
  download, metadata read, VRAM settle, process spawn, and health
  check. Done via a thin load_model wrapper around the existing
  body (renamed _load_model_impl) to avoid reindenting hundreds of
  lines.
- routes/models.py /delete-finetuned now checks
  loading_model_identifier so a pending HF GGUF download cannot
  have its destination directory rmtree'd before llama-server
  spawns.
- core/inference/diffusion.py stores the original caller-supplied
  gguf_filename (e.g. ``BF16/model.gguf``) in a new self._gguf_filename
  field and exposes it as active_gguf_filename. UI-facing
  gguf_filename still collapses to basename for the panel.
- routes/models.py /delete-cached llama guard now allows safe
  different-variant deletes when hf_variant differs, matching the
  diffusion path's variant-aware behaviour.
- core/inference/diffusion.py tracks self._cpu_offload_enabled and
  forces a CPU torch.Generator when offload is on, so seeded
  generation no longer crashes on CUDA hosts with the default offload
  enabled.

P2 fixes:
- core/inference/diffusion.py detect_family normalises mixed
  separators (``Qwen_Image-Edit-GGUF``, ``Qwen-Image_Edit-GGUF``,
  ``QwenImageEdit-GGUF``) so every Qwen-Image-Edit spelling is
  excluded from the base Qwen-Image family.
- core/inference/diffusion.py logger.info / logger.error in
  load_model run repo_id and effective_base through _redact_hf_tokens
  so URL-embedded ``hf_xxxxx`` tokens never reach structured-log
  sinks.
- core/inference/diffusion.py _release_other_gpu_owners_for_diffusion
  now raises RuntimeError when an export job is active instead of
  logging and continuing, so direct backend callers cannot bypass
  the route layer's 409 guard.
- core/inference/diffusion.py full-diffusers repo / base_repo paths
  expand ``~`` via _expand_existing_local_path so
  ``repo_id="~/models/my-flux"`` no longer falls through to the Hub.

Tests:
- 5 new regression cases (mixed Qwen-Image-Edit separators, token
  redaction, status full-filename, CPU offload generator device,
  staging Windows leaf already-set sanity).
- All 68 diffusion backend + route tests pass.
2026-05-25 06:34:10 +00:00
pre-commit-ci[bot]
f501ab8fc8 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 06:06:03 +00:00
Daniel Han-Chen
54adfdff53 Fix _resolve_local_gguf_child traversal check for Windows for PR #5754
Round 13 follow-up: on Windows Path('/etc/passwd').is_absolute()
returns False because POSIX absolute paths read as drive-relative,
which let the traversal check fall through to resolve(strict=True)
and crash with a raw FileNotFoundError instead of the friendlier
RuntimeError. Add a PurePosixPath check + explicit leading-separator
guard and wrap the resolve() in try/except so a missing path inside
the chosen repo is reported as 'Local repo path does not contain ...'
on every OS.

Pre-existing 59 diffusion backend + route tests still pass; staging
Windows Diffusion CI was failing on this exact case.
2026-05-25 06:03:02 +00:00
pre-commit-ci[bot]
ff98c6d160 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 05:57:51 +00:00
Daniel Han-Chen
ae41bfdbfd Fix/adjust diffusion: round 13 P1+P2 batch for PR #5754
Round 13 reviewer aggregate (logs/review_round13_aggregate.md):

P1 fixes:
- routes/export.py load_checkpoint refuses (409) when an export job
  is currently active, mirroring the chat/diffusion/training handoff
  guards. ``is_export_active`` absence is tolerated for older / mocked
  backends.
- core/inference/diffusion.py local-path GGUF loader now accepts
  relative directories (Studio exports surface as ``exports/my-flux``)
  and confines ``gguf_filename`` to the chosen repo via
  ``_resolve_local_gguf_child``: absolute filenames, ``..`` segments,
  and Windows separators are rejected before any file is opened.
- core/inference/diffusion.py status() exposes ``active_gguf_filename``
  alongside the pending variant so delete guards can pair each owned
  repo with the GGUF variant it actually owns.
- routes/models.py cache delete + finetuned delete adopt a shared
  ``_diffusion_owned_targets`` + ``_variant_delete_is_safe_for_owned_gguf``
  helper. Per-variant deletes during a swap-in-flight cannot remove
  the active variant while the pending variant is loading.
- core/inference/llama_cpp.py publishes ``loading_model_identifier``
  before ``_download_gguf`` starts and clears it in ``finally``. Cache
  delete (routes/models.py) and the cross-workload release helpers
  (routes/inference.py::_release_llama_for and
  diffusion.py::_release_chat_backend_for_diffusion) consult it so a
  multi-GB HF download cannot be rmtree'd or be ignored by /images/load
  while still in flight.

P2 fixes:
- core/inference/diffusion.py adds
  ``generate_image_with_metadata`` + ``async_generate_with_metadata``;
  /images/generate uses it so the response model/family reflect the
  pipeline that actually produced the image even if an unload races
  the route.
- core/inference/diffusion.py: ``base_repo`` only applies when picking
  a GGUF quant. Filling Base diffusers repo while loading a full
  diffusers repo no longer silently swaps the load target.
- core/inference/diffusion.py: failed device placement / offload now
  drops pipe + transformer references explicitly before drain so
  partial allocations cannot keep VRAM around.
- core/inference/diffusion.py: torch/diffusers imports surface as a
  clear RuntimeError naming the missing dependency.
- core/inference/diffusion.py: _smart_base_repo splits on both POSIX
  and Windows separators so ``C:\\Users\\me\\base\\FLUX.2-klein-4B-GGUF``
  no longer picks the Base 4B variant via the parent dir.

Tests:
- 6 new regression cases (Windows leaf, traversal/backslash rejection,
  relative-dir local load, metadata snapshot, lock serialisation).
- All 59 diffusion backend + route tests pass.
2026-05-25 05:57:32 +00:00
pre-commit-ci[bot]
d8b785a4e2 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 05:03:44 +00:00
Daniel Han-Chen
8b8980a607 Fix/adjust diffusion: round 12 local-path GGUF + per-variant delete + MPS + base namespace for PR #5754
Round 12 reviewer findings.

Backend correctness (P1)
  * core/inference/diffusion.py load_model: GGUF branch now
    handles an absolute local directory passed as repo_id by
    joining Path(repo_id) / gguf_filename directly instead of
    handing the path to hf_hub_download (which raises
    HFValidationError because the path is not 'namespace/repo').
    Closes round 12 review #1 -- the load request advertised
    'local path' support but actually only worked for Hub repo ids.

Delete guard precision (P1)
  * routes/models.py /delete-finetuned + /delete-cached:
    diffusion guard now consults gguf_filename from status()
    and ALLOWS per-variant deletes that target a different quant
    than the one the loaded pipeline is reading. Loading
    'Q4_K_S' no longer blocks deleting 'Q8_0' from the same
    repo / export directory (round 12 reviews #3 and #4).

Accelerator (P2)
  * core/inference/diffusion.py _drain_cuda_cache: also calls
    torch.mps.empty_cache() when the MPS backend is the
    active accelerator. Apple Silicon swaps now actually return
    held VRAM instead of leaving it pinned in the Metal
    allocator (round 12 review #10).

Smart base repo (P2)
  * core/inference/diffusion.py _smart_base_repo: only inspects
    the LAST segment of the repo id / path for the 'base' / '9b'
    tokens. A namespace like baseorg/FLUX.2-klein-4B-GGUF or
    a parent directory like /home/me/.cache/base/... no
    longer falsely selects the Base variant (round 12 review #9).
2026-05-25 05:03:29 +00:00
pre-commit-ci[bot]
921c60232e [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 04:33:06 +00:00
Daniel Han-Chen
4b1b149c0b Fix/adjust diffusion: round 11 export-active defense-in-depth + state/path/gguf for PR #5754
Round 11 reviewer findings.

Backend lifecycle (P1)
  * core/inference/diffusion.py _release_other_gpu_owners_for_
    diffusion: now re-checks is_export_active() locally before
    calling _shutdown_subprocess. The route layer already 409s on
    active exports, but defence-in-depth means direct backend
    callers (tests, scripts, future routes that forget the
    higher-level guard) can no longer terminate an in-flight
    export and corrupt the user's partial output.
  * routes/inference.py standard chat-load path: the duplicate
    inline 'if exp_backend.current_checkpoint -> _shutdown_subprocess'
    block was removed. _release_export_for above already handles
    settled checkpoints and skips active ones; the inline block
    was the round 11 #2 asymmetric fix surface.

Routing / error mapping (P2)
  * routes/training.py start_training: except HTTPException:
    raise was inserted before the broad except Exception:
    handler so the 409 raised by _raise_if_training_active /
    _raise_if_export_active reaches the client intact instead of
    being swallowed into a 500.

State publishing (P2)
  * core/inference/diffusion.py load_model: success path now
    clears _loading + _pending_* under _lock BEFORE returning
    self.status(), so the response payload reports the resident
    pipeline cleanly (no stale is_loading=true / pending_*). The
    finally block remains idempotent for error / early-raise paths.
  * core/inference/diffusion.py status(): nulls family /
    pipeline_class while a swap is in flight (pending_repo set
    and != active_repo). Previously the response paired pending
    model B's repo_id with model A's family, producing a
    combination that never existed.

Validation
  * models/inference.py: DiffusionLoadRequest.repo_id and
    base_repo length caps bumped from 256 to 1024; gguf_filename
    bumped from 256 to 512. The earlier caps rejected realistic
    Studio export paths (deeply nested outputs / exports
    directories, especially on Windows).

Dependencies
  * pyproject.toml huggingfacenotorch + studio/backend/
    requirements/no-torch-runtime.txt: floor gguf at >=0.10.0
    to match the diffusers requirement. Unconstrained pin allowed
    a resolver to install older gguf releases that raise at
    single-file load time.
2026-05-25 04:30:10 +00:00
Daniel Han-Chen
1698b66eb1 Fix/adjust diffusion: tolerate ExportBackend without is_export_active for PR #5754
Round 10's training-side _raise_if_export_active call broke
existing test mocks and older ExportBackend builds that only
expose current_checkpoint -- they raised AttributeError on
exp.is_export_active() and the outer guard converted that into
a 503, causing the prior Backend CI to fail
test_inference_route_returns_400_for_invalid_gpu_ids,
test_training_route_returns_400_for_invalid_gpu_ids, and
test_training_route_forwards_embedding_learning_rate.

Both _raise_if_export_active and _release_export_for now detect
the missing method with getattr(...) and treat absence as
'no async-job tracker available' (effectively 'not active').
The 503 fail-closed path still fires when the method exists but
the call itself raises, so production backends (the
ExportOrchestrator subclass that does expose is_export_active)
keep their stronger guard.
2026-05-25 04:17:00 +00:00
Daniel Han-Chen
641cdcc13a Fix/adjust diffusion: round 10 fix export-active asymmetry + GGUF chat helper for PR #5754
Round 10 reviewers found the round 9 export helpers had a
destructive bug: _release_export_for treated is_export_active=True
as a shutdown condition, so any caller (training, chat, diffusion)
could terminate an in-flight export and corrupt the user's output.
Conversely _raise_if_export_active raised 409 on a settled
checkpoint, blocking idle cleanup.

Backend (P1)
  * routes/inference.py: split the export-active surface in two:
      _raise_if_export_active() now ONLY raises when
      is_export_active() is True. A settled current_checkpoint is
      treated as held GPU memory, not an active job.
      _release_export_for() now ONLY shuts down when
      current_checkpoint is set AND is_export_active() is False
      (i.e. a previously completed checkpoint just holding memory).
      An unknown / unverifiable is_export_active is treated as
      'might still be active' so the helper refuses to drop.
  * routes/training.py: now calls _raise_if_export_active before
    _release_chat_for / _release_export_for, mirroring the chat
    and diffusion paths. The previous code went straight to
    _release_export_for and would kill an in-flight export.
  * routes/inference.py: split _release_chat_for into
    _release_llama_for and _release_safetensors_chat_for so the
    GGUF chat-load path can release only the OTHER chat backend
    (round 10 review #4: the previous inline 'if active_model_name'
    check skipped loading_models and let an in-flight safetensors
    load race the new GGUF allocation).
  * routes/inference.py: _raise_if_export_active now fails CLOSED
    (503) when is_export_active() raises, not only when
    get_export_backend() raises. Round 10 review #7.

Dependencies (P1)
  * pyproject.toml huggingfacenotorch extra: pin gguf. The
    Studio Images default curated picker is GGUF-only and
    diffusers.GGUFQuantizationConfig + from_single_file require
    the standalone gguf package at runtime; missing it would 500
    on the first /api/inference/images/load with
    'gguf>=0.10.0 is required'.
2026-05-25 03:56:50 +00:00
pre-commit-ci[bot]
b34fc6258f [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 03:21:30 +00:00
Daniel Han-Chen
1193c8144a Fix/adjust diffusion: round 9 shared release helpers + export-active guard for PR #5754
Round 9 reviewer flagged a pile of handoff asymmetries: every
GPU-owning lifecycle change (training, export, chat, images) needed
its own bespoke unload sequence and they had drifted out of sync.
Some skipped llama-server is_active; some missed safetensors
loading_models; export and training did not check is_export_active.

Backend handoff (P1)
  * routes/inference.py: new _release_chat_for / _release_export_for
    helpers. Both treat llama-server as held when is_loaded OR
    is_active, safetensors as held when active_model_name OR
    loading_models is non-empty, and export as held when
    current_checkpoint OR is_export_active. Both helpers run their
    unloads in worker threads so async routes do not block the
    event loop.
  * routes/training.py: replaces its bespoke inline llama / safe /
    export unload sequence with await _release_chat_for / _release_
    export_for.
  * routes/export.py: same swap for the chat unload chain (export
    still does NOT call _release_export_for on itself).
  * routes/inference.py GGUF + standard chat-load paths: now use
    _release_export_for to drop a settled export, and the standard
    path's llama unload now also handles is_active=True (round 9
    review #8).

Backend reject-on-active export (P1 #5)
  * routes/inference.py: new _raise_if_export_active. Symmetric
    with _raise_if_training_active: a long-running export is
    refused with HTTP 409 instead of being silently killed when
    /images/load or /load arrives. Diffusion / images load and
    both chat-load paths call it.
  * core/inference/diffusion.py _release_other_gpu_owners_for_
    diffusion: no longer tears down an in-flight export job. Only
    drops a SETTLED export checkpoint (current_checkpoint
    populated, is_export_active False). Round 9 review #5 -- the
    previous behavior could terminate an in-flight export and
    leave a partial output artifact.

Token leak via logger.exception (P1 #6)
  * core/inference/diffusion.py: load-failure logging now uses
    logger.error(..., exc_msg) with the already-scrubbed string
    and exc_info=False. logger.exception() with the raw Exception
    would expose any hf_... token that diffusers / huggingface_hub
    embedded in the message or traceback locals, defeating the
    earlier in-flight scrub.

Dependency pinning (P1 #11)
  * pyproject.toml: huggingfacenotorch optional extra now pins
    diffusers>=0.37.0. Previously the floor was only set in
    studio/backend/requirements/no-torch-runtime.txt, so a normal
    pip install would resolve diffusers 0.36.0 (no
    Flux2KleinPipeline) and the default curated FLUX.2 klein
    Images model would fail at runtime.

Cache-delete exact match (P1 #14)
  * routes/models.py /delete-cached: llama.cpp and safetensors
    guards now match on exact repo-id (case-insensitive) instead
    of prefix. Diffusion guard already does this; the chat guards
    were the remaining surface where loading org/model-v2
    blocked deleting org/model.
2026-05-25 03:20:59 +00:00
pre-commit-ci[bot]
0ae20554dc [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 02:48:14 +00:00
Daniel Han-Chen
c1f9aac510 Fix/adjust diffusion: round 8 async unloads + tighter handoffs for PR #5754
Round 8 reviewer surfaced event-loop stalls (blocking unload from
async routes), incomplete VRAM handoff coverage (is_active /
loading_models / is_export_active not checked), token leaks via
exception messages, /v1 exposure, and several fail-open paths.

Async / event-loop
  * routes/inference.py /images/unload, GGUF chat-load handoff,
    safetensors chat-load handoff: blocking DiffusionBackend.unload
    pushed onto asyncio.to_thread. unload takes _load_lock +
    _generate_lock and can block for the full duration of an
    in-flight load / generation, which was freezing the FastAPI
    worker, SSE stream, and hardware poller for minutes.
  * routes/export.py + routes/training.py: same to_thread wrap on
    diffusion unload during checkpoint / training start.

GPU-owner handoff completeness
  * core/inference/diffusion.py _release_chat_backend_for_diffusion:
    llama-server now also unloaded when is_active=True (mid-download
    / startup), not only when is_loaded; flushed in-flight
    safetensors loads from loading_models too.
  * core/inference/diffusion.py _release_other_gpu_owners_for_
    diffusion: export shutdown now also fires when
    is_export_active() returns True (checkpoint not yet assigned).

Security / scrubbing
  * core/inference/diffusion.py: load failure paths now scrub
    hf_token from both _last_error AND the raised RuntimeError
    message (the previous scrub only cleared frame locals).
    Falls back to a regex strip of hf_[A-Za-z0-9]{20,} to
    catch tokens that came in via huggingface_hub default caching.
  * routes/inference.py: image lifecycle endpoints moved from
    router to studio_router so they no longer answer under
    the /v1 OpenAI-compat prefix. Studio-only side effects
    (download multi-GB GGUFs, unload chat, etc.) should not be
    reachable via an OpenAI-compat client.
  * models/inference.py: control-char validator now also rejects
    tab. Some log sinks split fields on tab; allowing it left a
    log-injection surface.

Fail-closed delete guards
  * routes/models.py /delete-cached: llama.cpp and safetensors
    branches now fail closed with 503 when their status check
    raises (matches the diffusion-side guard added earlier).
  * routes/export.py: split the try/except around the training
    backend so import failure falls back to 'skip' (no
    core.training in this build) while a runtime failure of
    get_training_backend()/is_training_active() fails closed.
  * routes/models.py /delete-finetuned: diffusion guard now also
    compares against relative path candidates (Path.resolve() works
    on relative input). Previously a load with a relative repo_id
    bypassed the guard.

CUDA cleanup ordering
  * core/inference/diffusion.py: split _release() (drops local +
    gc.collect) from _drain_cuda_cache() (torch.cuda.empty_cache).
    Callers now drain AFTER nulling every reference so the
    allocator actually reclaims the freed slabs (previously
    empty_cache ran while caller still held a local, which left
    the cache pinned).

Generate response (P2 #16)
  * routes/inference.py: response uses status()['active_repo_id']
    instead of the UI-facing repo_id, so a queued /images/load
    promoting a pending model cannot mislabel the just-rendered
    image with the new model's identity.

Test wiring
  * tests/test_diffusion_routes.py: mount inf.studio_router on the
    test app so /images/* routes are reachable now that they live
    on the Studio-only router.
2026-05-25 02:48:00 +00:00
pre-commit-ci[bot]
92eccc3627 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 02:08:31 +00:00
Daniel Han-Chen
fa8efafcd8 Fix/adjust diffusion: round 7 swap-aware guards + race-free generate for PR #5754
Round 7 reviewer surfaced a handful of swap-window races, fail-open
guards, and seed precision mismatches. This commit closes them.

Lifecycle / state (P1)
  * core/inference/diffusion.py: status() now emits active_repo_id,
    active_base_repo, pending_repo_id, pending_base_repo, and
    pending_gguf_filename alongside the existing UI-facing fields.
    During a swap (model A loaded, model B loading) the previous
    coalesced 'repo_id or pending_repo_id' hid the loading target
    from delete guards. Splitting the fields lets guards block
    deletion of either repo currently owned by the backend.
  * core/inference/diffusion.py: generate_image() now takes
    _generate_lock BEFORE snapshotting _pipe / _device. Snapshotting
    outside the lock let a concurrent unload/load clear or replace
    the backend between the snapshot and the forward, so the freed
    or swapped pipeline would still run.

Symmetric handoffs (P1)
  * routes/export.py: training-active check now runs BEFORE the
    chat / inference / diffusion unload helpers, so a 409 does not
    leave the user's chat session torn down for nothing. Also
    explicitly fails CLOSED with 503 when is_training_active()
    raises.
  * routes/inference.py: _raise_if_training_active now fails closed
    with 503 when the training backend is importable but its status
    check raises. The previous best-effort log-and-continue could
    let chat / diffusion loads collide with unverifiable training.

Delete guards (P1)
  * routes/models.py /delete-cached: chat guard now also blocks
    when llama-server is_active (i.e. mid-download) and when the
    inference backend's loading_models set contains the target.
    Round 7 review #7 flagged that the PR's diffusion-side loading
    guard had no chat-side parallel, so deleting a chat repo while
    it was downloading could still race the cache.
  * routes/models.py /delete-cached: diffusion guard iterates the
    new active_* + pending_* status fields so a delete during a
    swap is refused on either repo.
  * routes/models.py /delete-finetuned: same active_+ pending
    handling, plus the guard now also refuses deletes of a parent
    directory that contains the loaded pipeline (round 7 review #6:
    rm -rf /exports/flux-model/ could unlink model_index.json that
    the live pipeline is reading via mmap).

Seed precision (P2)
  * models/inference.py + routes/inference.py: DiffusionGenerate-
    Response now carries seed_str alongside the existing numeric
    seed. Seeds above Number.MAX_SAFE_INTEGER are rounded by
    JSON.parse in the browser; seed_str ships full decimal
    precision for display and reproduction.
  * frontend/api.ts: DiffusionGenerateResponse types seed_str;
    images-page.tsx prefers seed_str over seed in the figure
    caption so the displayed value reproduces the image.
  * frontend/api.ts: stringifyWithBigInt no longer regex-replaces
    sentinel strings over the full JSON output. It pulls the seed
    BigInt out, JSON-serialises the remaining payload, and splices
    the seed's decimal digits into the resulting object literal at
    the known position. Avoids the round 7 #10 case where a
    user-supplied prompt equal to '__bigint__:123' was rewritten
    into a JSON integer and rejected as a non-string prompt.

Custom HF repo (P2)
  * frontend/images-page.tsx: custom panel now exposes a 'Base
    diffusers repo' input that maps to DiffusionLoadRequest.
    base_repo. Required when a private / mirrored GGUF needs a
    non-default base (e.g. a 9B Klein transformer would otherwise
    fall back to the 4B base default).
2026-05-25 02:08:18 +00:00
pre-commit-ci[bot]
0fd9e90c45 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 01:28:18 +00:00
Daniel Han-Chen
04de106e49 Fix/adjust diffusion: round 6 race-free lifecycle + delete guards for PR #5754
Round 6 reviewers identified several races between load / unload /
generate and several fail-open delete guards. This commit closes
them by widening the lock scope, publishing the pending load
target through status(), and switching delete guards to
fail-closed.

Lifecycle (P1)
  * core/inference/diffusion.py: load_model now also takes
    _generate_lock. Previous behavior released and reallocated the
    pipeline while a generation forward was still iterating
    denoising steps, corrupting scheduler state and stacking VRAM.
    The forward only briefly touches _lock, so taking it on the
    load path does not introduce a deadlock.
  * core/inference/diffusion.py: unload_model now also takes
    _generate_lock. Without it, /images/unload returned
    is_loaded=False while a slow forward was still running, which
    let chat / training / export handoffs allocate VRAM on top of
    the still-resident pipeline.
  * core/inference/diffusion.py: previous pipeline release now
    happens BEFORE from_single_file / from_pretrained. Switching
    FLUX.2 klein 4B -> 9B on a 16-24 GB GPU was failing because
    the new transformer allocation overlapped the old pipe's
    residency.
  * core/inference/diffusion.py: failed pipeline from_pretrained
    now explicitly releases the just-loaded transformer; previously
    its weights stayed pinned to GPU until GC and made the next
    load more likely to OOM.

Pending-target / delete guards (P1)
  * core/inference/diffusion.py: load_model now publishes
    _pending_repo_id / _pending_base_repo / _pending_gguf_filename
    under _lock at the start of the call (and refreshes
    _pending_base_repo when the smart-base / repo defaults resolve).
    status() exposes those as 'repo_id' / 'base_repo' /
    'gguf_filename' during is_loading=True so delete guards can see
    the target before _repo_id is set on success.
  * routes/models.py /delete-cached + /delete-finetuned: diffusion
    status check now fails CLOSED (HTTP 503) when status() raises.
    Both guards previously logged and continued, which could let a
    delete proceed against a repo whose status was unverifiable.
  * routes/models.py: is_loading is also blocked on both guards
    so a mid-download / mid-from_pretrained rmtree is refused.

Symmetric handoffs (P1)
  * routes/export.py: /load-checkpoint now refuses with HTTP 409
    when training is active instead of calling stop_training().
    Chat and /images/load did the same after round 5; export was
    the remaining asymmetry that would silently kill a long
    training run.
  * routes/training.py, routes/inference.py (GGUF and standard
    chat), routes/export.py: diffusion handoff now treats
    is_loading as is_loaded. The diffusion backend's unload waits
    on _load_lock + _generate_lock so an in-flight load completes
    first.

Requirements (P1)
  * requirements/studio.txt: pin python-multipart explicitly. The
    Studio routes package's eager router imports include
    routes/datasets.py whose FastAPI UploadFile/File validation
    crashes with RuntimeError without it in fresh test envs.

Frontend (P2)
  * features/images/api.ts + images-page.tsx: seed handling now
    accepts the full [-2^63, 2^64 - 1] range via BigInt. The
    previous safe-integer cap rejected valid uint64 seeds the
    backend accepts. A small stringify helper emits BigInts as JSON
    integers without touching the rest of the payload.

Tests
  * test_diffusion_routes.py: load routes/inference.py via
    importlib.spec_from_file_location to avoid triggering
    routes/__init__.py (which would pull in training / datasets /
    data_recipe imports unrelated to diffusion tests).
  * test_diffusion_backend.py: status() during is_loading shows
    pending repo + base; unload waits for in-flight generation.
2026-05-25 01:28:04 +00:00
pre-commit-ci[bot]
8858104b29 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 01:05:47 +00:00
Daniel Han-Chen
f06895b73e Fix/adjust diffusion: round 5 lifecycle + validation hardening for PR #5754
Round 5 reviewer findings, mostly symmetric-lifecycle and input
validation gaps the earlier rounds left open.

Backend lifecycle (P1)
  * routes/training.py: training start now also unloads the GGUF
    llama-server subprocess; was previously only unloading the
    safetensors backend, so starting training while a GGUF chat
    model was loaded kept the subprocess pinned to VRAM.
  * routes/inference.py: new _raise_if_training_active helper. Both
    GGUF and standard chat loads, plus /api/inference/images/load,
    now refuse with HTTP 409 when training is active instead of
    silently stopping training to free VRAM.
  * core/inference/diffusion.py: _release_other_gpu_owners_for_
    diffusion no longer stops active training. The route layer
    refuses the request first, so reaching the helper with training
    live would only happen from programmatic backend calls; better
    to surface OOM than terminate a long training run.
  * core/inference/diffusion.py: BF16 dtype is now gated on
    torch.cuda.is_bf16_supported. Pascal/Turing GPUs report
    is_available()=True but lack BF16 ALUs; FLUX kernels then fail
    inside from_pretrained. Falls back to FP16 instead of refusing.
  * core/inference/diffusion.py: GGUF transformer allocation and
    pipeline allocation now run AFTER releasing chat/export GPU
    owners; previously from_single_file ran first and could OOM
    before the intended VRAM handoff happened.
  * routes/models.py: /delete-cached now also blocks delete when
    diffusion is_loading=True (not just is_loaded); concurrent
    delete during hf_hub_download / from_single_file would have
    raced the rmtree.
  * routes/models.py: /delete-finetuned now also checks the
    diffusion backend before unlinking a Studio outputs/exports
    path. A user who exported a FLUX LoRA locally and loaded it via
    /images/load could previously rmtree the directory the
    diffusion backend was reading from.

Backend correctness / safety (P2)
  * core/inference/diffusion.py: _FAMILY_EXCLUDE for qwen-image now
    also covers qwen_image_edit / qwenimageedit underscore spellings
    so '...qwen_image_edit-GGUF' no longer misdetects as Qwen-Image.
  * core/inference/diffusion.py: detect_family now scans
    _FULL_REPO_FAMILIES in addition to _FAMILIES, so SDXL repos
    (stabilityai/stable-diffusion-xl-base-1.0) are auto-detected
    instead of failing with 'Could not infer a diffusion family'.
  * core/inference/diffusion.py: generate_image now uses a separate
    _generate_lock for the pipeline forward instead of holding
    _lock for the whole call. status() polls and concurrent unload
    requests no longer block for the full minutes-long generation.
  * routes/models.py: diffusion delete guard now uses exact repo-id
    match instead of prefix match; previously loading 'org/model-v2'
    would block deleting unrelated cached 'org/model'.
  * models/inference.py: DiffusionLoadRequest now rejects ASCII
    control characters in repo_id / gguf_filename / base_repo /
    family via field_validator (closes log-injection surface from
    authenticated callers). Also caps lengths at 256 chars.
  * models/inference.py: DiffusionGenerateRequest seed is now
    bounded to the int64/uint64 range; previously a huge seed
    (e.g. 2**100) passed Pydantic then crashed inside
    torch.Generator.manual_seed with 'Overflow when unpacking long
    long'.

Frontend (P2)
  * features/images/images-page.tsx: Custom HF repo panel now
    exposes a Pipeline family override dropdown; previously the
    backend supported it via DiffusionLoadRequest.family but the UI
    had no way to send it, so custom repos whose names did not
    contain a hard-coded substring failed to load.
  * features/images/images-page.tsx: handleLoad now re-fetches
    status on error. The backend clears its old pipeline before
    allocating the replacement; a failed swap previously left the
    UI showing 'Loaded:' with Generate enabled until manual
    refresh.

Tests (10 new)
  * underscore qwen-image-edit exclusion + SDXL full-repo detection
  * BF16 fallback when is_bf16_supported() returns False
  * status() does not block while generate_image holds _generate_lock
  * route layer rejects control chars in repo_id
  * route layer rejects 2**100 seeds (uint64-max boundary accepted)
  * route layer happy-path with negative-prompt true_cfg_scale
    forwarding (Qwen/Flux) and skip-when-no-neg (distilled CFG)
2026-05-25 01:05:27 +00:00
Daniel Han-Chen
f3f3f06dc1 Fix/adjust diffusion: pin requests chain in no-deps runtime for PR #5754
The Studio backend's no-torch-runtime.txt is installed via
pip --no-deps so the diffusion stack's transitive imports must
be pinned explicitly. huggingface_hub's blob downloader (used by
diffusers.GGUFQuantizationConfig and by every from_single_file
call) imports requests + urllib3 + charset_normalizer at module
load time; a fresh --no-deps install would 500 on the first
/api/inference/images/load with PackageNotFoundError: 'requests'.

Adds requests, urllib3, and charset_normalizer to the
transitive-deps block next to the existing httpx chain.
2026-05-25 00:49:14 +00:00
Daniel Han-Chen
fb0a31b9a5 Fix/adjust diffusion: forward true_cfg_scale on Qwen/Flux for negative prompt (PR #5754)
QwenImagePipeline and FluxPipeline treat guidance_scale as the
distilled CFG factor and expose true_cfg_scale as the real
classifier-free guidance knob. Negative prompts only steer the
output when true_cfg_scale > 1, so forwarding only guidance_scale
left Qwen-Image on the default true_cfg_scale=4.0 and the user's
slider value silently ineffective for negative prompts.

When the loaded pipeline accepts both negative_prompt and
true_cfg_scale and the caller supplies a non-empty negative
prompt, forward guidance_scale through both kwargs so the
negative prompt actually steers generation. When no negative
prompt is supplied, true_cfg_scale is left at the model default
to avoid switching distilled CFG models into real-CFG mode (which
would double inference cost and degrade quality).

Adds two regression tests covering the forward-when-negative and
skip-when-no-negative paths.
2026-05-25 00:46:25 +00:00
pre-commit-ci[bot]
ec507c5da7 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 00:36:49 +00:00
Daniel Han-Chen
0f3ed08351 Fix/adjust diffusion: token leak + cache guard + locked status + seed precision for PR #5754
- DiffusionBackend.status() now takes _lock so frontend polling
  cannot observe a torn snapshot mid-swap.
- Scrub hf_token / pipe_kwargs / single_file_kwargs from frame
  locals before logger.exception() so rich tracebacks and structlog
  formatters that render locals do not leak hf_... tokens into logs.
- routes/models.py delete_cached_repo: refuse to delete the cache
  underlying a currently-loaded diffusion pipeline (both the GGUF
  repo and the matching diffusers base_repo). Symmetric with the
  existing chat-load + GGUF guard.
- Frontend seed validation: reject non-integer and out-of-safe-
  integer-range inputs instead of silently rounding via Number(),
  which would otherwise send a different seed than what the user
  typed.
2026-05-25 00:36:35 +00:00
pre-commit-ci[bot]
18b50c1f8a [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 00:34:04 +00:00
Daniel Han-Chen
f44b55c796 Fix/adjust diffusion: clear stale metadata on failed swap for PR #5754
When a swap load fails after the previous pipeline is released,
status() previously reported is_loaded=false on top of the OLD
repo/family/base_repo metadata, which the frontend then rendered
as a misleading 'still loaded: X' label. Clear all metadata
atomically with the pipe drop so a failed swap reports a clean
empty status plus last_error. Add regression test.
2026-05-25 00:33:48 +00:00
pre-commit-ci[bot]
fbf06bec7a [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 00:31:35 +00:00
Daniel Han-Chen
65f7a2680d Fix/adjust diffusion lifecycle for round 3 findings (PR #5754)
- detect_family adds _FAMILY_EXCLUDE so 'stable-diffusion-3.5' no
  longer matches the SD3 Medium family and 'qwen-image-edit' no
  longer matches Qwen-Image. Both were misleading silent loads.
- from_single_file now forwards config=<effective_base>,
  subfolder='transformer', and the HF token. Diffusers-format GGUFs
  (FLUX.2 klein, Qwen-Image, SD3) need the matching base config or
  the transformer load picks the wrong shapes; gated GGUFs need the
  token both for download and config read.
- Move _release_chat_backend_for_diffusion + new
  _release_other_gpu_owners_for_diffusion to AFTER the GGUF download
  and pipeline class lookup so a typo or transient Hub error does
  not kill the user's currently-loaded chat model. Peak VRAM still
  stays at one model's worth because the releases run right before
  from_pretrained.
- _release_other_gpu_owners_for_diffusion: shut down the export
  subprocess and any active training subprocess before a diffusion
  load. Symmetric with the export load path.
- routes/training.py: unload diffusion before starting training so
  the new subprocess does not race FLUX/Qwen for VRAM.
- routes/export.py: also unload the GGUF llama-server before export
  load (the existing inference-backend unload only covered the
  safetensors path).
2026-05-25 00:31:21 +00:00
pre-commit-ci[bot]
0f9b19bb56 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 00:23:02 +00:00
Daniel Han-Chen
1601b7828f Fix safetensors chat backend unload for PR #5754
_release_chat_backend_for_diffusion was importing
get_inference_backend from core.inference.inference (the in-subprocess
class) and calling unload_model() without the required model_name
argument. The TypeError was swallowed and the active chat model
stayed resident, defeating the chat-to-diffusion lifecycle handoff.

Switch to the orchestrator's accessor at core.inference and pass
active_model_name through, mirroring the GGUF chat-load path. Add a
regression test that stubs both backends and verifies unload_model
is called with the active model name.
2026-05-25 00:20:32 +00:00
Daniel Han-Chen
6089720c0c Fix/adjust diffusion: export unload + sd3.5 alias for PR #5754
- routes/export.py load_checkpoint now unloads the diffusion
  pipeline alongside the existing inference + training unloads, so
  an export load after Images does not OOM the export subprocess.
- Remove the 'sd3.5' alias from the stable-diffusion-3 family.
  SD3.5 needs its own family + base_repo (and its own smoke test);
  pairing it with the SD3 Medium base produced a misleading load.
2026-05-25 00:06:24 +00:00
pre-commit-ci[bot]
8c10cf5f16 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-25 00:04:28 +00:00
Daniel Han-Chen
8074a2b67b Fix/adjust diffusion: smart base, safetensors, peak VRAM, GGUF guard
- _smart_base_repo: pick 9B base for unsloth/FLUX.2-klein-9B-GGUF
  and -base- variants per the repo id, instead of always falling
  back to the 4B family default.
- pipe_kwargs use_safetensors=True so diffusers refuses pickle .bin
  weights at load time (defends against compromised base_repo).
- Release the previous pipeline BEFORE allocating the new one so
  peak VRAM stays at one model's worth instead of two on swap.
- Reject empty gguf_filename when repo_id ends with -GGUF; the prior
  behavior tried from_pretrained on a GGUF-only repo and 500'd deep
  in diffusers with a confusing model-index error.
- Status returns gguf_filename (basename) instead of gguf_path so
  the local cache path / username does not leak to authenticated
  Studio sessions.
- requirements/no-torch-runtime.txt: pin diffusers>=0.37.0 so older
  installs cannot resolve a version without Flux2KleinPipeline.
- Frontend curated distilled klein entries now point at the
  matching non-base diffusers repos (FLUX.2-klein-4B / -9B) per
  the published model cards. Update api.ts to mirror the renamed
  status field.
2026-05-25 00:04:12 +00:00
Daniel Han-Chen
faa6822039 Fix/adjust diffusion symmetric chat handoff for PR #5754
_release_chat_backend_for_diffusion now unloads both the GGUF
chat backend (llama-server) and the safetensors / HF chat backend
(get_inference_backend) before a diffusion load. Mirror the
behaviour on the chat-load side: both the Unsloth/transformers
load path and the GGUF load path now unload the diffusion pipeline
before claiming GPU memory. Closes the OOM-on-swap path flagged
by reviewers in both directions.
2026-05-24 23:58:02 +00:00
Daniel Han-Chen
d6f2a238aa Fix/adjust diffusion lifecycle + UI for PR #5754
- unload_model now takes _load_lock so it cannot race with an in-flight
  load_model and have the load thread overwrite cleared state after
  unload returned is_loaded=false.
- Move stable-diffusion-xl out of _FAMILIES into _FULL_REPO_FAMILIES.
  SDXL uses a UNet (no transformer GGUF path is wired); listing it in
  the GGUF families panel was misleading. SDXL full-repo loads still
  work via family_override='stable-diffusion-xl'.
- Result gallery now uses h-auto + object-contain so portrait /
  landscape outputs render at their true aspect ratio instead of
  being cropped into a square thumbnail.
2026-05-24 23:48:57 +00:00
pre-commit-ci[bot]
669964f52c [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-24 23:41:10 +00:00
Daniel Han-Chen
bf5c4ac90b Fix/adjust diffusion review findings for PR #5754
Backend
- Fix FLUX.2 klein family default base_repo: black-forest-labs/FLUX.2-klein
  does not exist on the Hub. Point at the Apache 2.0 4B Base instead so
  the from_pretrained call works out of the box for ungated users.
- Serialise concurrent load_model calls with a dedicated _load_lock so
  two /images/load requests cannot both reach pipeline_cls.from_pretrained
  at the same time (would double-spend VRAM and corrupt _pipe).
- When the caller passes a full diffusers repo (no gguf_filename),
  use repo_id directly instead of silently substituting the family
  default. Closes the load-the-wrong-model regression flagged by review.
- Drop negative_prompt from the pipeline call when the loaded pipeline
  does not accept it (FLUX.2 / FLUX.2 klein). Inspect __call__ via
  inspect.signature so we do not maintain a manual class list.
- Best-effort unload the chat backend (llama-server) before a diffusion
  load so a 24 GB consumer GPU can swap between chat and diffusion
  without manual unload steps.

Frontend
- Replace the four curated entries with the actual filenames published
  on the Hub (lowercase flux-2-klein-Nb-Q4_K_S.gguf and flux2-dev*).
- Add an explicit base_repo per curated entry so the backend never
  falls back to the family default for the curated picker.
- Add the Apache 2.0 FLUX.2 klein base 4B entry so first-time users
  have an ungated, no-token-required default.
- Hide the negative prompt field for FLUX.2 / FLUX.2 klein and show a
  small explanatory note instead.

Tests
- Add 6 new backend tests: base_repo override, full-repo (no GGUF)
  no-substitution, concurrent serialise race, signature-based kwarg
  filter, negative_prompt strip on FLUX.2, negative_prompt preserved
  on supporting pipelines. 33 tests passing.
2026-05-24 23:40:50 +00:00
Daniel Han-Chen
f8504e3f3c Studio: fix Images page SectionCard required icon prop
SectionCard requires an icon prop. Pass GpuIcon, PaintBrush02Icon,
and SparklesIcon for the three sections so tsc -b stops failing on
TS2741 'Property icon is missing'.
2026-05-24 14:38:46 +00:00
pre-commit-ci[bot]
a08686cc46 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-24 14:33:52 +00:00
Daniel Han-Chen
b2b660f76f Studio: add local diffusion image generation page
Backend
- core/inference/diffusion.py: DiffusionBackend singleton that loads
  diffusion GGUFs from Hugging Face via diffusers.GGUFQuantizationConfig
  and runs them on the active CUDA / MPS / CPU device. Supports FLUX.2,
  FLUX.2 klein, FLUX.1, Qwen-Image, Stable Diffusion 3, and SDXL.
- routes/inference.py: POST /api/inference/images/load,
  POST /api/inference/images/generate, POST /api/inference/images/unload,
  GET /api/inference/images/status mirroring the llama-server lifecycle.
- models/inference.py: DiffusionLoadRequest, DiffusionGenerateRequest,
  DiffusionGenerateResponse pydantic schemas with prompt / step / size
  validation up front so callers get clear 422s rather than VAE crashes.
- requirements/no-torch-runtime.txt: pin gguf alongside the existing
  diffusers entry so GGUFQuantizationConfig works out of the box.
- tests/test_diffusion_backend.py + tests/test_diffusion_routes.py:
  27 unit tests covering family detection, validation, lifecycle, and
  the full FastAPI round trip with the backend stubbed. No torch /
  diffusers / GPU required to run.

Frontend
- features/images/: standalone images-page.tsx with curated model picker
  (FLUX.2 klein 4B / 9B, FLUX.2 dev, FLUX.1 dev), HF token field,
  prompt + negative prompt, resolution presets, steps + guidance
  sliders, seed input, and a result gallery that renders base64 PNGs
  inline.
- app/routes/images.tsx: lazy /images route wired into router.tsx.
- components/app-sidebar.tsx: PaintBrush02Icon nav item between
  Recipes and Export, hidden in chat-only mode.
2026-05-24 14:26:07 +00:00
29 changed files with 7836 additions and 172 deletions

View file

@ -81,9 +81,30 @@ huggingfacenotorch = [
"datasets>=3.4.1,!=4.0.*,!=4.1.0,<4.4.0",
"accelerate>=0.34.1",
"peft>=0.18.0,!=0.11.0",
# Round 33 P1: reverted the round-26 hub>=1.3.0 floor. studio.txt
# forces hub==0.36.2 to match the transformers 4.57.6 pin in
# extras-no-deps.txt; the 1.3.0 floor here was internally
# inconsistent and reviewers reproduced the resolver conflict.
# Align with the colab-new extra's 0.34.0 floor (line 610). The
# transformers-5.x is_offline_mode concern that motivated the
# original bump never triggers because transformers is pinned at
# 4.57.6 on the supported install path.
"huggingface_hub>=0.34.0",
"hf_transfer",
"diffusers",
# Studio Images page depends on Flux2KleinPipeline /
# Flux2Pipeline, both shipped in diffusers>=0.37.0. Floor was
# missing here so a `pip install unsloth[huggingfacenotorch]`
# could resolve to 0.36.0 and fail at runtime when the default
# curated FLUX.2 klein model loads.
"diffusers>=0.37.0",
# diffusers.GGUFQuantizationConfig + from_single_file rely on
# the standalone gguf package at runtime. Floor at 0.10.0 to
# match the diffusers requirement; older gguf releases raise
# at load time. Studio Images default curated picker is
# GGUF-only so this must install with the public
# huggingfacenotorch extra; missing / under-pinned it makes
# /api/inference/images/load 500.
"gguf>=0.10.0",
"transformers>=4.51.3,!=4.52.0,!=4.52.1,!=4.52.2,!=4.52.3,!=4.53.0,!=4.54.0,!=4.55.0,!=4.55.1,!=4.57.0,!=4.57.4,!=4.57.5,!=5.0.0,!=5.1.0,<=5.5.0",
"trl>=0.18.2,!=0.19.0,<=0.24.0",
"sentence-transformers",

File diff suppressed because it is too large Load diff

View file

@ -612,6 +612,18 @@ class LlamaCppBackend:
self._process: Optional[subprocess.Popen] = None
self._port: Optional[int] = None
self._model_identifier: Optional[str] = None
# Pending-load identifier: set BEFORE _download_gguf starts and
# cleared after the load finishes (success or failure). Delete
# guards and cross-workload handoff helpers read it via
# ``loading_model_identifier`` so a multi-GB HF download cannot
# have its cache rmtree'd or be ignored by /images/load,
# /training/start, /export/load while it is still resolving.
# ``_loading_hf_variant`` mirrors the same lifetime so the
# per-variant delete guard at routes/models.py:/delete-finetuned
# compares against the NEW variant rather than the previous
# loaded ``hf_variant`` (round 15 P1 #2).
self._loading_model_identifier: Optional[str] = None
self._loading_hf_variant: Optional[str] = None
self._gguf_path: Optional[str] = None
self._hf_repo: Optional[str] = None
self._hf_variant: Optional[str] = None
@ -713,6 +725,33 @@ class LlamaCppBackend:
def model_identifier(self) -> Optional[str]:
return self._model_identifier
@property
def loading_model_identifier(self) -> Optional[str]:
"""Identifier of a load currently in progress, or None.
Populated while ``_download_gguf`` is fetching the GGUF for a
new ``load_model`` call. Cleared in the surrounding
``finally`` block, so a failed load leaves it None. Delete
guards in ``routes/models.py`` and handoff helpers in
``routes/inference.py`` consult this so a long HF download
cannot have its destination rmtree'd or be ignored by a
concurrent /images/load that thinks llama-server is idle."""
return self._loading_model_identifier
@property
def loading_hf_variant(self) -> Optional[str]:
"""``hf_variant`` of the load currently in progress, or None.
Mirrors ``loading_model_identifier``'s lifetime so the
per-variant delete guards (routes/models.py /delete-cached and
/delete-finetuned) can compare against the NEW variant rather
than the previously-loaded one (round 15 P1 #2). Without this,
a directory with Q4 loaded and Q8 loading would still see the
stale Q4 ``hf_variant``, and a Q8 delete would be wrongly
allowed even though Q8 is being downloaded into the same
directory."""
return self._loading_hf_variant
@property
def is_vision(self) -> bool:
return self._is_vision
@ -2599,7 +2638,68 @@ class LlamaCppBackend:
# Serialise the whole load so concurrent /load calls never
# leave two llama-server processes alive (#5401 / #5161). Does
# not block /unload, /status, /load-progress.
#
# Publish ``_loading_model_identifier`` + ``_loading_hf_variant``
# AFTER acquiring ``_serial_load_lock``. Round 15 P1 #1: the
# previous round 14 version set them outside the lock so a
# second queued ``load_model`` would overwrite or clear the
# identifier of the load currently holding the lock, breaking
# the delete-safety and GPU handoff guards. Cleared in
# ``finally`` so failure / cancellation leaves the pending
# state empty. Round 15 P1 #2 added ``_loading_hf_variant``
# so per-variant delete guards can compare against the
# NEW variant rather than the previous loaded one.
with self._serial_load_lock:
self._loading_model_identifier = model_identifier
self._loading_hf_variant = hf_variant
try:
return self._load_model_impl_locked(
gguf_path = gguf_path,
mmproj_path = mmproj_path,
hf_repo = hf_repo,
hf_variant = hf_variant,
hf_token = hf_token,
model_identifier = model_identifier,
is_vision = is_vision,
n_ctx = n_ctx,
chat_template_override = chat_template_override,
cache_type_kv = cache_type_kv,
speculative_type = speculative_type,
spec_draft_n_max = spec_draft_n_max,
n_threads = n_threads,
n_gpu_layers = n_gpu_layers,
n_parallel = n_parallel,
extra_args = extra_args,
)
finally:
self._loading_model_identifier = None
self._loading_hf_variant = None
def _load_model_impl_locked(
self,
*,
gguf_path: Optional[str] = None,
mmproj_path: Optional[str] = None,
hf_repo: Optional[str] = None,
hf_variant: Optional[str] = None,
hf_token: Optional[str] = None,
model_identifier: str,
is_vision: bool = False,
n_ctx: int = 4096,
chat_template_override: Optional[str] = None,
cache_type_kv: Optional[str] = None,
speculative_type: Optional[str] = None,
spec_draft_n_max: Optional[int] = None,
n_threads: Optional[int] = None,
n_gpu_layers: Optional[int] = None,
n_parallel: int = 1,
extra_args: Optional[List[str]] = None,
) -> bool:
"""Internal body of ``load_model``. The caller is responsible
for holding ``_serial_load_lock`` and for publishing /
clearing ``_loading_model_identifier`` + ``_loading_hf_variant``
in the surrounding try/finally."""
if True:
# Duplicate /load that raced past the route-level check
# (the first one hadn't published _healthy=True yet). If the
# live server already satisfies this request, do nothing.

View file

@ -293,6 +293,70 @@ app = FastAPI(
lifespan = lifespan,
)
# ── Validation error scrubber ────────────────────────────────────
# Round 16 P2 #10: FastAPI's default RequestValidationError handler
# echoes the rejected ``input`` value back in the 422 body. A
# request like
# {"repo_id": "https://hf_token@huggingface.co/owner/repo"}
# is rejected by ``DiffusionLoadRequest._no_embedded_hf_tokens``,
# but the rejected URL would still appear in the response payload,
# leaking the token to the browser console / network log. Wrap the
# handler so any ``hf_xxxxx`` substring is replaced with
# ``<redacted>`` before serialisation. Scoped to the response body
# only; the underlying validator behaviour is unchanged.
from fastapi.exceptions import RequestValidationError as _RequestValidationError # noqa: E402
from fastapi.encoders import jsonable_encoder as _jsonable_encoder # noqa: E402
from fastapi.responses import JSONResponse as _JSONResponse # noqa: E402
import re as _re_validation # noqa: E402
_HF_TOKEN_VALIDATION_RE = _re_validation.compile(r"hf_[A-Za-z0-9]{20,}")
def _scrub_validation_obj(value):
"""Recursively scrub ``hf_xxxxx`` tokens out of a value tree.
Pydantic v2 nests raw ``ValueError`` (and other ``BaseException``)
instances under ``ctx.error``. Convert them to scrubbed strings
here; otherwise the default ``JSONResponse`` serializer raises
``TypeError: Object of type ValueError is not JSON serializable``
and the 422 turns into a 500 (round 17 P1 #1). Tuples become
lists so the downstream JSON encoder accepts them.
"""
if isinstance(value, str):
return _HF_TOKEN_VALIDATION_RE.sub("<redacted>", value)
if isinstance(value, BaseException):
return _scrub_validation_obj(str(value))
if isinstance(value, tuple):
return [_scrub_validation_obj(v) for v in value]
if isinstance(value, list):
return [_scrub_validation_obj(v) for v in value]
if isinstance(value, dict):
# Round 21 P2 #7: pydantic surfaces ``input`` for ``string_type``
# validation errors verbatim, including dict KEYS like
# ``{"hf_xxxxx": "owner/repo"}``. Scrub string keys too so the
# token does not leak through the 422 response body.
return {
(
_scrub_validation_obj(k) if isinstance(k, str) else k
): _scrub_validation_obj(v)
for k, v in value.items()
}
return value
@app.exception_handler(_RequestValidationError)
async def _validation_error_scrubbing_handler(request, exc):
# ``jsonable_encoder`` walks the scrubbed payload one more time
# to convert anything else Pydantic v2 surfaces (URL objects,
# Path objects, Url instances, etc.) into JSON-safe primitives.
return _JSONResponse(
status_code = 422,
content = _jsonable_encoder({"detail": _scrub_validation_obj(exc.errors())}),
)
# Initialize structured logging
from loggers.config import LogConfig
from loggers.handlers import LoggingMiddleware

View file

@ -9,7 +9,13 @@ from __future__ import annotations
from typing import Any
from pydantic import BaseModel, Field, model_validator
from pydantic import BaseModel, Field, field_validator, model_validator
# Round 23 P1 #5: identifier hardening reused from the chat models
# so /api/data_recipe/publish rejects control characters and
# URL-form ``hf_xxxxx`` tokens in ``repo_id`` before they reach
# log lines or the HF API.
from models.inference import _no_control_chars, _reject_embedded_hf_token
class RecipePayload(BaseModel):
@ -60,6 +66,16 @@ class PublishDatasetRequest(BaseModel):
description = "Execution artifact path captured by the UI for completed runs",
)
@field_validator("repo_id")
@classmethod
def _no_repo_id_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator("repo_id")
@classmethod
def _no_repo_id_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
class PublishDatasetResponse(BaseModel):
success: bool = True
@ -74,6 +90,20 @@ class SeedInspectRequest(BaseModel):
split: str | None = "train"
preview_size: int = Field(default = 10, ge = 1, le = 50)
# Round 26 P1 #11: dataset_name reaches HF + log/echo paths, so
# mirror the hardening other dataset request models already do.
# Round 27 P1 #7: split and subset also flow into HF dataset
# APIs / errors and must be guarded the same way.
@field_validator("dataset_name", "subset", "split")
@classmethod
def _no_dataset_name_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator("dataset_name", "subset", "split")
@classmethod
def _no_dataset_name_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
class SeedInspectUploadRequest(BaseModel):
# Legacy single-file flow (mutually exclusive with file_ids)
@ -89,6 +119,37 @@ class SeedInspectUploadRequest(BaseModel):
unstructured_chunk_size: int | None = Field(default = None, ge = 1, le = 20000)
unstructured_chunk_overlap: int | None = Field(default = None, ge = 0, le = 20000)
# Round 30 P1 #6: filename / file_names are reflected as dataset
# names + error/log messages; harden them the same way the sibling
# SeedInspectRequest hardens dataset_name.
@field_validator("filename")
@classmethod
def _no_filename_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator("filename")
@classmethod
def _no_filename_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
@field_validator("file_names")
@classmethod
def _no_file_names_control_chars(cls, v):
if v is None:
return v
for i, entry in enumerate(v):
_no_control_chars(entry, f"file_names[{i}]")
return v
@field_validator("file_names")
@classmethod
def _no_file_names_embedded_hf_tokens(cls, v):
if v is None:
return v
for i, entry in enumerate(v):
_reject_embedded_hf_token(entry, f"file_names[{i}]")
return v
@model_validator(mode = "after")
def _check_mutual_exclusivity(self) -> "SeedInspectUploadRequest":
has_legacy = self.content_base64 is not None

View file

@ -7,7 +7,12 @@ Dataset-related Pydantic models for API requests and responses.
from typing import Any, Dict, List, Optional
from pydantic import BaseModel, Field, model_validator
from pydantic import BaseModel, Field, field_validator, model_validator
# Round 24 P1 #11: reuse the chat / diffusion / export identifier
# hardening so dataset routes also reject control characters and
# URL-embedded HF tokens in user-controlled identifiers.
from models.inference import _no_control_chars, _reject_embedded_hf_token
class CheckFormatRequest(BaseModel):
@ -27,6 +32,18 @@ class CheckFormatRequest(BaseModel):
values.setdefault("train_split", values.pop("split"))
return values
# Round 27 P1 #6: subset / train_split also flow into HF dataset
# APIs and errors/responses, so they need the same hardening.
@field_validator("dataset_name", "subset", "train_split")
@classmethod
def _no_dataset_name_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator("dataset_name", "subset", "train_split")
@classmethod
def _no_dataset_name_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
class CheckFormatResponse(BaseModel):
"""Response for dataset format check"""
@ -57,6 +74,16 @@ class AiAssistMappingRequest(BaseModel):
model_name: Optional[str] = None
model_type: Optional[str] = None
@field_validator("dataset_name", "model_name")
@classmethod
def _no_identifier_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator("dataset_name", "model_name")
@classmethod
def _no_identifier_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
class AiAssistMappingResponse(BaseModel):
"""Response from LLM-assisted column classification and conversion advice."""

View file

@ -10,6 +10,13 @@ from pathlib import Path
from pydantic import BaseModel, Field, field_validator
from typing import List, Optional, Literal, Dict, Any
# Round 23 P1 #1 / #2 / #6: reuse the chat identifier validators
# so export requests reject newline / tab / control characters and
# URL-form ``hf_xxxxx`` tokens in any user-supplied identifier
# (Hub ``repo_id``, ``base_model_id``, the local
# ``checkpoint_path``) that flows into log lines or HF API calls.
from models.inference import _no_control_chars, _reject_embedded_hf_token
def _validate_save_directory(value: str) -> str:
"""Reject save_directory values that escape the export root."""
@ -18,9 +25,17 @@ def _validate_save_directory(value: str) -> str:
raw = str(value).strip()
if not raw:
raise ValueError("save_directory must not be empty")
# save_directory is logged verbatim by merged / base / GGUF export
# flows after resolution, so reject embedded HF tokens at the same
# boundary as the sibling identifier fields on export requests.
_reject_embedded_hf_token(raw, "save_directory")
if "\x00" in raw:
raise ValueError("save_directory may not contain null bytes")
if any(ch in raw for ch in ("\r", "\n")):
# Round 32 P1: reject ALL ASCII control characters (including
# TAB / VT / FF) so a caller cannot smuggle log-line breaks or
# subprocess argv splitters past the export worker. The earlier
# CR / LF check missed every other C0 byte.
if any(ord(ch) < 0x20 or ord(ch) == 0x7F for ch in raw):
raise ValueError("save_directory may not contain control characters")
if len(raw) > 255:
raise ValueError("save_directory must be <= 255 characters")
@ -54,6 +69,19 @@ class LoadCheckpointRequest(BaseModel):
description = "Allow loading models with custom code. Only enable for checkpoints/base models you trust.",
)
# Round 23 P1 #6: ``checkpoint_path`` is logged verbatim by the
# export route. Apply the same control-char + embedded-token
# rejection the chat / diffusion / training request models use.
@field_validator("checkpoint_path")
@classmethod
def _no_checkpoint_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator("checkpoint_path")
@classmethod
def _no_checkpoint_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
class ExportStatusResponse(BaseModel):
"""Current export backend status."""
@ -117,6 +145,20 @@ class ExportCommonOptions(BaseModel):
description = "HuggingFace model ID of the base model (for model card metadata)",
)
# Round 23 P1 #1: ``repo_id`` (Hub destination) and
# ``base_model_id`` (model card metadata) both feed log lines
# and the HF API. Reject control characters and URL-form
# ``hf_xxxxx`` tokens before they reach those sinks.
@field_validator("repo_id", "base_model_id")
@classmethod
def _no_identifier_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator("repo_id", "base_model_id")
@classmethod
def _no_identifier_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
class ExportMergedModelRequest(ExportCommonOptions):
"""Request for exporting a merged PEFT model."""
@ -163,6 +205,35 @@ class ExportGGUFRequest(BaseModel):
description = "Hugging Face token for GGUF upload",
)
# Round 23 P1 #2: GGUF export endpoint defines its own
# ``repo_id`` (does not inherit from ExportCommonOptions), so
# the chat-style hardening needs to be applied here separately.
# ``quantization_method`` is forwarded to the export worker
# command line, so it gets the control-char check too even
# though it does not normally carry tokens.
@field_validator("repo_id")
@classmethod
def _no_repo_id_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator("repo_id")
@classmethod
def _no_repo_id_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
@field_validator("quantization_method")
@classmethod
def _no_quantization_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
# Round 30 P1 #5: quantization_method is forwarded into worker
# command lines and reflected in error / success text, so also
# reject embedded HF tokens to mirror the repo_id hardening.
@field_validator("quantization_method")
@classmethod
def _no_quantization_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
class ExportLoRAAdapterRequest(ExportCommonOptions):
"""Request for exporting only the LoRA adapter (not merged)."""

View file

@ -60,6 +60,29 @@ class LoadRequest(BaseModel):
return None
return value
# Round 20 P1 #5: extend the diffusion-side identifier hardening
# (round 5 P2 / round 15 P1 #5) to the chat LoadRequest. Newline
# / tab / control characters in ``model_path`` or ``gguf_variant``
# would otherwise be echoed verbatim into structured-log lines
# ("Loading model %s") and let a caller smuggle in fake log
# entries, and an embedded ``hf_...`` token in a URL-form path
# would leak the credential into the same log sinks the
# diffusion route already redacts.
@field_validator("model_path", "gguf_variant")
@classmethod
def _no_identifier_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
# Round 21 P1 #1: also reject embedded HF tokens in
# ``gguf_variant``. A caller can pass a variant string like
# ``Q4_K_M-hf_xxxxxxxx`` that flows into log sinks via the
# GGUF resolver path; without this only ``model_path`` was
# protected.
@field_validator("model_path", "gguf_variant")
@classmethod
def _no_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
cache_type_kv: Optional[str] = Field(
None,
description = "KV cache data type for both K and V (e.g. 'f16', 'bf16', 'q8_0', 'q4_1', 'q5_1')",
@ -104,12 +127,47 @@ class LoadRequest(BaseModel):
),
)
# Round 28 P1 #13: each entry is forwarded verbatim to a logged
# subprocess command line and reflected in errors. Reject control
# chars and embedded HF tokens for every list entry; allow None.
@field_validator("llama_extra_args")
@classmethod
def _no_extra_args_control_chars(cls, v):
if v is None:
return v
for i, entry in enumerate(v):
_no_control_chars(entry, f"llama_extra_args[{i}]")
return v
@field_validator("llama_extra_args")
@classmethod
def _no_extra_args_embedded_hf_tokens(cls, v):
if v is None:
return v
for i, entry in enumerate(v):
_reject_embedded_hf_token(entry, f"llama_extra_args[{i}]")
return v
class UnloadRequest(BaseModel):
"""Request to unload a model"""
model_path: str = Field(..., description = "Model identifier to unload")
# Round 20 P1 #5: mirror the LoadRequest identifier hardening so
# /api/inference/unload also rejects control characters and
# URL-embedded HF tokens before the path reaches structured log
# sinks.
@field_validator("model_path")
@classmethod
def _no_identifier_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator("model_path")
@classmethod
def _no_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
class ValidateModelRequest(BaseModel):
"""
@ -130,6 +188,22 @@ class ValidateModelRequest(BaseModel):
None, description = "GGUF quantization variant (e.g. 'Q4_K_M')"
)
# Round 20 P1 #5: same identifier hardening as LoadRequest /
# UnloadRequest. /api/inference/validate flows directly into
# ``ModelConfig.from_identifier`` and the resulting log lines, so
# control characters and embedded HF tokens must not survive.
@field_validator("model_path", "gguf_variant")
@classmethod
def _no_identifier_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
# Round 21 P1 #2: extend embedded-token rejection to
# ``gguf_variant`` here too (mirrors LoadRequest).
@field_validator("model_path", "gguf_variant")
@classmethod
def _no_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
class ValidateModelResponse(BaseModel):
"""
@ -1421,3 +1495,193 @@ class AnthropicMessagesResponse(BaseModel):
stop_reason: Optional[str] = None
stop_sequence: Optional[str] = None
usage: AnthropicUsage = Field(default_factory = AnthropicUsage)
# ── Diffusion image generation ────────────────────────────────────
def _no_control_chars(value: Optional[str], field_name: str) -> Optional[str]:
"""Reject newlines, tabs, and other ASCII control chars in
identifiers that get logged before HF validates them.
Authenticated callers could otherwise inject ``\\n`` / ``\\r`` /
``\\t`` / NUL into ``logger.info("Loading diffusion model %s",
repo_id)`` and forge fake log lines. HF repo ids and filenames
legitimately contain only ``[A-Za-z0-9._/-]``, so this is also a
useful correctness check (catches accidental ``"my repo\\n"``
paste). Tab is included in the reject set because some logging
sinks split fields on tab; allowing it would still let an
attacker forge fake columns.
"""
if value is None:
return value
for ch in value:
if ch == "\x7f" or ord(ch) < 0x20:
raise ValueError(
f"{field_name} contains control characters; use a plain "
"Hugging Face repo / file name."
)
return value
import re as _re
_EMBEDDED_HF_TOKEN_RE = _re.compile(r"hf_[A-Za-z0-9]{20,}")
def _reject_embedded_hf_token(value: Optional[str], field_name: str) -> Optional[str]:
"""Refuse identifiers that contain an embedded ``hf_xxx`` token.
Round 15 P1 #5: ``repo_id`` and ``base_repo`` accept URL-style
strings (``https://hf_token@huggingface.co/owner/repo``). The
token would otherwise be stored in ``self._repo_id`` and echoed
back through ``status()`` to every authenticated browser session.
Log redaction (``_redact_hf_tokens``) covers the logger sink, but
the public status payload also needed to refuse the input. Use
the dedicated ``hf_token`` field for authentication.
"""
if value is not None and _EMBEDDED_HF_TOKEN_RE.search(value):
raise ValueError(
f"{field_name} must not embed a Hugging Face token; "
"pass it via the dedicated hf_token field instead."
)
return value
class DiffusionLoadRequest(BaseModel):
"""Load a diffusion image-generation model.
repo_id is the HF repo (either GGUF-only or full diffusers layout).
gguf_filename selects the quant when repo_id is a GGUF repo.
base_repo overrides the auto-picked diffusers base used for the
VAE / text encoders when loading a GGUF-only repo.
"""
# repo_id and base_repo are HF Hub identifiers in this release.
# Local-path support is gated behind a frontend / Tauri
# ``load-diffusion-model`` directory lease producer that has not
# shipped yet (round 32 P1 #3 in the PR reviewer trail). The
# 1024-char cap matches POSIX PATH_MAX so future local-path
# support can flip on without re-validating the field width.
repo_id: str = Field(
...,
min_length = 1,
max_length = 1024,
description = (
"HF repo id (owner/name). Local filesystem paths are reserved "
"for a future native-lease flow and currently rejected by the "
"route's _looks_like_local_diffusion_path guard."
),
)
# Round 30 P1 #4: chat /api/inference/load gates native local paths
# through a signed native_path_lease grant before the backend
# touches the filesystem. Mirror that here so /api/inference/images/
# load cannot be used as an authenticated probe for arbitrary
# local directories. Optional; Hub ids (no leading slash / tilde)
# skip the lease check entirely. The Images UI does not yet
# surface a local-path picker, so callers that omit this field
# always get the Hub-id code path.
native_path_lease: Optional[str] = Field(
None,
description = "Frontend-visible signed native path grant for a local repo_id",
)
gguf_filename: Optional[str] = Field(
None,
max_length = 512,
description = "GGUF filename inside repo_id (Q4_K_S, Q8_0, ...)",
)
base_repo: Optional[str] = Field(
None,
max_length = 1024,
description = (
"Diffusers base repo (HF id) for VAE + text encoders. Local "
"paths are gated on the same future native-lease flow as "
"repo_id."
),
)
base_repo_native_path_lease: Optional[str] = Field(
None,
description = "Frontend-visible signed native path grant for a local base_repo",
)
family: Optional[str] = Field(
None,
max_length = 64,
description = "Force pipeline family: flux.2-klein | flux.2 | flux.1 | qwen-image | stable-diffusion-3 | stable-diffusion-xl",
)
hf_token: Optional[str] = Field(
None, description = "HuggingFace token for gated models"
)
enable_model_cpu_offload: bool = Field(
True,
description = "Offload submodules to CPU between forwards. Trades a small speed hit for ~6 GB less VRAM on FLUX-class models.",
)
@field_validator("repo_id", "gguf_filename", "base_repo", "family")
@classmethod
def _no_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator("repo_id", "gguf_filename", "base_repo")
@classmethod
def _no_embedded_hf_tokens(cls, v, info):
# Round 17 P2 #12: ``gguf_filename`` is forwarded to the
# backend and stored on ``DiffusionBackend._gguf_filename``,
# which is later surfaced via ``status()`` / log lines. If a
# user pastes a URL-form quant path like
# ``https://hf_xxxxx@huggingface.co/.../flux.gguf`` we drop
# the embedded credential before it can leak.
return _reject_embedded_hf_token(v, info.field_name)
# torch.Generator.manual_seed packs into signed int64; values outside
# [-2**63, 2**63 - 1] raise ``Overflow when unpacking long long`` deep
# in the C++ layer. uint64 is also routinely cited online so accept
# any value the underlying RNG could store and bounce the rest at the
# Pydantic layer with a clean error.
_SEED_MIN = -(2**63)
_SEED_MAX = (2**64) - 1
class DiffusionGenerateRequest(BaseModel):
"""Generate a single image from the currently-loaded diffusion model."""
prompt: str = Field(..., min_length = 1, max_length = 4000)
negative_prompt: Optional[str] = Field(None, max_length = 4000)
num_inference_steps: int = Field(24, ge = 1, le = 200)
guidance_scale: float = Field(3.5, ge = 0.0, le = 20.0)
width: int = Field(1024, ge = 64, le = 2048)
height: int = Field(1024, ge = 64, le = 2048)
seed: Optional[int] = Field(
None,
ge = _SEED_MIN,
le = _SEED_MAX,
description = "Deterministic seed for reproducible outputs",
)
@field_validator("width", "height")
@classmethod
def _multiple_of_eight(cls, v: int) -> int:
if v % 8:
raise ValueError("width and height must be multiples of 8")
return v
class DiffusionGenerateResponse(BaseModel):
image_b64: str = Field(..., description = "Base64-encoded PNG")
image_mime: str = "image/png"
width: int
height: int
num_inference_steps: int
guidance_scale: float
# ``seed`` ships as a JSON number for backwards compatibility with
# the gallery and existing API consumers, but JavaScript rounds
# integers above Number.MAX_SAFE_INTEGER on JSON.parse so seeds
# bigger than 2**53 would render different from the value the
# backend actually used. ``seed_str`` is the exact decimal
# representation; the frontend reads it for reproducibility and
# falls back to ``seed`` when not supplied.
seed: Optional[int] = None
seed_str: Optional[str] = None
duration_ms: int
model: Optional[str] = None
family: Optional[str] = None

View file

@ -5,9 +5,11 @@
Pydantic schemas for Model Management API
"""
from pydantic import BaseModel, Field
from pydantic import BaseModel, Field, field_validator
from typing import Optional, List, Dict, Any, Literal
from models.inference import _no_control_chars, _reject_embedded_hf_token
ModelType = Literal["text", "vision", "audio", "embeddings"]
@ -206,6 +208,19 @@ class AddScanFolderRequest(BaseModel):
..., description = "Absolute or relative directory path to scan for models"
)
# path is reflected back in /scan-folders error details and logged
# via add_scan_folder_endpoint when the directory is missing, so
# apply the same identifier hardening used on other logged paths.
@field_validator("path")
@classmethod
def _no_path_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator("path")
@classmethod
def _no_path_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
class ScanFolderInfo(BaseModel):
"""A registered custom model scan folder."""

View file

@ -8,6 +8,13 @@ Pydantic schemas for Training API
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
from typing import Any, Optional, List, Dict, Literal
# Round 22 P1 #1: reuse the chat / diffusion identifier validators
# so /api/training/start rejects newline / tab / control characters
# and URL-form ``hf_xxxxx`` tokens in ``model_name``. Without these
# a caller could log-line-smuggle through "Loading model %s" lines
# and leak the bearer token into structured-log sinks.
from models.inference import _no_control_chars, _reject_embedded_hf_token
_MAX_BATCH_SIZE = 4096
_MAX_GRAD_ACCUM = 4096
@ -49,6 +56,52 @@ class TrainingStartRequest(BaseModel):
model_name: str = Field(
..., description = "Model identifier (e.g., 'unsloth/llama-3-8b-bnb-4bit')"
)
# Identifier hardening: extended progressively across analogous
# request models. format_type is copied into training_kwargs and
# written into trainer log lines, so it shares the same boundary.
@field_validator(
"model_name",
"hf_dataset",
"subset",
"train_split",
"eval_split",
"format_type",
)
@classmethod
def _no_model_name_control_chars(cls, v, info):
return _no_control_chars(v, info.field_name)
@field_validator(
"model_name",
"hf_dataset",
"subset",
"train_split",
"eval_split",
"format_type",
)
@classmethod
def _no_model_name_embedded_hf_tokens(cls, v, info):
return _reject_embedded_hf_token(v, info.field_name)
# local_datasets / local_eval_datasets are user-controlled lists
# reflected back in /api/training/start error details when
# _validate_local_dataset_paths fails, so the same control-char +
# embedded-token guards apply per entry.
@field_validator("local_datasets", "local_eval_datasets")
@classmethod
def _no_local_dataset_control_chars(cls, v, info):
for i, entry in enumerate(v or []):
_no_control_chars(entry, f"{info.field_name}[{i}]")
return v
@field_validator("local_datasets", "local_eval_datasets")
@classmethod
def _no_local_dataset_embedded_hf_tokens(cls, v, info):
for i, entry in enumerate(v or []):
_reject_embedded_hf_token(entry, f"{info.field_name}[{i}]")
return v
training_type: Literal["LoRA/QLoRA", "Full Finetuning", "Continued Pretraining"] = (
Field(
...,

View file

@ -43,15 +43,44 @@ safetensors>=0.4.3
datasets>=3.4.1,!=4.0.*,!=4.1.0,<4.4.0
accelerate>=0.34.1
peft>=0.18.0,!=0.11.0
# Round 33 P1: reverted the round-26 hub>=1.3.0 floor to the
# pre-PR >=0.34.0 floor. Studio's install_python_stack later
# forces hub==0.36.2 via studio.txt (constrained by
# transformers==4.57.6 in extras-no-deps.txt), so the 1.3.0
# floor was internally inconsistent. extras-no-deps holds
# transformers at 4.x, so the transformers-5.x is_offline_mode
# concern that motivated the original bump never actually
# triggers on the supported install path.
# Round 34 P1: the line itself must stay because install.sh
# --no-torch installs THIS file with --no-deps and does not run
# studio.txt afterward; without the package line a no-torch
# install ends with no huggingface_hub at all and the new
# diffusion / chat GGUF paths fail with ModuleNotFoundError.
# Verified live on B200: hub 0.36.2 + transformers 4.57.6 +
# diffusers 0.37.1 imports Flux2KleinPipeline cleanly and runs
# end-to-end image generation.
huggingface_hub>=0.34.0
hf_transfer
diffusers
# Floor 0.37.0 introduces Flux2KleinPipeline + Flux2Pipeline which the
# Studio Images page imports for the default curated picker.
diffusers>=0.37.0
# Required by diffusers.GGUFQuantizationConfig (used by the Images page
# to load FLUX.2 / FLUX.1 / Qwen-Image GGUFs from the Hub). Floor at
# 0.10.0 to match the diffusers requirement; older gguf releases raise
# at single-file load time.
gguf>=0.10.0
# Transitive deps required because this file is installed with --no-deps.
# Without these, `from transformers import AutoConfig` fails at import time.
regex
typing_extensions
filelock
# `requests` and its urllib3/charset chain are required by huggingface_hub's
# blob downloader; diffusers + GGUFQuantizationConfig 500 on first
# /api/inference/images/load otherwise.
requests
urllib3
charset_normalizer
httpx
httpcore
certifi

View file

@ -1,6 +1,11 @@
# Studio UI backend dependencies
typer
fastapi
# Required by FastAPI's multipart upload route validation
# (routes/datasets.py uploads files via UploadFile/File). Without
# this, importing the routes package raises RuntimeError on startup
# and CPU-only test environments fail before any test runs.
python-multipart
uvicorn
pydantic
packaging
@ -18,3 +23,10 @@ diceware
ddgs
cryptography>=42.0.0
httpx>=0.27.0
# Studio Images page runtime. Flux2KleinPipeline / Flux2Pipeline /
# QwenImagePipeline / StableDiffusion3Pipeline are available in
# diffusers>=0.37.0, and GGUFQuantizationConfig requires the gguf
# package (round 20 P1 #4: fresh standard Studio installs failed on
# /images/load because these were only listed in the extras files).
diffusers>=0.37.0
gguf>=0.10.0

View file

@ -433,6 +433,20 @@ async def upload_unstructured_file(
tracked_ids = [fid.strip() for fid in existing_file_ids.split(",") if fid.strip()]
original_filename = file.filename or "upload"
# Round 33 P1 #7: file.filename is reflected back to the client,
# persisted in the meta JSON, and echoed by error paths. Mirror
# the SeedInspectUploadRequest.filename hardening so a multipart
# upload cannot smuggle control characters or URL-form HF tokens
# through the path the JSON variant already rejects. Import
# locally to avoid a routes -> models cycle.
from models.inference import _no_control_chars, _reject_embedded_hf_token
try:
_no_control_chars(original_filename, "filename")
_reject_embedded_hf_token(original_filename, "filename")
except ValueError as exc:
raise HTTPException(status_code = 400, detail = str(exc)) from exc
ext = Path(original_filename).suffix.lower()
if ext not in UNSTRUCTURED_ALLOWED_EXTS:
raise HTTPException(

View file

@ -68,11 +68,27 @@ if str(backend_path) not in sys.path:
# Import dataset utilities
from utils.datasets import check_dataset_format
from auth.authentication import get_current_subject
from models.inference import _no_control_chars, _reject_embedded_hf_token
router = APIRouter()
logger = get_logger(__name__)
def _validate_logged_identifier(value: str, field_name: str) -> str:
"""Round 25 P1 #1: mirror the helper in routes/models.py so the
dataset ``/download-progress`` route never reaches logger/cache
paths with control characters or embedded HF tokens. Token-shaped
strings like ``owner/hf_abcdefghij0123456789`` would otherwise pass
the cheap ``_is_valid_repo_id`` regex and end up in warning logs.
"""
try:
value = _no_control_chars(value, field_name)
value = _reject_embedded_hf_token(value, field_name)
except ValueError as exc:
raise HTTPException(status_code = 422, detail = str(exc)) from exc
return value
from models.datasets import (
AiAssistMappingRequest,
AiAssistMappingResponse,
@ -320,7 +336,20 @@ async def upload_dataset(
file: UploadFile,
current_subject: str = Depends(get_current_subject),
) -> UploadDatasetResponse:
filename = _sanitize_filename(file.filename or "dataset_upload")
# Validate the raw multipart filename BEFORE sanitization so smuggled
# control characters and embedded HF tokens are rejected at the same
# boundary as the JSON path; sanitizing first would silently strip
# control chars and let raw inputs pass the validator.
raw_filename = file.filename or "dataset_upload"
from models.inference import _no_control_chars, _reject_embedded_hf_token
try:
_no_control_chars(raw_filename, "filename")
_reject_embedded_hf_token(raw_filename, "filename")
except ValueError as exc:
raise HTTPException(status_code = 400, detail = str(exc)) from exc
filename = _sanitize_filename(raw_filename)
ext = Path(filename).suffix.lower()
if ext not in LOCAL_UPLOAD_EXTS:
allowed = ", ".join(sorted(LOCAL_UPLOAD_EXTS))
@ -370,6 +399,11 @@ async def get_dataset_download_progress(
bytes are observable here. Returns ``cache_path`` so the UI can
show users where the dataset blobs landed on disk.
"""
# Round 25 P1 #1: harden ``repo_id`` before it reaches the
# ``logger.warning`` line at the bottom (or any future log/cache
# path). Matches ``GET /api/models/download-progress`` which
# already validates the same parameter in round 24.
repo_id = _validate_logged_identifier(repo_id, "repo_id")
_empty = {
"downloaded_bytes": 0,
"expected_bytes": 0,

View file

@ -50,6 +50,109 @@ router = APIRouter()
logger = get_logger(__name__)
import contextlib
def _raise_if_training_active_for_export() -> None:
"""409 if a training run is in flight; 503 if status check itself
raises. Mirrors the load_checkpoint guard so /export/* and /cleanup
never tear down or alter export state while training is using the
GPU. Missing core.training is treated as 'no tracker'."""
try:
from core.training import get_training_backend # type: ignore
except Exception as e:
logger.debug("core.training not importable, skipping training guard: %s", e)
return
try:
trn = get_training_backend()
active = trn.is_training_active()
except Exception as e:
logger.warning("Could not verify training status before export op: %s", e)
raise HTTPException(
status_code = 503,
detail = (
"Could not verify training status before the export "
"operation. Try again."
),
) from e
if active:
raise HTTPException(
status_code = 409,
detail = (
"Training is currently active. Stop the training run "
"before starting an export operation."
),
)
def _raise_if_export_active_for_export() -> None:
"""409 if another export job is already running; 503 if the status
check itself raises. Backends without is_export_active() are
treated as 'no tracker available' to stay compatible with mocked
backends in tests."""
backend = get_export_backend()
is_export_active_fn = getattr(backend, "is_export_active", None)
if is_export_active_fn is None:
return
try:
export_is_active = bool(is_export_active_fn())
except Exception as e:
logger.warning("Could not verify export status before export op: %s", e)
raise HTTPException(
status_code = 503,
detail = (
"Could not verify export status before starting the "
"export operation. Try again."
),
) from e
if export_is_active:
raise HTTPException(
status_code = 409,
detail = (
"An export job is currently active. Wait for it to "
"finish before starting another export operation."
),
)
@contextlib.asynccontextmanager
async def _export_public_window():
"""Publish the public-load window across an /export/* operation.
backend.export_*() runs in a worker thread and does not flip
``_export_active = True`` until the worker actually starts; during
that gap window another workload that calls ``_release_export_for``
would see ``is_export_active() == False`` and tear down the export
subprocess. Mirror the load_checkpoint guard so the pending counter
is set for the whole export call, and the helper-busy preflight
refuses if AI Assist is mid-handoff.
Also refuses 409 if training or another export is already active so
a queued /export/{merged,base,gguf,lora} or /cleanup cannot
double-own the GPU with a running training / export job (round 41
consensus: load_checkpoint already runs these checks but /export/*
and /cleanup were skipping them).
"""
from routes.inference import (
_clear_public_load_window,
_raise_if_helper_advisor_busy,
)
export_window_published = False
try:
_raise_if_training_active_for_export()
_raise_if_export_active_for_export()
_raise_if_helper_advisor_busy("export")
export_window_published = True
yield
finally:
if export_window_published:
try:
_clear_public_load_window("export")
except Exception:
pass
@router.post("/load-checkpoint", response_model = ExportOperationResponse)
async def load_checkpoint(
request: LoadCheckpointRequest,
@ -60,50 +163,123 @@ async def load_checkpoint(
Wraps ExportBackend.load_checkpoint.
"""
# Round 30 P1 #8: track whether we published a public-load pending
# entry so the outer finally clears it on either success or
# failure path.
export_load_window_published = False
try:
# Version switching is handled automatically by the subprocess-based
# export backend — no need for ensure_transformers_version() here.
# Free GPU memory: shut down any running inference/training subprocesses
# before loading the export checkpoint (they'd compete for VRAM).
# Symmetric lifecycle guard: refuse to load an export
# checkpoint while training is active so we do not silently
# terminate someone's long-running training job and possibly
# fail the export load on top of that. Mirrors the
# _raise_if_training_active checks in routes/inference.py for
# chat and /images/load.
# Run BEFORE the chat / inference / diffusion unload helpers
# below: otherwise a 409 from this guard would still leave
# the user's chat / inference / diffusion GPU owners freed
# for nothing, which is the asymmetry round 7 review #5
# flagged. Fail-CLOSED (503) when the training backend is
# importable but its status check raises.
try:
from core.inference import get_inference_backend
inf = get_inference_backend()
if inf.active_model_name:
logger.info(
"Unloading inference model '%s' to free GPU memory for export",
inf.active_model_name,
from core.training import get_training_backend # type: ignore
except Exception as e:
logger.debug(
"core.training not importable, skipping export training guard: %s",
e,
)
else:
try:
trn = get_training_backend()
active = trn.is_training_active()
except Exception as e:
logger.warning(
"Could not verify training status before export load: %s", e
)
raise HTTPException(
status_code = 503,
detail = (
"Could not verify training status before loading "
"an export checkpoint. Try again."
),
) from e
if active:
raise HTTPException(
status_code = 409,
detail = (
"Training is currently active. Stop the training "
"run before loading an export checkpoint."
),
)
inf._shutdown_subprocess()
inf.active_model_name = None
inf.models.clear()
except Exception as e:
logger.warning("Could not unload inference model: %s", e)
try:
from core.training import get_training_backend
trn = get_training_backend()
if trn.is_training_active():
logger.info("Stopping active training to free GPU memory for export")
trn.stop_training()
# Wait for training subprocess to actually exit before proceeding,
# otherwise it may still hold GPU memory when export tries to load.
for _ in range(60): # up to 30s
if not trn.is_training_active():
break
import time
time.sleep(0.5)
else:
logger.warning(
"Training subprocess did not exit within 30s, proceeding anyway"
)
except Exception as e:
logger.warning("Could not stop training: %s", e)
backend = get_export_backend()
# Refuse to reload the export checkpoint while an export job
# is still running. ``ExportBackend.load_checkpoint`` would
# terminate the running subprocess in order to spawn a new
# one, silently corrupting the partial output the user is
# waiting on (round 13 P1 #1). Runs BEFORE the chat /
# diffusion unloads below: a 409 from this guard must not
# leave the user's chat or diffusion GPU owners freed for
# nothing (round 14 P1 #1). ``is_export_active`` may be
# absent on older / mocked backends; treat missing as "no
# async-job tracker available" and skip rather than
# fail-closed.
is_export_active_fn = getattr(backend, "is_export_active", None)
if is_export_active_fn is not None:
try:
export_is_active = bool(is_export_active_fn())
except Exception as e:
logger.warning(
"Could not verify export status before export load: %s", e
)
raise HTTPException(
status_code = 503,
detail = (
"Could not verify export status before loading "
"an export checkpoint. Try again."
),
) from e
if export_is_active:
raise HTTPException(
status_code = 409,
detail = (
"An export job is currently active. Stop the "
"export job before loading another checkpoint."
),
)
# Free GPU memory: shut down any chat backend before loading
# the export checkpoint. Routes the unload through the shared
# helper so we cover llama-server is_active=True and
# safetensors loading_models -- the asymmetries round 9
# reviews #1, #8, #9 flagged.
from routes.inference import (
_clear_public_load_window,
_raise_if_helper_advisor_busy,
_release_chat_for,
_release_diffusion_for,
)
# Round 28 P1 #6: refuse before any release fires so AI Assist
# busy does not first tear down idle diffusion.
# Round 30 P1 #8: also publishes a public-load pending entry so
# a concurrent helper / advisor start cannot win the start
# lock between our snapshot and load_checkpoint flipping
# current_checkpoint / is_export_active.
_raise_if_helper_advisor_busy("export")
export_load_window_published = True
# Round 24 P1 #3: release diffusion BEFORE chat so a failing
# diffusion unload does not leave the user with no chat
# model loaded. Same reasoning as the training-start flow
# (round 18 P1 #8 / round 24 P1 #2). Earlier rounds kept the
# chat release first because the helper was best-effort;
# now that ``_release_diffusion_for`` is strict it must run
# while chat is still resident so a failure preserves it.
await _release_diffusion_for("export load")
await _release_chat_for("export")
# load_checkpoint spawns and waits on a subprocess and can take
# minutes. Run it in a worker thread so the event loop stays
# free to serve the live log SSE stream concurrently.
@ -127,6 +303,18 @@ async def load_checkpoint(
status_code = 500,
detail = f"Failed to load checkpoint: {str(e)}",
)
finally:
# Round 30 P1 #8: clear the public-load pending entry once the
# load attempt completes (success or failure). Skipped when
# the helper-busy check itself raised so the counter stays in
# sync with publishes.
if export_load_window_published:
try:
from routes.inference import _clear_public_load_window
except Exception:
pass
else:
_clear_public_load_window("export")
@router.post("/cleanup", response_model = ExportOperationResponse)
@ -140,7 +328,12 @@ async def cleanup_export_memory(
"""
try:
backend = get_export_backend()
success = await asyncio.to_thread(backend.cleanup_memory)
# Run the cleanup under the same public-load window /export/*
# uses so a queued export's handoff gap cannot race a cleanup
# call that tears down current_checkpoint. The window also
# refuses 409 if training or another export is in flight.
async with _export_public_window():
success = await asyncio.to_thread(backend.cleanup_memory)
if not success:
raise HTTPException(
@ -211,15 +404,16 @@ async def export_merged_model(
"""
try:
backend = get_export_backend()
success, message, output_path = await asyncio.to_thread(
backend.export_merged_model,
save_directory = request.save_directory,
format_type = request.format_type,
push_to_hub = request.push_to_hub,
repo_id = request.repo_id,
hf_token = request.hf_token,
private = request.private,
)
async with _export_public_window():
success, message, output_path = await asyncio.to_thread(
backend.export_merged_model,
save_directory = request.save_directory,
format_type = request.format_type,
push_to_hub = request.push_to_hub,
repo_id = request.repo_id,
hf_token = request.hf_token,
private = request.private,
)
if not success:
raise HTTPException(status_code = 400, detail = message)
@ -251,15 +445,16 @@ async def export_base_model(
"""
try:
backend = get_export_backend()
success, message, output_path = await asyncio.to_thread(
backend.export_base_model,
save_directory = request.save_directory,
push_to_hub = request.push_to_hub,
repo_id = request.repo_id,
hf_token = request.hf_token,
private = request.private,
base_model_id = request.base_model_id,
)
async with _export_public_window():
success, message, output_path = await asyncio.to_thread(
backend.export_base_model,
save_directory = request.save_directory,
push_to_hub = request.push_to_hub,
repo_id = request.repo_id,
hf_token = request.hf_token,
private = request.private,
base_model_id = request.base_model_id,
)
if not success:
raise HTTPException(status_code = 400, detail = message)
@ -291,14 +486,15 @@ async def export_gguf(
"""
try:
backend = get_export_backend()
success, message, output_path = await asyncio.to_thread(
backend.export_gguf,
save_directory = request.save_directory,
quantization_method = request.quantization_method,
push_to_hub = request.push_to_hub,
repo_id = request.repo_id,
hf_token = request.hf_token,
)
async with _export_public_window():
success, message, output_path = await asyncio.to_thread(
backend.export_gguf,
save_directory = request.save_directory,
quantization_method = request.quantization_method,
push_to_hub = request.push_to_hub,
repo_id = request.repo_id,
hf_token = request.hf_token,
)
if not success:
raise HTTPException(status_code = 400, detail = message)
@ -330,14 +526,15 @@ async def export_lora_adapter(
"""
try:
backend = get_export_backend()
success, message, output_path = await asyncio.to_thread(
backend.export_lora_adapter,
save_directory = request.save_directory,
push_to_hub = request.push_to_hub,
repo_id = request.repo_id,
hf_token = request.hf_token,
private = request.private,
)
async with _export_public_window():
success, message, output_path = await asyncio.to_thread(
backend.export_lora_adapter,
save_directory = request.save_directory,
push_to_hub = request.push_to_hub,
repo_id = request.repo_id,
hf_token = request.hf_token,
private = request.private,
)
if not success:
raise HTTPException(status_code = 400, detail = message)

File diff suppressed because it is too large Load diff

View file

@ -134,11 +134,30 @@ from models.responses import (
VisionCheckResponse,
EmbeddingCheckResponse,
)
from models.inference import _no_control_chars, _reject_embedded_hf_token
router = APIRouter()
logger = get_logger(__name__)
def _validate_logged_identifier(value: str, field_name: str) -> str:
"""Round 23 P1 #7 / #8 / #9 / #10: path / query parameters that
flow into ``logger.info("... %s", value)`` lines were the last
unguarded entry points. Newline / tab / control characters let
a caller smuggle forged log entries; URL-form ``hf_xxxxx``
tokens would leak into structured-log sinks. Mirror the
request-body validators by running both checks here and
mapping the validator's ``ValueError`` to HTTP 422 so the
client sees the same shape as a Pydantic validation failure.
"""
try:
value = _no_control_chars(value, field_name)
value = _reject_embedded_hf_token(value, field_name)
except ValueError as exc:
raise HTTPException(status_code = 422, detail = str(exc)) from exc
return value
def derive_model_type(
is_vision: bool, audio_type: Optional[str], is_embedding: bool = False
) -> ModelType:
@ -1571,6 +1590,7 @@ async def get_model_config(
This endpoint wraps the backend load_model_defaults function.
"""
model_name = _validate_logged_identifier(model_name, "model_name")
try:
if not is_local_path(model_name):
resolved = resolve_cached_repo_id_case(model_name)
@ -1580,7 +1600,11 @@ async def get_model_config(
resolved,
model_name,
)
model_name = resolved
# Round 23 P1 #7: re-validate the cache-resolved value
# (case-only resolver should be a no-op for these
# checks, but defend in depth in case the resolver
# ever broadens its match heuristic).
model_name = _validate_logged_identifier(resolved, "model_name")
logger.info(f"Getting model config for: {model_name}")
from utils.models.model_config import detect_audio_type
@ -1709,6 +1733,53 @@ def _is_path_under(path: Path, root: Path) -> bool:
return False
def _diffusion_owned_targets(diff_status: dict) -> list[tuple[str, str | None]]:
"""Return ``(owned_repo_or_path, owned_gguf_filename)`` pairs for
every diffusion target the backend currently holds.
Pairs the active / pending repo with the active / pending GGUF
filename (not the UI-facing collapsed ``gguf_filename``) so the
per-variant delete guards know which quant is actually owned by
each repo. Without this pairing, a swap in progress (active
``Q4_K_S``, pending ``Q8_0``) collapsed both to the pending
variant and the active ``Q4_K_S`` GGUF could be deleted while
still mmap'd by the resident pipeline (round 13 P1 #3-5).
Base repos are paired with ``None`` for the GGUF: the base /
component repo is loaded whole via ``from_pretrained`` and has no
per-variant delete to take advantage of.
"""
return [
(
diff_status.get("active_repo_id") or "",
diff_status.get("active_gguf_filename"),
),
(diff_status.get("active_base_repo") or "", None),
(
diff_status.get("pending_repo_id") or "",
diff_status.get("pending_gguf_filename"),
),
(diff_status.get("pending_base_repo") or "", None),
]
def _variant_delete_is_safe_for_owned_gguf(
requested_variant: str | None,
owned_gguf_filename: str | None,
) -> bool:
"""True iff a per-variant delete for ``requested_variant`` against
a repo that owns ``owned_gguf_filename`` cannot remove the owned
file.
Returns False (i.e. unsafe -> block the delete) when either
argument is missing so a NULL owned filename or a full-repo delete
(no variant) does not accidentally pass the guard."""
if not requested_variant or not owned_gguf_filename:
return False
loaded_label = (_extract_quant_label(owned_gguf_filename.lower()) or "").lower()
return bool(loaded_label and loaded_label != requested_variant.lower())
def _is_path_under_lexically(path: Path, root: Path) -> bool:
"""Check containment without resolving the final path's symlink target."""
try:
@ -1724,7 +1795,15 @@ def _loaded_model_matches_deleted_path(active_model: str, deleted_path: Path) ->
try:
active = Path(active_model).expanduser().resolve()
target = deleted_path.resolve()
return active == target or (target.is_dir() and active.is_relative_to(target))
# Round 27 P1 #8: match bidirectionally so deleting a child
# directory of a loaded local model (e.g. .../my-flux/text_encoder
# while .../my-flux is loaded) also trips the guard. Mirrors
# the diffusion delete-guard pattern.
return (
active == target
or (target.is_dir() and active.is_relative_to(target))
or (active.is_dir() and target.is_relative_to(active))
)
except (OSError, RuntimeError, ValueError) as e:
logger.debug(
"Could not resolve loaded/deleted model paths; falling back to string comparison: %s",
@ -1732,8 +1811,10 @@ def _loaded_model_matches_deleted_path(active_model: str, deleted_path: Path) ->
)
active_lower = active_model.lower()
target_lower = str(deleted_path).lower()
return active_lower == target_lower or active_lower.startswith(
f"{target_lower}{os.sep}"
return (
active_lower == target_lower
or active_lower.startswith(f"{target_lower}{os.sep}")
or target_lower.startswith(f"{active_lower}{os.sep}")
)
@ -1805,6 +1886,14 @@ async def delete_finetuned_model(
Only paths under Studio's outputs/exports roots are accepted. Exported
GGUF entries can delete one quantization variant at a time.
"""
# Round 24 P1 #7 + P2 #13: harden both ``model_path`` and
# ``gguf_variant`` for control characters and embedded HF
# tokens, mirroring the chat / diffusion / training request
# validators. Both fields end up in logger.info(...) lines.
model_path = _validate_logged_identifier(model_path, "model_path")
if gguf_variant is not None:
gguf_variant = _validate_logged_identifier(gguf_variant, "gguf_variant")
if source not in {"training", "exported"}:
raise HTTPException(
status_code = 400,
@ -1893,6 +1982,32 @@ async def delete_finetuned_model(
from routes.inference import get_llama_cpp_backend
llama_backend = get_llama_cpp_backend()
# Pending HF GGUF download targeting this path: round 14 P1 #3.
# ``loading_model_identifier`` is set before the download starts
# and cleared after the subprocess settles, so the user cannot
# rmtree the directory llama.cpp is writing into mid-flight.
# Round 15 P1 #2: compare against ``loading_hf_variant`` (the
# variant being downloaded) rather than ``hf_variant`` (the
# PREVIOUS loaded variant, which is stale until the new load
# completes its late-metadata update).
loading_identifier = getattr(llama_backend, "loading_model_identifier", None)
loading_variant = getattr(llama_backend, "loading_hf_variant", None)
if (
loading_identifier
and _loaded_model_matches_deleted_path(
loading_identifier,
target_path,
)
and (
not gguf_variant
or not loading_variant
or loading_variant.lower() == gguf_variant.lower()
)
):
raise HTTPException(
status_code = 409,
detail = "Cannot delete a model while it is loading",
)
if (
llama_backend.is_active
and not llama_backend.is_loaded
@ -1968,6 +2083,77 @@ async def delete_finetuned_model(
detail = "Could not verify model load status before deleting",
) from e
# Diffusion pipelines can also be loaded directly from a Studio
# outputs/exports path (e.g. user fine-tuned a FLUX LoRA, exported
# the merged repo locally, then loaded it via /images/load with a
# local path as repo_id). Without this guard /delete-finetuned
# could rmtree the directory the diffusion backend is reading from.
# is_loading is also blocked: status() exposes pending_repo_id /
# pending_base_repo during the load window so deletes during a
# mid-flight from_pretrained are refused. During a swap we still
# see the previous load's active_repo_id, so every owned path is
# checked rather than just the UI-facing one.
# Block both DIRECTIONS:
# * loaded path is the same as target (or a parent), and
# * loaded path is a child of target (so the user cannot rmtree
# a parent directory that contains the pipeline's mmap'd file).
# Fail-CLOSED on exception (503) like the llama.cpp / safetensors
# guards above: an unverifiable diffusion state means we cannot
# confirm the target is safe to rmtree.
try:
from core.inference.diffusion import get_diffusion_backend
diff_backend = get_diffusion_backend()
# include_internal=True so we can iterate active_*/pending_*
# raw paths against ``target_path`` (round 16 P1 #5).
diff_status = diff_backend.status(include_internal = True)
if diff_status.get("is_loaded") or diff_status.get("is_loading"):
target_str = str(target_path)
# Pair each owned repo / path with the GGUF variant it
# actually owns (round 13 P1 #5). For a swap in flight
# (active Q4_K_S, pending Q8_0) the active variant must
# NOT be deleted just because the pending variant uses
# a different quant.
for candidate, owned_gguf in _diffusion_owned_targets(diff_status):
if not candidate:
continue
try:
candidate_resolved = Path(candidate).expanduser().resolve()
except Exception:
continue
# Relative paths (the user can do
# `/images/load repo_id=exports/my-flux`) are still
# legitimate path candidates; resolve against the
# backend cwd so they can be compared with the
# absolute ``target_path``. Round 8 review #11.
overlaps = (
candidate_resolved == target_path
or str(candidate_resolved) == target_str
or _is_path_under(candidate_resolved, target_path)
or _is_path_under(target_path, candidate_resolved)
)
if not overlaps:
continue
if export_type == "gguf" and _variant_delete_is_safe_for_owned_gguf(
gguf_variant,
owned_gguf,
):
continue
raise HTTPException(
status_code = 400,
detail = "Unload the diffusion image model before deleting",
)
except HTTPException:
raise
except Exception as e:
logger.warning(
"Could not check diffusion backend loaded model before delete: %s", e
)
raise HTTPException(
status_code = 503,
detail = "Could not verify diffusion load status before deleting",
) from e
try:
if export_type == "gguf" and gguf_variant:
if not target_path.is_dir():
@ -2043,6 +2229,9 @@ async def get_lora_base_model(
This endpoint wraps the backend get_base_model_from_lora function.
"""
# Round 26 P1 #12: lora_path is echoed back in 404 detail and logs;
# harden it the same way other reflected identifiers are.
lora_path = _validate_logged_identifier(lora_path, "lora_path")
try:
base_model = get_base_model_from_lora(lora_path)
@ -2076,6 +2265,7 @@ async def check_vision_model(
This endpoint wraps the backend is_vision_model function.
"""
model_name = _validate_logged_identifier(model_name, "model_name")
try:
logger.info(f"Checking if vision model: {model_name}")
is_vision = is_vision_model(model_name)
@ -2104,6 +2294,7 @@ async def check_embedding_model(
This endpoint wraps the backend is_embedding_model function.
"""
model_name = _validate_logged_identifier(model_name, "model_name")
try:
logger.info(f"Checking if embedding model: {model_name}")
is_embedding = is_embedding_model(model_name, hf_token = hf_token)
@ -2141,6 +2332,7 @@ async def get_gguf_variants(
with file sizes, whether the model supports vision, and the recommended
default variant.
"""
repo_id = _validate_logged_identifier(repo_id, "repo_id")
try:
from utils.models.model_config import is_local_path, list_local_gguf_variants
@ -2248,6 +2440,13 @@ async def get_gguf_download_progress(
Tracks completed shard downloads in snapshots and in-progress downloads
in the blobs directory (incomplete files).
"""
# Round 28 P1 #14: mirror the hardening on the generic
# /download-progress route. Both repo_id and variant are echoed
# into the cache-scan path and can reach logs on the failure
# branch via the surrounding try/except.
repo_id = _validate_logged_identifier(repo_id, "repo_id")
if variant:
variant = _validate_logged_identifier(variant, "variant")
try:
if not _is_valid_repo_id(repo_id):
return {
@ -2335,6 +2534,10 @@ async def get_download_progress(
"progress": 0,
"cache_path": None,
}
# Round 24 P1 #9: ``repo_id`` flows into log lines deep in
# ``_get_repo_size_cached`` on lookup failure, so the same
# hardening the request-body models use applies here too.
repo_id = _validate_logged_identifier(repo_id, "repo_id")
try:
if not _is_valid_repo_id(repo_id):
return _empty
@ -2598,39 +2801,283 @@ async def delete_cached_model(
are removed (e.g. ``UD-Q4_K_XL``). Otherwise the entire repo is deleted.
Refuses if the model is currently loaded for inference.
"""
# Round 24 P1 #8 + #10: harden both ``repo_id`` and ``variant``
# against control characters / embedded HF tokens before they
# reach logger.info(...) lines or the HF cache scan.
repo_id = _validate_logged_identifier(repo_id, "repo_id")
if variant is not None:
variant = _validate_logged_identifier(variant, "variant")
if not _is_valid_repo_id(repo_id):
raise HTTPException(status_code = 400, detail = "Invalid repo_id format")
# Check if model is currently loaded
# Round 25 P1 #2 / #3: round 15 added a path-ownership check to
# the diffusion guard below, but the llama.cpp and safetensors
# guards still only compared logical ``owner/repo`` strings to
# the loaded/loading identifier. If a chat or safetensors model
# was loaded via a LOCAL HF snapshot path (e.g. through the
# ``/load-local-path`` flow), the loaded identifier is the
# absolute snapshot path -- ``owner/repo`` never appears there,
# the guards passed, and ``DELETE /api/models/delete-cached``
# could rmtree an actively mmap'd snapshot.
#
# Build the HF cache roots for ``repo_id`` ONCE up front and reuse
# them in all three guards (llama, safetensors, diffusion). Failure
# to scan the cache fails CLOSED on the assumption that we cannot
# verify ownership safely; mirrors the diffusion path-scan guard.
needle = repo_id.lower()
cache_repo_roots: list[Path] = []
try:
for hf_cache in _all_hf_cache_scans():
for repo_info in hf_cache.repos:
if (
repo_info.repo_type == "model"
and repo_info.repo_id.lower() == needle
):
try:
cache_repo_roots.append(
Path(repo_info.repo_path).expanduser().resolve()
)
except Exception:
pass
except Exception as cache_scan_exc:
logger.warning(
"Could not scan HF cache during delete guard preflight: %s",
cache_scan_exc,
)
raise HTTPException(
status_code = 503,
detail = ("Could not verify cache ownership before deleting. Try again."),
) from cache_scan_exc
def _owned_cache_path_matches(value: Optional[str], roots: list[Path]) -> bool:
"""Return True if ``value`` resolves to (or contains, or is a
child of) any of the HF cache repo roots for the target repo.
Used by the llama / safetensors guards to catch local snapshot
paths the same way the diffusion guard already does.
"""
if not value or not roots:
return False
try:
owned = Path(value).expanduser().resolve()
except Exception:
return False
for root in roots:
try:
if (
owned == root
or _is_path_under(owned, root)
or _is_path_under(root, owned)
):
return True
except Exception:
continue
return False
# Round 26 P1 #13 / #14: helper/advisor GGUF loads run on a
# PRIVATE LlamaCppBackend, so the global backend below cannot see
# them. utils/datasets/llm_assist.py publishes the active repo
# via helper_advisor_owns_repo() for exactly this guard. Fail
# closed on the variant question (block any variant of the repo)
# because helper/advisor flows do not pass a variant through.
try:
from utils.datasets.llm_assist import helper_advisor_owns_repo
if helper_advisor_owns_repo(repo_id):
raise HTTPException(
status_code = 409,
detail = "Cannot delete a model while AI Assist is using it",
)
except HTTPException:
raise
except Exception as e:
logger.warning(
"Could not check helper/advisor backend status before cache delete: %s", e
)
raise HTTPException(
status_code = 503,
detail = "Could not verify AI Assist load status before deleting cache",
) from e
# Check if model is currently loaded OR loading. is_active and
# not is_loaded means an llama-server download / startup is in
# flight; the cache delete would race the hf_hub_download / mmap.
# Fail CLOSED on exception (503) like the diffusion guard below:
# unverifiable load state means we cannot confirm the delete is
# safe.
try:
from routes.inference import get_llama_cpp_backend
llama_backend = get_llama_cpp_backend()
if llama_backend.is_loaded and llama_backend.model_identifier:
loaded_id = llama_backend.model_identifier.lower()
if loaded_id == repo_id.lower() or loaded_id.startswith(repo_id.lower()):
loaded_id_raw = llama_backend.model_identifier or ""
loaded_id = loaded_id_raw.lower()
loading_id_raw = getattr(llama_backend, "loading_model_identifier", None) or ""
loading_id = loading_id_raw.lower()
loading_variant = (
getattr(llama_backend, "loading_hf_variant", None) or ""
).lower()
# Also consult the pending-load identifier: a multi-GB HF
# download stays in ``loading_model_identifier`` until the
# download completes, before ``model_identifier`` is set
# (round 13 P1 #6). Without this check the cache directory
# the download was writing into could be rmtree'd mid-flight.
# Round 16 P1 #1: pair against ``loading_hf_variant`` so a
# delete of a DIFFERENT cached quant from the same repo
# (loading Q4_K_M, deleting cached Q8_0) is allowed; only
# block when the requested variant matches what is being
# downloaded. Mirrors the /delete-finetuned pairing.
requested_variant = (variant or "").lower()
# Round 25 P1 #2: also match by HF cache snapshot path so
# local-path GGUF chat loads block the cache delete that
# owns their snapshot.
loading_matches_repo = loading_id == needle or _owned_cache_path_matches(
loading_id_raw, cache_repo_roots
)
if loading_matches_repo:
same_loading_variant = (
not requested_variant
or not loading_variant
or requested_variant == loading_variant
)
if same_loading_variant:
raise HTTPException(
status_code = 409,
detail = "Cannot delete a model while it is loading",
)
# Exact match only (case-insensitive). Prefix match would
# block deleting unrelated ``org/model`` while
# ``org/model-v2`` is loaded -- same surface the diffusion
# guard fixed in round 5. Per-variant deletes that target a
# DIFFERENT quant than the loaded one are allowed so the
# llama and diffusion paths stay symmetric (round 14 P1 #7).
loaded_matches_repo = loaded_id == needle or _owned_cache_path_matches(
loaded_id_raw, cache_repo_roots
)
if loaded_matches_repo and (
llama_backend.is_loaded or getattr(llama_backend, "is_active", False)
):
loaded_variant = (getattr(llama_backend, "hf_variant", None) or "").lower()
same_variant = (
not requested_variant
or not loaded_variant
or requested_variant == loaded_variant
)
if same_variant:
raise HTTPException(
status_code = 400,
detail = "Unload the model before deleting",
)
except HTTPException:
raise
except Exception:
pass
except Exception as e:
logger.warning(
"Could not check llama.cpp backend status before cache delete: %s", e
)
raise HTTPException(
status_code = 503,
detail = "Could not verify llama.cpp load status before deleting cache",
) from e
try:
inference_backend = get_inference_backend()
if inference_backend.active_model_name:
active = inference_backend.active_model_name.lower()
if active == repo_id.lower() or active.startswith(repo_id.lower()):
loading_models = getattr(inference_backend, "loading_models", set()) or set()
# Loading set holds model identifiers currently being
# downloaded / instantiated; treat them like active loads
# so a delete cannot race a partial mmap.
# Exact match only on the logical ``owner/repo`` side, but
# also match local snapshot paths (round 25 P1 #3) so a
# safetensors model loaded from a local HF snapshot path
# cannot have its cache rmtree'd out from under it.
for loading_model in loading_models:
ml_raw = loading_model or ""
ml = ml_raw.lower()
if ml == needle or _owned_cache_path_matches(ml_raw, cache_repo_roots):
raise HTTPException(
status_code = 409,
detail = "Cannot delete a model while it is loading",
)
active_model_raw = inference_backend.active_model_name
if active_model_raw:
active = active_model_raw.lower()
if active == needle or _owned_cache_path_matches(
active_model_raw, cache_repo_roots
):
raise HTTPException(
status_code = 400,
detail = "Unload the model before deleting",
)
except HTTPException:
raise
except Exception:
pass
except Exception as e:
logger.warning(
"Could not check safetensors backend status before cache delete: %s", e
)
raise HTTPException(
status_code = 503,
detail = "Could not verify safetensors load status before deleting cache",
) from e
# Also refuse to delete the cache underlying a loaded OR loading
# diffusion pipeline. The diffusion backend mmap's the GGUF + base
# repo weights and continues to read from the cache long after
# load; deleting them out from under it would corrupt generation.
# is_loading=True is also blocked because a mid-flight
# hf_hub_download / from_single_file would race the rmtree.
# Match exactly on repo_id (case-insensitive) instead of prefix to
# avoid blocking unrelated deletes like "org/model" while
# "org/model-v2" is loaded.
# During a swap (model A loaded, model B loading), status()
# exposes both via ``active_*`` and ``pending_*`` so we check
# every repo the backend currently owns.
# Fail-CLOSED on exception (return 503) like the neighboring
# llama.cpp / safetensors guards: we cannot verify whether the
# delete is safe, so refuse rather than risk corrupting the
# pipeline's mmap.
try:
from core.inference.diffusion import get_diffusion_backend
diff_backend = get_diffusion_backend()
# include_internal=True so we can pair owned raw paths against
# the HF cache snapshot root (round 16 P1 #5).
diff_status = diff_backend.status(include_internal = True)
if diff_status.get("is_loaded") or diff_status.get("is_loading"):
# ``needle`` and ``cache_repo_roots`` come from the
# preflight scan above; round 25 deduplicated the
# diffusion-specific rescan and now all three guards
# share the same fail-closed cache view.
#
# Pair each owned repo with the GGUF variant it actually
# owns (active or pending) so a swap in progress does not
# collapse both quants into the pending one (round 13
# P1 #4). Per-variant delete is still allowed if the
# requested variant differs from the variant that owns
# the matched repo.
for owned_id, owned_gguf in _diffusion_owned_targets(diff_status):
if not owned_id:
continue
owned_matches_repo = owned_id.lower() == needle
if not owned_matches_repo and _owned_cache_path_matches(
owned_id, cache_repo_roots
):
owned_matches_repo = True
if not owned_matches_repo:
continue
if _variant_delete_is_safe_for_owned_gguf(variant, owned_gguf):
continue
raise HTTPException(
status_code = 400,
detail = "Unload the diffusion image model before deleting",
)
except HTTPException:
raise
except Exception as e:
logger.warning(
"Could not check diffusion backend status before cache delete: %s",
e,
)
raise HTTPException(
status_code = 503,
detail = "Could not verify diffusion load status before deleting cache",
) from e
try:
cache_scans = _all_hf_cache_scans()

View file

@ -127,6 +127,11 @@ async def start_training(
This endpoint initiates training in the background and returns immediately.
Use the /status endpoint to check training progress.
"""
# Round 30 P1 #7: track whether we published a public-load pending
# entry so the outer finally clears it on either success or
# failure (including any early HTTPException raised by the helper
# check itself).
training_load_window_published = False
try:
logger.info(f"Starting training job with model: {request.model_name}")
@ -265,37 +270,48 @@ async def start_training(
)
training_kwargs["trust_remote_code"] = True
# Free GPU memory: shut down any running inference/export subprocesses
# before training starts (they'd compete for VRAM otherwise)
try:
from core.inference import get_inference_backend
# Symmetric lifecycle guard: refuse to start training while
# an export job is in flight. Round 10 review #1 -- the
# previous code went straight to ``_release_export_for``,
# which would terminate the in-flight export and corrupt
# the user's output artifact. Now we 409 first; the user
# stops the export and re-submits.
from routes.inference import (
_clear_public_load_window,
_raise_if_export_active,
_raise_if_helper_advisor_busy,
_release_chat_for,
_release_diffusion_for,
_release_export_for,
)
inf_backend = get_inference_backend()
if inf_backend.active_model_name:
logger.info(
"Unloading inference model '%s' to free GPU memory for training",
inf_backend.active_model_name,
)
inf_backend._shutdown_subprocess()
inf_backend.active_model_name = None
inf_backend.models.clear()
except Exception as e:
logger.warning("Could not unload inference model: %s", e)
_raise_if_export_active("training")
# Round 28 P1 #5: refuse before any release fires so AI Assist
# busy does not first tear down idle diffusion/export.
# Round 30 P1 #7: also publishes a public-load pending entry so
# a concurrent helper / advisor start cannot win the start
# lock between our snapshot and start_training flipping
# is_training_active. Paired clear lives in the outer
# ``finally`` below.
_raise_if_helper_advisor_busy("training")
training_load_window_published = True
# Round 18 P1 #8: release settled export FIRST so an export
# cleanup failure preserves the user's currently loaded chat
# model. The previous order (chat -> export) would drop chat
# and then refuse training when a wedged idle export raised,
# leaving the user with nothing loaded.
# Round 24 P1 #2: same reasoning extended to diffusion ->
# chat. A wedged diffusion unload used to fire AFTER the chat
# backend was already gone, so the user lost both chat and
# diffusion on a single failure mode. Order is now
# export -> diffusion -> chat, with chat as the last drop so
# earlier failures preserve it.
await _release_export_for("training")
await _release_diffusion_for("training")
await _release_chat_for("training")
try:
from core.export import get_export_backend
exp_backend = get_export_backend()
if exp_backend.current_checkpoint:
logger.info(
"Shutting down export subprocess to free GPU memory for training"
)
exp_backend._shutdown_subprocess()
exp_backend.current_checkpoint = None
exp_backend.is_vision = False
exp_backend.is_peft = False
except Exception as e:
logger.warning("Could not shut down export subprocess: %s", e)
# (Diffusion release moved above chat in round 24 P1 #2;
# the old trailing call was removed to avoid double-unload.)
# start_training now spawns a subprocess (non-blocking)
success = backend.start_training(job_id = job_id, **training_kwargs)
@ -319,12 +335,31 @@ async def start_training(
except ValueError as e:
logger.warning("Rejected training GPU selection: %s", e)
raise HTTPException(status_code = 400, detail = str(e))
except HTTPException:
# Preserve the intended status code from
# _raise_if_training_active / _raise_if_export_active
# (409) and the gpu-id 400 raises above. Without this
# explicit re-raise the broad ``except Exception`` below
# converts a deliberate 409 into a 500.
raise
except Exception as e:
logger.error(f"Error starting training: {e}", exc_info = True)
raise HTTPException(
status_code = 500,
detail = f"Failed to start training: {str(e)}",
)
finally:
# Round 30 P1 #7: clear the public-load pending entry once the
# start attempt has finished. Skipped when the helper-busy
# check itself raised (no publish to clear) so the counter
# stays in sync with publishes.
if training_load_window_published:
try:
from routes.inference import _clear_public_load_window
except Exception:
pass
else:
_clear_public_load_window("training")
@router.post("/stop", response_model = TrainingStopResponse)

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,336 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
"""Route-level tests for ``/api/inference/images/*``.
Mounts the actual ``inference_router`` on a fresh FastAPI app with the
auth dependency replaced by a stub so we exercise the same FastAPI
handlers Studio ships in production. The diffusion backend is replaced
with an in-memory stub so we don't need diffusers / GPUs to run these.
To stay runnable in a minimal CPU-only env, ``routes/inference.py``
is loaded directly via ``importlib`` so we do NOT trigger
``routes/__init__.py`` -- that file eagerly imports training /
datasets / data_recipe / export and would drag in heavy deps
(matplotlib, etc.) that the diffusion tests do not need.
"""
from __future__ import annotations
import importlib.util
import sys
from pathlib import Path
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from PIL import Image
_BACKEND_ROOT = Path(__file__).resolve().parents[1]
if str(_BACKEND_ROOT) not in sys.path:
sys.path.insert(0, str(_BACKEND_ROOT))
def _import_inference_module():
"""Load ``routes/inference.py`` without executing ``routes/__init__``.
The package init imports training / datasets / data_recipe / export
routers, which pull in matplotlib / pandas / training stack. The
diffusion tests only need the inference module so we side-step the
package import via importlib.spec_from_file_location.
"""
# If a previous test already imported routes the normal way, reuse
# the cached module instead of re-loading.
cached = sys.modules.get("routes.inference")
if cached is not None:
return cached
target = _BACKEND_ROOT / "routes" / "inference.py"
spec = importlib.util.spec_from_file_location(
"routes.inference",
target,
# We do NOT set submodule_search_locations for routes itself
# because that would re-trigger routes/__init__.py. The module
# uses relative imports sparingly; absolute imports resolve via
# sys.path[0] = backend root.
)
assert spec and spec.loader, "could not build spec for routes/inference.py"
module = importlib.util.module_from_spec(spec)
sys.modules["routes.inference"] = module
# Round 15 P3 #9: drop the half-initialised module from
# sys.modules if exec_module() raises, otherwise later tests pick
# up the poisoned entry and report a misleading AttributeError
# instead of the original ImportError.
try:
spec.loader.exec_module(module)
except Exception:
sys.modules.pop("routes.inference", None)
raise
return module
class _FakeBackend:
def __init__(self) -> None:
self._loaded = False
self._repo: str | None = None
self.calls: list[dict] = []
@property
def is_loaded(self) -> bool:
return self._loaded
def status(self) -> dict:
return {
"is_loaded": self._loaded,
"is_loading": False,
"repo_id": self._repo,
"family": "flux.2-klein" if self._loaded else None,
"pipeline_class": "Flux2KleinPipeline" if self._loaded else None,
"base_repo": "black-forest-labs/FLUX.2-klein" if self._loaded else None,
"gguf_filename": None,
"active_repo_id": self._repo,
"active_base_repo": (
"black-forest-labs/FLUX.2-klein" if self._loaded else None
),
# Round 14: guard-facing GGUF filename is now the full
# caller-supplied value, but this fake never sets one so
# both active and pending stay None.
"active_gguf_filename": None,
"pending_repo_id": None,
"pending_base_repo": None,
"pending_gguf_filename": None,
"device": "cpu",
"dtype": "torch.bfloat16",
"loaded_at": 0,
"last_error": None,
"supported_families": [],
}
def load_model(self, repo_id, **kw):
self.calls.append({"op": "load", "repo_id": repo_id, **kw})
self._loaded = True
self._repo = repo_id
return self.status()
def unload_model(self) -> dict:
self._loaded = False
self._repo = None
return {"is_loaded": False}
def generate_image(self, **kw):
self.calls.append({"op": "generate", **kw})
return Image.new("RGB", (kw["width"], kw["height"]), color = (123, 45, 67))
def generate_image_with_metadata(self, **kw):
image = self.generate_image(**kw)
meta = {
"model": self._repo,
"family": "flux.2-klein" if self._loaded else None,
}
return image, meta
@pytest.fixture
def app_with_stub(monkeypatch):
"""Build a FastAPI app that mounts the real inference router with
auth disabled and the diffusion backend swapped for a stub."""
inf = _import_inference_module()
import core.inference.diffusion as d
stub = _FakeBackend()
# Override the singleton accessor the route uses.
monkeypatch.setattr(d, "get_diffusion_backend", lambda: stub)
monkeypatch.setattr(inf, "_get_diffusion_backend", lambda: stub)
app = FastAPI()
# Diffusion image routes live on studio_router so they are NOT
# exposed under /v1 (which would let OpenAI-compat clients
# trigger Studio-only side effects).
app.include_router(inf.router, prefix = "/api/inference")
app.include_router(inf.studio_router, prefix = "/api/inference")
# Bypass auth by overriding the dependency.
from auth.authentication import get_current_subject
app.dependency_overrides[get_current_subject] = lambda: "test-user"
return app, stub
def test_status_when_unloaded(app_with_stub):
app, _ = app_with_stub
c = TestClient(app)
r = c.get("/api/inference/images/status")
assert r.status_code == 200
body = r.json()
assert body["is_loaded"] is False
assert body["repo_id"] is None
def test_generate_without_load_returns_400(app_with_stub):
app, _ = app_with_stub
c = TestClient(app)
r = c.post(
"/api/inference/images/generate",
json = {"prompt": "a red sphere"},
)
assert r.status_code == 400
assert "No diffusion model" in r.json()["detail"]
def test_load_then_generate_round_trip(app_with_stub):
app, stub = app_with_stub
c = TestClient(app)
r = c.post(
"/api/inference/images/load",
json = {
"repo_id": "unsloth/FLUX.2-klein-4B-GGUF",
"gguf_filename": "flux-2-klein-4b-Q4_K_S.gguf",
},
)
assert r.status_code == 200, r.text
assert r.json()["is_loaded"] is True
r = c.post(
"/api/inference/images/generate",
json = {
"prompt": "a tiny synth-pop album cover",
"width": 256,
"height": 256,
"num_inference_steps": 4,
"seed": 7,
},
)
assert r.status_code == 200, r.text
body = r.json()
assert body["image_b64"]
assert body["image_mime"] == "image/png"
assert body["width"] == 256
assert body["height"] == 256
assert body["seed"] == 7
assert body["duration_ms"] >= 0
# Round-trip the base64 -> PIL to confirm it is a real PNG of the
# right size and not, say, an empty string.
import base64
import io
raw = base64.b64decode(body["image_b64"])
decoded = Image.open(io.BytesIO(raw))
assert decoded.format == "PNG"
assert decoded.size == (256, 256)
# Backend stub should have recorded both calls.
ops = [c["op"] for c in stub.calls]
assert ops == ["load", "generate"]
def test_generate_rejects_off_grid_size(app_with_stub):
app, stub = app_with_stub
c = TestClient(app)
c.post(
"/api/inference/images/load",
json = {
"repo_id": "unsloth/FLUX.2-klein-4B-GGUF",
"gguf_filename": "x.gguf",
},
)
r = c.post(
"/api/inference/images/generate",
json = {"prompt": "x", "width": 513, "height": 512},
)
# Pydantic v2 wraps validator errors in 422 by default.
assert r.status_code in (400, 422), r.text
def test_unload_clears_state(app_with_stub):
app, _ = app_with_stub
c = TestClient(app)
c.post(
"/api/inference/images/load",
json = {"repo_id": "unsloth/FLUX.2-klein-4B-GGUF", "gguf_filename": "x.gguf"},
)
r = c.post("/api/inference/images/unload")
assert r.status_code == 200
assert r.json()["is_loaded"] is False
r = c.get("/api/inference/images/status")
assert r.json()["is_loaded"] is False
def test_load_rejects_embedded_hf_token(app_with_stub):
"""Round 15 P1 #5: URL-embedded ``hf_xxxxx`` tokens in repo_id /
base_repo must be rejected with 422 so they never reach
``self._repo_id`` and get echoed back by ``status()``."""
app, _ = app_with_stub
c = TestClient(app)
r = c.post(
"/api/inference/images/load",
json = {
"repo_id": "https://hf_abcdefghij0123456789@huggingface.co/owner/repo",
},
)
assert r.status_code == 422, r.text
body = r.json()
text = repr(body).lower()
assert "hf_token" in text or "embed" in text
# base_repo is also rejected.
r = c.post(
"/api/inference/images/load",
json = {
"repo_id": "owner/repo",
"gguf_filename": "x.gguf",
"base_repo": "https://hf_abcdefghij0123456789@huggingface.co/base/repo",
},
)
assert r.status_code == 422, r.text
def test_load_rejects_control_chars_in_repo_id(app_with_stub):
"""Newline-laden repo ids must be rejected by Pydantic BEFORE the
log line that echoes them. Catches log-injection from authenticated
callers (issues a 422 instead of forging a fake log line)."""
app, _ = app_with_stub
c = TestClient(app)
r = c.post(
"/api/inference/images/load",
json = {"repo_id": "owner/model\nFAKE_LOG_LINE"},
)
assert r.status_code == 422, r.text
body = r.json()
text = repr(body).lower()
assert "control" in text or "repo_id" in text
def test_generate_rejects_oversize_seed(app_with_stub):
"""Huge seeds raise inside torch.Generator.manual_seed; Pydantic
must clamp first with a 422 instead of a 500 traceback."""
app, _ = app_with_stub
c = TestClient(app)
c.post(
"/api/inference/images/load",
json = {"repo_id": "unsloth/FLUX.2-klein-4B-GGUF", "gguf_filename": "x.gguf"},
)
r = c.post(
"/api/inference/images/generate",
json = {"prompt": "x", "seed": 2**100},
)
assert r.status_code == 422, r.text
def test_generate_accepts_uint64_max_seed(app_with_stub):
"""Boundary value: 2**64 - 1 (uint64 max) is the largest seed
torch.Generator on CPU accepts; reject would frustrate users
who paste large seeds from other tooling."""
app, _ = app_with_stub
c = TestClient(app)
c.post(
"/api/inference/images/load",
json = {"repo_id": "unsloth/FLUX.2-klein-4B-GGUF", "gguf_filename": "x.gguf"},
)
r = c.post(
"/api/inference/images/generate",
json = {"prompt": "x", "seed": (2**64) - 1},
)
# The fake backend returns 200 on success; we only care that the
# request did NOT 422 on seed bounds.
assert r.status_code != 422, r.text

View file

@ -66,15 +66,24 @@ from core.inference import llama_cpp as llama_cpp_module
def _load_model_source() -> str:
"""Return the source of ``LlamaCppBackend.load_model``.
"""Return the source of ``LlamaCppBackend.load_model`` PLUS the
internal ``_load_model_impl_locked`` body it delegates to.
Using ``inspect.getsource`` instead of reading the file directly
scopes the assertions to the function that actually launches
llama-server, so neither the presence check nor the location check
can be fooled by a stray occurrence of ``"--no-context-shift"``
elsewhere in the module.
Studio's diffusion PR split ``load_model`` into a thin wrapper
that publishes ``_loading_model_identifier`` under
``_serial_load_lock`` and an inner ``_load_model_impl_locked``
body that actually spawns llama-server. The launch flags and the
``_wait_for_vram_settle`` call now live in the inner method, so
inspecting only ``load_model`` would miss them. Concatenating the
two sources keeps these source-inspection regression tests
working without weakening the scope (we still only look at the
two load entry points, not the entire module).
"""
return inspect.getsource(llama_cpp_module.LlamaCppBackend.load_model)
parts = [inspect.getsource(llama_cpp_module.LlamaCppBackend.load_model)]
impl = getattr(llama_cpp_module.LlamaCppBackend, "_load_model_impl_locked", None)
if impl is not None:
parts.append(inspect.getsource(impl))
return "\n".join(parts)
def test_no_context_shift_is_in_load_model():

View file

@ -271,10 +271,19 @@ def test_load_model_calls_helper_outside_lock_and_uses_last_kill_timestamp():
"""Pin the call site: outside Phase 3 lock, gated on the timestamp,
no ``had_live_process`` in-band flag regression. Mirrors the
``inspect.getsource`` pattern from ``test_llama_cpp_no_context_shift``.
Studio's diffusion PR split ``load_model`` into a thin wrapper +
``_load_model_impl_locked`` that actually launches llama-server, so
look at both sources to keep the assertions scoped to the load entry
points and not the entire module.
"""
import inspect
src = inspect.getsource(LlamaCppBackend.load_model)
parts = [inspect.getsource(LlamaCppBackend.load_model)]
impl = getattr(LlamaCppBackend, "_load_model_impl_locked", None)
if impl is not None:
parts.append(inspect.getsource(impl))
src = "\n".join(parts)
assert "_wait_for_vram_settle" in src
assert "since_kill" in src
assert "self._last_kill_monotonic" in src

View file

@ -18,7 +18,9 @@ import logging
import os
import re
import textwrap
import threading
import time
from collections import Counter
from itertools import islice
from typing import Any, Optional
@ -31,6 +33,137 @@ DEFAULT_HELPER_MODEL_VARIANT = "UD-Q4_K_XL"
README_MAX_CHARS = 1500
# Round 26 P1 #13 / #14: helper/advisor run on PRIVATE LlamaCppBackend
# instances. Expose loading repo ids through thread-safe Counters so
# DELETE /api/models/delete-cached can block while a helper or
# advisor still owns the cache.
#
# Round 28 P1 #2: split into CACHE vs GPU refcounts. precache_helper_gguf
# downloads files (cache ownership) without occupying VRAM (GPU
# ownership), so collapsing them caused the public GPU handoffs to
# 503 during a background precache that did not need the GPU.
# * CACHE: blocks delete-cache for any active downloader / loader
# * GPU : blocks public chat / training / export / diffusion loads
_HELPER_ADVISOR_CACHE_REFCOUNT: Counter[str] = Counter()
_HELPER_ADVISOR_GPU_REFCOUNT: Counter[str] = Counter()
# Round 30 P1 #7-#10: counter of public GPU workloads (chat /
# diffusion / training / export) that have passed the helper-busy
# snapshot but have not yet flipped their public ownership flags
# (``llama.is_loaded`` / ``loading_model_identifier`` /
# ``current_checkpoint`` / ``is_training_active``). Helper / advisor
# starts consult this so they cannot win the start lock and race a
# public load that already destroyed the previous owner.
_PUBLIC_LOAD_PENDING_COUNT: Counter[str] = Counter()
_HELPER_ADVISOR_LOCK = threading.Lock()
# Round 28 P1 #7 / #8 / #10: serialize helper / advisor STARTS so two
# concurrent invocations cannot both pass the busy precheck before
# either registers. Held only across the precheck + register window,
# not across the full helper run.
# Round 30 P1 #7-#10: public GPU loads also enter under this lock to
# publish their pending counter so a concurrent helper / advisor
# start sees the pending public owner and refuses VRAM.
_HELPER_ADVISOR_START_LOCK = threading.Lock()
def helper_advisor_owns_repo(repo_id: str) -> bool:
"""Return True if any helper/advisor activity (precache OR live
helper / advisor load) currently owns this HF repo id."""
if not repo_id:
return False
needle = repo_id.lower()
with _HELPER_ADVISOR_LOCK:
return _HELPER_ADVISOR_CACHE_REFCOUNT.get(needle, 0) > 0
def helper_advisor_busy() -> bool:
"""True if any helper/advisor load is currently OCCUPYING THE GPU.
Round 28 P1 #2: must not return True for a precache-only download
(it owns disk cache, not VRAM)."""
with _HELPER_ADVISOR_LOCK:
return sum(_HELPER_ADVISOR_GPU_REFCOUNT.values()) > 0
def _register_helper_advisor_repo(repo_id: str, *, gpu_owner: bool = True) -> None:
"""Register a helper/advisor activity. Set ``gpu_owner=False`` for
precache-only downloads that need cache-delete protection but do
not load weights into VRAM."""
if not repo_id:
return
needle = repo_id.lower()
with _HELPER_ADVISOR_LOCK:
_HELPER_ADVISOR_CACHE_REFCOUNT[needle] += 1
if gpu_owner:
_HELPER_ADVISOR_GPU_REFCOUNT[needle] += 1
def _unregister_helper_advisor_repo(repo_id: str, *, gpu_owner: bool = True) -> None:
if not repo_id:
return
needle = repo_id.lower()
with _HELPER_ADVISOR_LOCK:
_HELPER_ADVISOR_CACHE_REFCOUNT[needle] -= 1
if _HELPER_ADVISOR_CACHE_REFCOUNT[needle] <= 0:
_HELPER_ADVISOR_CACHE_REFCOUNT.pop(needle, None)
if gpu_owner:
_HELPER_ADVISOR_GPU_REFCOUNT[needle] -= 1
if _HELPER_ADVISOR_GPU_REFCOUNT[needle] <= 0:
_HELPER_ADVISOR_GPU_REFCOUNT.pop(needle, None)
def _publish_public_load_pending(workload: str) -> None:
"""Mark a public GPU workload as mid-handoff. Must be called under
``_HELPER_ADVISOR_START_LOCK`` immediately after the helper-busy
snapshot succeeded (round 30 P1 #7-#10)."""
if not workload:
return
needle = workload.lower()
with _HELPER_ADVISOR_LOCK:
_PUBLIC_LOAD_PENDING_COUNT[needle] += 1
def _release_public_load_pending(workload: str) -> None:
"""Decrement the pending public-load counter once per matched
publish. Safe to call in finally even if the load failed."""
if not workload:
return
needle = workload.lower()
with _HELPER_ADVISOR_LOCK:
_PUBLIC_LOAD_PENDING_COUNT[needle] -= 1
if _PUBLIC_LOAD_PENDING_COUNT[needle] <= 0:
_PUBLIC_LOAD_PENDING_COUNT.pop(needle, None)
def public_load_pending(*, excluding: str | None = None) -> bool:
"""True if any public GPU workload has passed its helper-busy
snapshot but not yet flipped its public ownership flags. Helper /
advisor starts treat this as busy so they cannot race a public
load mid-handoff.
Round 38 P1: ``excluding`` lets a route-wrapped backend call
skip the marker its own route layer already published (e.g. the
diffusion route publishes ``diffusion`` before calling into
``backend.load_model``, which publishes ``diffusion-backend`` --
the backend should ignore its own ``diffusion`` marker so the
parity check does not self-block) while still seeing every
OTHER in-flight public workload."""
ignored = excluding.lower() if excluding else None
with _HELPER_ADVISOR_LOCK:
return any(
count > 0 and workload != ignored
for workload, count in _PUBLIC_LOAD_PENDING_COUNT.items()
)
def public_load_pending_for(workload: str) -> bool:
"""True if a specific public GPU workload is mid-handoff. Used by
release helpers to refuse a destructive teardown while the matching
/export/* or /chat /load_* route is still in its publish window."""
if not workload:
return False
needle = workload.lower()
with _HELPER_ADVISOR_LOCK:
return _PUBLIC_LOAD_PENDING_COUNT.get(needle, 0) > 0
def _strip_think_tags(text: str) -> str:
"""Strip <think>...</think> reasoning blocks emitted by some models.
@ -72,6 +205,12 @@ def precache_helper_gguf():
"UNSLOTH_HELPER_MODEL_VARIANT", DEFAULT_HELPER_MODEL_VARIANT
)
# Round 27 P1 #4: register the repo so DELETE /api/models/delete-cached
# cannot rmtree the cache directory while we are mid-download.
# Round 28 P1 #2: precache only downloads files; it does NOT occupy
# VRAM. Use gpu_owner=False so helper_advisor_busy() does not block
# public GPU workloads during a background pre-cache.
_register_helper_advisor_repo(repo, gpu_owner = False)
try:
from huggingface_hub import HfApi, hf_hub_download
from huggingface_hub.utils import disable_progress_bars, enable_progress_bars
@ -103,12 +242,143 @@ def precache_helper_gguf():
except Exception as e:
logger.warning(f"Failed to pre-cache helper GGUF: {e}")
finally:
_unregister_helper_advisor_repo(repo, gpu_owner = False)
try:
enable_progress_bars()
except Exception as e:
pass
def _diffusion_image_model_busy() -> bool:
"""Round 22 P1 #2 / #3: helper / advisor GGUFs share VRAM with
the Images page diffusion pipeline. Public chat / training /
export routes call the strict ``_release_diffusion_for`` helper
before allocating, but these dataset-side helpers used to load
llama-server directly with no diffusion guard at all. Skip the
helper GGUF when ``DiffusionBackend.status()`` reports loaded /
loading so we do not double-own VRAM. Fail closed (treat as
busy) on any status() error to preserve the resident image
model rather than racing it for memory.
"""
try:
from core.inference.diffusion import get_diffusion_backend
except Exception:
return False
try:
status = get_diffusion_backend().status()
except Exception:
return True
return bool(status.get("is_loaded") or status.get("is_loading"))
def _gpu_workload_busy_for_helper() -> bool:
"""Round 23 P1 #3 / #4: the diffusion-only guard from round 22
let the helper / advisor GGUF run on top of a live training run
or a resident export checkpoint. Extend the busy check to those
workloads too so any GPU owner (Images, Training, Export)
blocks the helper instead of double-owning VRAM. Each step
fails closed: an unverifiable status counts as busy so the
user's primary workload is preserved over the optional helper.
Round 24 P1 #1: extended to also catch a Chat-backend GPU owner.
The helper GGUF used to run on top of a loaded GGUF chat model
(llama-server) or safetensors chat model and OOM their shared
GPU; mirror the diffusion check by inspecting llama
``is_loaded`` / ``is_active`` / ``loading_model_identifier`` and
safetensors ``active_model_name`` / ``loading_models``.
Round 28 P1 #9: also catch another helper / advisor that already
owns a private LlamaCppBackend. Without this two concurrent
helpers could both pass the precheck and OOM each other.
"""
if helper_advisor_busy():
logger.info(
"Skipping helper GGUF while another helper/advisor is using the GPU"
)
return True
# Round 30 P1 #7-#10: a public GPU load (chat / diffusion / training /
# export) that has passed its busy snapshot but not yet flipped its
# public ownership flags is still mid-handoff. Refuse so the helper
# does not race it for VRAM after the previous owner was torn down.
if public_load_pending():
logger.info("Skipping helper GGUF while a public GPU load is mid-handoff")
return True
if _diffusion_image_model_busy():
return True
try:
from routes.inference import get_llama_cpp_backend
except Exception:
pass
else:
try:
llama = get_llama_cpp_backend()
if (
getattr(llama, "is_loaded", False)
or getattr(llama, "is_active", False)
or getattr(llama, "loading_model_identifier", None)
):
logger.info(
"Skipping helper GGUF while a GGUF chat model is loaded/loading"
)
return True
except Exception:
logger.info(
"Skipping helper GGUF because llama-server status is unavailable"
)
return True
try:
from core.inference import get_inference_backend
except Exception:
pass
else:
try:
inf = get_inference_backend()
active = getattr(inf, "active_model_name", None)
loading = set(getattr(inf, "loading_models", set()) or set())
if active or loading:
logger.info(
"Skipping helper GGUF while a safetensors chat model is loaded/loading"
)
return True
except Exception:
logger.info(
"Skipping helper GGUF because safetensors chat status is unavailable"
)
return True
try:
from core.training import get_training_backend
except Exception:
pass
else:
try:
if get_training_backend().is_training_active():
logger.info("Skipping helper GGUF while training is active")
return True
except Exception:
logger.info("Skipping helper GGUF because training status is unavailable")
return True
try:
from core.export import get_export_backend
except Exception:
return False
try:
exp = get_export_backend()
is_active = getattr(exp, "is_export_active", None)
if (is_active and is_active()) or getattr(exp, "current_checkpoint", None):
logger.info("Skipping helper GGUF while export owns the GPU")
return True
except Exception:
logger.info("Skipping helper GGUF because export status is unavailable")
return True
return False
def _run_with_helper(prompt: str, max_tokens: int = 256) -> Optional[str]:
"""
Load helper model, run one chat completion, unload.
@ -118,13 +388,28 @@ def _run_with_helper(prompt: str, max_tokens: int = 256) -> Optional[str]:
if os.environ.get("UNSLOTH_HELPER_MODEL_DISABLE", "").strip() in ("1", "true"):
return None
# Round 23 P1 #3: round 22 only guarded against a busy
# diffusion pipeline. Training / export own the same GPU too,
# so use the broader helper that gates on all three workloads.
# Round 28 P1 #7 / #10: serialize the busy check + register pair
# so two concurrent helper invocations cannot both pass the
# precheck before either registers and then OOM each other.
repo = os.environ.get("UNSLOTH_HELPER_MODEL_REPO", DEFAULT_HELPER_MODEL_REPO)
variant = os.environ.get(
"UNSLOTH_HELPER_MODEL_VARIANT", DEFAULT_HELPER_MODEL_VARIANT
)
with _HELPER_ADVISOR_START_LOCK:
if _gpu_workload_busy_for_helper():
return None
_register_helper_advisor_repo(repo)
backend = None
try:
# Round 26 P1 #1 / #3 / #13 / #14: use a PRIVATE backend so the
# helper can never preempt or be preempted by the user's
# chat backend and cannot accidentally unload it in finally.
# The active repo is published via _register_helper_advisor_repo
# above so DELETE /api/models/delete-cached can still block the
# cache rmtree while the helper is downloading or mmap'ing.
from core.inference.llama_cpp import LlamaCppBackend
backend = LlamaCppBackend()
@ -176,6 +461,7 @@ def _run_with_helper(prompt: str, max_tokens: int = 256) -> Optional[str]:
logger.info("Helper model unloaded")
except Exception:
pass
_unregister_helper_advisor_repo(repo)
# ─── Public API ───────────────────────────────────────────────────────
@ -508,13 +794,26 @@ def _run_multi_pass_advisor(
if os.environ.get("UNSLOTH_HELPER_MODEL_DISABLE", "").strip() in ("1", "true"):
return None
# Round 23 P1 #4: extend the round 22 diffusion-only check to
# training + export so the advisor cannot race the user's
# active workload for GPU memory.
# Round 28 P1 #8 / #10: serialize the precheck + register pair so
# two concurrent advisor invocations cannot both pass before
# either registers and then OOM each other.
repo = os.environ.get("UNSLOTH_HELPER_MODEL_REPO", DEFAULT_HELPER_MODEL_REPO)
variant = os.environ.get(
"UNSLOTH_HELPER_MODEL_VARIANT", DEFAULT_HELPER_MODEL_VARIANT
)
with _HELPER_ADVISOR_START_LOCK:
if _gpu_workload_busy_for_helper():
return None
_register_helper_advisor_repo(repo)
backend = None
try:
# Round 26 P1 #2 / #4 / #13 / #14: mirror ``_run_with_helper``
# and use a PRIVATE backend. Round 25's global-backend swap
# introduced chat-evict races and finally-eviction bugs.
# The registry above keeps delete-cache safe.
from core.inference.llama_cpp import LlamaCppBackend
backend = LlamaCppBackend()
@ -849,6 +1148,7 @@ def _run_multi_pass_advisor(
logger.info("Advisor model unloaded")
except Exception:
pass
_unregister_helper_advisor_repo(repo)
def llm_conversion_advisor(

View file

@ -9,6 +9,7 @@ import { Route as dataRecipeRoute } from "./routes/data-recipes.$recipeId";
import { Route as chatRoute } from "./routes/chat";
import { Route as exportRoute } from "./routes/export";
import { Route as gridTestRoute } from "./routes/grid-test";
import { Route as imagesRoute } from "./routes/images";
import { Route as indexRoute } from "./routes/index";
import { Route as loginRoute } from "./routes/login";
import { Route as onboardingRoute } from "./routes/onboarding";
@ -26,6 +27,7 @@ const routeTree = rootRoute.addChildren([
studioRoute,
chatRoute,
exportRoute,
imagesRoute,
dataRecipesRoute,
dataRecipeRoute,
]);

View file

@ -0,0 +1,21 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { createRoute } from "@tanstack/react-router";
import { lazy } from "react";
import { requireAuth } from "../auth-guards";
import { Route as rootRoute } from "./__root";
const ImagesPage = lazy(() =>
import("@/features/images").then((m) => ({
default: m.ImagesPage,
})),
);
export const Route = createRoute({
getParentRoute: () => rootRoute,
path: "/images",
staticData: { title: "Images" },
beforeLoad: () => requireAuth(),
component: ImagesPage,
});

View file

@ -50,6 +50,7 @@ import {
Globe02Icon,
HelpCircleIcon,
Logout01Icon,
PaintBrush02Icon,
Search01Icon,
PowerIcon,
PencilEdit02Icon,
@ -497,6 +498,18 @@ export function AppSidebar() {
}}
/>
<NavItem
icon={PaintBrush02Icon}
label="Images"
active={pathname === "/images" || pathname.startsWith("/images/")}
disabled={chatOnly}
onClick={() => {
if (chatOnly) return;
navigate({ to: "/images" });
closeMobileIfOpen();
}}
/>
<NavItem
icon={DownloadSquare01Icon}
label="Export"

View file

@ -0,0 +1,138 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
// Thin client for the diffusion image-generation routes exposed by
// studio/backend/routes/inference.py (images/load, images/generate,
// images/status, images/unload). Mirrors the shape returned by
// DiffusionBackend.status() and DiffusionGenerateResponse so the
// page can render results without re-deriving fields client-side.
import { authFetch } from "@/features/auth";
import { readFastApiError } from "@/lib/format-fastapi-error";
export interface DiffusionFamily {
name: string;
pipeline_class: string;
base_repo: string;
}
export interface DiffusionStatus {
is_loaded: boolean;
is_loading: boolean;
repo_id: string | null;
family: string | null;
pipeline_class: string | null;
base_repo: string | null;
gguf_filename: string | null;
device: string | null;
dtype: string | null;
loaded_at: number | null;
last_error: string | null;
supported_families: DiffusionFamily[];
}
export interface DiffusionLoadRequest {
repo_id: string;
gguf_filename?: string;
base_repo?: string;
family?: string;
hf_token?: string;
enable_model_cpu_offload?: boolean;
}
export interface DiffusionGenerateRequest {
prompt: string;
negative_prompt?: string;
num_inference_steps?: number;
guidance_scale?: number;
width?: number;
height?: number;
// bigint when the seed exceeds Number.MAX_SAFE_INTEGER, otherwise
// number. The wire format is always a JSON integer; see
// ``stringifyWithBigInt`` below.
seed?: number | bigint;
}
export interface DiffusionGenerateResponse {
image_b64: string;
image_mime: string;
width: number;
height: number;
num_inference_steps: number;
guidance_scale: number;
/**
* Numeric seed. Safe ONLY for values <= Number.MAX_SAFE_INTEGER.
* For larger seeds, prefer ``seed_str`` (full-precision decimal).
*/
seed: number | null;
/** Decimal string with full uint64 precision. Use this for display
* and reproduction when the user pastes the seed back in. */
seed_str: string | null;
duration_ms: number;
model: string | null;
family: string | null;
}
async function parseJson<T>(res: Response): Promise<T> {
if (!res.ok) throw new Error(await readFastApiError(res));
return (await res.json()) as T;
}
export async function fetchDiffusionStatus(): Promise<DiffusionStatus> {
return parseJson<DiffusionStatus>(
await authFetch("/api/inference/images/status"),
);
}
export async function loadDiffusionModel(
payload: DiffusionLoadRequest,
): Promise<DiffusionStatus> {
return parseJson<DiffusionStatus>(
await authFetch("/api/inference/images/load", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(payload),
}),
);
}
export async function unloadDiffusionModel(): Promise<{ is_loaded: boolean }> {
return parseJson<{ is_loaded: boolean }>(
await authFetch("/api/inference/images/unload", { method: "POST" }),
);
}
/** JSON.stringify cannot serialise BigInt directly. Pull the seed
* BigInt out, stringify the rest of the payload normally, then
* splice the seed's decimal digits back into the JSON literal at the
* exact ``"seed":<int>`` slot.
*
* Avoids the previous regex-over-JSON approach, which could be
* tripped by a user-supplied prompt that exactly matched the
* sentinel string. With this approach the only thing we touch is
* the literal ``"seed":<number>`` substring we wrote ourselves.
*/
function stringifyWithBigInt(value: DiffusionGenerateRequest): string {
const { seed, ...rest } = value;
if (typeof seed !== "bigint") {
return JSON.stringify(value);
}
// Serialise the rest without seed, then inject the seed at the end
// of the object literal as a JSON integer. Strip the trailing "}"
// and re-append once the field is added.
const base = JSON.stringify(rest);
const inner = base.length === 2 /* '{}' */ ? "" : base.slice(1, -1) + ",";
return `{${inner}"seed":${seed.toString()}}`;
}
export async function generateDiffusionImage(
payload: DiffusionGenerateRequest,
): Promise<DiffusionGenerateResponse> {
return parseJson<DiffusionGenerateResponse>(
await authFetch("/api/inference/images/generate", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: stringifyWithBigInt(payload),
}),
);
}

View file

@ -0,0 +1,620 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { Label } from "@/components/ui/label";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { SectionCard } from "@/components/section-card";
import { Slider } from "@/components/ui/slider";
import { Spinner } from "@/components/ui/spinner";
import { Textarea } from "@/components/ui/textarea";
import { toast } from "@/lib/toast";
import { PaintBrush02Icon, SparklesIcon, GpuIcon } from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
import {
fetchDiffusionStatus,
generateDiffusionImage,
loadDiffusionModel,
unloadDiffusionModel,
type DiffusionGenerateResponse,
type DiffusionStatus,
} from "./api";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
// Curated short list of working diffusion GGUFs. Picked to span
// size + license so any GPU class has at least one viable option:
// FLUX.2 klein 4B -> ~13 GB VRAM with Q4_K_S, Apache 2.0
// FLUX.2 klein 9B -> ~17 GB VRAM, FLUX [klein] non-commercial (gated)
// FLUX.2 dev -> ~24+ GB VRAM, FLUX [dev] non-commercial (gated)
// FLUX.1 dev -> ~12 GB VRAM, older but widely tested (gated)
//
// Filenames mirror the Hub canonical case (lowercase 'flux-2-klein-4b')
// and base_repo is set explicitly so the backend never falls back to the
// family default. The CLI on the backend can load anything supported by
// detect_family(); this list just keeps the picker compact for the v1 UI.
const CURATED_MODELS: Array<{
label: string;
repo_id: string;
default_gguf: string;
base_repo: string;
family: string;
notes: string;
}> = [
{
label: "FLUX.2 klein base 4B (Q4_K_S, Apache 2.0)",
repo_id: "unsloth/FLUX.2-klein-base-4B-GGUF",
default_gguf: "flux-2-klein-base-4b-Q4_K_S.gguf",
base_repo: "black-forest-labs/FLUX.2-klein-base-4B",
family: "flux.2-klein",
notes: "13 GB VRAM, fastest. Apache 2.0, ungated.",
},
{
label: "FLUX.2 klein 4B (Q4_K_S, distilled)",
repo_id: "unsloth/FLUX.2-klein-4B-GGUF",
default_gguf: "flux-2-klein-4b-Q4_K_S.gguf",
// Distilled GGUF must pair with the distilled base, not the Base
// checkpoint. The Hub model card for the GGUF lists
// base_model: black-forest-labs/FLUX.2-klein-4B.
base_repo: "black-forest-labs/FLUX.2-klein-4B",
family: "flux.2-klein",
notes: "13 GB VRAM. Distilled klein 4B. Requires HF access to FLUX.2 klein 4B.",
},
{
label: "FLUX.2 klein 9B (Q4_K_S, gated)",
repo_id: "unsloth/FLUX.2-klein-9B-GGUF",
default_gguf: "flux-2-klein-9b-Q4_K_S.gguf",
base_repo: "black-forest-labs/FLUX.2-klein-9B",
family: "flux.2-klein",
notes: "17 GB VRAM. Higher quality distilled. Requires HF access to FLUX.2 klein 9B.",
},
{
label: "FLUX.2 dev (Q4_K_S, gated)",
repo_id: "unsloth/FLUX.2-dev-GGUF",
default_gguf: "flux2-dev-Q4_K_S.gguf",
base_repo: "black-forest-labs/FLUX.2-dev",
family: "flux.2",
notes: "24+ GB VRAM. Requires HF access to FLUX.2 dev.",
},
{
label: "FLUX.1 dev (Q4_K_S, city96, gated)",
repo_id: "city96/FLUX.1-dev-gguf",
default_gguf: "flux1-dev-Q4_K_S.gguf",
base_repo: "black-forest-labs/FLUX.1-dev",
family: "flux.1",
notes: "12 GB VRAM. Older but widely tested. Requires HF access to FLUX.1 dev.",
},
];
const DEFAULT_PRESET = CURATED_MODELS[0];
const RESOLUTION_PRESETS: Array<{ label: string; w: number; h: number }> = [
{ label: "Square 1024", w: 1024, h: 1024 },
{ label: "Square 768", w: 768, h: 768 },
{ label: "Square 512", w: 512, h: 512 },
{ label: "Portrait 832x1216", w: 832, h: 1216 },
{ label: "Landscape 1216x832", w: 1216, h: 832 },
];
export function ImagesPage() {
const [status, setStatus] = useState<DiffusionStatus | null>(null);
const [refreshingStatus, setRefreshingStatus] = useState(false);
const [busy, setBusy] = useState<"idle" | "loading" | "unloading" | "generating">("idle");
const [presetIndex, setPresetIndex] = useState(0);
const [customRepoId, setCustomRepoId] = useState("");
const [customGguf, setCustomGguf] = useState("");
const [customBaseRepo, setCustomBaseRepo] = useState("");
const [customFamily, setCustomFamily] = useState<string>("auto");
const [useCustom, setUseCustom] = useState(false);
const [hfToken, setHfToken] = useState("");
const [prompt, setPrompt] = useState("a tiny ginger sloth coding in a sunlit treehouse, photorealistic");
const [negativePrompt, setNegativePrompt] = useState("");
const [steps, setSteps] = useState(24);
const [guidance, setGuidance] = useState(3.5);
const [resolutionIdx, setResolutionIdx] = useState(0);
const [seed, setSeed] = useState<string>("");
const [results, setResults] = useState<DiffusionGenerateResponse[]>([]);
const lastErrorRef = useRef<string | null>(null);
const preset = CURATED_MODELS[presetIndex] ?? DEFAULT_PRESET;
const resolution = RESOLUTION_PRESETS[resolutionIdx];
// Round 30 P2 #12: split the fetch from the spinner toggle so the
// mount + auto-poll effects can call the fetch without the
// synchronous setRefreshingStatus(true) that tripped
// react-hooks/set-state-in-effect.
const fetchAndUpdateStatus = useCallback(async () => {
try {
const next = await fetchDiffusionStatus();
setStatus(next);
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
if (lastErrorRef.current !== msg) {
lastErrorRef.current = msg;
toast.error("Could not fetch image-model status", { description: msg });
}
}
}, []);
const refreshStatus = useCallback(async () => {
setRefreshingStatus(true);
try {
await fetchAndUpdateStatus();
} finally {
setRefreshingStatus(false);
}
}, [fetchAndUpdateStatus]);
useEffect(() => {
// Defer the mount fetch out of the synchronous effect body so the
// setStatus call inside fetchAndUpdateStatus does not trip the
// react-hooks/set-state-in-effect rule.
const id = window.setTimeout(() => {
void fetchAndUpdateStatus();
}, 0);
return () => window.clearTimeout(id);
}, [fetchAndUpdateStatus]);
// Round 27 P2: when the backend is mid-load (is_loading=true) the
// status label froze at "Loading..." until the user clicked
// Refresh. Auto-poll every 2 s while a load is in flight so the
// UI tracks real backend progress.
useEffect(() => {
if (!status?.is_loading) return;
const id = window.setInterval(() => {
void fetchAndUpdateStatus();
}, 2000);
return () => window.clearInterval(id);
}, [status?.is_loading, fetchAndUpdateStatus]);
const handleLoad = useCallback(async () => {
setBusy("loading");
try {
const repo = useCustom ? customRepoId.trim() : preset.repo_id;
const gguf = useCustom ? customGguf.trim() || undefined : preset.default_gguf;
// Custom mode lets the user pin a family explicitly because
// detect_family is substring-based and exotic repo names (custom
// fine-tunes, third-party mirrors) frequently fail to match.
// "auto" leaves the override blank and lets the backend infer.
const family = useCustom
? customFamily === "auto"
? undefined
: customFamily
: preset.family;
// Always pass base_repo for curated entries; custom-repo mode
// now also lets the user pin one because private / mirrored
// GGUFs (e.g. a 9B klein transformer) would otherwise fall
// back to the family-default 4B base and 500 on load. Empty
// string still falls back to the backend's smart-base /
// repo-id defaults.
const baseRepo = useCustom
? customBaseRepo.trim() || undefined
: preset.base_repo;
if (!repo) {
toast.error("Pick a model first");
return;
}
const next = await loadDiffusionModel({
repo_id: repo,
gguf_filename: gguf,
base_repo: baseRepo,
family,
hf_token: hfToken.trim() || undefined,
});
setStatus(next);
toast.success("Loaded image model", { description: next.repo_id ?? undefined });
} catch (err) {
toast.error("Failed to load image model", {
description: err instanceof Error ? err.message : String(err),
});
// Backend clears its old pipeline before allocating the new one;
// a failed swap leaves status.is_loaded=false while our local
// copy still says loaded. Re-fetch so Generate disables and the
// user does not see a stale "Loaded:" label.
await refreshStatus();
} finally {
setBusy("idle");
}
}, [useCustom, customRepoId, customGguf, customBaseRepo, customFamily, preset, hfToken, refreshStatus]);
const handleUnload = useCallback(async () => {
setBusy("unloading");
try {
await unloadDiffusionModel();
await refreshStatus();
} catch (err) {
toast.error("Failed to unload image model", {
description: err instanceof Error ? err.message : String(err),
});
// Round 27 P2: a partial unload (subprocess refused to terminate,
// 503 from the backend) used to leave the UI showing the old
// "Loaded:" label even though the backend state was half torn
// down. Refresh so the button states match reality (mirrors
// handleLoad above which always re-fetches on catch).
await refreshStatus();
} finally {
setBusy("idle");
}
}, [refreshStatus]);
const handleGenerate = useCallback(async () => {
if (!prompt.trim()) {
toast.error("Prompt is empty");
return;
}
setBusy("generating");
try {
// Reject non-integer seeds and clamp to the [-2^63, 2^64 - 1]
// range the backend's torch.Generator can actually pack. JSON
// serialises BigInts as plain integers, so we keep the wire
// format compatible and avoid the Number(seed) precision loss
// (>= 2^53 silently rounds, producing a different image than
// the seed the user typed). When the seed fits a safe integer
// it goes through unchanged; larger seeds ride along as their
// BigInt-derived string via the wire-format BigInt JSON helper
// in the api layer.
const seedStr = seed.trim();
let parsedSeed: number | bigint | undefined;
if (seedStr) {
if (!/^-?\d+$/.test(seedStr)) {
toast.error("Seed must be an integer");
return;
}
let big: bigint;
try {
big = BigInt(seedStr);
} catch {
toast.error("Seed must be an integer");
return;
}
const SEED_MIN = -(BigInt(2) ** BigInt(63));
const SEED_MAX = BigInt(2) ** BigInt(64) - BigInt(1);
if (big < SEED_MIN || big > SEED_MAX) {
toast.error(
"Seed must be in [-2^63, 2^64 - 1] (the torch.Generator range)",
);
return;
}
// Use a plain Number when it fits a safe integer so the
// existing api.ts JSON serialiser does not break on BigInt;
// otherwise pass the BigInt and let api.ts emit it as a JSON
// number via a custom replacer.
const SAFE_MAX = BigInt(Number.MAX_SAFE_INTEGER);
const SAFE_MIN = -SAFE_MAX;
parsedSeed = big >= SAFE_MIN && big <= SAFE_MAX ? Number(big) : big;
}
const out = await generateDiffusionImage({
prompt,
negative_prompt: negativePrompt.trim() || undefined,
num_inference_steps: steps,
guidance_scale: guidance,
width: resolution.w,
height: resolution.h,
seed: parsedSeed,
});
setResults((prev) => [out, ...prev].slice(0, 12));
} catch (err) {
toast.error("Image generation failed", {
description: err instanceof Error ? err.message : String(err),
});
} finally {
setBusy("idle");
}
}, [prompt, negativePrompt, steps, guidance, resolution, seed]);
const statusLabel = useMemo(() => {
if (!status) return refreshingStatus ? "Checking..." : "Not loaded";
if (status.is_loading) return "Loading...";
if (status.is_loaded) {
const dev = status.device ? ` on ${status.device}` : "";
return `Loaded: ${status.repo_id ?? "(unknown)"} (${status.family ?? "unknown"})${dev}`;
}
return "Not loaded";
}, [status, refreshingStatus]);
// FLUX.2 / FLUX.2 klein pipelines do NOT accept negative_prompt and
// would 500 if we sent one through. The backend strips the field
// defensively but hiding it client-side keeps the UI honest.
// Round 29 P2 #12: also honour the user-picked customFamily when no
// model is loaded yet, so a Custom HF repo with family flux.2 /
// flux.2-klein hides the negative-prompt field correctly.
const supportsNegativePrompt = useMemo(() => {
const family = status?.family;
if (!family) {
let candidate: string | undefined;
if (useCustom) {
candidate = customFamily === "auto" ? undefined : customFamily;
} else {
candidate = preset.family;
}
if (!candidate) return true;
return !candidate.startsWith("flux.2");
}
return !family.startsWith("flux.2");
}, [status, useCustom, customFamily, preset.family]);
return (
<div className="flex flex-1 flex-col gap-4 overflow-y-auto p-4 sm:p-6">
<SectionCard
icon={<HugeiconsIcon icon={GpuIcon} className="size-5" strokeWidth={1.5} />}
title="Local image generation"
description={
"Run diffusion GGUFs from Hugging Face on your own GPU. " +
"Pick a curated FLUX.2 model or paste any unsloth/* GGUF repo."
}
>
<div className="flex flex-col gap-3">
<div className="flex flex-col gap-2">
<Label>Model</Label>
<Select
value={useCustom ? "custom" : String(presetIndex)}
onValueChange={(v) => {
if (v === "custom") {
setUseCustom(true);
} else {
setUseCustom(false);
setPresetIndex(Number(v));
}
}}
>
<SelectTrigger>
<SelectValue placeholder="Pick a model" />
</SelectTrigger>
<SelectContent>
{CURATED_MODELS.map((m, idx) => (
<SelectItem key={m.repo_id} value={String(idx)}>
{m.label}
</SelectItem>
))}
<SelectItem value="custom">Custom HF repo...</SelectItem>
</SelectContent>
</Select>
{!useCustom && (
<p className="text-xs text-muted-foreground">{preset.notes}</p>
)}
</div>
{useCustom && (
<div className="flex flex-col gap-2">
<Label>HF repo id</Label>
<Input
value={customRepoId}
onChange={(e) => setCustomRepoId(e.target.value)}
placeholder="unsloth/FLUX.2-klein-4B-GGUF"
/>
<Label>GGUF filename (optional)</Label>
<Input
value={customGguf}
onChange={(e) => setCustomGguf(e.target.value)}
placeholder="FLUX.2-klein-4B-Q4_K_S.gguf"
/>
<Label>Base diffusers repo (optional)</Label>
<Input
value={customBaseRepo}
onChange={(e) => setCustomBaseRepo(e.target.value)}
placeholder="black-forest-labs/FLUX.2-klein-9B"
/>
<p className="text-xs text-muted-foreground">
{"Optional. Defaults to the family base. Set this when "}
{"your GGUF expects a non-default base (for example a 9B "}
{"transformer that would otherwise fall back to a 4B base)."}
</p>
<Label>Pipeline family (override)</Label>
<Select
value={customFamily}
onValueChange={setCustomFamily}
>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="auto">Auto-detect from repo id</SelectItem>
<SelectItem value="flux.2-klein">FLUX.2 klein</SelectItem>
<SelectItem value="flux.2">FLUX.2</SelectItem>
<SelectItem value="flux.1">FLUX.1</SelectItem>
<SelectItem value="qwen-image">Qwen-Image</SelectItem>
<SelectItem value="stable-diffusion-3">Stable Diffusion 3</SelectItem>
<SelectItem value="stable-diffusion-xl">Stable Diffusion XL</SelectItem>
</SelectContent>
</Select>
<p className="text-xs text-muted-foreground">
{"Set this when your repo name does not contain "}
{"a recognised family substring (e.g. private fine-tunes)."}
</p>
</div>
)}
<div className="flex flex-col gap-2">
<Label>Hugging Face token (only for gated repos)</Label>
<Input
type="password"
value={hfToken}
onChange={(e) => setHfToken(e.target.value)}
placeholder="hf_..."
autoComplete="off"
/>
</div>
<div className="flex flex-wrap items-center gap-2">
<Button
onClick={handleLoad}
disabled={busy !== "idle"}
data-testid="diffusion-load"
>
{busy === "loading" ? <Spinner className="mr-2 size-4" /> : null}
Load model
</Button>
<Button
variant="outline"
onClick={handleUnload}
disabled={busy !== "idle" || !status?.is_loaded}
data-testid="diffusion-unload"
>
Unload
</Button>
<Button
variant="ghost"
onClick={() => void refreshStatus()}
disabled={refreshingStatus}
>
Refresh status
</Button>
<span
className="ml-auto text-xs text-muted-foreground"
data-testid="diffusion-status"
>
{statusLabel}
</span>
</div>
</div>
</SectionCard>
<SectionCard
icon={<HugeiconsIcon icon={PaintBrush02Icon} className="size-5" strokeWidth={1.5} />}
title="Prompt"
description="The pipeline runs on the GPU you launched Unsloth Studio on."
>
<div className="flex flex-col gap-3">
<div className="flex flex-col gap-1">
<Label htmlFor="diffusion-prompt">Prompt</Label>
<Textarea
id="diffusion-prompt"
value={prompt}
onChange={(e) => setPrompt(e.target.value)}
rows={3}
data-testid="diffusion-prompt"
/>
</div>
{supportsNegativePrompt ? (
<div className="flex flex-col gap-1">
<Label htmlFor="diffusion-negative">Negative prompt (optional)</Label>
<Textarea
id="diffusion-negative"
value={negativePrompt}
onChange={(e) => setNegativePrompt(e.target.value)}
rows={2}
/>
</div>
) : (
<p className="text-xs text-muted-foreground">
{"FLUX.2 and FLUX.2 klein do not accept a negative prompt. "}
{"Steer the output via the main prompt instead."}
</p>
)}
<div className="grid grid-cols-1 gap-3 sm:grid-cols-3">
<div className="flex flex-col gap-1">
<Label>Resolution</Label>
<Select
value={String(resolutionIdx)}
onValueChange={(v) => setResolutionIdx(Number(v))}
>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
<SelectContent>
{RESOLUTION_PRESETS.map((r, idx) => (
<SelectItem key={r.label} value={String(idx)}>
{r.label}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
<div className="flex flex-col gap-1">
<Label>Steps: {steps}</Label>
<Slider
aria-label="Inference steps"
min={1}
max={60}
step={1}
value={[steps]}
onValueChange={(v) => setSteps(v[0] ?? steps)}
/>
</div>
<div className="flex flex-col gap-1">
<Label>Guidance: {guidance.toFixed(1)}</Label>
<Slider
aria-label="Guidance scale"
min={0}
max={15}
step={0.1}
value={[guidance]}
onValueChange={(v) => setGuidance(v[0] ?? guidance)}
/>
</div>
</div>
<div className="flex flex-col gap-1">
<Label htmlFor="diffusion-seed">Seed (optional)</Label>
<Input
id="diffusion-seed"
value={seed}
onChange={(e) => setSeed(e.target.value)}
placeholder="leave empty for random"
inputMode="numeric"
/>
</div>
<div>
<Button
size="lg"
onClick={handleGenerate}
disabled={busy !== "idle" || !status?.is_loaded}
data-testid="diffusion-generate"
>
{busy === "generating" ? <Spinner className="mr-2 size-4" /> : null}
Generate image
</Button>
</div>
</div>
</SectionCard>
{results.length > 0 && (
<SectionCard
icon={<HugeiconsIcon icon={SparklesIcon} className="size-5" strokeWidth={1.5} />}
title="Results"
description="Most recent first."
>
<div className="grid grid-cols-1 gap-4 sm:grid-cols-2 lg:grid-cols-3">
{results.map((r, idx) => (
<figure key={idx} className="flex flex-col gap-2">
<img
src={`data:${r.image_mime};base64,${r.image_b64}`}
alt={`Generated image ${idx + 1}`}
// h-auto + object-contain so portrait / landscape
// outputs render at their true aspect ratio instead
// of being cropped into a square thumbnail.
className="h-auto w-full rounded-md border border-border object-contain"
data-testid="diffusion-result-image"
/>
<figcaption className="text-xs text-muted-foreground">
{r.width}x{r.height} - {r.num_inference_steps} steps - g={(r.guidance_scale ?? 0).toFixed(1)}
{/* Prefer seed_str (full uint64 precision) since the
numeric seed gets rounded by JSON.parse above
Number.MAX_SAFE_INTEGER and would otherwise
display a value that does not reproduce. */}
{r.seed_str
? ` - seed ${r.seed_str}`
: r.seed !== null && r.seed !== undefined
? ` - seed ${r.seed}`
: ""} -
{` ${(r.duration_ms / 1000).toFixed(1)}s`}
</figcaption>
</figure>
))}
</div>
</SectionCard>
)}
</div>
);
}

View file

@ -0,0 +1,5 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
export { ImagesPage } from "./images-page";
export * from "./api";