unsloth/studio/backend/core/inference
Daniel Han-Chen fa8efafcd8 Fix/adjust diffusion: round 7 swap-aware guards + race-free generate for PR #5754
Round 7 reviewer surfaced a handful of swap-window races, fail-open
guards, and seed precision mismatches. This commit closes them.

Lifecycle / state (P1)
  * core/inference/diffusion.py: status() now emits active_repo_id,
    active_base_repo, pending_repo_id, pending_base_repo, and
    pending_gguf_filename alongside the existing UI-facing fields.
    During a swap (model A loaded, model B loading) the previous
    coalesced 'repo_id or pending_repo_id' hid the loading target
    from delete guards. Splitting the fields lets guards block
    deletion of either repo currently owned by the backend.
  * core/inference/diffusion.py: generate_image() now takes
    _generate_lock BEFORE snapshotting _pipe / _device. Snapshotting
    outside the lock let a concurrent unload/load clear or replace
    the backend between the snapshot and the forward, so the freed
    or swapped pipeline would still run.

Symmetric handoffs (P1)
  * routes/export.py: training-active check now runs BEFORE the
    chat / inference / diffusion unload helpers, so a 409 does not
    leave the user's chat session torn down for nothing. Also
    explicitly fails CLOSED with 503 when is_training_active()
    raises.
  * routes/inference.py: _raise_if_training_active now fails closed
    with 503 when the training backend is importable but its status
    check raises. The previous best-effort log-and-continue could
    let chat / diffusion loads collide with unverifiable training.

Delete guards (P1)
  * routes/models.py /delete-cached: chat guard now also blocks
    when llama-server is_active (i.e. mid-download) and when the
    inference backend's loading_models set contains the target.
    Round 7 review #7 flagged that the PR's diffusion-side loading
    guard had no chat-side parallel, so deleting a chat repo while
    it was downloading could still race the cache.
  * routes/models.py /delete-cached: diffusion guard iterates the
    new active_* + pending_* status fields so a delete during a
    swap is refused on either repo.
  * routes/models.py /delete-finetuned: same active_+ pending
    handling, plus the guard now also refuses deletes of a parent
    directory that contains the loaded pipeline (round 7 review #6:
    rm -rf /exports/flux-model/ could unlink model_index.json that
    the live pipeline is reading via mmap).

Seed precision (P2)
  * models/inference.py + routes/inference.py: DiffusionGenerate-
    Response now carries seed_str alongside the existing numeric
    seed. Seeds above Number.MAX_SAFE_INTEGER are rounded by
    JSON.parse in the browser; seed_str ships full decimal
    precision for display and reproduction.
  * frontend/api.ts: DiffusionGenerateResponse types seed_str;
    images-page.tsx prefers seed_str over seed in the figure
    caption so the displayed value reproduces the image.
  * frontend/api.ts: stringifyWithBigInt no longer regex-replaces
    sentinel strings over the full JSON output. It pulls the seed
    BigInt out, JSON-serialises the remaining payload, and splices
    the seed's decimal digits into the resulting object literal at
    the known position. Avoids the round 7 #10 case where a
    user-supplied prompt equal to '__bigint__:123' was rewritten
    into a JSON integer and rejected as a non-string prompt.

Custom HF repo (P2)
  * frontend/images-page.tsx: custom panel now exposes a 'Base
    diffusers repo' input that maps to DiffusionLoadRequest.
    base_repo. Required when a private / mirrored GGUF needs a
    non-default base (e.g. a 9B Klein transformer would otherwise
    fall back to the 4B base default).
2026-05-25 02:08:18 +00:00
..
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_html_to_md.py fix: studio web search SSL failures and empty page content (#4754) 2026-04-01 06:12:02 -07:00
anthropic_compat.py Studio: Claude Code Anthropic API tool compatibility (#5390) 2026-05-21 16:45:05 +04:00
audio_codecs.py Add native GGUF intake to Studio (#5246) 2026-05-04 11:46:18 +02:00
chat_template_helpers.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
defaults.py studio: engage draft-mtp on vision MTP GGUFs (drop incorrect vision gate) (#5560) 2026-05-18 08:42:55 -07:00
diffusion.py Fix/adjust diffusion: round 7 swap-aware guards + race-free generate for PR #5754 2026-05-25 02:08:18 +00:00
external_provider.py Studio: PDF / document attachments for Anthropic + OpenAI (#5689) 2026-05-22 06:22:57 -07:00
inference.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
key_exchange.py studio: API external provider support for chat (OpenAI, Mistral, Gemini, Cohere, Anthropic, OpenRouter, DeepSeek, custom providers) (#4706) 2026-05-14 16:13:59 +04:00
llama_cpp.py studio: settle GPU VRAM after killing llama-server before the next reload (#5693) 2026-05-22 05:50:39 -07:00
llama_server_args.py studio: add --spec-draft-n-max toggle for MTP speculative decoding (#5582) 2026-05-19 06:17:04 -07:00
mlx_inference.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
orchestrator.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
pricing.py Studio: per-session cost calculator + /api/providers/pricing endpoint (#5690) 2026-05-22 06:03:43 -07:00
providers.py Studio: expand Connections model picker for local inference server (#5643) 2026-05-20 15:06:06 +04:00
safetensors_agentic.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
tool_call_parser.py Revert "studio: tool calling for Llama-3, Mistral, Gemma 4 on safetensors + MLX (#5615)" (#5619) 2026-05-19 07:26:39 -07:00
tools.py studio: tighten sandbox blocklist precision (bash, hf upload, NOFILE) (#5487) 2026-05-18 00:01:17 -07:00
worker.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00