unsloth/studio/backend/core/inference
Daniel Han-Chen 6c528fb013 Fix/adjust diffusion: round 27 P1 + P2 batch for PR #5754
Round 27 findings (Opus parallel concurrency + frontend reviews).

Backend P1 fixes:

1. utils/datasets/llm_assist.py: the round 26 helper/advisor active
   registry used a plain set, so two concurrent helper / advisor
   loads of the same DEFAULT_HELPER_MODEL_REPO would both
   set.add() (no-op the second time) and then the first finally
   set.discard() would underflow the registration while the second
   call was still mmap'ing the GGUF. Switch to a Counter with
   proper refcount increment/decrement so the repo stays registered
   until the last user releases it.

2. routes/inference.py _release_chat_for and
   core/inference/diffusion.py _release_chat_backend_for_diffusion:
   helper/advisor GGUF runs on a PRIVATE LlamaCppBackend (round 26
   P1 #1), so the global llama checks below could not see them.
   A user-driven /training/start, /export/load-checkpoint, or
   /images/load would skip the unload and allocate FLUX VRAM on top
   of the helper's resident weights, OOMing on 16-24 GB consumer
   GPUs. Both release paths now consult helper_advisor_busy() and
   fail 503 (or RuntimeError for the in-backend path) so the user
   retries instead of double-owning VRAM.

Frontend P2 fixes:

3. studio/frontend/src/features/images/images-page.tsx: handleUnload
   now calls refreshStatus() in the catch path so a partial unload
   (503 from the backend) does not leave the UI showing a stale
   "Loaded:" label. Matches the handleLoad pattern.

4. images-page.tsx: when status.is_loading is true, auto-poll
   refreshStatus every 2 s so the user sees real progress instead
   of a frozen "Loading..." label until they manually click Refresh.

5. images-page.tsx: aria-label="Inference steps" / "Guidance scale"
   on the two sliders so screen readers can announce them.

6. images-page.tsx: defensive (r.guidance_scale ?? 0).toFixed(1)
   in the results caption so a future backend that serialises
   NaN/None for guidance does not throw at render.

Tests: 105 targeted (diffusion + cached_gguf + inference_validation)
and 1768 broader backend tests pass locally. Frontend
`npm run typecheck` passes.
2026-05-25 13:39:40 +00:00
..
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_html_to_md.py fix: studio web search SSL failures and empty page content (#4754) 2026-04-01 06:12:02 -07:00
anthropic_compat.py Studio: Claude Code Anthropic API tool compatibility (#5390) 2026-05-21 16:45:05 +04:00
audio_codecs.py Add native GGUF intake to Studio (#5246) 2026-05-04 11:46:18 +02:00
chat_template_helpers.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
defaults.py studio: engage draft-mtp on vision MTP GGUFs (drop incorrect vision gate) (#5560) 2026-05-18 08:42:55 -07:00
diffusion.py Fix/adjust diffusion: round 27 P1 + P2 batch for PR #5754 2026-05-25 13:39:40 +00:00
external_provider.py Studio: PDF / document attachments for Anthropic + OpenAI (#5689) 2026-05-22 06:22:57 -07:00
inference.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
key_exchange.py studio: API external provider support for chat (OpenAI, Mistral, Gemini, Cohere, Anthropic, OpenRouter, DeepSeek, custom providers) (#4706) 2026-05-14 16:13:59 +04:00
llama_cpp.py Fix/adjust diffusion: round 15 P1+P2+P3 batch for PR #5754 2026-05-25 07:00:29 +00:00
llama_server_args.py studio: add --spec-draft-n-max toggle for MTP speculative decoding (#5582) 2026-05-19 06:17:04 -07:00
mlx_inference.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
orchestrator.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
pricing.py Studio: per-session cost calculator + /api/providers/pricing endpoint (#5690) 2026-05-22 06:03:43 -07:00
providers.py Studio: expand Connections model picker for local inference server (#5643) 2026-05-20 15:06:06 +04:00
safetensors_agentic.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
tool_call_parser.py Revert "studio: tool calling for Llama-3, Mistral, Gemma 4 on safetensors + MLX (#5615)" (#5619) 2026-05-19 07:26:39 -07:00
tools.py studio: tighten sandbox blocklist precision (bash, hf upload, NOFILE) (#5487) 2026-05-18 00:01:17 -07:00
worker.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00