unsloth/studio/backend/core/inference
Daniel Han-Chen e03ed3dd29 Fix/adjust diffusion: round 14 P1+P2 batch for PR #5754
Round 14 reviewer aggregate (logs/review_round14_aggregate.md):

P1 fixes:
- routes/export.py /load-checkpoint now runs the active-export 409
  guard BEFORE the chat / diffusion unloads, so a rejected request
  no longer tears down unrelated GPU state.
- core/inference/llama_cpp.py wraps the WHOLE load_model body in a
  single try/finally that publishes loading_model_identifier across
  download, metadata read, VRAM settle, process spawn, and health
  check. Done via a thin load_model wrapper around the existing
  body (renamed _load_model_impl) to avoid reindenting hundreds of
  lines.
- routes/models.py /delete-finetuned now checks
  loading_model_identifier so a pending HF GGUF download cannot
  have its destination directory rmtree'd before llama-server
  spawns.
- core/inference/diffusion.py stores the original caller-supplied
  gguf_filename (e.g. ``BF16/model.gguf``) in a new self._gguf_filename
  field and exposes it as active_gguf_filename. UI-facing
  gguf_filename still collapses to basename for the panel.
- routes/models.py /delete-cached llama guard now allows safe
  different-variant deletes when hf_variant differs, matching the
  diffusion path's variant-aware behaviour.
- core/inference/diffusion.py tracks self._cpu_offload_enabled and
  forces a CPU torch.Generator when offload is on, so seeded
  generation no longer crashes on CUDA hosts with the default offload
  enabled.

P2 fixes:
- core/inference/diffusion.py detect_family normalises mixed
  separators (``Qwen_Image-Edit-GGUF``, ``Qwen-Image_Edit-GGUF``,
  ``QwenImageEdit-GGUF``) so every Qwen-Image-Edit spelling is
  excluded from the base Qwen-Image family.
- core/inference/diffusion.py logger.info / logger.error in
  load_model run repo_id and effective_base through _redact_hf_tokens
  so URL-embedded ``hf_xxxxx`` tokens never reach structured-log
  sinks.
- core/inference/diffusion.py _release_other_gpu_owners_for_diffusion
  now raises RuntimeError when an export job is active instead of
  logging and continuing, so direct backend callers cannot bypass
  the route layer's 409 guard.
- core/inference/diffusion.py full-diffusers repo / base_repo paths
  expand ``~`` via _expand_existing_local_path so
  ``repo_id="~/models/my-flux"`` no longer falls through to the Hub.

Tests:
- 5 new regression cases (mixed Qwen-Image-Edit separators, token
  redaction, status full-filename, CPU offload generator device,
  staging Windows leaf already-set sanity).
- All 68 diffusion backend + route tests pass.
2026-05-25 06:34:10 +00:00
..
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_html_to_md.py fix: studio web search SSL failures and empty page content (#4754) 2026-04-01 06:12:02 -07:00
anthropic_compat.py Studio: Claude Code Anthropic API tool compatibility (#5390) 2026-05-21 16:45:05 +04:00
audio_codecs.py Add native GGUF intake to Studio (#5246) 2026-05-04 11:46:18 +02:00
chat_template_helpers.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
defaults.py studio: engage draft-mtp on vision MTP GGUFs (drop incorrect vision gate) (#5560) 2026-05-18 08:42:55 -07:00
diffusion.py Fix/adjust diffusion: round 14 P1+P2 batch for PR #5754 2026-05-25 06:34:10 +00:00
external_provider.py Studio: PDF / document attachments for Anthropic + OpenAI (#5689) 2026-05-22 06:22:57 -07:00
inference.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
key_exchange.py studio: API external provider support for chat (OpenAI, Mistral, Gemini, Cohere, Anthropic, OpenRouter, DeepSeek, custom providers) (#4706) 2026-05-14 16:13:59 +04:00
llama_cpp.py Fix/adjust diffusion: round 14 P1+P2 batch for PR #5754 2026-05-25 06:34:10 +00:00
llama_server_args.py studio: add --spec-draft-n-max toggle for MTP speculative decoding (#5582) 2026-05-19 06:17:04 -07:00
mlx_inference.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
orchestrator.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
pricing.py Studio: per-session cost calculator + /api/providers/pricing endpoint (#5690) 2026-05-22 06:03:43 -07:00
providers.py Studio: expand Connections model picker for local inference server (#5643) 2026-05-20 15:06:06 +04:00
safetensors_agentic.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
tool_call_parser.py Revert "studio: tool calling for Llama-3, Mistral, Gemma 4 on safetensors + MLX (#5615)" (#5619) 2026-05-19 07:26:39 -07:00
tools.py studio: tighten sandbox blocklist precision (bash, hf upload, NOFILE) (#5487) 2026-05-18 00:01:17 -07:00
worker.py Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00