unsloth/studio/backend/core/inference
Daniel Han 9af9c064c5 Studio: throttle repeated progress logs to a ~10s heartbeat
Some progress logs are needed but repeat redundantly: during a long model load
the inference subprocess streams the same keepalive status (e.g. "Downloading
(xet transport)...", "Loading model...") every tick, and the orchestrator logged
each one. Add a ProgressThrottle that logs the first message for a key, any time
the message changes (a phase change), then at most once per interval while it
stays the same; start/completion/errors keep logging at their own sites.

- loggers/progress.py: ProgressThrottle (dedupe + heartbeat), verbose-aware,
  interval via UNSLOTH_STUDIO_PROGRESS_LOG_INTERVAL_S (default 10s; 0 = log all).
- core/inference/orchestrator.py: throttle the two "Subprocess status" sites; the
  deadline reset still runs on every tick (only the log line is throttled); reset
  the key on load completion so the next load logs immediately.
- core/training/training.py: per-step metrics already stream over SSE (not logged
  per step), so add a throttled progress heartbeat ("step N/M, loss=..., epoch")
  so training progress is visible in the log without a line per step.
- tests/test_progress_throttle.py: first/changed/heartbeat/reset/verbose/zero.
2026-06-27 07:48:53 +00:00
..
__init__.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
_html_to_md.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
anthropic_compat.py Studio: improve OpenAI- and Anthropic-compatible API spec compliance (#6010) 2026-06-09 17:13:25 +02:00
api_monitor.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
audio_codecs.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
chat_template_helpers.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
chat_templates.py Studio: bundle Gemma 4 chat templates (E2B/E4B + larger) and auto-apply to unsloth/gemma-4-*-GGUF (#6245) 2026-06-12 05:49:39 -07:00
defaults.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
external_provider.py Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) 2026-06-26 03:31:33 -07:00
inference.py Studio: trim log noise (scanner probes, library chatter, load narration) with a --verbose escape hatch 2026-06-27 07:47:29 +00:00
key_exchange.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
llama_cpp.py [pre-commit.ci] auto fixes from pre-commit.com hooks 2026-06-27 07:47:30 +00:00
llama_http.py Studio: serialize non-streaming responses once and pool the proxy client (#6393) 2026-06-17 22:38:02 -07:00
llama_server_args.py studio: return a clean model id from the OpenAI API instead of the local .gguf path (#6518) 2026-06-26 16:07:53 -03:00
llama_stats.py Studio: trim serving-log noise and surface llama-server engine stats (#6377) 2026-06-17 05:37:57 -07:00
mcp_client.py Studio: enable stdio MCP servers on a loopback bind (#6295) 2026-06-15 03:02:32 +01:00
mcp_config_import.py studio: show MCP "Import config" on the add-server form (#6030) 2026-06-11 16:17:22 +01:00
message_content.py fix(studio): handle multimodal list content in inference text paths (#4383) (#6480) 2026-06-23 01:26:11 -07:00
mlx_inference.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
model_ids.py studio: list the full local model catalog from /v1/models (#6519) 2026-06-26 20:42:06 -03:00
orchestrator.py Studio: throttle repeated progress logs to a ~10s heartbeat 2026-06-27 07:48:53 +00:00
pricing.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
providers.py Studio: Add custom provider option to Connections (#6112) 2026-06-12 13:09:35 +02:00
runtime_context.py Expose runtime context length for hub models (#6154) 2026-06-11 22:13:53 +03:00
safetensors_agentic.py Studio: show tool-call progress for large GGUF tool arguments (#6484) 2026-06-22 05:50:10 -07:00
tensor_fallback.py studio: deterministic VRAM auto-fit for GGUF (MTP reserve, compute buffer, total-based budget) (#6312) 2026-06-17 03:10:22 -07:00
tool_call_parser.py Fix Gemma 4 GGUF OpenAI API streams (#6476) 2026-06-23 06:13:56 -07:00
tool_loop_controller.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
tools.py Studio: trim log noise (scanner probes, library chatter, load narration) with a --verbose escape hatch 2026-06-27 07:47:29 +00:00
worker.py Generalize transformers tier selection by probing AutoConfig (#6550) 2026-06-22 08:20:06 -07:00