* Studio: trim serving-log noise and surface llama-server engine stats
Studio prints one structured line per HTTP request, so the SPA's polling and
per-invalidation fan-out bury the lines that matter.
- Dedup identical successful GETs within a short window (default 300ms,
UNSLOTH_STUDIO_ACCESS_LOG_DEDUP_MS) so a burst logs once. The dedup key
includes the query string, so distinct query-driven GETs are not collapsed.
Runs after the response is sent, so it adds no request latency; mutations,
non-2xx, and loading polls are untouched.
- Collapse pure-liveness polls (/api/health, /api/auth/status,
/api/inference/status, /api/inference/monitor) to a longer heartbeat
(default 10s, UNSLOTH_STUDIO_ACCESS_LOG_POLL_DEDUP_MS). The API monitor
console polls /monitor every 1.5s while open.
- Translate llama-server's Prometheus /metrics into a periodic vLLM-style
engine_stats line (generation/prompt throughput and requests in flight) from
a daemon poller, gated on UNSLOTH_STUDIO_ENGINE_STATS. Throughput uses
llama-server's predicted_tokens_seconds / prompt_tokens_seconds gauges, with
a tokens_predicted_total / prompt_tokens_total counter-delta fallback; it does
not use n_decode_total (which counts llama_decode() calls, not tokens). No KV
field is emitted, since llama.cpp does not expose kv_cache_usage_ratio.
--metrics is added only when probe_server_capabilities reports the binary
supports it, so older/custom binaries still load. The poller keeps retrying
through transient scrape failures (stop() drives shutdown) and a malformed
sample cannot crash its thread.
- api_monitor.append_reply: once the preview cap is reached, skip the per-chunk
re-concat (avoids O(n^2) on long generations) while still recording the "..."
truncation marker for a reply that lands exactly on the cap.
- unsloth studio --verbose and unsloth studio run --verbose both restore every
per-request log; --verbose before a subcommand is rejected with guidance
(matching --secure / --parallel). run --verbose still forwards --log-verbose
to llama-server, preserving the pre-existing pass-through verbosity.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add Studio API activity monitor
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Mark disconnected monitor streams cancelled
* Fix monitor lifecycle, parsing, and clock issues for PR #5558
Backend:
- Finalize the monitor entry as error on the chat-completions validation
reject paths (addresses the codex P2 comment around line 2337). Adds a
small _reject helper next to the api_monitor.start so each early-raise
in the GGUF tool-passthrough, non-empty-messages, GGUF vision/audio,
PIL image decode, and text-only model branches no longer leaves the
entry stuck running until eviction.
- _monitor_openai_chunk is now defensive about malformed shapes
(non-list choices, non-dict choice/delta) so a misbehaving provider
chunk does not raise into the streaming generator and abort the
user's response.
- _monitor_openai_sse_line accepts both data:value and data: value per
SSE spec; previously the single-space-only prefix silently dropped
tokens from compact emitters.
- openai_completions stream now keeps a residual buffer between
aiter_bytes chunks so a data: line whose newline lands in the next
TCP frame is reconstructed before parsing, instead of being dropped
by the per-chunk splitlines call.
api_monitor:
- duration_ms is derived from time.monotonic anchors and clamped at 0,
so NTP / manual clock steps no longer produce negative durations.
- finish() and fail() are idempotent: a second call (for example [DONE]
arriving after the generator's finally block already ran) no longer
moves finished_at or finished_monotonic.
- set_usage only derives total_tokens when no authoritative total has
been recorded, so a later partial-usage chunk does not clobber a
provider-reported total from an earlier chunk.
- _trim guards limit < 3 so the slice cannot underflow.
Tests:
- Adds regression coverage for the new idempotency, total preservation,
monotonic clock, and _trim guard behaviour.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Mark monitored stream failures correctly
* Fix completions monitor failures
* Fix responses stream monitor cleanup
* Fix audio input API monitor lifecycle
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix API monitor review follow-ups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix completions monitor usage accounting
* Fix API monitor cancellation and usage gaps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move API monitor into settings
* Fix API monitor review followups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Lazy load API monitor details
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix API monitor review issues
* Finalize passthrough monitor on clean EOF
* Finalize monitor on chat validation rejects
* Fix API monitor review follow-ups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix API monitor follow-up review issues
* Monitor embeddings and tool-call replies
* Cover tool-call monitor replies and cancellations
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope API monitor entries by subject
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Finalize monitor entries on cancellation
* fix: address PR 5558 CI failures
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: update provider proxy test stub
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Daniel Han