unsloth/studio/backend
danielhanchen 0cc69de210 studio: scope agentic-loop boundary marker so normal completions stream cleanly
Codex P2 on #5549 flagged that the cycle-5 cursor-reset (efa43c4) keyed
on every empty-status event, but generate_chat_completion_with_tools
emits empty status events in five places, only two of which are real
iteration boundaries:

  - Line 4497: emitted right before `continue` after a re-prompt /
    auto-continue. The next iteration starts a fresh assistant turn.
    BOUNDARY: reset cursor.
  - Line 4766: emitted right before `continue` after a tool call. The
    next iteration regenerates with tool results in history. BOUNDARY:
    reset cursor.
  - Line 4501: emitted at metadata-yield after normal streaming. Stream
    is about to end, no new iteration follows. NOT a boundary.
  - Line 4584: emitted in DRAINING-no-tool-call fallback path. Stream
    is about to end with buffered content_accum. NOT a boundary.
  - Line 4794: emitted at the final exit of the generator. NOT a
    boundary.

Treating all five as boundaries gave every Anthropic-streaming response
an extra content_block_stop + content_block_start pair around its final
text and around every tool call.

Fix by tagging the two real boundary sites with `"boundary": True` and
tightening both the Anthropic emitter (`anthropic_compat.py`) and the
OpenAI-compat tool path + Anthropic non-streaming path
(`routes/inference.py`) to reset the cumulative-text cursor only when
that flag is set. Plain empty-status events keep their existing badge-
clear semantics on the frontend (`tool_status` SSE with content "").

Add two regression tests in
`backend/tests/test_anthropic_messages.py::TestAnthropicStreamEmitter`:

  - test_boundary_flag_closes_block_and_resets_cursor: a boundary=True
    status closes the open text block and the next content delta
    streams from zero.
  - test_empty_status_without_boundary_does_not_close_block: a plain
    empty status leaves block_index unchanged and the next content
    delta is diffed against the previous text length.

107 tests pass across the four anthropic + trailing-plan test files.
2026-05-19 04:35:39 +00:00
..
assets Add Qwen3.6 inference defaults for Studio (#5065) 2026-04-16 11:42:42 -07:00
auth studio: security and hardening pass (auth rate-limit, sandbox, path containment, schema validation, headers) (#5375) 2026-05-13 06:12:18 -07:00
core studio: scope agentic-loop boundary marker so normal completions stream cleanly 2026-05-19 04:35:39 +00:00
loggers Studio: stop truncating long log lines as suspected base64 (#5335) 2026-05-08 13:07:18 +04:00
models Studio: warn when llama.cpp prebuilt is at least 3 days behind (#5529) 2026-05-18 00:21:50 -07:00
plugins fix(gh_client): fail fast on 401/403 auth errors instead of retrying forever (#5325) (#5329) 2026-05-08 21:57:41 +04:00
requirements studio: API external provider support for chat (OpenAI, Mistral, Gemini, Cohere, Anthropic, OpenRouter, DeepSeek, custom providers) (#4706) 2026-05-14 16:13:59 +04:00
routes studio: scope agentic-loop boundary marker so normal completions stream cleanly 2026-05-19 04:35:39 +00:00
state unsloth run: add --enable-tools/--disable-tools server-side tool policy (#5277) 2026-05-05 12:45:15 +04:00
storage studio: API external provider support for chat (OpenAI, Mistral, Gemini, Cohere, Anthropic, OpenRouter, DeepSeek, custom providers) (#4706) 2026-05-14 16:13:59 +04:00
tests studio: scope agentic-loop boundary marker so normal completions stream cleanly 2026-05-19 04:35:39 +00:00
utils studio: extend offline DNS auto-detect to inference parent + training (#5512) 2026-05-18 00:31:33 -07:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_platform_compat.py Fix Studio crash on Anaconda/conda-forge Python (#4484) 2026-03-22 05:36:55 -07:00
colab.py Fix/studio colab button message: Add fallback message for Colab Studio button when proxy URL fails (#4866) 2026-04-05 21:57:45 -07:00
main.py Studio: warn when llama.cpp prebuilt is at least 3 days behind (#5529) 2026-05-18 00:21:50 -07:00
run.py Studio: stop hint, Uvicorn log rename, reachability check + Mac UI CI retry hardening (#5503) 2026-05-17 07:44:06 -07:00
startup_banner.py Studio: stop hint, Uvicorn log rename, reachability check + Mac UI CI retry hardening (#5503) 2026-05-17 07:44:06 -07:00