* Replace standalone Studio wording with Unsloth
Replace the single word Studio with Unsloth wherever it is used as
shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n
locales, workflow display names, comments and docstrings.
Kept unchanged: the full name Unsloth Studio, third party product
names (LM Studio, Visual Studio, Mac Studio), feature names
(Recipe Studio, Fine-tuning Studio and its translations), and all
identifiers such as env vars, commands, paths and filenames.
* Address review feedback on the Studio wording rename
Use "an" before Unsloth where the rename left the article as "a".
Restore the split brand where Unsloth and Studio render as two halves
of the full product name: the onboarding sidebar subtitle and the
IPv6 localhost warning. Scope two messages to the full name Unsloth
Studio where plain Unsloth was misleading: the AMD README bullet and
the CLI studio setup error.
* Studio: stream live tool output with SSE heartbeats and fix web page extraction
Server-side python/terminal tools now stream incremental stdout to the chat
UI while running (new tool_output SSE event), and every blocking tool
execution emits heartbeat keepalives so reverse proxies (Cloudflare tunnels
cap idle streams at ~100s) cannot drop the connection mid-turn. The tool
loop routes also emit a stall keepalive during silent prompt prefill between
tool iterations. The final role=tool message the model sees is byte-identical
to before, so tool-call parsing, nudging, and healing are untouched.
web_search page fetches now extract main content: GitHub repo root pages are
rewritten to the README API (with HTML fallback), hidden/aria-hidden client
error placeholders are dropped, conversion scopes to article/main, and known
boilerplate fragments are stripped. Non-HTML responses are returned raw
instead of being run through the HTML converter.
The frontend renders live-scrolling tool output inside running python and
terminal cards, and a chat stream that ends without a terminal signal now
surfaces an explicit interrupted state with a Retry action instead of
silently ending the turn.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix content-type sniffing, unlimited-timeout drain, and env parity in tool streaming
Content-Type sniffing: get_content_type() defaults to text/plain when the
header is absent, so the sniffing fallback never fired and header-less HTML
came back as raw markup. Report an empty type for a missing header and sniff
the body whenever the declared type is not HTML, so mislabeled text/plain
HTML pages are converted like before the extraction change.
Unlimited timeout drain: with tool_call_timeout disabled the old path used
communicate(timeout=None) and waited for EOF, but the streaming drain capped
the post-exit drain at a 5 second join, truncating output from a grandchild
that holds stdout open. When timeout is None, drain until EOF or the cancel
event fires; finite timeouts keep the bounded remaining-budget join.
Env parity: drop the PYTHONUNBUFFERED=1 injection on the streaming path so
the child invocation is byte-identical with and without streaming (the env
var was model-visible via os.getenv). Live streaming granularity now depends
on the child flushing; unflushed output arrives in ~8 KB chunks or at exit
and the final result is unchanged, with SSE heartbeats covering the gaps.
* Studio: stream tool-call arguments while the model writes them
A model writing a large tool call (a full python game is minutes of
generation) produced nothing on the stream: the structured path
accumulated delta.tool_calls fragments silently after the provisional
card, and the text path's DRAINING state consumed everything until
stream end. The user saw a dead Running spinner while the model was in
fact writing code, and the byte-silent SSE segment was also the window
where proxies drop the connection.
New tool_args SSE events stream the arguments as they generate. The
structured path forwards each fragment once a provisional card exists
(backlog first, so the card starts from the top of the call). The text
path sniffs the drained call for an enabled tool name and streams the
raw call text under the id the stream-end parser assigns its first call
(call_0), so the final tool_start reconciles the same card; the sniff is
gated on enabled names plus the provisional size floor, and prose or
ordinary JSON answers never spawn a card. The safetensors loop streams
the drained render_html call to its existing provisional card the same
way.
The chat adapter accumulates the raw stream per card and feeds a partial
JSON parse (call envelopes and stringified arguments unwrapped) into the
part's args, so the python and terminal cards render the code live and
the render_html canvas builds while streaming; both cards now say
Writing code / Writing command during this phase via useToolArgsStatus.
Display only: the parser input, the executed call, and the conversation
the model sees are byte-identical, covered by new loop-level tests for
the structured path, the text path, and the no-tool JSON answer.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep full tool output visible past the model cap; heal /mnt/data habits
Live testing surfaced two issues in the tool streaming UX.
First, a long python stdout ended in '... (truncated' in the finished
card: the model-visible result is capped by tools._truncate
(_MAX_OUTPUT_CHARS, previously 8000 chars) to protect the context
window, and the card rendered that capped text even though the live
stream had already shown everything. The cap stays (raised to 16000,
overridable via UNSLOTH_TOOL_RESULT_MAX_CHARS) but display and model
concerns are now split: the adapter preserves the accumulated live
stream on tool_end whenever it captured more than the final result, and
the finished python/terminal cards prefer it. The live-stream ceiling
rises from 16 KB to 400 KB (chunks batch per poll, so SSE stays cheap),
and both the live pane and the finished card render only the last 2000
lines with a Show all control so a huge output cannot jank the DOM. The
truncation notice now tells the model the user saw the full output and
that written files persist in the working directory. The final result
string remains byte-identical with and without streaming.
Second, models trained on ChatGPT code-interpreter transcripts write to
/mnt/data, which does not exist here (the sandbox CWD is a per-thread
persistent dir). Three layers, all identical across streaming and
non-streaming paths: the python/terminal tool descriptions gain one
sentence saying to use relative paths in the persistent CWD; a failed
execution whose output shows a missing-file error on a known
code-interpreter prefix (/mnt/data, /mnt/outputs, /home/sandbox,
/workspace) gets a model-visible retry hint appended after truncation
so it always survives; and a sitecustomize shim on the sandbox
PYTHONPATH remaps those prefixes onto the CWD in open()/os.makedirs()
with a one-line stderr notice, covering the python tool and any Python
launched from the terminal tool without touching the exec wrapper (so
tracebacks keep their line numbers). Bash-level file operations cannot
be redirected without root or mount namespaces, so they rely on the
description and the hint.
* Studio: fix hidden-element parsing, heartbeat gaps, and tool output id collisions
Review follow-ups on the tool streaming work:
- _html_to_md: treat any present hidden attribute value as hidden (it is an
enumerated attribute whose invalid value default is the Hidden state, so
hidden="false" is still hidden), and implement HTML5 optional end tags so
an unclosed <p hidden> or <li hidden> ends at the next sibling start tag
instead of swallowing every following sibling until the parent closes
- tool_stream_exec: keep heartbeats flowing after the live-output cap; a
tool that keeps printing past the cap kept the queue non-empty, so neither
tool_output nor heartbeat events were emitted and the SSE stream went
silent past proxy idle timeouts
- routes/inference: forward tool heartbeats before the
disable_parallel_tool_use drop window swallows events, so a dropped call
that executes server-side cannot leave the Anthropic stream silent
- llama_cpp: close the provisional text tool card with a tool_end when the
drained call fails to parse (DRAINING false-positive path), so the card
cannot spin forever while the text is delivered as content
- tools: decode terminal output as utf-8 with errors=replace like the python
tool; invalid bytes used to raise UnicodeDecodeError from communicate() on
the non-streaming path and silently truncate the streaming reader, so the
two paths diverged
- sitecustomize: patch io.open alongside builtins.open; pathlib Path.open,
read_text and write_text call io.open directly and bypassed the remap
- frontend: scope the toolLiveOutput/toolFullOutput store keys by pane
(modelType and pairId) and clear stale entries on tool_start; backend ids
like call_0 repeat across turns and across concurrently streaming panes
(compare mode), so a later turn or another pane could display the wrong
preserved output, and run-end cleanup now clears only its own keys
Each backend fix carries a regression test that fails on the previous code;
the byte-identity tests between streaming and non-streaming stay green.
* Studio: keep tool failure status visible and truncation/remap notices truthful
Finished python/terminal cards preferred the fuller live stream by length
alone, so a tool that printed a lot then timed out or exited non-zero showed
the captured stdout but dropped the final result's status (timeout notice,
Exit code N). preferFullToolOutput now shows the stream when the result is
just its truncated prefix, and appends the result otherwise so the failure
tail always survives and the copy button copies both.
The result truncation notice claimed the user was shown the full output, but
the same wrapper serves non-streaming chat/API and direct execute_tool()
callers where nothing is streamed to anyone. The notice is now mode-neutral
and stays byte-identical with and without an output_callback, keeping the
streaming vs non-streaming invariant intact.
The sandbox sitecustomize shim now remaps /tmp/outputs into the working
directory only while it does not already exist, so a real /tmp/outputs the
user's own code created is never shadowed; /tmp/outputs also joins the
missing-path retry-hint list.
* Studio: suppress hidden void elements and keep live output scroll pinned only when at bottom
* Studio: drop capped tool output without concatenating; remap pathlib mkdir
Past the live-output cap stream_tool_execution built item + _drain_pending()
(the current chunk joined with every queued sibling) only to discard it in the
capped branch, so a chatty tool (yes, a tight print loop) could enqueue far
more than one poll interval of text and blow past the memory/CPU ceiling the
cap exists to enforce. Drain and drop queued items without building a combined
string, still counting each drain toward the heartbeat cadence so the SSE
keepalive survives.
Generated code often prepares code-interpreter paths with
Path('/mnt/data').mkdir(parents=True, exist_ok=True); pathlib drives that
through os.mkdir (not the patched os.makedirs) per component and, on
FileExistsError, probes the unpatched os.stat via Path.is_dir(), so the setup
raised before open() ever ran. Patch os.mkdir with the same remap and patch
Path.mkdir so the whole parents/exist_ok dance lands on the mapped working
directory and stays idempotent; real paths still pass through.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: generalize sandbox write remap and hint to any hallucinated absolute path
Models invent absolute paths from seeing their CWD (a DeepSeek run did
open('/home/ubuntu/Sandbox/flappy_bird.html', 'w') and died with
FileNotFoundError). A prefix list cannot enumerate these, so the sitecustomize
shim gains a write-mode fallback in open()/io.open(): when a write/create-mode
open targets an absolute path outside the CWD whose parent directory does not
exist, redirect it to the basename in the CWD and emit the same one-line stderr
notice, echoing the original path. The prefix remaps still run first (they cover
reads and preserve subpaths); read modes never hit the fallback so real system
files fail or succeed truthfully; bytes paths pass through. The fallback is not
applied to mkdir/makedirs/Path.mkdir, since creating an arbitrary absolute
directory can legitimately succeed on the host, and that decision is documented
in a comment.
The model-visible retry hint now echoes the real failing path (parsed from the
traceback tail) instead of the canned /mnt/data example, and fires for any
absolute path outside the working directory, not just the enumerated prefixes,
while a relative miss still gets no hint.
The shim wrapper still adds one frame to tracebacks that surface open() errors;
suppressing only our frame has no clean standard mechanism (a wrapper always
adds a frame), so the frame is left as an accepted compromise.
Tests: hallucinated absolute write remaps to the CWD basename across w/a/x/w+;
reads of a missing absolute path pass through untouched; writes to an existing
external dir pass through; prefix subpaths still preserved; end-to-end write
fallback lands the file in the sandbox workdir identically with and without
streaming; the hint echoes the actual path for convention and non-convention
absolute paths alike.
* Studio: kill exited process groups on drain; bound the over-cap output batch
_drain_process_output killed the process only via _kill_process_tree, which
short-circuits once the parent has exited, so a grandchild that inherited
stdout and outlived the parent was never signaled: a finite-timeout run could
return while it kept holding the pipe, and a timeout=None cancel left it
behind. Capture the setsid process group before waiting and SIGKILL that group
at both give-up points so the whole tree is torn down.
The streaming wrapper's first over-cap batch joined the current chunk with the
entire pending backlog before enforcing the live-output cap, so a chatty tool
could allocate far past the cap on the crossing batch. Bound the drain to the
remaining budget and drop the surplus in place, keeping the truncated output
byte-identical to joining everything.
* Studio: harden sandbox path healing and process/generator cleanup
Sandbox sitecustomize shim:
- Make the generalized write fallback collision-safe: never redirect an
invented absolute path onto an already-present CWD file (refuse and let the
original open raise FileNotFoundError, preserving the workspace file).
- Only w/a/x create a file; r+/rb+ are read-update modes that require the
target to exist, so a bare + no longer trips the write fallback.
- Gate every convention-prefix remap (/mnt/data, /mnt/outputs, /home/sandbox,
/workspace) on the prefix root being absent, so a real host mount is never
shadowed; a miss under an existing real prefix passes through.
- Patch os.open so Path.touch and other low-level creators heal convention
paths too, matching the Path.mkdir patch.
Local code execution (tools.py):
- Capture the setsid process group right after Popen (before any watcher can
poll/reap the leader) and thread it through the cancel watcher and drain.
- Kill the captured group in the non-streaming python/terminal timeout branch
so an exited leader no longer leaks a stdout-holding grandchild (matches the
streaming drain path).
- Guard os.getpgid/os.killpg by platform so streamed execution no longer
raises on Windows; fall back to single-pid kill.
- Judge missing-path hints against the executor's real workdir so a legitimate
miss inside a project workspace outside the sandbox root is not mislabeled.
Tool streaming routes (routes/inference.py):
- Drain a pending next(gen) worker before closing the generator in the
safetensors and Anthropic tool streams, so a disconnect no longer races
gen.close() (generator already executing) or leaks the thread/generator.
HTML to markdown:
- Only drop boilerplate lines composed entirely of known furniture phrases so
real prose that merely quotes one (for example "we use cookies to
authenticate requests") is preserved.
Adds hermetic tests for each change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep aside callouts, contain sandbox path remaps, and keepalive dropped Anthropic tool events
_html_to_md: stop dropping <aside> unconditionally. Documentation pages
render notes/warnings/examples as aside admonition callouts; those inside
the selected article/main scope are real content. A furniture aside outside
the scope is already excluded by the main-content pass.
sitecustomize: contain the code-interpreter path remap under the sandbox
CWD. A hallucinated habit path such as /mnt/data/../other_session/file no
longer escapes the per-conversation workdir; parent-traversal components in
the suffix are dropped and a '.'/'..' write-fallback basename is refused.
routes/inference: emit a rate-limited comment keepalive when the Anthropic
Messages stream drops tool_output/tool_args events. A chatty tool keeps the
generator busy so the stall keepalive never fires and the tool wrapper emits
heartbeats only while idle, which left the SSE stream silent past proxy idle
caps; the OpenAI passthrough paths forward these events, this path now keeps
the connection alive.
* Studio: bound the tool-output chunk that first crosses the live cap
_drain_queue joined the entire chunk that first crossed the live-output
cap before dropping the rest, so a single multi-megabyte line (or any
chunk dequeued once the budget was already met at max_chars <= 0) was
materialized in full only to be truncated away, defeating the memory
ceiling the cap enforces. Slice the crossing chunk to one character past
the budget: that preserves the caller's overflow signal and its
byte-identical truncation while dropping the arbitrarily large remainder
in place.
* Studio: scope missing-path hint to the failing line, keepalive dropped-call output, and preserve truncated tool streams over byte length
- tools._missing_path_hint: the code-interpreter convention-prefix trigger
scanned the whole output, so a convention prefix mentioned only in a
traceback frame (a /workspace project root) or printed by the user's code
would add a misleading 'use a relative path' hint even when the actual
FileNotFoundError was a relative or in-workdir path. Scope the convention
test to the failing-path error line(s), matching _extract_missing_abs_path.
- _anthropic_tool_stream: the tool_output/tool_args rate-limited keepalive sat
after the drop_until_tool_end skip, so under disable_parallel_tool_use a
chatty second-or-later tool call was dropped whole with no keepalive, letting
an idle proxy kill the SSE stream. Check the keepalive branch before the drop
skip (like the heartbeat branch) so dropped-call output keeps the stream alive.
- preferFullToolOutput / chat-adapter: a truncated result can be longer than
the live stream by byte count once its footer, an 'Exit code N:' notice, or an
__IMAGES__ base64 tail is appended, so the length-only gate discarded the full
stream and the finished card fell back to the truncated text. Add a shared
truncation-aware shouldPreserveFullOutput used by both the write and read
sites: preserve the stream whenever the result carries the truncation footer.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: skip the habit-path hint for real project paths under a convention prefix
* Kill captured process group on streamed wait-timeout
The streamed drain path's proc.wait() timeout branch only called
_kill_process_tree(proc). If the leader exits in the narrow window between
the wait timing out and _kill_process_tree sampling its pgid, that helper
short-circuits on the reaped leader and a stdout-holding grandchild in the
same group survives. Also kill the captured pgid there, matching the
non-streaming communicate() timeout path. Adds a hermetic regression test
that models the reaped-leader race by stubbing _kill_process_tree.
* Fix 3.10 pathlib write_text remap and honor cancel in finite drain
On Python < 3.11 pathlib routes Path.open / read_text / write_text through
a module-level accessor singleton whose open attribute captured the original
io.open at import time (_NormalAccessor.open = io.open). Patching io.open in
the sandbox shim therefore never reached that captured reference, so a
Path('/mnt/data/x').write_text(...) raised FileNotFoundError on 3.10 while
passing on 3.11+ (which dropped the accessor and calls io.open at call time).
Repoint _NormalAccessor.open at the same io.open wrapper via a staticmethod,
guarded so it is an idempotent no-op on 3.11+. Keep the test save/restore
helpers symmetric so the accessor is restored too, and add a hermetic
write_text/read_text remap test that covers every version.
Also honor cancellation while draining inherited stdout after the leader
exits. Once the leader is reaped the cancel watcher returns (its loop is
while proc.poll() is None), so the finite-timeout drain did one blocking
reader.join(timeout=remaining) that ignored cancel_event and kept draining a
chatty grandchild for the whole budget after a disconnect/Stop. Poll
cancel_event in 0.5s slices against a deadline like the timeout=None branch
and kill the captured process group promptly on cancel. The normal path still
reaches EOF on its own, so the streamed vs non-streamed result is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: port no-tool stream keepalive/drain and fix subprocess/queue/extraction asymmetries
Streaming no-tool paths now match their tool twins:
- _anthropic_plain_stream, safetensors/MLX no-tool stream, and standard GGUF
no-tool stream run next(gen) in a worker with a timed SSE keepalive loop so a
long prompt prefill cannot leave the stream idle past a proxy cap.
- The Anthropic plain and safetensors/MLX no-tool teardowns now drain the
pending next(gen) worker and close the generator on disconnect instead of
leaking the suspended generator.
Other asymmetries:
- Non-streaming _python_exec/_bash_exec always drain via _drain_process_output
(output_callback may be None) so a cancelled run reaps a stdout-holding
grandchild that outlived the leader instead of blocking in communicate(). The
joined bytes are identical to communicate(), so streamed vs non-streamed
results stay byte-identical.
- _build_bypass_env installs the sitecustomize path shim on PYTHONPATH (prepend,
keeping the operator's entries) so /mnt/data remap works in bypass mode too.
- GGUF forwards output_callback to execute_tool only when the callable accepts
it (shared accepts_output_callback), matching safetensors and preserving
legacy monkey-patched signatures.
- tool_stream_exec bounds accepted live output at the producer boundary so a
chatty tool cannot grow the queue without limit under consumer backpressure
and cannot keep the drain spinning and starve heartbeats.
- html_to_md implicit-close now searches past unclosed inline descendants so a
hidden <p>/<li> is closed by a following block; main-content scoping gates on
the largest single <article>/<main> so a swarm of tiny cards cannot pass the
threshold in aggregate and displace the real main.
- preferFullToolOutput re-attaches the "Exit code N:" prefix to the fuller
stream instead of appending the still-prefixed result, so a failed truncated
tool no longer duplicates its stdout in the finished card.
Adds hermetic tests for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Preserve short live output on timed-out tools; strip inline-CSS-hidden subtrees and score truncated main-content scopes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten chat tool streaming comments and docstrings
* Keep HTML READMEs from the GitHub API and preserve interrupted tool output
Convert a 200 HTML README body from the GitHub README API to Markdown
instead of discarding it and falling back to the repo page chrome, and
promote captured live stdout to full output when a tool never reaches
tool_end (stream interrupted or cancelled) so the partial diagnostics
stay on the finished card.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: anchor HTML sniff, keep repeated sandbox writes, reuse textual tool ids
Anchor _looks_like_html to the leading doctype/tag so a Markdown README that
opens with a fenced HTML example stays Markdown (no html_to_markdown
corruption), while bare HTML fragments (<body>/<article>/<section>) are still
detected and converted on a missing/wrong Content-Type.
Let the sandbox write fallback re-serve a target it already healed for the same
invented absolute path, so iterative overwrites of a generated artifact stop
failing with FileNotFoundError while the anti-clobber guard still refuses
unrelated same-basename files.
Reconcile the first textual tool call carrying an explicit id onto the open
provisional TEXT card instead of spawning a duplicate card under that id.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: run implicit-close before skipping tags and keep leading README tables as Markdown
A skipped block (<nav>/<footer>) is an HTML5 optional-end-tag closer of an
open <p>, but handle_starttag returned before the implicit-close bookkeeping,
so a never-closed <p hidden> kept its hidden mark and swallowed every following
sibling. Run _close_implicit before the skip decision so the hidden mark is
released and trailing content renders.
Drop <table> (and its <thead>/<tbody>/<tr>/<td>/<th> children) from the
_looks_like_html leading set: Markdown READMEs routinely open with a raw HTML
<table> badge/layout row, and sniffing that as HTML collapsed the whole
Markdown body through html_to_markdown, exactly like the already-excluded
<div align>/<p align> layout headers.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make bypass-permissions Popen double faithful to the unified drain path
The non-streaming _python_exec/_bash_exec now share _drain_process_output,
which reads proc.stdout in a reader thread and calls proc.wait(); the test
double only implemented communicate(), so bypass-mode bash returned an
AttributeError instead of the faked output. Give _FakeProc a readable stdout
pipe (yields the fake line then EOF), wait()/poll()/pid, so the test exercises
the real drain path on both the python and bash bypass branches.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: persist sandbox path heals across runs and suppress nested hidden lists
* Studio: run tool Python child unbuffered (-u) so unflushed prints stream live
A long-running snippet doing bare print() without flush=True never reached
the live-output pane: CPython block-buffers stdout when writing to a pipe, so
_drain_process_output's readline() saw nothing until the buffer filled or the
process exited. Launch the child with the interpreter -u flag so stdout is
unbuffered and each print streams as it is produced.
-u is applied unconditionally on both the streaming and non-streaming path, so
the child invocation stays byte-identical with and without streaming and the
final joined result is unchanged (buffering/timing only). Unlike the earlier
PYTHONUNBUFFERED=1 env injection that was removed, -u does not pollute the
child's os.environ and is not visible via os.getenv.
* Render only the selected main-content subtree in html_to_markdown
The main-content heuristic sized each <article>/<main> candidate
individually to pick the largest subtree, but then rendered every
matching tag in the document. A page with one real article plus
sibling related-post cards or comment threads passed the size gate on
the real article yet still emitted the unrelated siblings.
Size and render the same chosen subtree so only the selected
main-content subtree reaches the output.
* Studio: tighten chat-tool-streaming fix comments
* Studio: store tool-output-scope separators as unicode escapes
The pane-scope and tool-output-key separators were literal NUL (0x00) bytes, which made git treat the file as binary and hide its diff and blame. Write them as \u0000 escapes instead; the runtime key value is unchanged.
* Studio: bound tool-stream teardown when the client disconnects
stream_tool_execution ran its yield loop with no try/finally, so a gen.close() on client disconnect (GeneratorExit at a yield) skipped the worker join and never signalled cancellation. A tool that does not poll cancel_event mid-flight (web_search, MCP, search_knowledge_base) then kept request teardown blocked until the tool's own timeout. Thread the request cancel_event into the wrapper, set it only on the abnormal-exit path so a clean multi-tool turn is unaffected, and bound the worker join to a few seconds; the daemon worker cannot outlive the process.
* Studio: sandbox path remap no longer masks missing reads
The sandbox sitecustomize shim remapped code-interpreter prefixes (/mnt/data, /workspace, ...) onto the working directory for every open mode, including reads. A read of a path that truly did not exist was silently redirected onto a same-basename workdir file instead of raising on the path the model used, hiding real missing-input errors. Remap writes and creates as before, but remap a read only when the mapped workdir target already exists (re-reading a just-written artifact); otherwise keep the original absolute path so the failure stays truthful.
* Studio: bound web fetch with one overall deadline and cancellation
The web fetch applied timeouts per network operation, so a GitHub README API attempt plus its HTML fallback plus up to five redirect hops could run well past the tool timeout, and nothing aborted once the client had disconnected. Add a single wall-clock deadline shared across the API attempt, the fallback, every redirect hop and the body read, cap each hop's socket timeout at the time left on the budget, and poll cancel_event. SSRF host pinning, per-hop redirect revalidation, the five-hop cap and the size cap are unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep tool-stream teardown off the event loop on disconnect
The bounded worker join added for disconnect safety still ran on an abnormal close, so a client disconnect could wait the full join timeout; and the safetensors and Anthropic tool streams closed their generator synchronously on the event loop, unlike the GGUF path. On abnormal exit the daemon worker is abandoned, so join with a zero timeout instead of waiting; offload the safetensors and Anthropic gen.close to a thread to match GGUF; and surface a heartbeat as soon as cancel_event is set while the worker is silent so the route regains control at once instead of after a heartbeat interval.
* Studio: extend the web-fetch deadline to DNS, the body read, and search
The overall fetch deadline did not cover host resolution or the response body read, and query-mode web_search ignored cancellation. Resolve hosts (initial and every redirect) on a budget-polled helper so a slow or pre-cancelled getaddrinfo aborts on time; read the capped body in chunks with the budget re-checked between them so a slow-drip server cannot stretch a single read past the deadline; and gate the blocking DDGS query on cancel_event on both sides. SSRF host pinning, per-hop redirect revalidation, the five-hop cap and the size cap are unchanged.
* Studio: defer the sandbox remap notice and tighten os.open create flags
The one-shot remap notice fired while computing the mapping, so a read that kept its original path emitted a false notice and spent the notice a later genuine remap needed. Only emit it once _remap_open commits to the redirect. Separately, os.open classified O_TRUNC / O_APPEND without O_CREAT as creating, but those cannot create a missing file, so a missing target now stays truthful (only O_CREAT maps to the creating mode).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: convert only genuine HTML README bodies, not Markdown with a leading block tag
The GitHub README API returns the raw file, almost always Markdown. _looks_like_html classified a Markdown README opening with a block tag (<ul>, <ol>, <dl>, <pre>, <blockquote>) as HTML, so _fetch_page_text ran it through html_to_markdown and collapsed its headings, lists and fenced code into a single line. Sniff the README body with a stricter document-level check (doctype or a leading <html>/<head>/<body>) so only a real .html README is converted; the general page path is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Surface unclassified mid-stream Anthropic errors as SSE error events
The local Anthropic tool-stream and plain-stream paths called
_anthropic_stream_error_event(e) with force defaulting to False, so an
unclassified mid-stream failure (llama-server crash, decode OOM, a
dropped upstream socket) returned no event. The except block then fell
through to emitter.finish(), emitting a normal message_delta and
message_stop that masked a truncated turn as a clean finish.
Pass force = True at both fall-through sites so an unclassified failure
emits a 500 SSE error event and returns, matching the Anthropic
passthrough path that already forces it. Add regression tests covering
both stream paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: give each tool run a unique part id so finished cards keep their own output
Backend tool ids restart at call_0 every assistant response, and the
transient toolLiveOutput/toolFullOutput store maps were keyed by pane
scope plus that bare backend id. Two turns in the same pane therefore
shared one key: the stale-clear at tool_start only guards the forward
direction, so when a later call_0 finished and wrote its preserved full
output, every earlier still-mounted finished card reading the same key
re-rendered and displayed the newer tool's output instead of its own.
Mint one per-run-unique part id per backend id (call_0:<uuid>) and route
tool_start/output/args/end through a single resolver so all events for a
call resolve the same id. The durable part carries the unique id, so the
finished-card readers derive a collision-free key with no change, and the
awaiting-confirmation path keeps its own synthesized id. Outbound replay
stays paired (the assistant tool_call id and the role=tool result
tool_call_id both come from the part id) and gains unique ids across
turns, which strict providers require.
* Studio: tighten PR comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: permission levels for chat tool calls (Ask, Approve for me, Off, Full access)
Replace the Bypass permissions on/off toggle with a four level permission
selector, available in Settings > General (new Permissions section above
Notifications), the chat settings panel, the composer plus menu, and a new
always visible composer pill.
Levels:
- Ask for approval: every local tool call pauses for allow/deny.
- Approve for me: only calls detected as potentially unsafe pause; the
python/terminal sandbox stays on.
- Off: never pauses; sandbox stays on (previous default behavior).
- Full access: never pauses and the sandbox is disabled. Still requires
the danger confirmation and is never restored across reloads.
Backend adds permission_mode to the OpenAI compatible and Anthropic
passthrough payloads and threads it through both tool loops. Auto mode
uses a fail closed classifier in tools.py: terminal commands must be on
a read only allowlist with no redirection or substitution, python code
is AST scanned for writes, exec, process and network use, MCP tools
auto run only with read only style names. Unknown tools always ask.
Legacy bypass_permissions and confirm_tool_calls keep their exact
behavior for existing API callers.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio permissions: Off is a plain toggle below Full access
Off moves to the bottom of the level menu with a short description and
acts as the feature-off state: the composer pill is hidden entirely
while Off, and reselecting the active level toggles back to Off.
* Studio permissions: higher contrast composer pill text
The permission pill uses a foreground based grey instead of the shared
muted pill color, so it reads darker in light mode and lighter in dark
mode. Full access keeps the danger yellow.
* Studio permissions: panel dropdown layout and shorter tooltip
Chat settings panel: the Bypass permissions label sits on one line with
a full width dropdown underneath, styled like the other panel selects.
Tooltip shortened and wording uses Unsloth instead of Studio.
* Studio permissions: harden auto-mode unsafe detection
Extend the Approve for me classifier to catch write and exec paths that
slipped through:
- terminal: sort -o, tree -o, xxd -r, find -exec/-execdir/-ok/-delete
and find -fprint/-fprintf/-fls now ask; plain read-only forms still
auto-run. awk is no longer allowlisted since its program can write and
call system().
- python: from-imports of mutating names (from os import remove [as rm])
and star imports now ask.
Found by a fuzz and edge-case simulation matrix; pinned in
test_permission_mode.py.
* Studio permissions: split multi-line terminal commands in auto detection
A shell runs each line as its own command, but shlex reads newlines as
whitespace, so "ls\nrm -rf x" demoted rm to argument position and
auto-ran. Normalize newlines and CR to separators, and treat any all
separator token as a command boundary so runs of blank lines still
split. Found by the simulation matrix; pinned in tests.
* Studio permissions: address review feedback on auto-mode detection
Auto-mode (Approve for me) safety classifier hardening:
- Python: flag any reference to a mutating attribute, not only direct
calls, so indirect refs (f = os.remove; f(x)) and aliases ask. Detect
Path.open(mode) write modes and wrap the AST walk to fail closed.
- Terminal: match attached short output flags (sort -o/tmp/out) and keep
find context across grouping parens so find ( -delete ) asks.
- Both: ask before reads that escape the sandbox workdir via parent
traversal or hit credential paths (.ssh, .aws, id_rsa, .pem, etc.).
permission_mode plumbing:
- Fold permission_mode=full into bypass_permissions at the request model
so route-level confirm-gate guards see it as bypass.
- Reject ask/auto on the Anthropic Messages server-tools path, which has
no confirmation channel (mirrors the confirm_tool_calls rejection).
- Keep forced RAG autoinject in auto mode: the safe search_knowledge_base
retrieval never gates, so derive the skip from the real confirm need.
- Reset all local preferences now also clears the legacy confirm key so a
reset restores the fresh default instead of the old level.
Regression tests added for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio permissions: close auto-mode classifier gaps from review round 2
Auto mode ("Approve for me") let a few mutating calls through as safe:
- os.open(...) always creates/writes a descriptor, so treat it as unsafe
even though builtin open in read mode stays safe.
- fd -x/--exec/-X/--exec-batch runs a command per match; scan for these
alongside find's -exec/-delete.
- tempfile writes artefacts and hands back writable handles, so importing
it now asks.
- Calling the result of a call (getattr(os, "remove")("x"), partials) is a
dynamic target the AST can't vet, so fail closed.
- An MCP tool whose name pairs a read verb with a mutating one
(get_or_create_issue, read_and_delete_file) no longer auto-runs on the
read prefix alone.
Also fold permission_mode="off" into confirm_tool_calls=False on both
request models so the non-stream route guard sees the disabled gate, and
drive the Confirm tool calls toggle off permission_mode="ask" so auto no
longer shows it on.
* Harden auto-mode classifier and normalize bypass to full for PR #7079
Approve for me now asks for a few cases it previously auto-ran:
- os.open via an os alias (import os as o; o.open(path, O_CREAT))
- pathlib symlink_to / hardlink_to / link_to
- importlib.import_module dynamic imports
- os.mkfifo / os.mknod / os.utime
Also fold bypass_permissions into full when a stale ask/auto permission_mode
is sent alongside it, so the Anthropic route guard no longer 400s those legacy
callers. Adds classifier and request-model regression tests.
* Close more auto-mode classifier gaps for PR #7079
Approve for me now asks for cases the review surfaced:
- builtin open aliased to a name (f = open; from builtins import open as w)
or looked up dynamically (globals()['open'])
- pickle / marshal / shelve / dill deserialization
- io.FileIO write handles
- sort --compress-program (runs an external program)
- MCP names carrying save/archive/submit/commit/push/sync/register verbs
Also refine the attribute open() write check so an explicit read mode
(ZipFile.open(name, "r")) stays auto while os.open flags still ask. Adds
test coverage for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close three more auto-mode gaps for PR #7079
- rg runs an arbitrary program per file via --pre / --hostname-bin, so
"Approve for me" now asks for those flags (rg is on the read-only
allowlist).
- A path-qualified command token (./ls, /tmp/cat) is an arbitrary
executable, not the trusted utility its basename matches, so it asks
before running.
- A direct /chat/completions caller that sets permission_mode ask/auto
but omits the legacy confirm_tool_calls flag now self-enables the
confirmation gate, so tools can no longer run ungated on that path.
Adds classifier and request-model tests for each case.
* Close auto-mode classifier gaps from review round 3 for PR #7079
Approve for me now asks for cases the latest pass surfaced:
- short-option clusters bundling a write flag (sort -uo out => -u -o)
- procfs reads that leak a process env/args/memory
(cat /proc/self/environ, /proc/PID/cmdline, maps)
- env-assignment prefixes that change command lookup/loading
(LD_PRELOAD=x ls, PATH=. ls, IFS=x ls); benign FOO=1 cmd stays auto
- os.open imported as a bare callable (from os import open as o)
Also drops ps from the safe terminal allowlist: its BSD environment
flags (ps auxe, ps eww) dump a parent process's unscrubbed env and
cannot be flag-parsed reliably, so ps always asks now. Adds classifier
tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 4 for PR #7079
Terminal (Approve for me now asks for these):
- cd dropped from the safe allowlist: cd /; cat etc/passwd moves the
shell out of the session workdir so a later relative read escapes it
- env -C/--chdir (workdir escape) and -S/--split-string (builds a fresh
command line); wrapper flags are now checked
- /etc//passwd and /etc/./passwd normalize to /etc/passwd before the
sensitive-path scan
- a sensitive path split across an assignment and an argument
(p=/etc; cat $p/passwd) via best-effort NAME=value expansion
Python:
- builtins.exec / builtins.eval attribute calls (dynamic code execution)
- destructured open aliases (f, _ = (open, print); f('out', 'w'))
- a sensitive path composed from literals (os.path.join('/etc','passwd'),
'/etc' + '/passwd')
- ZipFile/TarFile write modes (ZipFile(name, 'w')); the reader stays auto
Adds classifier tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 5 for PR #7079
Terminal (Approve for me now asks for these):
- procfs reads hidden by shell quotes (cat /proc/$PPID/enviro''n) or
quoted/nested-variable assignments (p="/proc/$PPID"; cat $p/environ):
quotes are stripped and NAME=value prefixes expanded before the scan
- LESSOPEN/LESSCLOSE, which make less run an input preprocessor command
Python:
- os.chdir / os.fchdir, which move the cwd so a later relative read
escapes the sandbox workdir
- sensitive paths composed via a pathlib / chain (Path('/etc') / 'passwd')
or an f-string of literals (f'/proc/{pid}/environ')
- runpy (import) and runpy.run_path / run_module, which run arbitrary code
Adds classifier tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 6 for PR #7079
Approve for me now asks for these:
- a mutating callable reached through a getattr alias
(rm = getattr(os, "remove"); rm("f")): calls through a getattr-bound
name fail closed
- compound MCP tool names carrying clone/checkout/comment/fork/tag/
invite/share, which start with a read verb but still mutate
- a sensitive path hidden behind a glob (cat /e??/passwd,
cat /e[t]c/passwd): a ? / * / [..] token is matched against the
sensitive-file set and bracket classes are de-obfuscated; benign
globs (ls *.py) stay auto
Also run first-pass RAG retrieval in off mode: like auto, off never
prompts, so a direct caller passing a stale confirm flag should not lose
document retrieval (both tool loops).
Adds classifier tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 7 for PR #7079
Approve for me now asks for these:
- __builtins__.exec / __builtins__.eval (dynamic code via the dunder)
- terminal reads that hide a credential path behind a backslash escape
(cat /et\c/passwd)
- read-named MCP filesystem calls pointed at a credential path
(mcp__fs__read_file {"path": "/etc/passwd"})
- compound MCP names carrying append / prepend
- open aliased through a subscript or builtins attribute
(f = globals()["open"]; f = builtins.open) then called to write
- open(..., **{"mode": "w"}) where a kwargs splat hides the write mode
- a sensitive path with a dynamic segment (open(f"/etc/{name}"),
os.path.join("/etc", name)); /tmp/{name} stays auto
- urllib3 networking
Also stop folding permission_mode ask/auto into confirm_tool_calls for
external-provider requests: that branch rejects confirm_tool_calls with
tools, and the mode only governs local tool calls. Local requests still
self-gate. Adds tests for each case.
* Close auto-mode classifier gaps from review round 8 for PR #7079
Approve for me now asks for these:
- dbm on the unsafe-module list: dbm.open(file, "c"/"n") creates files,
and importing the family signals a persistence writer
- reads of ~/.azure and ~/.config/gh credential stores (Azure/GitHub
tokens), in terminal, MCP arguments, and Python literals
- compound MCP names carrying upsert / assign
Adds classifier tests for each case.
* Gate secret mounts and fix the composer pill count for PR #7079
- Add Docker/Kubernetes secret mount dirs (/run/secrets,
/var/run/secrets) to the sensitive-path checks, so Approve for me asks
before reading injected credentials (terminal, MCP args, Python).
- Count the always-visible permission pill in the composer's compact
threshold so labels collapse at the intended width instead of
overflowing by one pill.
Adds classifier tests for the secret mount paths.
* Close auto-mode classifier gaps from review round 10 for PR #7079
Approve for me now asks for these:
- qualified pathlib constructors (pathlib.Path('/etc') / name), folded
the same as bare Path(...), so a dynamic sensitive path is detected
- open aliased through an annotated assignment (f: object = open;
f('out', 'w')), tracked like a plain assignment
- recursive searches rooted at an absolute path (grep -R TOKEN /home,
rg TOKEN /, fd pattern /etc), which read host files outside the
sandbox tree; sandbox-relative searches stay auto
Adds classifier tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 11 for PR #7079
Approve for me now asks for these terminal reads, which bash would
expand into a sensitive path only after the classifier had approved:
- a glob that resolves into a secret mount or credential dir
(cat /r?n/secrets/hf_token, cat /root/.s??/id_rsa)
- a recursive search rooted at a tilde home (grep -R TOKEN ~root,
grep -R TOKEN ~/logs)
- a brace expansion that builds a credential path (cat /etc/pass{w,}d)
- a default/alternate parameter expansion that builds one
(cat /etc/pass${x:-wd})
- an input redirection that hides a glob (cat </e??/passwd)
And these python calls:
- a str.format-built sensitive path (open('/etc/{}'.format('passwd')))
- writer methods that persist to disk without open() (numpy.save,
Image.save, plt.savefig, DataFrame.to_csv, json.dump)
Segment-wise directory matching keeps benign globs (ls /home/*/projects)
auto. Adds regression tests for each case and its safe counterpart.
* Close auto-mode classifier gaps from review round 12 for PR #7079
Approve for me now asks for these too:
- a terminal read whose parent traversal hides behind a redirection with
no following space (cat <../../notes)
- a python read whose path is built with str.join
(open(''.join(['/etc', '/passwd']))), told apart from os.path.join
- a dynamic-code builtin reached through an alias
(from builtins import eval as e; e(...); x = builtins.exec; x(...))
Adds regression tests for each case and its safe counterpart.
* Close auto-mode classifier gaps from review round 13 for PR #7079
Approve for me now asks for these too:
- a recursive search whose root is hidden behind an assignment
(p=/; grep -R TOKEN $p): the recursive-root test now runs on the
assignment-expanded tokens as well
- a python read whose sensitive path is split through a literal variable
(base = '/etc'; open(base + '/passwd')), including via an f-string
- numpy ndarray.tofile, which persists without open()
- a sequence brace read (cat /etc/pass{w..w}d), expanded alongside the
comma brace form before the sensitive-path scan
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 14 for PR #7079
Approve for me now asks for these python reads that assemble a sensitive
path in a form the fold did not yet recognize:
- a pathlib object reused through a name (p = Path('/etc'); p / 'passwd')
- old-style percent formatting ('%s/%s' % ('/etc', 'passwd'))
- Path.joinpath ('/etc'.joinpath('passwd'))
- a bytes path literal (open(b'/etc/passwd'))
And these terminal reads, which bash expands into a sensitive path only
after the classifier had approved:
- a substring parameter expansion off an assignment
(p=passwd; cat /etc/${p:0:6})
- an ANSI-C quoted path (cat $'/etc/pass\x77d')
- a glob into an Azure or GitHub CLI config dir
(cat /home/*/.az?re/..., cat /home/*/.config/g?/...)
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 15 for PR #7079
Approve for me now asks for these terminal reads, which bash expands into
a sensitive path only after the classifier had approved:
- a per-thread procfs env alias (cat /proc/$PPID/task/$PPID/environ)
- a recursive root behind a default parameter (grep -R TOKEN ${root:-/home})
- a path built by pattern replacement (p=passXd; cat /etc/${p/X/w})
And these python reads:
- a pathlib .parent/.parents chain that escapes the session workdir
((Path.cwd().parent / 'other' / 'notes').read_text())
- a sensitive path resolved through glob (glob.glob('/e??/passwd')[0])
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 16 for PR #7079
Approve for me now asks for these terminal reads, which bash expands into
a sensitive path only after the classifier had approved:
- a case-modifying parameter expansion (p=PASSWD; cat /etc/${p,,})
- a mutating find action hidden behind an assignment (f=-delete; find . $f)
- a glob assembled through an assignment (g=e??; cat /$g/passwd)
- a POSIX bracket class glob (cat /etc/pass[[:lower:]]d)
And these python reads/writes:
- a glob pattern folded from a literal variable
(base='/e??'; glob.glob(base + '/passwd'))
- a directly imported os.path.join (from os.path import join; join('/etc', 'passwd'))
- a directly imported writer (from numpy import save; save(...))
- an aliased pathlib constructor (from pathlib import Path as P; P('/etc') / 'passwd')
The find/fd and glob scans now run on the assignment/parameter-expanded
command, and pathlib/join/writer import aliases are tracked. Adds
regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode gaps from review round 17 for PR #7079
Two fixes:
- Gate sqlite3 in auto mode. sqlite3.connect(path) creates or mutates a
database file (and runs DDL/DML) with no open()/writer attribute for
the AST checks to catch, so treat the module like dbm and ask.
- Only self-enable confirm_tool_calls for Studio's own tool loop. The
ask/auto fold previously set confirm on every non-provider request,
including a plain client-tool passthrough (client-supplied tools that
Studio does not execute), which then tripped the local-tool
streaming-confirm route guard and rejected the passthrough. Restrict
the fold to requests that actually ask Studio to run tools
(enable_tools / enabled_tools / mcp_enabled).
Adds regression tests for the sqlite3 write and for the passthrough vs
tool-loop confirm behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode gaps from review round 18 for PR #7079
Classifier (auto mode asks for these):
- os.open through a module alias (import os as o; o.open(...)); os/posix
aliases are tracked like the literal module name.
- less/more pagers, whose escapes (+cmd, !shell, -o/--log-file, LESSOPEN)
can run a command or write a file the command-name allowlist cannot
see, so they are no longer auto-approved.
- a read-named MCP tool carrying a mutating query
(query_database {"query": "DELETE FROM runs"}); DML/DDL statements are
matched as whole statements so a natural-language query that merely
contains "delete" stays safe.
- ML persistence helpers (save_pretrained / save_file / save_model /
save_weights / save_lora / save_checkpoint) that export weights to disk.
Route:
- Honor CLI-forced tools when deriving the confirm gate. When a process
policy (unsloth run --enable-tools) opens the local tool loop without a
request-level tool signal, a permission_mode ask/auto request now
derives confirm at the route (GGUF and safetensors paths) so the mode
still gates the call, and a non-streaming ask/auto request is rejected
rather than running unprompted. A plain client-tool passthrough (no
local loop) is unaffected.
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 19 for PR #7079
Approve for me now asks for these too:
- a terminal read whose path is built by indirect parameter expansion
(x=passwd; p=x; cat /etc/${!p})
- a bash /dev/tcp or /dev/udp redirection, which opens a network socket
(cat </dev/tcp/host/port)
- a python read via pathlib's receiver-plus-pattern glob
(Path('/etc').glob('passw?'))
- a python read whose sensitive root passes through a normalizer
(os.path.abspath('/etc'), Path('/etc').resolve())
- a pickle-backed loader that can execute code on load
(torch.load, joblib.load, pandas.read_pickle), tracked through module
import aliases
- compiled code wrapped into a callable (compile(...) + types.FunctionType)
Adds regression tests for each case and its safe counterpart.
* Honor unset permission_mode as ask across the local tool loop for PR #7079
Three gaps where an omitted permission_mode did not behave as the
documented default ("ask"):
- The frontend only sent permission_mode / confirm_tool_calls /
bypass_permissions when a tool pill was on. A process policy
(unsloth run --enable-tools) can open the tool loop with no pill, so
the backend never saw the selected gate. Send the three permission
fields at the top level of every local chat payload instead.
- The backend read payload.confirm_tool_calls directly at the
pre-switch guard and both late per-backend derivations, so an unset
mode fell through as no-gate even for an explicit ask/auto. Add
_permission_mode_confirm(payload): explicit confirm_tool_calls wins,
explicit ask/auto engage the gate, off/full never prompt, and an
unset mode defaults to ask only where realizable (streaming), keeping
the legacy no-gate run for non-streaming unset requests.
- A forced ask/auto tool loop (CLI --enable-tools) with no stream now
400s at the pre-switch guard before evicting the resident model,
matching the existing confirm-without-stream rejection.
Adds test_permission_mode_confirm_derivation covering the derivation
truth table.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Declare permission_mode and bypass_permissions on the local chat request type
The previous change moved permission_mode, confirm_tool_calls and
bypass_permissions to the top level of the local chat payload. They had
lived inside a conditional spread, which is not subject to excess
property checking, so the fields were never declared on
OpenAIChatCompletionsRequest. At the top level tsc flagged
permission_mode as unknown (TS2322), failing the frontend build and
every job whose Studio install builds the frontend.
Add permission_mode and bypass_permissions to the request interface
(confirm_tool_calls was already present).
* Close auto-mode classifier gaps from review round 21 for PR #7079
Auto mode ("Approve for me") now asks for these too:
- a pathlib read built from a concrete constructor (PosixPath, WindowsPath
and their Pure* forms), which the folder previously ignored so
PosixPath('/etc') / 'passwd' lost its /etc root and ran unprompted
- a terminal or python read of the ssh host keys under /etc/ssh, which
the sensitive-path regex only covered for passwd/shadow/sudoers
- a read whose path variable is reassigned: the whole-tree pre-scan kept
the last binding, so base = '/etc'; open(base + '/passwd'); base = 'data'
folded to data/passwd and ran even though execution reads /etc/passwd;
any multiply-bound name now folds to the escape sentinel and asks
Also stop the pre-switch guard from rejecting a plain client-tool
passthrough. permission_mode only implies the confirm gate for Studio's
own local tool loop (enable_tools / enabled_tools / mcp_enabled); a
non-streaming client-tool passthrough that carries permission_mode
ask/auto (confirm_tool_calls left unset by the validator) must forward to
the provider branch. Only an explicit confirm_tool_calls=True still forces
the local-confirm rejection there.
Adds regression tests for each case and its safe counterpart.
* Fix permission-pill compaction count and Full-access confirm sync for PR #7079
Two frontend consistency issues in the permission-level UI:
- The composer collapses tool pills to icons above four, but the count
left out the permission pill, which renders in every mode except off.
With one optional pill also shown the row reached five pills without
collapsing and could overflow. Count the pill when it is visible
(permission_mode != off).
- Entering Full access via setPermissionMode('full') or
setBypassPermissions(true) left confirmToolCalls at its previous value,
so a Full-access run (which sends confirm_tool_calls=false) could still
report confirmations as enabled in response metadata. Set
confirmToolCalls false at both entry points.
* Close auto-mode classifier gaps from review round 23 for PR #7079
Auto mode ("Approve for me") now asks for these too:
- a command using an abbreviated GNU long option that reaches a
write/exec action (sort --out= for --output, env --ch= for --chdir,
fd --base-dir= for --base-directory); a prefix of an unsafe long flag
now fails closed
- printf -v NAME, which assigns to a shell variable, so
printf -v PATH %s .; ls can rewrite PATH and run ./ls unprompted
- fd --base-directory / --search-path, which move the search root
outside the session workdir without any positional slash token
- an MCP tool whose compound read name carries a copy-style mutator
(read_and_copy_file, get_and_snapshot_volume): copy, duplicate,
import, export, download, backup, restore, snapshot, mirror
Also treat an omitted permission_mode as its documented default ("ask")
on the Anthropic Messages server-tool path. That branch has no
confirmation channel and already rejects explicit ask/auto, so an
omitted mode now falls into the same rejection instead of silently
running server tools unprompted, unless the caller opted out with
confirm_tool_calls=false (the legacy equivalent of "off"). off/full and
that opt-out still run; the two routing tests that relied on the old
implicit run now set permission_mode="off".
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refine permission gating from review round 24 for PR #7079
Four fixes from the latest review:
- Anthropic Messages server tools: an omitted permission_mode no longer
rejects a request that only runs safe server tools (web_search), so
existing Anthropic callers keep working. It still rejects an omitted
mode when a local tool (terminal/python) is selected, and an explicit
ask/auto is still rejected outright. off/full and a
confirm_tool_calls=false opt-out always run.
- Pre-switch confirm-without-stream guard: use
_explicit_studio_tool_loop_requested (the same predicate the
passthrough router uses) instead of the policy-inclusive
_effective_enable_tools, so a process --enable-tools policy no longer
turns a client-tool passthrough into a local-loop rejection.
- Auto mode now asks for `uniq INPUT OUTPUT`: uniq writes its second
file positional, so a second positional (numeric flag values skipped)
is treated like `sort -o`. A lone `uniq file` or piped `... | uniq`
stays safe.
- MCP mutation check now strips SQL comments before matching, so
DELETE/**/FROM and UPDATE/**/users (comment-as-whitespace) no longer
slip past the DML/DDL denylist.
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode gaps from review round 25 for PR #7079
Auto mode ("Approve for me") now asks for these Python cases too:
- a bare archive constructor with a write mode (from zipfile import
ZipFile; ZipFile('out.zip', 'w')), tracked through import aliases like
the zipfile.ZipFile attribute call already was
- a dynamic lookup aliased through getattr (g = getattr;
rm = g(os, 'remove'); rm('file')), not just direct getattr(...) calls
- a callable that wraps open or a writer via functools.partial
(w = partial(open, mode='w'); w('out.txt')), which hides the write mode
Also:
- Always-safe tools (render_html) stream their early provisional canvas
card in auto mode again. The provisional-card guard mirrored the raw
confirm flag, which suppressed the early card under Approve-for-me; it
now reuses the auto-mode safety decision (is_always_safe_tool).
- The assistant-ui composer no longer counts the permission pill toward
its collapse threshold when the level is Off (the pill renders null
there), matching the other composer.
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align permission-mode confirm guards with the router (review round 26)
Three pre-switch confirm-gate checks disagreed with how the tool
loop actually enters, so a valid request could 400 (or an invalid
one could evict the resident model) at the wrong point:
- The /chat/completions pre-switch guard only looked at explicit
request fields, so a process --enable-tools policy that forces the
loop on (request omits enable_tools, no client tools) slipped past
it and only 400ed after _maybe_auto_switch_model had swapped the
model. It now mirrors the router's own loop-entry gate
(_effective_enable_tools or mcp, tool_choice="none" disabling it
unless explicitly asked) while still deferring to client-tool
passthrough, so the policy-forced case is caught before the switch.
- The ChatCompletionRequest full/off fold treated enabled_tools by
itself as a local-loop request and set confirm_tool_calls=True.
The router never starts the loop on enabled_tools alone (it only
filters which tools run), so a non-streaming passthrough carrying
client tools plus enabled_tools 400ed instead of routing verbatim.
The fold now keys off the same enable_tools / mcp_enabled signals.
- The Anthropic /v1/messages unsupported-mode rejection (ask/auto,
or an omitted mode selecting terminal/python) ran inside the
post-switch server-tools block, so an invalid request evicted the
resident model before the 400. It now runs before the auto-switch,
determined from the requested server tools, like the neighboring
malformed- and mixed-tool guards.
Adds regressions for each: a policy-forced non-streaming ask/auto
guard rejection that never reaches the switch, an enabled_tools-only
passthrough that keeps confirm unset, and an Anthropic rejection that
precedes _maybe_auto_switch_model.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 27 for PR #7079
Auto mode ("Approve for me") now asks for these host-mutating or
host-reading cases it previously ran unprompted (the sandbox does not
jail filesystem reads, and terminal commands can change host state):
- Destructured string literals fold into the scanned path now, so
base, leaf = ('/etc', 'passwd'); open(base + '/' + leaf).read()
resolves to /etc/passwd and asks, like the single-assignment form
already did. The tuple/list unpacking branch tracked only aliases to
open; it now also binds literal and folded-path elements.
- pathlib name rewrites fold to the rewritten path:
Path('/etc/x').with_name('passwd').read_text() (and with_stem /
with_suffix) spell no literal /etc/passwd but resolve to it, so they
are folded and caught. Benign in-sandbox rewrites stay safe.
- hostname NAME (or -F/--file, -b/--boot) sets the hostname, so a
positional or a set flag asks; bare hostname and the display flags
(-f/-i/-I/...) stay read-only.
- date -s/--set STRING and the bare MMDDhhmm... positional set the
system clock and now ask; the display forms stay read-only (+FORMAT,
-u/-R, and -d/-r/-f whose following value is skipped so date -d
tomorrow is not mistaken for a clock-setting positional).
Adds regression rows for each gap and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close more auto-mode classifier gaps from review round 28 for PR #7079
Auto mode ("Approve for me") now asks for these cases too:
- Mapping-style %-formatted paths. '/etc/%(f)s' % {'f': 'passwd'} folds
to /etc/passwd and asks; a dynamic value or a non-literal mapping
leaves the NUL marker so /etc/<dynamic> still fails closed. The path
folder previously handled only tuple/scalar % right-hand sides and
returned None for a dict, hiding the sensitive segment.
- A read-named MCP database tool carrying PostgreSQL COPY. COPY ... FROM
bulk-loads a table and COPY ... TO writes a server-side file, so both
are matched as mutating queries like DELETE/UPDATE already were. A
'copy' substring in a column name stays safe (word boundary).
- logging file handlers. logging.FileHandler('out.log', mode='w') (and
the default append mode, RotatingFileHandler/TimedRotatingFileHandler/
WatchedFileHandler, and the bare from-import form) create or truncate
a file like open(..., 'w'), so they are classified as writer calls.
StreamHandler / NullHandler and logging reads stay safe.
Adds regression rows for each gap and its safe counterpart.
* Fix writer aliases, GraphQL mutations, and auto server tools (review round 29)
- Auto-mode Python: an aliased writer or archive constructor is tracked
like the existing open alias, so from numpy import save; s = save;
s('out.npy', arr) (and z = ZipFile; z('a.zip', 'w'), incl. the
destructured forms) ask instead of running the write unprompted. A
benign builtin alias (x = len) stays safe.
- Auto-mode MCP: a read-named tool carrying a GraphQL mutation now asks.
query_graphql {"query": "mutation { deleteIssue(id: 1) }"} matches a
leading mutation keyword (GraphQL uses # comments, so it scans the raw
payload); GraphQL read queries stay safe.
- Anthropic /v1/messages: permission_mode "auto" no longer 400s a
safe-only server-tool selection. auto only needs a confirmation
channel for an unsafe call, so like the omitted default it runs for
web_search / RAG / render and rejects only when a gate-needing local
terminal/python tool is selected. ask still always rejects (it asks
per call, which this passthrough cannot honor). The rejection stays
ahead of the model auto-switch.
Adds regression rows/cases for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate asyncio spawn, net clients, default-captured open; allow safe-only auto (round 30)
Auto-mode Python now asks for more process/network/write vectors:
- asyncio process spawners (asyncio.create_subprocess_exec/shell and a
loop's subprocess_exec/shell) run an arbitrary program without the
terminal blocklist, so they gate like os.system/subprocess.
- stdlib network clients imaplib / poplib / nntplib / xmlrpc(.client) /
webbrowser open outbound connections the sandbox does not namespace
off, so their import asks like the other network modules.
- a callable captured as a function or lambda parameter default
(def f(o=open): o('out', 'w')) now binds that parameter into the same
alias set, so the later write through it is gated. A benign default
(o=len) stays safe.
Also, permission_mode "auto" no longer 400s a non-streaming local tool
request whose selection is always-safe-only (web_search / RAG / render).
auto only prompts for a classifier-flagged call, so a safe-only auto
request needs no stream, while ask, an explicit confirm_tool_calls=true,
MCP, and an unrestricted or unsafe selection still require it. Applied
via a shared _confirm_gate_needs_stream helper at the pre-switch, GGUF,
and safetensors confirm-stream guards; the loop's per-call confirm flag
is unchanged.
Adds regression rows/cases for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Catch brace-glob paths and attribute writer aliases; unfold auto (round 31)
- Terminal auto mode now runs the glob-sensitive scan over every
expansion candidate, so a brace-expanded glob (cat /e{t,}c/pass?d,
which bash expands to /etc/pass?d and then globs to /etc/passwd) asks.
Brace expansion alone spells no literal /etc/passwd and the glob only
resolves once the brace group is expanded, so scanning both together
is required. A benign brace + glob stays safe.
- Python auto mode now tracks a mutating attribute captured as a plain
name: s = np.save; s('out.npy', arr) binds a writer alias, a captured
.open bound method (p = Path('out').open; p('w')) fails closed on any
call since its mode position varies, and z = zipfile.ZipFile is gated
like the bare import. A benign attribute alias (x = np.mean) stays safe.
- permission_mode "auto" is no longer folded to confirm_tool_calls=true
on the request model. Folding it defeated the safe-only-selection
exception in _confirm_gate_needs_stream (an explicit confirm forces
stream=true), so a non-streaming safe-only auto request was rejected.
Leaving it unset lets the route apply the exception; the mode still
drives the loop's per-call gate. "ask" still folds (it gates every
call).
Adds regression rows/cases for each.
* Harden SQL/GraphQL/writer classification and passthrough guards (round 32)
MCP argument mutation detection (read-named query tools):
- CREATE DDL now matches modifiers and the broader object set, so
CREATE OR REPLACE VIEW, CREATE UNIQUE INDEX, CREATE TEMP TABLE,
CREATE MATERIALIZED VIEW and CREATE FUNCTION ask.
- Stored-procedure invocation (CALL proc(...), EXEC/EXECUTE) and VACUUM
ask; a natural-language "call me back" stays safe via the trailing
"(" / ";" / end lookahead.
- GraphQL # comments are stripped before the mutation match, so
mutation # note\n { deleteIssue(id: 1) } no longer hides the mutation.
Python auto-mode classification:
- numpy.memmap / open_memmap and pandas ExcelWriter / HDFStore create or
truncate a file on construction, so they gate like open(..., "w").
- asyncio networking (asyncio.open_connection, loop.create_connection /
create_server and unix variants) opens outbound connections/listeners
the sandbox does not isolate, so it gates like socket.connect.
Terminal auto-mode: file -C / --compile writes a compiled magic database.
Routing:
- A JSON-schema response_format is guided-decoding passthrough, not a
local tool loop, so a --enable-tools policy no longer 400s a
non-streaming ask/auto structured-output request at the confirm guard.
- An explicit confirm_tool_calls=False opts out of the Anthropic Messages
server-tool gate entirely (it wins over the mode, mirroring
_permission_mode_confirm and the GGUF path), so it runs even under ask.
Adds regression rows/cases for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Track path-ctor aliases, exempt empty selection and safe safetensors card (round 33)
- Python auto mode now propagates path constructor / join aliases, so
assigning Path or os.path.join to another local name is still folded:
P = Path; (P('/etc') / 'passwd').read_text() and j = os.path.join;
open(j('/etc', 'passwd')) ask, while a benign /tmp alias stays safe.
- _confirm_gate_needs_stream now distinguishes an omitted enabled_tools
(None, all tools) from an explicit empty list ([], no tools). An empty
selection runs no built-in tool and cannot prompt, so a non-streaming
auto request with enable_tools=true, enabled_tools=[] is no longer
400ed under a --enable-tools policy.
- The safetensors provisional render_html card now uses permission_mode:
render_html is always safe and never prompts, so its early canvas card
streams under auto (which ships confirm_tool_calls=true) instead of
being suppressed, matching the GGUF path's is_always_safe_tool exemption.
Adds regression rows/cases for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Extend auto-mode classifier: SQLite mutations, more net/xattr/compressed writers
Additional fail-closed gaps found by a fresh adversarial pass, each with a
reproduction and a benign control:
- MCP read-named tools now ask on SQLite-flavored writes the base DML/DDL regex
missed: ATTACH / DETACH DATABASE, a write-form PRAGMA (PRAGMA journal_mode=WAL
/ user_version=42 / foreign_keys(0), while the read-form PRAGMA journal_mode
stays safe), and load_extension() which loads and runs an arbitrary shared
library.
- Python auto mode now gates the remaining asyncio network entry points
(start_server, open_unix_connection, loop.create_datagram_endpoint,
sock_connect), os.setxattr / os.removexattr metadata writes, the gzip / bz2 /
lzma single-stream writers (GzipFile / BZ2File / LZMAFile, mode-gated like
ZipFile so a read stays safe), pandas to_xml, and the websockets client.
Benign controls (SELECT 1, read-form PRAGMA, asyncio.sleep, gzip read, numpy
read, natural-language "attach"/"analyze") stay safe. Regression rows added to
test_permission_mode.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close follow-up auto-mode gaps: SQLite/GraphQL variants, more writers and net
A fresh adversarial pass on the previous round found consistent extensions of
the same fail-closed rules, each reproduced with a benign control:
- MCP read-named tools: DROP / ALTER now cover the same broad object set as
CREATE (DROP FUNCTION, ALTER INDEX, DROP MATERIALIZED VIEW); ATTACH is caught
without the optional DATABASE keyword via its quoted-path form; a
schema-qualified write PRAGMA (PRAGMA main.user_version=1) is matched; and a
GraphQL mutation carrying directives (mutation M @audit { ... }) is treated as
a mutation.
- Python auto mode: os.startfile (Windows program launch), asyncio
start_unix_server, and the socketserver framework now ask; a gzip/bz2/lzma
open imported under an alias (from gzip import open as gopen) is gated like
builtin open; and a dynamic path prefix that can form a sensitive absolute
root (open(chr(47) + "etc/passwd"), open(os.sep + "etc/passwd")) is treated as
sensitive, while a dynamic prefix with a benign suffix stays safe.
Benign controls (read-form PRAGMA, natural-language "attach ... as", "drop the
idea", SELECT dropped_at, query @cached, gzip read alias, dynamic prefix +
data/file suffix) stay safe. Regression rows added to test_permission_mode.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate GNU time -o, basicConfig/methodcaller/fileinput, and more SQL mutations
Another adversarial pass surfaced further consistent fail-closed gaps, each
reproduced with a benign control:
- Terminal: GNU time -o/--output/-a/--append truncate or append to a file with
timing output; time is a wrapper, so the flag is checked before the wrapped
command like env -C.
- Python auto mode: logging.basicConfig(filename=...) opens a log file for
write; operator.methodcaller("write_text"/...) hides a writer method behind a
string and is now treated as dynamic dispatch (like getattr/partial);
fileinput.input(..., inplace=True) rewrites a file in place (the default read
form stays safe).
- MCP read-named tools: UPDATE now matches quoted, bracketed, and
schema-qualified targets (UPDATE "users" / public.users / ONLY public.users /
[users] / `users` SET); SELECT ... INTO OUTFILE/DUMPFILE writes a server file;
and state-changing SQL functions inside a SELECT (pg_terminate_backend,
setval, pg_write_file, lo_export, ...) ask.
Benign controls (time ls / time -p, basicConfig(level=), methodcaller("upper"),
fileinput read, NL "update ... set", setval_col column, PL/pgSQL SELECT INTO
var) stay safe. Regression rows added to test_permission_mode.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten auto-mode classifier comments
Collapse the multi-line rationale blocks in the permission classifier to one or
two lines each without dropping the exploit each branch closes. Comments and
whitespace only (no code change); the classifier tests are unchanged and pass.
* Retry transient SSE stalls in the tool-calling smoke probes
The tool-calling job flaked with a bare "TimeoutError: timed out": the
server-side python/bash probes stream over post_sse(), which (unlike
post()) had no transport-level retry, so a single stalled stream on a
shared CI runner hard-failed the whole step even though function calling
had already passed.
post_sse() now mirrors post(): a transport-level stall (stream open or a
mid-stream read timing out) is retried once with a fresh request capped
at 300s, while HTTP status errors still surface immediately. The
Linux _run_tool_probe caps each attempt at 360s and treats a stall that
outlives the retry as a failed attempt (rotate to the next seed) instead
of raising, and the web_search probe uses the same 360s cap. A genuine
server wedge still fails (the retry also times out), so real regressions
are not masked. Applied to the Linux, macOS, and Windows inference-smoke
workflows, which share the probe.
* Close five more auto-mode classifier gaps from review
Each reproduces with a benign control:
- Path constructor aliased through an attribute (P = pathlib.Path) now folds
like the bare-name alias, so (P('/etc') / 'passwd').read_text() asks while a
/tmp alias stays safe.
- Callable defaults that are not plain names now bind the parameter: an
attribute writer (def f(s=np.save)), an archive constructor, a captured .open,
and partial(open, mode='w') fold like the equivalent assignment; a benign
default (np.mean) does not.
- A dynamic piece inside a sensitive name (open('/et' + chr(99) + '/passwd'),
which folds to '/et\x00/passwd') now asks: the literals around each dynamic
segment are matched against a credential target with the segment as any run of
non-separator chars, so an all-dynamic ('1 + 1') or segment-spanning
(a + '/' + b) path stays safe.
- MCP read-named tools now ask on REFRESH MATERIALIZED VIEW and REINDEX; a
'refresh' column or natural-language 'refresh' stays safe.
- A writer/open alias handed to a higher-order invoker (map(open, names, modes),
starmap(np.save, ...)) is gated even without a direct call site; a benign
map(len, ...) is unaffected.
Regression rows added to test_permission_mode.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Default tool pills off on model load so tool execution is opt-in
resolveToolsEnabledOnLoad turned the web-search and code pills on for
any tool-capable model when the user had expressed no preference. Default
them off instead, so tool execution is enabled only when the person
clicks the pill to turn it on; a saved preference (on or off) is still
honoured, so a user who already enabled tools keeps them on.
* Gate mark/subscribe MCP verbs and qualified higher-order writer invokers
- A read-prefixed MCP tool name carrying mark / subscribe / unsubscribe
(get_and_mark_read, get_and_subscribe) now asks; a 'mark' substring inside
one token (list_bookmarks) stays safe.
- The higher-order writer check now also fires for a qualified invoker
(itertools.starmap(open, ...), functools.reduce(open, ...)), matching the
bare-name map/filter form; the writer-check on the first arg keeps a benign
itertools.starmap(len, ...) or itertools.chain(...) safe.
Regression rows added to test_permission_mode.py.
* Close more auto-mode gaps and align the ask confirm fold across paths
Each classifier change reproduces with a benign control:
- MCP read-named tools now ask on reply / notify verbs (get_and_reply_email,
list_and_notify_users), on catalog writes COMMENT ON / SECURITY LABEL / LOCK
TABLE and CREATE|DROP|ALTER POLICY, and on state-changing PostgreSQL functions
inside a read-shaped SELECT (nextval, set_config, pg_notify, the advisory-lock
family). A 'comment' column, a 'locks' table, and a 'nextval' column prefix
stay safe; the natural-language NOTIFY/SET ROLE statement forms are left out
because SET/NOTIFY overlap ordinary prose.
- Python auto mode now gates loader.exec_module (runs a module's code), archive
extractall (zip-slip file writes), the ensurepip / venv modules (install pip /
build an environment), and pydoc.writedoc. The Hugging Face login token
(~/.cache/huggingface/token and stored_tokens) is now a sensitive path, while
the rest of that cache (model data) stays readable.
- ChatCompletionRequest no longer overwrites an explicit confirm_tool_calls=false
when permission_mode='ask': the fold only self-enables the gate when the flag
is unset, so an explicit opt-out wins on the chat path exactly as it already
does via _permission_mode_confirm and the Anthropic pre-switch guard.
Regression rows added to test_permission_mode.py.
* Gate sort -T, xxd outfile positional, and the legacy HF token path
- sort -T / --temporary-directory writes spill files to a caller-chosen dir,
so it joins -o / --output in sort's unsafe-flag set.
- xxd [infile [outfile]] writes its second positional, like uniq; xxd now uses
the same second-positional-write handling (xxd in.bin out.hex asks, xxd
in.bin and xxd -c 16 in.bin stay read-only).
- The sensitive-path regex now also covers the legacy ~/.huggingface/token
location (optional leading dot), not just ~/.cache/huggingface/token; an
unrelated dir like myhuggingface/token stays safe.
Regression rows added to test_permission_mode.py.
* Catch multi-char SQL mutation targets, globbed credential names, digit outfiles
Three fail-open gaps in the auto-mode classifier, each with a benign control:
- SQL: the trailing word boundary on the MCP mutation regex meant a bare \w
stopped at the first character, so TRUNCATE users, GRANT SELECT ON t, and
REVOKE ALL ON t (multi-character names) slipped through while single-letter
targets matched. Match the whole identifier instead, and accept an explicit
AS alias on UPDATE (UPDATE users AS u SET). The implicit-alias form is left
out because it is indistinguishable from the prose "update <noun> <noun> set".
A truncate_log column and a grants table stay safe.
- A glob that resolves to a credential basename anywhere (cat ~/.huggingface/tok?n
-> token, cat proj/.netr? -> .netrc, cat repo/.aws/cred*) now asks; the fixed
target list only covered a handful of home paths. notes/dra?t.txt and
token_counts.tx? stay safe.
- uniq / xxd counted file positionals but skipped every numeric token to ignore
a flag value, so a file literally named with digits (uniq 123 out) hid the
output positional. Track each command's value-taking flags and consume only
the value, so uniq -f 2 in stays safe while uniq 123 out asks.
Regression rows added to test_permission_mode.py.
* Isolate the permission-mode loop tests from process-global state
The loop-driving tests (auto/off/full/bypass) drove run_safetensors_tool_loop
against a process-global approval registry (state.tool_approvals._pending)
keyed by a single shared session id, and read os.environ. Other backend test
modules mutate both, some at import time, so in the full-suite ordering a stale
pending approval or a leaked env var could make the loop deny or skip a call
these tests expect to run. It passed when the file ran alone but failed only in
the complete tests/ run on CI.
Add an autouse fixture that snapshots and restores os.environ and the approval
registry around each test, and give every _drive call a unique session id so a
leaked approval can never collide. Attach a compact event-stream dump to the
loop assertions so any residual full-suite-only failure reports what the loop
actually did instead of a bare diff.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden auto-mode classifier for recursive listers, sort file lists, aliased invokers, single-member extract
Close four fail-open gaps in is_potentially_unsafe_tool_call:
- terminal: tree/du (always recursive) and ls -R rooted at an absolute or
tilde path now ask, matching the existing grep/rg/find recursive-read gate;
relative walks stay safe.
- terminal: sort --files0-from=F reads the file list named in F, so it can
read arbitrary host files indirectly; added to sort's unsafe flags.
- python: track aliases of the higher-order invokers (m = map;
from itertools import starmap as sm) so an aliased invoker handed open/a
writer is still gated; a benign callable (map(len, ...)) stays safe.
- python: single-member archive extract (ZipFile/TarFile.extract) writes to
disk like extractall and is vulnerable to a crafted member path, so gate it.
Also update the stale _FakeExecuteTool in test_permission_mode.py to accept
the thread_id keyword that run_safetensors_tool_loop now forwards to
execute_tool after the main merge, which had broken the five tool-loop tests.
Adds regression rows covering each gap plus benign controls.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: normalize unknown permission_mode to 'ask' instead of a 422
The request models validated permission_mode with Literal[ask, auto, off,
full], so an unrecognized value from a newer UI/client was rejected with a 422
before the tool loops could apply their unknown -> ask fallback
(safetensors_agentic.py:464, llama_cpp.py:9001). That made the intended
forward-compat degradation unreachable at the API boundary for both Chat
Completions and the analogous Anthropic field.
Accept a plain string on both ChatCompletionRequest and AnthropicMessagesRequest
and normalize in a before-validator: None stays unset, the four known modes pass
through, and any other value degrades to the safest gate ('ask'), matching the
loops. Adds a regression test covering unknown/None/known across both models.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: close five more auto-mode classifier gaps
- terminal: xargs is no longer a safe wrapper. It appends arguments read from
stdin that the scan never sees, so `echo -o out /etc/passwd | xargs sort`
forwards to `sort -o out /etc/passwd` (a write + sensitive read) while only
the allow-listed literals are visible. Any xargs command now asks.
- terminal: ionice -p/-P/-u change the I/O priority of an already running
process / group / user instead of forwarding to a wrapped read-only command,
so `ionice -c 3 -p <pid>` now asks. ionice -c 3 <cmd> stays safe.
- MCP: gate ALTER SYSTEM, which persists PostgreSQL server configuration and was
not one of the DDL objects the mutation detector matched.
- MCP: a credential noun in a read-named tool (read_secret, list_tokens,
get_credentials, fetch_api_key) is a sensitive disclosure, so it asks even
without a mutating verb or a path/SQL argument. Scoped *_key nouns keep a
primary_key / keyboard lookup safe.
- render_html: no longer unconditionally safe. A static canvas still auto-runs,
but one whose HTML/JS reaches the network (fetch/WebSocket/remote script) asks,
since it can egress under the canvas CSP when artifact network access is on.
Its early provisional card is suppressed under the auto confirm gate, and the
confirm-without-stream guard now requires a stream when render_html is
selectable.
Adds regression rows and benign controls for each, and updates the render_html
provisional-card and confirm-gate tests to the new behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: extend auto-mode gates for indirect file lists, dynamic lookups, HTML network loads, and Anthropic render_html
Follow-ups on the previous classifier round:
- terminal: wc/du/find --files0-from (and find's -files0-from primary) read a
NUL-separated list of input paths from a file, the same indirect mechanism as
sort --files0-from, so a crafted list reads arbitrary host files past the
literal path/root checks. Gate them like sort.
- python: a namespace lookup through a dict-style call (f =
__builtins__.__dict__.get('open'), globals().get('open'), vars(x).get(...))
can return open/eval/a mutator, so poison the bound name like getattr/subscript
lookups already are. An ordinary dict .get or os.environ.get stays safe.
- render_html: broaden the network detector so a canvas that loads a resource
via CSS url()/@import, srcset, or a root-relative (/path) or protocol-relative
(//host) src/href is treated as networked, not just fetch/WebSocket/remote
script. Relative ./x and url(#id)/data: refs stay static/safe.
- Anthropic /v1/messages: drop render_html from the unprompted-safe server-tool
set. Since it can prompt (networked canvas) and this channel invokes the loop
without confirm, selecting it under ask/auto/omitted now rejects like
terminal/python; off/full (or an explicit confirm opt-out) run it.
Adds regression rows and benign controls for each, plus an Anthropic route test.
* Studio: close six more auto-mode classifier gaps
- terminal: a glob that expands to a project .env (cat .e?v) now asks; .env
joins the sensitive glob-basename set, matching the literal-path gate.
- python: an open bound onto an attribute (box.f = open; box.f('out','w'))
is tracked by attribute name, and open invoked via .__call__
(open.__call__('out','w'), unwrapped to the underlying callable) is gated,
so neither slips past the name-based open-alias checks. Benign attribute
callables and .__call__ on non-writers stay safe.
- python: a namespace lookup via .get/.pop/.setdefault already covered the
builtins case; unchanged here.
- MCP: a mutating HTTP verb in a method/verb argument (get_url
{"method": "DELETE"|"POST"|"PUT"|"PATCH"}) now asks, so a generic HTTP
tool cannot mutate an external service unprompted; GET/HEAD stay safe.
- MCP: a credential/secret environment-variable value (get_env
{"name": "OPENAI_API_KEY"}) is treated as a sensitive read via the same
credential-noun match used for tool names; PATH/HOME stay safe.
- render_html: self-navigation sinks (location.assign/replace, window.open,
assigning a URL to (window.)location(.href)) join the network detector, so a
canvas that navigates itself to an external URL asks; location.reload() /
history.back() stay static.
Adds regression rows and benign controls for each.
* Studio: gate obfuscated canvas egress, sensitive-dir iteration, and MCP metadata-host reads
- render_html: strip block comments before the network scan so fetch/*x*/(...)
cannot hide egress, and match bracket-access forms (window['fetch'](...),
self['open'](...)). Line // comments are left alone so the // in an https URL
is not eaten. A comment-only canvas stays static.
- python: enumerating a directory outside the sandbox (Path('/etc').iterdir(),
os.scandir('/etc'), os.listdir('/home'), os.walk('/')) reads host filenames
the direct /etc/passwd checks would prompt for, so gate it when the target dir
folds to an absolute/tilde/sensitive path; a relative dir stays safe and an
unresolved dynamic dir is left to other checks.
- MCP: a read-named HTTP tool pointed at a cloud-metadata / link-local host
(fetch_url {"url": "http://169.254.169.254/..."}, metadata.google.internal)
reads instance credentials, so classify those URL arguments as sensitive,
mirroring the sandbox SSRF blocklist; ordinary and localhost URLs stay safe.
Adds regression rows and benign controls for each.
* Studio: gate meta-refresh navigation, pandas HTML/markdown exporters, absolute glob roots, and checksum verify mode
* Studio: gate starred open writes, builtins.__import__, computed render_html sinks, and procfs fd reads in auto mode
* Studio: gate remote worker canvases, huggingface_hub downloads, and write callables passed to user helpers in auto mode
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio chat: tool-call nudging on by default (API stays opt-in)
Healing is already default-on everywhere and the nudge retry from the
client-tool passthrough is opt-in on the API. Studio chat had neither
signal: the frontend never sent nudge_tool_calls, and the safetensors
and MLX server-side loop lacked the GGUF loop's plan-without-action
re-prompt entirely.
Backend: the re-prompt helpers move from llama_cpp.py into
tool_call_parser.py (shared, cycle-free; the GGUF loop imports them
under its old names with zero behavior change) and
run_safetensors_tool_loop now re-prompts once at the streaming
no-tool-call exit, gated on Auto-Heal, active tools, nothing executed
yet, and short forward-looking text. Re-prompts do not consume tool
iterations.
Frontend: the chat adapter sends nudge_tool_calls from a new
nudgeToolCalls runtime setting (default true) with the same
persistence, hydration, and settings toggle plumbing as Auto-Heal.
Request-model defaults are untouched, so raw API callers stay opt-in.
* Address review: persist the nudge setting, consume the flag in the loops, skip the re-prompt after RAG autoinject
ChatSettingsPayload uses extra forbid, so a settings patch containing
nudgeToolCalls failed to persist any settings; the field is now typed
and round-trips. nudge_tool_calls now plumbs into both server-side tool
loops and gates the plan-without-action re-prompt with None meaning on,
so API callers keep today's behavior, explicit false disables it, and
Studio's default-on flag actually controls the path Studio chat runs.
The safetensors loop no longer re-prompts after RAG autoinject: the
injected retrieval bypasses the tool controller, so the nothing-executed
gate saw an empty history and re-asked after a successful retrieval.
* Safetensors loop: the plan-without-action retry requires an explicit nudge flag
The retry is new on this loop, so an omitted nudge_tool_calls must not
change existing API behavior; Studio opts in explicitly. The GGUF loop
keeps None as on because its re-prompt predates the flag.
* Suppress the plan-without-action re-prompt after a denied tool confirmation
A denial appends TOOL_REJECTED_MESSAGE but records nothing in the tool
controller history, so the nothing-executed gate re-prompted the model
to call the tool the user had just rejected, producing another
confirmation prompt. A denial now suppresses the re-prompt for the rest
of the request, mirroring the RAG autoinject handling.
* Tighten plan-without-action re-prompt comments
* Tighten plan-without-action re-prompt comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: match unified plan-without-action nudge cap to GGUF default of 3
The shared MAX_ACT_REPROMPTS was set to 1, but GGUF's established default
(llama_cpp.py) has re-prompted a stalling model up to 3 times since #5620.
Restore the GGUF-matched cap so safetensors and MLX inherit the same
behavior, and update the safetensors cap test to assert the cap dynamically.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: parse Mistral [TOOL_CALLS] and rehearsal tool-call shapes
Extends the rescue parsers in core/tool_healing.py and
core/inference/tool_call_parser.py to recognise two extra serialisations
local models commonly emit when bypassing native function calling:
* [TOOL_CALLS]name{json_args} (Devstral-Small-2, Mistral-Small-3.x).
* name[ARGS]{json_args} (reasoning-model rehearsal).
Both extractors use a brace-balance scan that honours escapes and
quoted strings so nested JSON args stay intact.
Also pre-strips <think>...</think> and [THINK]...[/THINK] blocks before
matching so calls emitted after a reasoning preamble are recognised
regardless of position.
Streaming gates (TOOL_XML_SIGNALS, llama_cpp.py _TOOL_XML_SIGNALS) and
the SSE strip regex (routes/inference.py _TOOL_XML_RE) gain the new
sentinels so the parser is actually invoked and the raw markup never
leaks to the UI.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip unclosed think blocks and catch rehearsal [ARGS] mid-buffer
The pre-existing ``_THINK_TAG_RE`` only matched closed thinking
blocks (``<think>...</think>`` or ``[THINK]...[/THINK]``). During
streaming the model is still inside the open block when the parser
runs, so any tool-shaped markup the model is REHEARSING inside that
block survived the strip and could be executed as a real call.
Switch both copies of the regex (parser + healing) to accept the
trailing block being terminated by end-of-string in addition to
the explicit closer.
The ``_TOOL_XML_SIGNALS`` list on the llama_cpp streaming buffer
included ``[ARGS]`` to catch rehearsal syntax, but the gate used a
``startswith`` check against the buffer head -- rehearsal is shaped
``name[ARGS]{json}``, so the buffer never STARTS with ``[ARGS]``
and the signal had no effect. Add a substring fallback for the
bracket-style signals so the BUFFERING window can still divert the
stream into DRAINING when rehearsal markup arrives mid-buffer.
Adds three regression tests covering rehearsal inside unclosed
``<think>`` / ``[THINK]`` blocks (must yield no calls) and the
positive case after a closed think block (still parsed).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden bracket-tag tool-call parsing and streaming strip
Address review findings on the Mistral [TOOL_CALLS] / rehearsal [ARGS] paths:
- Accept hyphenated tool names in the bracket parsers and strip patterns.
_MISTRAL_BRACKET_RE and _REHEARSAL_RE used \w+, which dropped or truncated
MCP function names containing dashes (mcp__srv__list-issues). Use [\w-]+ to
match the XML and Gemma parsers.
- Strip a partial bracket marker streamed before its opening brace. The
trailing-unclosed patterns required the {, so a [TOOL_CALLS]web_search or
python[ARGS] split across deltas leaked the raw marker to the UI. Match the
bare marker to end-of-text, mirroring how the bare open tags are stripped.
Closed pairs are unchanged so in-progress markup stays buffered until parsed.
- Strip a truncated bracket tail in the route-level display regex. _TOOL_XML_RE
required a balanced JSON object; a tool call truncated by EOS now strips up
to \Z, like the orphan-opening XML shapes. Complete calls still strip only
their balanced JSON so following prose survives.
Add regression tests for hyphenated names, the streaming partial-marker strip,
and the unclosed-tail route strip.
* Studio: preserve XML parameter indentation in tool_healing
The chat template emits <parameter=k>\nVALUE\n</parameter>; the parameter-start
regex consumed the wrapping newline AND the value's first-line indentation via a
trailing \s*, then str.strip() removed the rest, corrupting code/diff arguments.
Narrow the trailing class to horizontal whitespace and trim exactly one wrapping
newline (_trim_param_value), preserving indentation. Matches SGLang's qwen3_coder
detector and the same fix on the multi-format parser. Add a regression test.
* Studio: tighten Mistral/rehearsal tool-call comments
Compress the comments in the Mistral [TOOL_CALLS] / rehearsal [ARGS] healing shim
and its callers to one or two lines, keeping the bracket-tag stripping rationale,
the thinking-block handling note, and the forge attribution intact.
Comment-only: no code or behavior change (verified with comment_tools.py check
--strip-docstrings; tests green).
* Studio: fix think-strip arg corruption and nested bracket-JSON strip
Review follow-up for the Mistral/rehearsal healing shim:
- The <think>/[THINK] strip ran unconditionally over the whole content before
parsing, so a real tool argument that legitimately contained a <think> /
[THINK] literal was silently corrupted. Don't delete the blocks: compute the
reasoning-block spans and skip any tool-call candidate that STARTS inside one,
across all parse paths (JSON, Gemma, XML, bracket, rehearsal). A rehearsed call
inside reasoning is still ignored; a real call after </think> still parses.
- The bracket-tag display strip used a fixed one-level-nesting regex, so a call
with two-level-nested JSON args either leaked raw markup or, in final mode, let
the catch-all eat the trailing prose. Add a balanced-brace
_strip_bracket_tag_calls pass (any nesting depth) used by strip_tool_call_markup
and the route display strip.
Add regressions: <think>/[THINK] literal inside a real argument, rehearsal-inside-
think with a real call after, and two-level-nested bracket/rehearsal strip keeping
trailing prose.
* Studio: correct think-block comments to match span-skip behavior
The think-strip fix replaced the unconditional think-block strip with a
span-skip (the block is kept and any tool-call candidate starting inside it is
ignored), but two comments still described the old strip-first behavior. Update
the _THINK_TAG_RE comment and the parse_tool_calls_from_text docstring.
* Studio: parse Mistral arrays and call-ids, unify bracket parse/strip, keep it linear
- Parse the canonical Mistral array form (TOOL_CALLS followed by a JSON list of
calls) and emit every call; parse the v11 shape that carries an opaque CALL_ID
token between the name and ARGS (the function name is the token after
TOOL_CALLS, never the call-id); and parse a Mistral call plus a rehearsal call
in one message (the second was dropped yet still stripped from display).
- One shared balanced forward scan (_iter_bracket_spans) backs both the parser
and the strip path, so they no longer diverge. It is linear: each regex is
re-searched only once its cached match falls behind the cursor, replacing the
per-match full-tail re-scan that was O(n^2) (O(n^3) over a stream). A length cap
before the scan is a backstop.
- strip_tool_call_markup preserves think/reasoning blocks verbatim (the parser
skips tool markup inside them), stripping only the visible text around them.
- _in_think uses bisect over the sorted think spans (was a linear scan per
candidate).
- GGUF streaming strip runs the balanced bracket pre-pass before the regex
patterns so nested-arg calls do not leak or eat trailing prose, and the
BUFFERING ARGS detector requires the rehearsal name-ARGS shape.
- Tests: canonical array, array string-args, array strip keeps prose, Mistral
plus rehearsal multi-call, v11 call-id name, think-rehearsal strip
preservation, and bracket-strip linearity.
* Studio: preserve reasoning blocks in the route and streaming strip paths too
Addresses Gemini/Codex review: making strip_tool_call_markup preserve think
blocks left the route display strip and the GGUF streaming strip inconsistent,
so a rehearsed call inside a reasoning block was still deleted from the visible
text on those paths.
- Extract the think-block segmentation into one shared helper (strip_outside_think)
and route all three strip paths through it: strip_tool_call_markup,
_strip_tool_xml_for_display, and the GGUF _strip_tool_markup_streaming closure.
- Add a route-strip regression test that a rehearsal inside a reasoning block is
preserved while a real call outside it is still stripped.
* Studio: fix bracket-tag strip/buffer review findings
Address the live code-review findings on the Mistral bracket-tag / rehearsal
tool-call rescue path:
- tool_healing: a literal think block inside a tool-call argument is no longer
treated as a reasoning block. strip_outside_think now excludes think spans
that sit inside a complete tool-call span, so the call is stripped whole
instead of the split hiding its open/close pair and leaking the raw call.
- tool_healing: the rehearsal trailing-strip pattern requires a following brace
or end-of-text, so prose that merely mentions name[ARGS] is not truncated as
a phantom call. The bracket strip patterns are aligned with the parser
regexes (whitespace, v11 [CALL_ID]/[ARGS] metadata, and the [CALL_ID]
lookbehind).
- routes: strip a truncated canonical Mistral array ([TOOL_CALLS] [{... with no
closing bracket) that the balanced scan cannot remove, align the display
regex with the parser regexes, and apply the same rehearsal-prose guard.
- safetensors loop: mirror the GGUF [ARGS] rehearsal-substring check during
BUFFERING so a rehearsal name does not stream before its [ARGS] arrives.
Adds regression tests for each; existing parser suite stays green.
* Studio: hold split rehearsal tool-name prefix in both streaming loops
A reasoning-model rehearsal call can stream the tool name and its [ARGS] arm in
separate chunks (web_search then [ARGS]{...}). The buffering detector only
recognised the rehearsal once [ARGS] was present, so the bare tool name was
emitted as visible content before the call drained and executed.
Add _is_rehearsal_prefix (mirrored in the safetensors loop and the GGUF loop):
when a no-signal buffer is a bare active-tool name -- or a partial prefix of
NAME[ARGS] -- hold it as a prefix instead of streaming it, so the next chunk's
[ARGS] flips it to a drain. A whitespace in the buffer means prose, not a split
call, so ordinary text still streams.
Adds regression tests for the split rehearsal in both loops and a guard that a
plain non-tool word still streams.
* Studio: route Anthropic tool-call cleanup through the protected display strip
The Anthropic stream, non-stream, and passthrough paths cleaned content with raw
_TOOL_XML_RE.sub instead of _strip_tool_xml_for_display, so a rehearsal call
inside <think> was deleted from the reasoning and a nested [TOOL_CALLS] call
dropped its trailing prose (the OpenAI-compatible paths already use the helper).
Route all four sites (prior-assistant cleanup, streaming content events,
non-stream aggregation, passthrough conversion) through the protected helper, and
add a source-level guard test so raw _TOOL_XML_RE.sub stays confined to the
helper itself.
* Studio: stop split rehearsal tool names leaking once streaming, uncapped, or unrestricted
The split-rehearsal guard (NAME in one chunk, [ARGS]{...} in the next) only held
the name in the initial BUFFERING state. Three gaps remained where the bare tool
name still streamed as visible content before the call drained:
- STREAMING: after prose had already streamed, both loops emitted a trailing
active-tool-name token (and the GGUF/safetensors [ARGS] boundary was not pulled
back over the name). Hold the trailing rehearsal token and release it on the
next chunk, with an end-of-stream flush so a plain answer that merely ends on a
tool-name word is never dropped.
- Buffer cap: a realistic MCP name longer than the 32-char _MAX_BUFFER_CHARS cap
defeated the BUFFERING hold. A rehearsal prefix is self-bounding (it stops
matching once it grows past NAME[ARGS]), so the generic cap no longer applies to
it.
- Unrestricted mode (tools=[]): with no declared tool list, any bare identifier
may be a NAME[ARGS] rehearsal, so the prefix check now recognises one instead of
leaking the name and mis-parsing the call.
Regression tests cover the streaming, long-name, and unrestricted cases plus the
plain-prose paths that must not be held or corrupted.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio tools: protect think blocks in safetensors streaming, hold split rehearsal on initial flush, advertise Mistral tools
Pass-3 review follow-ups on the Mistral [TOOL_CALLS] / rehearsal [ARGS] work:
- Safetensors streaming display strip now preserves think / [THINK] reasoning
verbatim (routes through strip_outside_think like the GGUF path). A call
rehearsed inside a reasoning block was stripped mid-stream and then restored by
the final strip, a non-monotonic shrink/grow that corrupted append-by-length
stream consumers and the visible reasoning.
- The first flush out of BUFFERING (safetensors and GGUF) now applies the same
trailing-name hold the STREAMING branch uses, so a split rehearsal (prose plus a
trailing active tool name in one chunk, [ARGS]{...} in the next) no longer leaks
the bare name before the call drains.
- Safetensors capability gate no longer suppresses tools for Mistral [TOOL_CALLS]
templates, which the shared bracket-tag parser now handles end to end. Llama
python_tag stays suppressed (still unparseable).
- Route display strip applies the open-ended / bare-marker tail arms only on the
segment after the last reasoning block (closed-only regex before it), matching
strip_tool_call_markup, so a bare foo[ARGS] before a reasoning block is preserved
while complete calls are still removed in every segment.
Adds regression tests for each and updates the now-stale Mistral capability test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix tool-call think-marker and bracket-wrapper edge cases
Round-1 review follow-ups on the Mistral/rehearsal tool-call healing:
- tool_healing: a reasoning marker that opens INSIDE a tool call's
arguments is argument data, not a reasoning block. Add
_think_spans_outside_tool_markup (start-inside test) and use it in
both parse_tool_calls_from_text and strip_outside_think so a literal
marker in one call's args no longer hides a later call (parse) or
leaks the raw markup (strip) when the greedy match runs past the
call's closer.
- tool_healing: strip the orphan Mistral v11 [/TOOL_CALLS] closer left
behind after the balanced scan removes the call body. Add a route arm
for the same closer in _TOOL_XML_RE / _TOOL_XML_CLOSED_RE.
- safetensors + llama_cpp streaming strip: run the open-ended (EOS
anchored) tail patterns only on the last segment; segments before a
reasoning block use the closed-only patterns, matching the final
strip and the route strip. A bare foo[ARGS] before a reasoning block
is prose, not a truncated call.
- safetensors streaming detector: validate each [ARGS] hit before
draining. A bare foo[ARGS] in prose (no active tool name in front)
no longer drains the rest of the turn; a later real NAME[ARGS] call
is still found and the prose in between is preserved.
Regression tests added for each case across the parser, strip helpers,
and both streaming loops.
* Strip incomplete-XML tool markup with literal think tags; widen render-html detector
Round-2 review follow-ups.
- tool_healing: an UNCLOSED <tool_call> / <function= call that the parser still
executes via allow_incomplete leaked its markup when an argument contained a
literal think marker. _tool_call_markup_spans only covered closed calls, so the
literal was treated as a reasoning block to preserve. Extend it to the
open-ended XML tail forms (shared as _TOOL_OPEN_XML_TAIL_PATS) so a think marker
inside an unclosed call is argument data and the call's markup is stripped. A
complete call's opener stays bounded to its closed span, and a real reasoning
block with no tool call is still preserved.
- safetensors render-html provisional card: _detect_render_html_tool_start was
XML-only, so a Mistral [TOOL_CALLS]render_html or rehearsal render_html[ARGS]
call executed but skipped the early card. Detect the earliest tool-call marker
across every serialization the loop executes and fire when it is render_html.
Regression tests added for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio tools: gate [ARGS] on active tools and skip think-block render_html rehearsal
Round 3 review fixes for the Mistral / rehearsal tool-call parsing path. Both are
asymmetric-fix bugs where one code path applied a guard the analogous paths did not.
- [ARGS] active-tool gating: the streaming state already validates a rehearsal
NAME[ARGS] against the active tool list before draining, but the BUFFERING
detection and the end-of-stream safety-net checks (safetensors and GGUF) treated
any word[ARGS] substring as a tool boundary. An answer containing a literal
foo[ARGS]{...} in prose, where foo is not an enabled tool, was drained, parsed into
a disabled foo no-op, and forced an extra generation turn. Gate those checks on the
active tool name too (unrestricted mode still accepts any name), so inactive-name
prose is neither drained nor parsed. Adds a shared _has_genuine_tool_signal helper
(safetensors) and _gguf_rehearsal_signal_pos / _gguf_has_genuine_tool_signal (GGUF).
- render_html provisional card vs think blocks: the parser skips tool candidates that
start inside a <think>/[THINK] reasoning block, but the provisional render_html
detector scanned raw content. A render_html rehearsed inside <think> followed by a
real non-render_html call emitted a provisional render_html tool_start (reusing the
later call's id) that the loop never executed. Drop candidates that start inside a
think span and use the first marker of each shape outside the blocks. Also resolve
the [TOOL_CALLS] [{...}] array shape through the parser so a nested "name" argument
key no longer fires a false provisional card ahead of the real top-level tool name.
Adds regression tests for both loops: inactive-name foo[ARGS]{...} is not drained into
a disabled no-op or a retry turn, a think-block render_html rehearsal emits no
provisional card, and the array top-level name is read correctly.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate ambiguous bare-rehearsal parse and strip on the active tool list
A bare NAME[ARGS]{json} is a genuine rehearsal call only when NAME is an
active tool; otherwise it is prose. The earlier round gated only detection
(so an inactive foo[ARGS] no longer drained the buffer or forced a retry
turn), but the parse and strip stayed unrestricted, which produced two
regressions:
1. An inactive foo[ARGS]{...} placed immediately before a real
web_search[ARGS]{...} in the same content span made the real call fail
to execute (parse consumed the phantom foo call).
2. An inactive foo[ARGS]{...} in a prose answer had its markup stripped
from the visible text, corrupting the sentence to " is just syntax."
Thread enabled_tool_names through the shared parser/strip so parse and
strip apply the SAME active-tool gate as detection:
- core/tool_healing.py: _iter_bracket_spans skips an inactive rehearsal
span; parse_tool_calls_from_text, _strip_bracket_tag_calls,
_strip_markup_segment and strip_tool_call_markup accept and thread the
gate; apply_tool_strip_patterns keeps an inactive rehearsal match.
- core/inference/tool_call_parser.py: wrappers forward the gate.
- core/inference/safetensors_agentic.py and core/inference/llama_cpp.py:
compute the gate from the active tool list (None when unrestricted, to
keep the legacy strip-all behavior) and thread it into every parse and
streaming/final strip site.
- routes/inference.py: _strip_tool_xml_for_display accepts the gate and
keeps an inactive rehearsal via a capture group on its rehearsal arm, so
the display cleanup does not re-strip the already-correct loop output.
The [TOOL_CALLS] control-token arms still strip unconditionally. Wire
the current turn's active tool names into the GGUF and safetensors
content-display sites.
Tests: parse and strip gate coverage in test_tool_call_parser_strict.py,
test_tool_xml_strip.py and test_safetensors_tool_loop.py; end-to-end GGUF
coverage for the real-call-after-inactive-rehearsal case and a
strengthened assertion that the inactive rehearsal prose survives intact.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: render the reasoning block for safetensors and MLX like GGUF
enable_thinking chat templates (Qwen3/Qwen3.5/GLM) prefill an unclosed <think>
into the generation prompt, so the model emits only the closing </think> then
the answer. The safetensors/MLX chat stream emitted that as plain content, so
the reasoning showed inline with no collapsible thinking block, while GGUF
(which surfaces reasoning via reasoning_content) rendered one. This brings
safetensors and MLX to parity.
- _ResponsesReasoningExtractor gains a reasoning_prefilled mode that starts
inside the reasoning block and splits on the first </think>; default False
keeps GGUF and every existing caller byte-identical. It suppresses a stray
re-emitted <think> and holds partial markers back across chunk boundaries.
- _sf_reasoning_prefill_mode gates the mode on reasoning being enabled for the
request, an enable_thinking or enable_thinking_effort style, and the template
actually using the standard <think>/</think> markers. Models with a bespoke
reasoning channel (e.g. gemma's <|think|>/<|channel>) are excluded so their
answer is never swallowed; gpt-oss (Harmony) and thinking-off requests are
excluded too.
- sf_tool_stream and stream_chunks (the latter also serves MLX) feed text
through the extractor, emitting reasoning_content then content deltas, with a
per-turn reset in the tool loop and a flush before each tool_start; only the
visible delta reaches the monitor reply. The two non-streaming drains split
reasoning_content the same way.
- Tests: extractor prefilled mode (streaming and edge cases), the gate matrix
including the gemma-style exclusion, and a route-replay of the tool-loop
reasoning stream.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: skip tool calls rehearsed in prefilled reasoning
Reasoning models (Qwen3.5 enable_thinking) open <think> in the prompt, so the
generated text starts inside the thought and emits only a closing </think> with
no opener. _think_spans_outside_tool_markup only found spans with an explicit
opener, so a NAME[ARGS]{...} or [TOOL_CALLS] call rehearsed in that leading
thought was parsed and executed as a real call.
Add a leading think span (offset 0 through the first close marker) when the
content opens with a bare close, so the rehearsed call is skipped and the
reasoning is preserved by strip_outside_think. Guarded by the existing call-span
check: a literal </think> inside a real call's arguments does not trigger the
span, so a genuine leading call still fires. Tests for both cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: do not start prefilled reasoning mode when reasoning_effort is none
enable_thinking_effort models (e.g. GLM-5.2) express thinking-off via
reasoning_effort="none" rather than enable_thinking=False, but
_sf_reasoning_prefill_mode only looked at enable_thinking, so such a request
started the extractor in prefilled mode. With thinking off the model never emits
</think>, so the whole answer was captured as reasoning_content and the visible
content/stream came back empty. Thread reasoning_effort through and return False
when it is "none". Tests for none vs a real effort level.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: only treat a leading bare </think> as prefilled reasoning when a real call follows
The prefilled-reasoning virtual span fired on any unmatched leading close marker,
so a non-prefilled turn that emits a real call before a stray </think> (for
example "Now web_search[ARGS]{...}</think> answer") had the call swallowed by the
span and dropped. Require that a real tool call also appear after the close (the
actual turn that follows the thought) before adding the span, so a stray close in
a normal answer no longer suppresses a genuine leading call. The rehearse-then-
call case still skips the rehearsal. Test for the stray-close case.
* Studio: trim redundant comments (comment-only, AST-verified)
* studio: keep tool_healing importable on Python 3.9
_balanced_json_span was annotated -> int | None. With no
from __future__ import annotations, that PEP 604 union is evaluated at
import time, so on Python 3.9 (which the package still supports,
requires-python >=3.9, and where external inference servers import this
module standalone) the def raises TypeError and the whole module fails
to import before any parsing runs.
Add from __future__ import annotations so annotations stay lazy strings,
matching the prevailing convention across studio/backend. No behavior
change: the module has no runtime annotation introspection.
* Studio: gate the Anthropic tool-stream display strip on declared tools
The Anthropic streaming and non-streaming tool paths called
_strip_tool_xml_for_display without enabled_tool_names, so with the default
strip-all behavior a final answer that literally contains an inactive-name
NAME[ARGS]{json} (prose, not a call) lost those bytes in the delivered text.
The GGUF and safetensors paths already pass _display_tool_name_gate(tools);
these two sites were missed when that gate was threaded through.
Compute the gate from the declared tools and pass it at both sites (threading
openai_tools into _anthropic_tool_non_streaming and its caller), so an
inactive-name rehearsal survives while an active-name one is still stripped.
Add a regression test.
* Studio: hold a split unrestricted rehearsal prefix at the bracket
In unrestricted tool mode (tools=[]) the rehearsal-prefix regex required
[A after the bracket, so a chunk boundary landing right after NAME[ (e.g.
web_search[ then ARGS]{...}) failed the prefix check and streamed the
partial tool markup web_search[ to the client before the call drained.
Restricted mode already holds this via a startswith check. Make the bracket
and each ARGS letter individually optional so NAME[ is held too, matching
the documented intent. Add a regression test.
* Studio: gate rehearsal detection and history strip on the original tool set
Two display/loop gate fixes so a spent one-shot tool is handled consistently:
- Rehearsal DETECTION (safetensors and GGUF loops) now uses the ORIGINAL tool
list, matching the strip gate, instead of the post-removal active_tools. After a
one-shot tool (render_html) runs it is dropped from active_tools; a repeat
render_html[ARGS]{...} while another tool is still active was stripped from
display yet never detected, so it was not routed to the render_html_repeat no-op
and the turn ended as a blank continuation. Detection now fires for it.
- The GGUF assistant-history sanitiser forwards the enabled-tool-name gate (like
the live-response strip), so a prior turn documenting an inactive foo[ARGS]{...}
shape is preserved in the replayed prompt context instead of being deleted.
Add regression tests for both loops and the history strip.
* Studio: thread the tool-name gate through the remaining rehearsal/history sites
Follow-up to the rehearsal-detection and history-strip gate fixes, covering the
sibling sites that were missed:
- GGUF loop: the rehearsal-prefix and trailing-name hold checks now use the
original tool list (_detect_tools) like the detection path, so a spent one-shot's
split repeat (bare render_html then [ARGS]{...}) is held instead of flushed as
visible text.
- The safetensors and Anthropic assistant-history sanitisers and the Anthropic
non-streaming passthrough now forward the enabled-tool-name gate to
_strip_tool_xml_for_display, matching the GGUF history sanitiser and the live
strips, so a prior turn documenting an inactive foo[ARGS]{...} example is
preserved in the replayed prompt / final text instead of deleted.
Add regression tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tile bracket-call spans per array item and include the v11 closer
Two with_spans fixes for the Mistral bracket parser, both hit through the
client-tool passthrough healers:
- A multi-call [TOOL_CALLS] array carried its whole markup span on the first
call and zero-width spans after, so a consumer that filters promotions by
the declared tool set either re-emitted the full raw array as text next to
the promoted call or silently dropped a filtered call's bytes. The region is
now tiled across the call-producing items (each call's span covers its own
JSON object plus the separator bytes before it; the last span runs to the
region end), so promoted markup strips exactly once and a skipped call's
bytes stay visible.
- The v11 wrapper closer [/TOOL_CALLS] sat outside the reported span and
leaked as stray text after promotion; the region now extends over an
immediately-following closer.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: decouple healer signals from the loop signal set
The passthrough healer buffered on every TOOL_XML_SIGNALS entry, so the bare
[ARGS] rehearsal marker this branch adds for the loops (where it is gated on
active tool names) put legitimate prose like 'Use foo[ARGS] in templates'
into the holding state and stalled the stream until finalization. The healer
can never promote a bare rehearsal call, so it now buffers only on formats
its parser promotes: <tool_call>, <|tool_call>, <function=, [TOOL_CALLS].
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Condense comments in the Mistral tool-call rescue to contract essentials
* verify_import_hoist: exempt __future__ imports and same-diff relocations
Two false positives fired on this PR's refactor. A from __future__ import
is a compiler directive whose name never appears as a runtime load, so
HOISTED-IMPORT-UNUSED can never see it used, yet the file requires it for
PEP 604 annotations on Python 3.9. TARGET-CHANGED flagged the deliberate
move of the strip-pattern constants into core.inference.tool_call_parser
as a silent re-point even though the old module-level target was removed
and the new one added in the same diff. Both get narrow exemptions; a
re-point to a pre-existing target is still caught, and the self-test
negative controls all pass unchanged.
* Drain the whole Mistral [TOOL_CALLS] array in streaming passthrough healing
StreamToolCallHealer._drain promoted only the first parsed call per pass and
dropped the rest of the buffer past that one span. For a well-formed Mistral
parallel-tool-call array streamed through client-tool passthrough
([TOOL_CALLS][{...},{...}]), the per-item spans are contiguous, so after the
first call was promoted the residue began with ,{...}] (no leading signal) and
was flushed as raw text: every call after the first was lost.
_drain now walks the contiguous run of parsed calls (adjacent tiled spans =
one array), promoting each declared call and relaying undeclared ones as data,
and stops at the first gap (prose) or incomplete trailing block so separate
blocks still stream incrementally in document order. This mirrors the
non-streaming heal_openai_message / finalize promote-or-flush loop and the
server-side safetensors loop, which already handled multi-call arrays.
Added regression tests: 2-call array in one feed and char-by-char, an
undeclared middle call kept as text, and an array followed by trailing prose.
* Drain comma-less Mistral tool-call arrays and normalize null arguments
The array branch fed the whole body to a single json.loads, which rejects the
comma-less multi-call form the repo's own Mistral/Ollama templates render (the
range loop in ollama_template_mappers.py emits the objects with no separator) and
so dropped every call. Decode elements individually with the existing
comma-tolerant raw_decode helper, now _decode_array_items, which also returns the
objects, so all calls are recovered while the span tiling is unchanged.
Also normalize a non-object array argument such as arguments null to an empty
object, matching the wrapped tool_call path, instead of serializing None to the
string "null" that auto-heal would turn into a bogus query of "null".
* Gate safetensors reasoning prefill on the rendered generation prompt
reasoning_always_on fires on any paired <think></think> in the template,
including markup that only renders PAST assistant history (Kimi-K2-Thinking)
while the generation prompt opens no <think>. Starting the reasoning extractor
in prefilled mode there captured a normal answer entirely as reasoning_content
and returned blank visible content. Prefill only when rendering the generation
prompt actually leaves <think> open (DeepSeek-R1 / QwQ / Qwen3-Thinking);
history-only templates start the extractor in normal mode and parse the model's
own <think>...</think>. Adds a Kimi-shape regression test.
* Keep bare scalar Mistral array arguments raw instead of double-encoding
A scalar string argument in the canonical Mistral [TOOL_CALLS] array
(for example [TOOL_CALLS][{"name":"web_search","arguments":"weather"}])
was run through json.dumps, turning weather into the JSON string
"weather". The downstream argument healer then wrapped that quoted
form, so a single-string tool like web_search searched for the literal
"weather" with quotes. The <tool_call> path already keeps a scalar
argument raw; mirror it here so only a dict is serialized. Add a
regression test asserting both paths yield the same healed arguments.
* Tighten tool-call rescue and reasoning-prefill comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio: tool calling for Llama-3, Mistral, Gemma 4 on safetensors + MLX (#5615)
Adds tool calling for Llama-3, Mistral (pre-v11 + v11+ + [ARGS]), and Gemma 4 to the safetensors / transformers and MLX backends. Parser patched against llama.cpp / vLLM / SGLang per-family parsers and normalises to OpenAI shape. 96 targeted unit tests + cross-OS staging CI (ubuntu / macos-14 / windows) green on the multi-format probe.
* studio: tool-call healing parity between safetensors / MLX and GGUF
After the multi-format parser landed in #5615, the safetensors / MLX
agentic loop and the GGUF loop still differed on healing behaviour.
This commit closes the gaps in both directions so the two backends
react the same way to identical model output.
Changes:
1. core/inference/llama_cpp.py -- the GGUF BUFFERING state machine
now wakes on every emission marker the shared parser knows. Was
("<tool_call>", "<function="); is now the five-tuple imported
from core.inference.tool_call_parser (Qwen / Qwen3.5 / Llama-3
<|python_tag|> / Mistral [TOOL_CALLS] / Gemma 4 <|tool_call>).
Stream cleanup is delegated to the same shared strip_tool_markup
so leaked markup from any family is removed from assistant
content.
2. core/inference/llama_cpp.py -- per-tool canonical heal key. When
a tool arguments field is a bare string and JSON parsing fails,
the GGUF path now heals to {"code": raw_args} for python,
{"command": raw_args} for terminal, and {"query": raw_args} for
everything else. Was hard-coded to {"query": raw_args}, which
silently routed every python / terminal emission through
web_search. Mirrors safetensors_agentic._CANONICAL_HEAL_ARG.
3. core/inference/safetensors_agentic.py -- re-prompt on plan-
without-action. When the model emits a short forward-looking
intent ("I'll search for that", "Let me check", "First, I
will...") and no tool call, the loop nudges the model to act
instead of silently returning a plan-only answer. Up to
_MAX_REPROMPTS=3 (matches GGUF). The intent regex, character
cap, and instruction text are byte-identical to the GGUF path.
The buffer-end fall-through is unified so a buffered intent
emission that never exits the BUFFERING state still triggers
the re-prompt.
4. core/inference/safetensors_agentic.py -- extra iteration slots
for re-prompts. The loop now budgets max_tool_iterations +
_MAX_REPROMPTS + 1 total iterations and tracks the tool-call
count separately, so a stalling model can be nudged 3x without
eating the caller's tool-call budget. Mirrors the _extra slot
reservation in the GGUF path.
Tests (14 new safetensors-side units; 5 GGUF parity pins):
TestLoopRePrompt -- intent-trigger, plain-answer,
no-tools, cap-at-three, budget
preserved, buffer-end intent.
TestLoopCanonicalHealKey -- python / terminal / unknown.
TestGGUFSafetensorsHealingParity -- shared markers used, shared
strip used, canonical heal keys
identical, intent regex matches
same phrases, _MAX_REPROMPTS
equal on both backends.
All 110 targeted tests pass locally; the broader tool / inference /
model-config / sandbox / anthropic / mlx suites stay green.
Why this matters
Without this parity, Llama-3.2 / Mistral / Gemma 4 emissions on Mac
(MLX) and Linux-safetensors stop the agentic loop as soon as the
model says "Let me...", because the GGUF re-prompt logic never
existed on these backends. The two-marker GGUF BUFFERING tuple also
let non-Qwen tool emissions stream out as plain prose when
llama-server's structured channel did not pick them up. Both paths
now drain the same way, heal the same way, and re-prompt the same
way -- so a tool call that works on GGUF works identically on
safetensors / MLX.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix tool-call parser bugs from gemini review on #5620
Three high-priority gemini findings on the tool-call parsing additions:
1. unicode_escape on UTF-8 bytes corrupts non-ASCII literals
(e.g. ✨ becomes â\x9c¨). Replace with json.loads on a quoted
string -- preserves emoji / CJK / RTL while still handling
\n \t \uXXXX escapes.
2. Llama-3 sentinel stripping is order-dependent. A leading
`<|eot_id|><|begin_of_text|>` left `<|begin_of_text|>` behind
because the loop had already passed that sentinel. Loop until
no sentinel matches at the start.
3. Mistral v11+ `[TOOL_CALLS] name { json }` regex uses non-greedy
`\{.*?\}` which truncates at the first `}` of a nested JSON
argument, leaking the tail (e.g. `}}`) into user-visible
streamed text. Same problem for the v0.3 array pattern with
nested brackets. Strip those with balanced brace/bracket
scanning via a new `_strip_mistral_closed_calls` helper called
from `strip_tool_markup`.
Also fix the inference routes' parallel `_TOOL_XML_RE`:
- Same nested-JSON truncation in the Mistral patterns; route the
strip through the parser's balanced-scan helper via a thin
`_strip_tool_xml` wrapper that all existing callers now use.
- Llama-3 `<|python_tag|>[^\n<]*` stopped at any `<`, leaking the
tail of any tool call whose argument contained a literal `<`
(queries, code snippets). Relax to `[^\n]*` which keeps the
strip confined to the actual end-of-line.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tool calling for DeepSeek (R1/V3/V3.1), GLM 4.x, Kimi K2
Adds three more emission-family parsers to tool_call_parser.py so the
shared safetensors / MLX / GGUF agentic loop covers the major open-
weight reasoning families. Patterns ported from llama.cpp
(common/chat-parser.cpp legacy pre-PEG branch), vLLM
(tool_parsers/deepseekv3*, glm4_moe, kimi_k2), and SGLang
(function_call/deepseekv31_detector, glm4_moe_detector, kimik2_detector).
All three references are MIT (llama.cpp) or Apache-2.0 (vLLM, SGLang).
Formats covered:
DeepSeek R1 <｜tool▁calls▁begin｜><｜tool▁call▁begin｜>function
<｜tool▁sep｜>NAME\n```json\n{...}\n```<｜tool▁call▁end｜>
<｜tool▁calls▁end｜>
-- args wrapped in a Markdown json fence, ``function``
literal prefix per llama.cpp common_chat_parse_
deepseek_r1 (chat-parser.cpp:801-820)
DeepSeek V3/V3.1
<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>NAME
<｜tool▁sep｜>{json}<｜tool▁call▁end｜><｜tool▁calls▁end｜>
-- bare JSON, no code fence, no ``function`` prefix
per llama.cpp common_chat_parse_deepseek_v3_1
(chat-parser.cpp:822-879)
GLM 4.5/4.6/4.7 <tool_call>NAME\n<arg_key>k1</arg_key>
\n<arg_value>v1</arg_value>...</tool_call>
-- strings raw, non-strings JSON-encoded per
chat_template.jinja; multi-call is back-to-back
blocks. Per llama.cpp common_chat_parse_glm_4_5
(chat-parser.cpp:1040-1052)
Kimi K2 <|tool_calls_section_begin|><|tool_call_begin|>
functions.NAME:IDX<|tool_call_argument_begin|>{json}
<|tool_call_end|><|tool_calls_section_end|>
-- bare name recovered by stripping ``functions.``
prefix and ``:IDX`` suffix; full id preserved as
tool_calls[i].id so the roundtrip replays verbatim.
Per llama.cpp common_chat_parse_kimi_k2
(chat-parser.cpp:896-913)
Marker collisions
GLM uses the same ``<tool_call>`` opener as Qwen but with a bare
function name + ``<arg_key>`` body (Qwen has ``\s*{`` after the tag).
The dispatch keeps Qwen first; Qwen's _TC_JSON_START_RE returns no
matches on a GLM emission, so the fall-through to _parse_glm_tool_
calls handles it correctly. Existing Qwen tests confirm zero
regression.
Streaming buffer
TOOL_XML_SIGNALS extended from 5 markers to 12 so the BUFFERING state
machine wakes on every new family's section opener. Added the
DeepSeek alternative markers (ASCII underscores, short ``<｜tool▁calls｜>``
form) because real checkpoints emit those variants.
Strip patterns
_TOOL_CLOSED_PATS adds DeepSeek envelope (``<｜tool▁calls▁begin｜>...
<｜tool▁calls▁end｜>``) and Kimi section (``<|tool_calls_section_begin|>
...<|tool_calls_section_end|>``). _TOOL_ALL_PATS adds the same plus
the unclosed-tail variants so a truncated stream does not leak
markup.
Route gate
_detect_safetensors_features._PARSER_MARKERS grows to include
DeepSeek and Kimi markers plus ``<arg_key>`` (the unique GLM signal).
_TOOL_XML_RE (the route-layer markup-strip regex) gets DeepSeek and
Kimi closed-pair patterns. _TOOL_TEMPLATE_MARKERS in llama_cpp.py
adds ``message['role'] == 'tool'``, ``message['tool_calls']``, and
``tool_calls is defined`` so the classifier recognises DeepSeek's
subscripted-access template style (it has no top-level
``{% if tools %}`` block).
Tests (39 new):
TestParserDeepSeek (7) -- R1 fence, short-form opener, V3.1 bare,
multi-call, with-reasoning, strip,
signal-wakes-streaming
TestParserGLM (6) -- single, mixed types, multi-call,
unclosed-heal, no-Qwen-regression, strip
TestParserKimi (6) -- single, multi-call, dotted-name, unclosed,
strip, signal-wakes-streaming
TestParserCrossFormatRouting (2) -- dispatch routing, signal coverage
TestLoopBasic loop integration (3) -- DeepSeek / GLM / Kimi end-to-end
Capability advertise (3) -- DeepSeek / GLM / Kimi templates flip
supports_tools=True
All 398 targeted tests pass locally (115 safetensors + 27 capability
+ rest of tool / inference / sandbox / model-config suites). Builds
on PR #5620 (parser + healing parity for Llama-3 / Mistral / Gemma 4);
will rebase cleanly onto main once #5620 lands. PR opened as draft -
do not merge until validated against real models for each family.
Sources
- llama.cpp common/chat-parser.cpp lines 801-913, 1040-1052 (MIT)
- vLLM vllm/tool_parsers/deepseekv31_tool_parser.py (Apache-2.0)
- vLLM vllm/tool_parsers/glm4_moe_tool_parser.py (Apache-2.0)
- vLLM vllm/tool_parsers/kimi_k2_tool_parser.py (Apache-2.0)
- SGLang python/sglang/srt/function_call/{deepseekv31,glm4_moe,kimik2}_
detector.py (Apache-2.0)
- Live chat templates: deepseek-ai/DeepSeek-V3.1, zai-org/GLM-4.6,
moonshotai/Kimi-K2-Instruct, unsloth/DeepSeek-V3-0324,
unsloth/GLM-4.5-Air, unsloth/Kimi-K2-Instruct
* studio/routes: make python_tag strip multi-line aware
Earlier revisions of _TOOL_XML_RE in studio.backend.routes.inference
oscillated between two bug shapes:
5615 r"<\|python_tag\|>[^\n<]*" -- stopped at any literal "<"
so code='if x < 10: pass'
leaked '< 10: pass)' to the
user.
5620.1 r"<\|python_tag\|>[^\n]*" -- single-line only; the second
line of
python.call(code="a\nb")
leaked.
The full parser (_parse_llama3_python_tag) already handles both via
balanced-brace scanning, so the parsing path was fine; the LEAK was
in the streaming strip path that runs on every cumulative emission
while content is still arriving.
Switch to r"<\|python_tag\|>(?:[^<]|<(?!\|))*" so the strip consumes:
* any character that is not a "<" (newlines, JSON, code, ...),
* a "<" only when it is NOT followed by "|" (i.e. NOT a Llama-3
sentinel start like <|eot_id|>, <|eom_id|>, <|begin_of_text|>).
This means:
* code='if x < 10' stays inside the strip (5615 fix preserved),
* multi-line code stays inside the strip (5620 round 2),
* the strip terminates at the next Llama-3 sentinel so trailing
assistant content survives.
Tests: TestRoutesPythonTagStrip (8 cases)
pytest test_safetensors_tool_loop.py test_safetensors_capability_advertise.py
-> 118 passed in 1.81s (was 110).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: review follow-ups for DeepSeek / GLM / Kimi tool calling
Four fixes addressing review of the parent commit:
1. GLM <arg_value> coercion: tighten the
json.loads -> ast.literal_eval -> raw cascade to only deserialize
when the body unambiguously looks like a JSON literal (object,
array, JSON-encoded string, true/false/null, or numeric). Strings
like ``True`` / ``None`` (Python literals, not JSON) and arbitrary
prose now stay raw. The bare-numeric / bare-boolean ambiguity with
string args remains an inherent limitation of the template without
schema access -- documented in the new comment. Drops the ast
import entirely (closes Gemini's :1036 suggestion).
2. Kimi K2 bare-counter ids (e.g. ``<|tool_call_begin|>3``) are now
dropped rather than surfaced as a tool literally named "3". Matches
vLLM behaviour; SGLang's schema-infer fallback is out of scope at
the parse site. Real Kimi K2 emissions use ``functions.NAME:IDX``
so this is the exception path.
3. Restore the elaborate ``<|python_tag|>(?:[^<]|<(?!\|))*`` clause in
routes.inference._TOOL_XML_RE -- the simpler ``[^\n<]*`` form
regressed PR #5620's multi-line / literal-``<`` python_tag fix.
Restore ``TestRoutesPythonTagStrip`` (8 tests) adapted to call
``_TOOL_XML_RE.sub`` directly since the ``_strip_tool_xml`` helper
was inlined this PR.
4. Add the spaced and backslash-escaped DeepSeek opener variants
(``<｜tool calls begin｜>``, ``<｜tool\_calls\_begin｜>``) to
``TOOL_XML_SIGNALS`` for streaming-gate parity with
``_DEEPSEEK_BEGIN_RE``.
Also updates the llama.cpp / vLLM citations in the parser docstrings:
``common/chat-parser.cpp`` was split into ``common/chat.cpp`` +
``common/chat-peg-parser.cpp`` by llama.cpp PR #18675, and vLLM
moved the tool parsers from ``vllm/entrypoints/openai/tool_parsers/``
to ``vllm/tool_parsers/``. Pin to pre-refactor commit ``51fa458a92d6``
where the cited line numbers still resolve.
New regression tests in ``test_pr5624_regressions.py`` cover the GLM
coercion heuristic shapes, GLM literal-``<`` in arg_value, Kimi K2
dotted name, Kimi K2 bare-counter drop, DeepSeek V3.1 truncated
mid-stream, and routes-layer strip across all three new families.
Tests:
pytest studio/backend/tests/test_safetensors_tool_loop.py
studio/backend/tests/test_safetensors_capability_advertise.py
studio/backend/tests/test_pr5624_regressions.py -q
-> 170 passed in 1.91s
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten verbose comments in tool-call parser sections
Comments were narrating what the code already says. Cut historical
"earlier revisions used X, then Y" narratives down to one-line WHY
notes where the footgun still matters (canonical heal-key parity,
balanced-brace vs non-greedy regex, ``(?:[^<]|<(?!\|))*`` over
``[^\n<]*``/``[^\n]*``). Drop section-header banners.
No behaviour change. Re-ran:
pytest studio/backend/tests/test_safetensors_tool_loop.py \
studio/backend/tests/test_safetensors_capability_advertise.py -q
-> 118 passed.
Regression replay (parser + _coerce_arguments on the 5 #5615 inputs)
-> 21/21.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: GLM 4.7 no-newline emission + Kimi multi-section parity
Two fixes surfaced by triple-confirm verification against the live
HF chat templates and upstream llama.cpp / vLLM / SGLang parsers.
1. GLM 4.7 silent drop
``zai-org/GLM-4.7/chat_template.jinja`` line 65 uses
``{{- '<tool_call>' + tc.name -}}`` which Jinja strips trailing
whitespace from, so the first ``<arg_key>`` follows the function
name with NO ``\n`` between them. Real emissions look like
``<tool_call>get_weather<arg_key>city</arg_key><arg_value>London
</arg_value></tool_call>``. The previous ``_GLM_TC_OPEN_RE`` ended
the name with ``\n`` so GLM-4.7 calls were silently dropped
(parser returned ``[]``).
Fix: relax the name terminator to a lookahead that accepts EITHER
``\n`` OR the next ``<arg_key>``:
_GLM_TC_OPEN_RE = re.compile(
r"<tool_call>\s*([^\n<{][^\n<]*?)\s*(?=\n|<arg_key>)"
)
The first-char restriction ``[^\n<{]`` still excludes Qwen's
``<tool_call>{json}`` form so the Qwen-vs-GLM dispatch remains
mutually exclusive.
2. Kimi multi-section parity with vLLM / SGLang
``vllm/tool_parsers/kimi_k2_tool_parser.py`` and SGLang's
``kimik2_detector.py`` both use ``re.findall`` and so collect every
``<|tool_calls_section_begin|>...<|tool_calls_section_end|>`` block
in a single stream. The previous implementation stopped at the
first ``<|tool_calls_section_end|>``. Kimi K2 doesn't emit
multi-section in practice, but parity is cheap.
Fix: wrap the existing per-call body parser in an outer loop that
advances past each ``<|tool_calls_section_end|>`` and continues to
the next ``<|tool_calls_section_begin|>``. Body parsing extracted
to ``_parse_kimi_section_body`` for clarity. Truncated final
section is still surfaced via the existing in-body balanced-brace
walk.
Verified independently against the live HF templates:
* GLM-4.7 emission constructed from the live template parses to the
expected ``{name, arguments}`` shape.
* GLM-4.5 / 4.6 newline shape continues to parse (the lookahead also
matches ``\n``).
* Qwen ``<tool_call>{json}`` still dispatches to the Qwen path -- the
first-char restriction stops the GLM regex from biting JSON bodies.
* Kimi two-section stream surfaces both calls in order with full ids
preserved.
* Bare-counter Kimi ids still drop.
Tests added in ``test_pr5624_regressions.py``:
* ``test_glm_4_7_no_newlines_between_name_and_arg_key``
* ``test_glm_4_7_no_newlines_multi_call``
* ``test_glm_4_7_does_not_break_qwen_path``
* ``test_kimi_two_sections_in_one_stream_both_parse``
pytest studio/backend/tests/test_safetensors_tool_loop.py
studio/backend/tests/test_safetensors_capability_advertise.py
studio/backend/tests/test_pr5624_regressions.py -q
-> 174 passed in 1.93s
pytest studio/backend/tests/ -q -k 'not gpu and not llama_cpp_integration'
-> 2038 passed, 15 failed (pre-existing CI gaps).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: parser robustness fixes for PR #5620
Three surgical extensions to the multi-format tool-call parser, each
covering a real fine-tune / template emission shape that the current
parser silently drops. No path narrows; all changes widen what is
accepted.
1. `_parse_tool_call_json` now accepts both `arguments` and
`parameters` keys. A Hermes / Qwen `<tool_call>{json}</tool_call>`
wrapper around a Llama-3.2 fine-tune that emits the `parameters`
key was extracting the tool name and silently discarding the
args, producing a working-shaped call with an empty payload. The
bare-JSON and python_tag paths already accepted both keys; this
path now matches them.
2. `_TC_FUNC_START_RE`, `_TC_PARAM_START_RE`, and `_TC_PARAM_CLOSE_RE`
now also match the attribute form
`<function name="..."><param name="...">v</param></function>` used
by MiniCPM-5 and MiniMax-M2. Names land in either capture group,
and `</param>` is accepted as a short close.
3. `_parse_llama3_bare_json` sentinel-strip now consumes the role
label inserted between `<|start_header_id|>` and
`<|end_header_id|>` by Meta's official Llama-3.x chat template.
Without this, every assistant turn re-fed through the template
prefix `<|start_header_id|>assistant<|end_header_id|>\n\n{json}`
parsed to zero calls, so any history-with-tool-call round-trip
in production silently dropped.
Tests in `studio/backend/tests/test_safetensors_tool_loop.py`:
* `TestParserRobustness::test_tool_call_json_accepts_parameters_key`
* `TestParserRobustness::test_function_xml_attribute_form`
* `TestParserRobustness::test_function_xml_attribute_form_multi_param`
* `TestParserRobustness::test_function_xml_legacy_equals_form_still_works`
(regression guard for the existing `<function=name>` syntax)
* `TestParserRobustness::test_llama3_chat_template_round_trip`
* `TestParserRobustness::test_llama3_round_trip_all_roles`
* `TestParserRobustness::test_llama3_round_trip_with_eot_prefix`
`pytest studio/backend/tests/test_safetensors_tool_loop.py
studio/backend/tests/test_safetensors_capability_advertise.py -q`
goes from 118 to 125 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim verbose comments in tool-call parser sections for PR #5624
Pure comment / docstring tightening on top of the GLM 4.7 + Kimi
multi-section fixes. No behavioural change.
* Drop multi-paragraph prelude and post-refactor citation chatter in
the DeepSeek, GLM and Kimi parser docstrings; keep the shape and
upstream-commit pin.
* Collapse ``parse_tool_calls_from_text``'s 9 per-family blocks into
a single ordered loop with one combined comment.
* Tighten the GLM coercion, Kimi bare-counter and ``_TOOL_XML_RE``
comments to one or two lines each.
* Same trim pass on ``_PARSER_MARKERS`` and the regression-test
docstrings.
Tests:
pytest studio/backend/tests/test_safetensors_tool_loop.py
studio/backend/tests/test_safetensors_capability_advertise.py
studio/backend/tests/test_pr5624_regressions.py -q
-> 174 passed in 2.00s
* Fix O(N^2) DeepSeek V3.1 backtracking for PR #5624
Adversarial input ``<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>fn<｜tool▁sep｜>``
followed by a long body that does NOT contain a closing brace caused
the V3 path's ``([^\n<]+?)<｜tool▁sep｜>`` regex to backtrack
quadratically: at each position the lazy quantifier extends one char
at a time looking for a sep that isn't there, taking ~19s on 50k
chars.
Replace the regex search with ``str.find`` on the sep marker plus a
left-walk to recover the name. ``str.find`` is O(N); the walk stops
on ``\n`` (turn boundary), ``<`` (start of a tag), or ``>`` (end of
an optional ``<｜tool▁call▁begin｜>`` prefix). Same observable
behaviour as the regex on every canonical input.
Tests:
test_deepseek_v3_1_huge_truncated_body_is_linear (new) -- 50k chars
must parse in < 1s.
pytest studio/backend/tests/test_safetensors_tool_loop.py
studio/backend/tests/test_safetensors_capability_advertise.py
studio/backend/tests/test_pr5624_regressions.py -q
-> 175 passed in 1.97s
pytest studio/backend/tests/ -q -k 'not gpu and not llama_cpp_integration'
-> 2038 passed, 15 pre-existing failures unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: terminate function-XML body at </function>, not just </tool_call>
`_parse_function_xml` was looking for `</tool_call>` (the Hermes
wrapper) as the body terminator. When a model emits a standalone
`<function=NAME><parameter=K>v</parameter></function>` followed by
explanatory prose (which models routinely do), no `</tool_call>` is
present, so the body extended to end-of-string and the trailing
prose leaked into the LAST parameter value.
Pre-existing on main (the legacy `<function=NAME>` form had this
bug too). Same affects PR #5620's new attribute-form
`<function name="NAME"><param name="K">v</param></function>`
emission used by MiniCPM-5 / MiniMax-M2.
Fix: `_TC_END_TAG_RE` now matches either `</tool_call>` OR
`</function>`. The existing `_TC_FUNC_CLOSE_RE` / `_TC_PARAM_CLOSE_RE`
strips are unchanged. Multi-call inputs still bound each function
at the next `<function=` start, so no over-eager consumption.
New tests:
* `test_function_xml_followed_by_prose` (legacy form + prose)
* `test_function_attribute_xml_followed_by_prose` (attribute form + prose)
Existing `test_code_with_embedded_xml` still passes (a parameter
value containing literal `<a></a>` is preserved because the
embedded close tag is `</a>`, not `</function>`).
`pytest studio/backend/tests/test_safetensors_tool_loop.py
studio/backend/tests/test_safetensors_capability_advertise.py -q`
goes from 125 to 127 passed.
* Studio: tighten Llama-3.2 bare-JSON guard
A fuzz pass on PR #5811 turned up that ``_parse_llama3_bare_json``
accepted ``parameters`` as a string, contradicting the docstring's
"parameters or arguments is a dict" guard. Prose JSON like
``{"name":"foo","parameters":"a sentence"}`` would wrongly fire the
parser, which the agentic loop would then heal into a real
``foo(query="a sentence")`` call.
Same code lives on this branch, so the same fix applies here.
Tightened guard:
- ``parameters`` must be a dict (Llama-3 spec).
- ``arguments`` may be a dict, or a JSON-encoded string that
decodes to a dict (OpenAI shape, e.g.
``"arguments":"{\"q\":\"x\"}"``). Plain non-JSON strings or
JSON-strings of lists / scalars / null no longer pass.
Mirrors the fix landed in PR #5811 commit 615b8608. Adds the same
4 regression tests under TestParserMultiFormat.
Existing test suite stays green: 127 -> 131 passing.
* Studio: skip non-scalar args in python_tag JSON form
The JSON sub-path of ``_parse_llama3_python_tag`` was fabricating
``{"value": args}`` when the model emitted a non-dict / non-string
``arguments`` value (e.g. ``42``, ``[1,2,3]``, ``null``, ``true``).
This silently turned a malformed emission into a real tool call,
which the agentic loop would then execute with arguments the model
never intended.
Tightened: skip the call instead of fabricating. The same
behaviour now matches the bare-JSON guard tightened earlier
(strict-guard merge from PR #5620, inherited via merge here).
Added a regression test covering the four non-scalar shapes.
Pass count on this branch: 158 -> 159.
Sites in ``_parse_tool_call_json`` and ``_consume_mistral_call``
keep the existing looser behaviour for now; both are reached
only after explicit ``<tool_call>`` / ``[TOOL_CALLS]`` markers
so the false-positive surface there is much narrower.
* studio: fix safetensors tool-call parser gaps vs llama.cpp (Mistral CALL_ID / THINK, attribute-form signal)
Three GGUF-parity fixes to the safetensors tool-call parser, each matching
llama.cpp's reference behaviour:
- Mistral Small 3.2 emits [TOOL_CALLS]name[CALL_ID]<id>[ARGS]{json}. The
parser stopped after the name on seeing [CALL_ID] (neither [ARGS] nor {),
dropping the call. Skip an optional [CALL_ID]<id> segment in both the
parse and strip paths. llama.cpp parses this (test-chat.cpp:4785).
- Magistral wraps reasoning in [THINK]...[/THINK]. A [TOOL_CALLS] inside the
reasoning was parsed as a real call, producing a phantom call. Strip a
leading [THINK] block before scanning so only the post-reasoning call
counts (test-chat.cpp:2285); a literal [THINK] inside a later argument is
left intact.
- The standalone MiniCPM-5 / MiniMax-M2 <function name="..."> attribute form
parsed correctly but was absent from TOOL_XML_SIGNALS and the markup strip
patterns, so the streaming safety-net parse was gated off (dropping the
call) and markup leaked into displayed text. Add the signal and broaden
the strip regexes.
Adds regression tests for all three.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix GLM and Kimi K2 safetensors tool-call parser gaps vs llama.cpp
Four GGUF-parity fixes for the GLM and Kimi K2 families:
- GLM 4.7 zero-argument inline call <tool_call>name</tool_call> was dropped:
the open-tag lookahead only allowed \n or <arg_key> after the name. Allow
</tool_call> too so a no-arg call parses to empty args (vLLM / SGLang /
llama.cpp all parse it).
- GLM string argument values were stripped, losing significant leading /
trailing whitespace in code / diff arguments. Keep the raw value for the
string fallback and only strip the copy used to probe for a JSON literal,
matching vLLM glm4_moe which never strips string args.
- Kimi K2 calls emitted without the <|tool_calls_section_begin|> wrapper
were dropped. llama.cpp makes the section optional (Kimi can call a tool
straight after reasoning without opening a section); parse a bare
<|tool_call_begin|> when no section is present.
- Kimi K2 malformed / truncated JSON in one call dropped every later call in
the section. Skip the bad call and keep parsing so valid subsequent calls
are recovered (vLLM parity).
Adds regression tests for all four.
* studio: fire safetensors tool calls for the bare-JSON (Llama-3.2) form
The agentic loop's streaming safety-net parse was gated on
has_tool_signal(), which is False for the Llama-3.1 / 3.2 bare-JSON tool
form {"name":..,"parameters":..} (no XML marker). Real tool calls were
therefore dropped: the loop logged "model planned without calling tools",
re-prompted three times, then gave up with zero tool calls, while GGUF's
llama-server parses the same emission natively.
Run parse_tool_calls_from_text() unconditionally in the safety net. The
parser is strict (only fires on a valid tool-call shape) so plain answers
are unaffected. Reproduced on a real unsloth/Llama-3.1-8B-Instruct run:
the model emits {"name":"web_search","parameters":{...}} which now
executes the tool instead of being re-prompted into a no-op.
Adds a loop regression test for the bare-JSON form.
* studio: fire safetensors tool calls for Gemma 4 (native template + stripped parser)
Gemma-4 safetensors fired no tools while its GGUF fired reliably. Three gaps:
- The Studio swaps in the Unsloth "gemma-4" chat template, which does not
render the tools schema (the model's native template does), so the model
never saw the tools. Fall back to the model's native template when the
override template renders identically with and without tools. Same fix
helps any family whose override template drops tools.
- skip_special_tokens strips the <|tool_call> wrapper and <|"|> string
markers, so a streamed Gemma-4 call arrives as a bare call:NAME{k:v, ...}
with unquoted values. Parse that form, keeping commas/braces inside a
code or command value, normalising surrounding quotes, and stripping the
leaked markup from the final answer.
- Without a grammar a small model can loop, repeating one call for the whole
tool budget. Collapse exact-duplicate calls within a turn and force a final
answer after a turn that made no new tool progress (llama-server's lazy
grammar prevents this loop on the GGUF side).
Adds parser tests for the bare/stripped Gemma-4 form.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: complete strict-mode contract and fix parser import paths
Address review findings on the multi-format tool-call parser:
- Honor allow_incomplete=False in the remaining sub-parsers. The Llama-3
<|python_tag|>NAME.call(...) parser, the pre-v11 Mistral [TOOL_CALLS] array
parser, and the Gemma 4 <|tool_call> parser ignored strict mode, so a
truncated call (missing closing paren, ], or <tool_call|>) was still healed
and executed with Auto-Heal disabled. Thread strictness through and reject
the unclosed forms, matching the JSON and function-XML paths.
- Drop the duplicate tool_call_parser import block in llama_cpp.py and the
redundant un-aliased TOOL_XML_SIGNALS; only the _SHARED_TOOL_XML_SIGNALS
alias is used as a value.
- Import _strip_mistral_closed_calls from core.inference.tool_call_parser in
routes/inference.py instead of studio.backend.core... The self-contained
run.py launch mode only puts studio/backend on sys.path, so the absolute
package path raised ModuleNotFoundError on the server-tool strip path.
Add strict-mode regression tests for the truncated Llama-3 dot-call and the
unclosed Mistral array.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden DeepSeek/Kimi tool-call parsing and strip
Address review findings on the DeepSeek and Kimi parsers:
- Honor allow_incomplete=False for DeepSeek. An envelope with no closing
<｜tool▁calls▁end｜> is truncated mid-stream; reject it in strict mode
instead of healing the body out to EOF, matching the strict XML and Mistral
paths.
- Do not skip a following tool call when the current call's end marker is
missing. The DeepSeek V3 and Kimi loops advanced by searching forward for the
next <｜tool▁call▁end｜> / <|tool_call_end|>, which could land on a later
call's end marker and drop the call in between. Advance by the JSON end; the
loop re-locates the next call marker from there.
- Strip truncated DeepSeek and Kimi section blocks in the route-level display
regex. The patterns required the closing marker; add the end-of-text
alternative so a block truncated by EOS does not leak raw markup to the UI.
Add regression tests for the truncated DeepSeek envelope, and for DeepSeek and
Kimi multi-call recovery when the first call's end marker is missing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: preserve XML param indentation and alias Mistral array parameters
Two parser-correctness fixes found by auditing against the model chat templates
and the SGLang / vLLM reference parsers:
- Qwen3.5 XML parameter values lost their leading indentation. The chat template
emits <parameter=k>\nVALUE\n</parameter>, but the parameter-start regex ate the
wrapping newline AND the value's first-line indentation with a trailing \s*,
then str.strip() removed the rest. Narrow the trailing class to horizontal
whitespace only and trim exactly one wrapping newline (via _trim_param_value),
preserving indentation in code/diff arguments. Matches SGLang's qwen3_coder
detector. Applies to both _parse_function_xml (tool_call_parser.py) and the XML
path in tool_healing.py.
- Mistral pre-v11 array objects keyed on parameters dropped their payload.
_consume_mistral_call read only the arguments key; alias parameters the same way
the JSON/XML paths and SGLang's base detector do.
Add regression tests for preserved multi-line indentation and the array
parameters alias.
* Studio: DeepSeek strip sync, Gemma nested args, GLM/Kimi strict mode
Parser-correctness fixes found by auditing DeepSeek/GLM/Kimi against vLLM,
SGLang, and the model chat templates:
- DeepSeek: the short <｜tool▁calls｜> opener (and the space / escaped-underscore
spellings) was parsed but never stripped, so a short-opener envelope leaked raw
markup to the UI. Share one opener alternation between _DEEPSEEK_BEGIN_RE and
the strip patterns (and the route-level display regex) so a signal we parse can
never be left un-stripped.
- Gemma wrapper-less stream: a nested object/array argument (loc:{city:NYC},
labels:[bug,ui]) was kept as a literal string. Parse it recursively when the
bare value is a balanced {} / [], falling back to the raw string for a
truncated value.
- GLM and Kimi ignored allow_incomplete. With Auto-Heal off, a GLM block with no
</tool_call>, a Kimi section with no <|tool_calls_section_end|>, or a Kimi call
with no <|tool_call_end|> are truncated and must be rejected, matching the
strict behavior of the JSON/XML/Mistral/DeepSeek paths and vLLM/SGLang.
Add regression tests for the short-opener strip, the Gemma nested args, and GLM /
Kimi strict-mode rejection.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten tool-call parser comments
Make the comments in the multi-format tool-call parser and its callers succinct:
compress verbose docstrings/blocks to one or two lines, drop ones that restate the
code, and trim the tiny balanced-scanner helpers. Correctness rationale and
upstream provenance (SGLang/llama.cpp parity, the strict-mode / Auto-Heal
contract, whitespace-preservation, and the Unicode / full-width-pipe notes) are
kept in compact form.
Comment-only: no code or behavior change (verified with comment_tools.py check
--strip-docstrings; parser suite green).
* Studio: tighten DeepSeek/GLM/Kimi parser comments
Compress the comments added for the DeepSeek/GLM/Kimi parsers and the Gemma
wrapper-less helpers to one or two lines, keeping the upstream provenance
(llama.cpp 51fa458a92d6), the O(N^2) / strict-mode rationale, and the vLLM parity
notes intact.
Comment-only: no code or behavior change (verified with comment_tools.py check
--strip-docstrings; parser suite green).
* Studio: make DeepSeek R1 / GLM parsing linear and close routes strip gaps
Review follow-up for the DeepSeek/GLM/Kimi parser:
- DeepSeek R1 detection used a greedy ``([^\n]+)\n```json`` regex that backtracks
O(N^2) on a fence-less truncated body; scan with str.find instead (mirrors the
V3 path).
- GLM arg pairs used a lazy-group finditer that rescanned to EOF from each bare
<arg_key> in an unclosed body (O(N^2)); walk pairs with str.find.
- The route display strip (_TOOL_XML_RE) accepted fewer DeepSeek openers than the
parser (missed the space / escaped-underscore spellings) and missed bare
section-less Kimi calls, so a call we parse could leak raw markup to the UI.
Reuse the parser's shared _DEEPSEEK_OPEN_RE_SRC and add a bare-Kimi arm.
Add ReDoS-linearity regressions for the R1 and GLM paths, a positive R1
fenced-json parse test, and routes-strip tests for the space/escaped DeepSeek
openers and the bare Kimi call.
* Studio: fix test_mcp_servers _TOOL_XML_RE reconstruction after _DS_OPEN_SRC reuse
The routes strip fix made _TOOL_XML_RE reference the module-level
_DS_OPEN_SRC variable. test_mcp_servers reconstructs the regex by exec-ing
the extracted compile() source in a namespace that only defined _re, so it
raised NameError. Inject _DS_OPEN_SRC into that namespace, matching the same
fix already applied in test_tool_xml_strip.
* Studio: make Llama-3 .call and Mistral-array healing parsing linear
Two more O(n^2) ReDoS paths in the multi-format parser, both reachable from
the agentic loop on a long truncated body with no length cap:
- _LLAMA3_KV_RE.finditer over a .call(...) body retried at every offset of a
long word run / unterminated quote (40K -> 14s). Replace with a hand-scan
that reuses the same key/number/literal sub-regexes via anchored match and
walks the string body by hand, so an unterminated quote is O(n). Verified
byte-identical to the old regex over 200K fuzzed inputs.
- _parse_mistral_array healing ran _balanced_brace_end from every { in the
body (20K -> 17s). Walk top-level objects, advancing past each balanced
{...}; this also drops the phantom call the old scan emitted from a nested
argument object.
Add adversarial-length linearity regressions plus positive .call kwargs and
unclosed-array recovery coverage.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: strengthen #5624 regression assertions and strip-test harness guards
- test_strip_tool_markup_handles_deepseek_envelope used `A or B` where B was the
preservation property the next line already asserts, masking the real check.
Replace with an explicit assertion that the call name and args are stripped.
- The test_tool_xml_strip source-extraction harness reconstructs _TOOL_XML_RE and
_strip_tool_xml_for_display from routes/inference.py via lazy regexes that could
silently grab a shorter slice. Assert the extracted regex carries the DeepSeek /
bare-Kimi arms and the helper body reached the _TOOL_XML_RE.sub call.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: honor strict mode in safety-net, keep empty Gemma args, strip attribute-form function XML
- safetensors safety-net parser now forwards allow_incomplete=auto_heal_tool_calls,
matching the draining path, so a late incomplete tool call is not healed and
executed when Auto-Heal is off.
- Gemma empty bare value ({k:}) now serialises as "" instead of invalid {"k":},
which previously dropped the whole call.
- Route _TOOL_XML_RE also strips the <function name="..."> attribute form
(MiniCPM-5 / MiniMax-M2) so it no longer leaks to the UI.
* Studio: linearize wrapper-less Gemma nested-arg parsing and correct parser provenance
- _gemma_parse_value/_gemma_parse_mapping/_gemma_parse_array now parse nested
{}/[] in a single forward pass instead of pre-scanning each subtree with a
balanced-brace walk and re-parsing it. Deeply nested wrapper-less Gemma args
were O(n^2); they are now ~linear (and ~40x faster at depth 400).
- Correct the DeepSeek/GLM/Kimi provenance comments: the cited commit
51fa458a92d6 is unrelated, and GLM/Kimi were never standalone
common_chat_parse_* functions (llama.cpp uses common_chat_params_init_glm_4_5
plus a generalized XML parser, PRs #15904 / #16932).
- Add tests: Gemma deep-nesting linearity, nested object/array preservation,
same-turn distinct-call cap, and the native-template tool-render fallback.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: guard Gemma value parser against non-advancement and missing tokenizer
Addresses Gemini review:
- _gemma_parse_value now consumes one character when a stray }/]/, sits where a
value is expected, so _gemma_parse_array can never stall at the same index on
malformed input (a latent infinite loop).
- _render_with_native_template returns None when neither a tokenizer nor a
processor is present instead of raising AttributeError.
- Tests for both.
* Studio: fix attribute-form function-XML literal close tag and zero-arg strict call
Addresses Codex review of the <function name="..."> attribute form in
_parse_function_xml (MiniCPM-5 / MiniMax-M2):
- End the call body at the LAST </function> / </tool_call> within the call's
window, so a literal close tag inside a code/search argument (e.g.
print("</function>")) is preserved instead of truncating the call.
- Accept a closed call with no parameters as a valid zero-argument call in strict
mode (the function close is already required), instead of rejecting it as a
truncated call.
- Tests for both, mirroring the legacy <function=...> coverage.
* Studio: drop scratch review/planning artifacts from the branch
* Studio: fix tool-call parser/loop review findings on the multi-format path
Address the live code-review findings on the safetensors/MLX + GGUF tool path:
- routes: include the attribute form <function name="..."> in the safetensors
capability whitelist so MiniCPM-5 / MiniMax-M2 templates keep the tool pill
(parser already handles the form; the post-filter wrongly suppressed it).
- safetensors loop: build the plan-without-action re-prompt from the active
tools instead of a hardcoded web_search/python string, and gate it on
auto_heal_tool_calls, matching the GGUF loop.
- safetensors loop: hold a leading bare-JSON object ({"name":..,"parameters":..})
during BUFFERING until it closes, then drain it as a tool call instead of
streaming the raw JSON to clients. The DRAINING/STREAMING resolvers still
recover a plain JSON answer, so this can never drop content.
- parser: anchor the Llama-3 <|python_tag|>NAME.call(...) scan to the tag and
chain ; -separated calls, so all semicolon-separated built-ins parse and a
literal <|python_tag|>x.call(...) inside a JSON string argument no longer
fires the wrong tool.
- parser: consume the optional trailing </s> after a named Mistral
[TOOL_CALLS]name{json} call, mirroring the array shape.
- GGUF streaming strip: use the shared parser patterns (which know
[TOOL_CALLS] and <|python_tag|>) so a textual tool call entering DRAINING is
stripped instead of leaking the marker to streaming clients.
- routes: hoist the _strip_mistral_closed_calls import to module level.
Adds regression tests covering each fix; existing parser suite stays green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix DeepSeek/GLM/Gemma tool-call review findings
Address the live code-review findings specific to the DeepSeek / GLM / Kimi
and native-template additions:
- parser: in strict mode (Auto-Heal off) require the per-call
<｜tool▁call｜end｜> terminator for DeepSeek V3 calls instead of executing on
a bare balanced object closed only by the envelope end.
- parser: keep GLM string arguments that begin with a quote verbatim (drop
the leading-quote case from the JSON-decode probe) so a quoted search query
is not decoded down to its inner text.
- parser: reject a GLM call with an unclosed <arg_value> in strict mode, and
under Auto-Heal keep the partial value rather than dropping it to a no-arg
call.
- parser: add a balanced wrapper-less Gemma strip (call:NAME{...}) so a nested
object/array argument is removed whole instead of leaving a trailing brace;
run the balanced Mistral and Gemma strips on the streaming display paths too.
- safetensors loop: buffer a leading wrapper-less Gemma call:NAME{...} so it
drains and executes instead of streaming the raw call text.
- inference: render the native-template fallback on a shallow tokenizer copy
instead of mutating the shared tokenizer outside the generation lock, and
load the native template from base_model for LoRA adapters.
Adds regression tests for each; existing parser suite stays green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden multi-format tool-call detection from review findings
Apply five targeted fixes from the review pass over the multi-format tool
path:
- routes: route display strip delegates to _strip_tool_xml so Mistral
[TOOL_CALLS] blocks with nested JSON are removed from streamed display
text, not just the XML forms.
- tool_call_parser: skip function/parameter starts that fall inside an
already-open parameter block (_inside_open_parameter) so nested example
payloads are not mis-parsed as new calls; extract
strip_llama3_leading_sentinels so the bare-JSON guard is shared.
- safetensors_agentic: probe bare JSON through strip_llama3_leading_sentinels
before the balanced-brace check so a leaked header sentinel does not defeat
the guard.
- tool_healing: allow dotted tool names in the Gemma wrapped start pattern.
- llama_cpp (GGUF): buffer wrapper-less Llama-3.2 {"name":..} calls that carry
no XML signal, drain a complete object silently and hold an incomplete one,
and run the end-of-stream safety net unconditionally so markerless calls are
detected and never leak the raw JSON (including truncated fragments).
Adds regression tests for the GGUF bare-JSON streaming path and the Mistral
display strip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: stop bare-JSON tool calls leaking at EOF, oversized, and into history
The second review pass flagged that the Llama-3.2 bare-JSON tool-call handling
still leaked raw JSON in several spots; ``strip_tool_markup`` only knows
XML/bracket markup, so the bare-JSON form survived it. Fix them symmetrically
across the safetensors and GGUF loops:
- Safetensors stream-end resolver now routes a held bare-JSON fragment to
DRAINING (mirroring GGUF) so a truncated ``{"name":..`` cut off by the end of
the stream is dropped instead of flushed as assistant content. The 7/10
reviewer finding.
- Both loops now drain (suppress) an oversized still-open bare-JSON call once it
passes ``_MAX_BARE_JSON_BUFFER`` instead of streaming the raw prefix, gated on
a ``"name"`` key so a giant plain JSON answer still streams; a complete
oversized call still executes via the safety net.
- Add a shared ``strip_leading_bare_json_call`` helper and apply it to the
content kept for the assistant turn in both loops, so an executed bare-JSON
call is not replayed as visible text or fed back as next-turn history.
Plain JSON answers without a ``"name"`` key are untouched throughout. Adds
regression tests for the EOF, oversized, and next-turn cases on both backends
plus unit tests for the helper.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: bound the Llama-3 python_tag strip on real control sentinels
The route display strip's <|python_tag|> arm ran to the next <| of any kind.
A tool-call argument carrying a literal <|...|> token (for example <|cite|>
inside a string value) truncated the strip early and leaked the call tail into
the visible response. Narrow the stop condition to the genuine Llama control
sentinels (eot_id, eom_id, python_tag, start/end_header_id, begin_of_text,
finetune_right_pad_id) so embedded markup and JSON are consumed while real
header/turn boundaries still bound the strip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden GLM/Gemma parsing, cap GGUF textual calls, share native-template fallback
GLM 4.x parser walked a body pre-bounded by the first </tool_call>, so a string
argument containing a literal </tool_call> (e.g. code that prints it) was
truncated. Walk arg_key/arg_value pairs against the full content instead, since
each <arg_value> is delimited by its own </arg_value> and the call's real close
is the </tool_call> that precedes the next <arg_key>.
Add a truncated wrapper-less Gemma pattern (call:NAME{... with no closing brace)
to the markup strip so a call cut off mid-arguments does not leak raw into the
visible stream. It runs after the closed form, so a complete call keeps trailing
prose.
Cap and dedup tool calls parsed from the GGUF TEXTUAL fallback at
_MAX_TOOL_CALLS_PER_TURN, mirroring the safetensors loop. Structured
delta.tool_calls are grammar-bounded by llama-server, but text parsed straight
from content is not, so one runaway turn could fan out into dozens of
executions.
Extract the native-chat-template fallback into chat_template_helpers
(render_native_template / render_with_native_template_fallback) so the
transformers and MLX text backends share one implementation. The MLX text path
now applies it too, so an Unsloth override template that drops the tools schema
no longer silently stops MLX from advertising tools. The MLX VLM path renders
via the processor for image tokens and is intentionally left on its own render.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: gate markerless bare JSON on enabled tools and close parser/strip asymmetries
The Llama-3.2 custom_tools bare-JSON form has no marker, so any JSON object with a
name key was read as a tool call. An ordinary JSON answer like
{"name":"Alice","parameters":{"age":30}} was misclassified as a call to a
disabled tool and dropped from the visible response. Gate the markerless form on
the enabled tool names (threaded through parse_tool_calls_from_text and
strip_leading_bare_json_call, supplied by both streaming loops): an object whose
name is not an enabled tool is ordinary content. The marker-based forms keep
their name-agnostic behaviour (an explicit signal is a real call attempt), and
unrestricted mode stays ungated.
Also fix two parser/strip asymmetries the parser already tolerated:
- A literal </function> inside a parameter value (print("</function>")) truncated
both the core and route strips at the first close, leaking the tail. Extend the
strip to the call's real close (last </function> before the next opener),
mirroring the parser, without merging separate calls.
- The single-object Mistral [TOOL_CALLS]{...} shape parsed but _strip_mistral_closed_calls
left it, leaking the raw object into display. Strip the balanced object while
keeping trailing prose, matching the array and name shapes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio tools: fix strip/parse symmetry and native-template token for DeepSeek/GLM/Kimi
Pass-3 review follow-ups on the multi-format tool parser:
- Bare Kimi call (<|tool_call_begin|>...<|tool_call_end|> with no section
wrapper) is accepted by the parser, so add it to the closed strip patterns
so the streaming (non-final) display strip removes it instead of leaking the
markup mid-generation.
- Route display strip now also runs the wrapper-less Gemma cleanup, so a
Gemma 4 call:NAME{..} no longer leaks into the visible answer.
- MLX model record carries base_model for a LoRA adapter so the native-template
fallback loads the base repo template rather than the adapter's
(often template-less) tokenizer.
- Native-template reload forwards the load-time HF token so a gated/private
model's repo template can still be fetched (transformers and MLX text paths).
- GGUF end-of-stream bare-call heuristic is gated on the enabled tool names so a
truncated ordinary JSON object ({"name":"Alice","age":) streams as the answer
instead of being dropped as a tool call.
Adds regression tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio tools: gate GGUF bare-JSON suppression on enabled tools and fix python-tag exponent parsing
Pass-4 review follow-ups on the GGUF tool loop and Llama-3 parser:
- The GGUF bare-JSON suppression sites still keyed off a raw "name" substring,
so an ordinary JSON answer whose name is not an enabled tool was dropped when
it was truncated, oversized, or reached the no-tool DRAINING fallback (the
parser, helper, and safetensors paths were already gated). All three sites now
use the shared enabled-name gate, and a held bare-JSON buffer that turns out not
to be an enabled call is shown as the answer instead of dropped at stream end.
- The Llama-3 python-tag numeric kwarg regex matched only the mantissa, so
scientific notation was truncated to its leading digits (1e-3 parsed as 1) and a
tool executed with the wrong value. The regex now accepts exponent and decimal
forms, and the int/float classification keys off the exponent too.
Adds regression tests for the truncated / oversized disabled-name JSON cases (and
a counterpart that a truncated enabled call still does not leak) plus the
scientific-notation kwargs.
* Studio: drop accidentally committed async worker transcripts
Eight generated reviewer / async-worker transcripts were committed under
studio/backend/async_task_outputs/. They are not imported or referenced by any
code and carry only internal task state, so they should never ship in the repo.
Remove them and gitignore the directory so they cannot be re-added.
* Studio tools: gate safetensors bare-JSON drain, fix nested-name gate and function-XML strip
Pass-4 review follow-ups on the shared parser / safetensors loop:
- The safetensors oversized and end-of-stream bare-JSON drain branches keyed off
a raw "name" substring, so a large or truncated ordinary JSON answer whose name
is not an enabled tool was drained instead of streamed. Both now use the shared
enabled-tool-name gate, matching the GGUF path.
- strip_leading_bare_json_call matched the first "name" anywhere, so a plain JSON
answer with a nested name equal to an enabled tool ({"result":{"name":"web_search"}})
was wrongly suppressed. It now extracts the TOP-LEVEL name only, walking past
nested objects/arrays and keeping the text when a top-level value is truncated.
- The function-XML display strip used a regex negative-lookahead that stopped at a
literal <function=...> opener inside a parameter value and then dropped the rest
of the answer to EOF. A scan-based strip mirrors the parser (ignores openers
inside an open <parameter> via _inside_open_parameter) and closes each call at its
real </function>, so trailing assistant text after such a call survives.
Adds regression tests for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep tools prompt when native-template probe raises; make helper tests hermetic
Pass-4 review follow-ups on the native-template fallback:
- render_with_native_template_fallback re-renders the live template with tools=None
to detect whether it dropped the schema. A template that requires tools can raise
on that probe; that must not discard the already-valid tools prompt. The probe is
now wrapped so any error returns the original formatted_prompt (transformers would
otherwise fall back to manual formatting and lose the schema; MLX would let the
exception escape).
- The native-template helper tests imported InferenceBackend just to reach the
thin wrapper, which pulls in unsloth and its optional vllm package metadata. They
now call the dependency-light render_native_template helper directly so they pass
in a backend/test environment without vllm. Adds a probe-raises regression test.
* Tool parsing: 3.9 import safety, disabled-Auto-Heal contract, capability gate
Round-2 review follow-ups on the multi-format tool-call parser:
- tool_call_parser: add `from __future__ import annotations`. The module
is dependency-light by design (external llama-server wrappers import it
standalone) and the package targets python >=3.9, where its PEP 604
`int | None` return annotations would raise TypeError on import.
- safetensors + GGUF drain fallback: gate the leading bare-JSON strip on
auto_heal_tool_calls. With Auto-Heal off, a truncated enabled-name
fragment that did not parse now stays visible, matching the XML strip
in the same branch and the disabled-Auto-Heal contract. With Auto-Heal
on it is still suppressed.
- safetensors capability gate: match the bare-JSON `{"name":` template
marker with a whitespace/escape-tolerant regex so a pretty-printed
`{ "name" :` or JSON-escaped `{\"name\":` template is not mis-classified
as tool-less. The parser already accepts that whitespace via
raw_decode, so the gate must too.
Regression tests added for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* GLM tool-call display strip: treat literal close tag in arg value as data
Round-2 review follow-up on the GLM 4.x tool-call format.
The GLM call shape is <tool_call>NAME<arg_key>k</arg_key><arg_value>v
</arg_value>...</tool_call>. The parser was hardened to walk arg_key /
arg_value pairs so a literal </tool_call> inside an argument value (e.g.
print("</tool_call>")) is treated as data and the call's real close is the
</tool_call> that precedes the next <arg_key>. The display strips still used a
non-greedy <tool_call>.*?</tool_call> regex, which stopped at the literal and
leaked the call's tail into visible content and stale history.
Add _strip_glm_calls, a scan that mirrors the parser's close detection, and run
it before the regex arms in every strip pipeline: the core strip_tool_markup,
the route _strip_tool_xml display/history cleanup, and the safetensors + GGUF
streaming strips. Qwen / Hermes <tool_call>{json} has no NAME token after the
opener, so it is left to the regex arms unchanged.
Regression tests cover the literal-close-tag leak (core + route), normal GLM
calls, back-to-back GLM calls, zero-arg GLM, truncated GLM, and untouched Qwen.
* Tool parsing: symmetric "function" bare-JSON alias and route strip parity
Round-3 review follow-ups, all parser/strip symmetry fixes.
- Bare-JSON "function" alias: the markerless parser accepts a call name via
obj.get("name") or obj.get("function"), but the strip/gates only knew "name",
so a {"function":<enabled tool>} call executed while its raw JSON leaked. Teach
_top_level_bare_json_name the alias (with "name" precedence and the same nested
and truncated-name guards), and widen the guards in strip_leading_bare_json_call,
the safetensors and GGUF _looks_like_enabled_bare_json gates, and the route
capability marker regex.
- Route display/history cleanup: strip a tail-only </param> alias close (the
parser accepts <param name="...">...</param>), and run the parser's guarded
function-XML scan (_inside_open_parameter) before _TOOL_XML_RE so a literal
nested <function=...></function> inside an argument value does not truncate the
strip and leak the tail.
Regression tests added for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio tools: fix DeepSeek strict recovery, Kimi dotted names, Gemma spaced streaming
Round 3 review fixes for the DeepSeek / GLM / Kimi tool-call parsing path.
- DeepSeek R1 and V3/V3.1 strict parsing (Auto-Heal off): when a call is
truncated (missing closing fence or <tool_call_end> terminator), skip it
and keep scanning for later well-formed calls instead of breaking out and
dropping the rest of the envelope. This matches the Kimi strict parser's
recovery behaviour.
- Kimi dotted tool names: keep the full name after stripping only the
functions. prefix and :idx suffix, e.g. functions.mcp.server-list:0 stays
mcp.server-list. The previous split on "." truncated dotted MCP names to
their last segment. This matches current vLLM
(tool_id.split(":")[0].removeprefix("functions.")) and SGLang
(^(?:functions\.)?(?P<name>[\w.\-]+):(?P<index>\d+)$).
- Gemma wrapper-less call streaming: hold the whitespace-tolerant prefix
(call : NAME) in the streaming suppression buffer, matching the parser's
_GEMMA_BARE_TC_RE, so the spaced spelling split across chunks is buffered
instead of leaking as visible text. Applied to both the safetensors and
llama.cpp streaming paths.
- Remove dead _render_with_native_template method and the now-unused copy
import from inference.py; the live path uses render_with_native_template_fallback.
Adds regression tests for DeepSeek R1/V3 strict recovery, Kimi full dotted
name preservation, and the Gemma spaced-call streaming suppression.
* Studio tools: honor tool budget in GGUF loop and guard function-XML streaming strip
Round 4 review fixes. Both are asymmetric-fix bugs where the final/steady path got a
guard the analogous streaming/loop path did not.
- GGUF tool-call budget: the safetensors loop counts real tool-call turns against
max_tool_iterations (re-prompt stalls excepted), but the GGUF loop only bounded the
turn count by the enlarged range (max_tool_iterations + _MAX_REPROMPTS). Since this
PR raised _MAX_REPROMPTS from 1 to 3, a model that keeps making valid tool calls
could run up to three extra tool rounds (with max_tool_iterations=1, four rounds
instead of one). Add a _tool_iters_done counter that increments only when a tool
actually executed in the turn, and stop once the caller's budget is spent so the
post-loop final-answer nudge fires. A duplicate/disabled no-op turn is a correction
turn (like a plan-without-action re-prompt) and does not consume budget, preserving
the existing "already completed" re-prompt behavior.
- Streaming display strip: the final strip runs the guarded _strip_function_xml_calls
scanner (a literal <function=...> inside a parameter value is data, not a nested
call), but the GGUF and safetensors streaming strips still used only the open-ended
regex arms. When a tool-call argument contained literal function markup, the regex
tail ate everything to end-of-text and dropped the real trailing prose after the
call's true </function>. Run the guarded scanner (and the balanced Mistral strip)
before the regex arms in both streaming paths so streaming and final display agree.
Adds regression tests: GGUF valid tool calls respect max_tool_iterations, and the
streaming strip keeps trailing prose after a function-XML call with a literal marker.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio tools: safetensors tool budget counts only executed turns (GGUF parity)
Follow-up to the GGUF budget fix. The safetensors loop charged max_tool_iterations
per non-re-prompt iteration (iteration + 1 - reprompt_count), so a duplicate/disabled
no-op turn spent a budget slot even though no tool ran. With a small cap this dropped
real work: for max_tool_iterations=2, a model that made a valid call, repeated it (an
internal no-op correction turn), then made a distinct valid call executed only the
first -- the third turn was sent with no tools and the distinct call was ignored.
Track whether a turn actually executed a tool (set on record_result) and count only
those turns against the cap, matching the GGUF loop. A duplicate/disabled no-op is a
correction turn -- like a plan-without-action re-prompt -- and no longer consumes
budget, so the model still gets its "already completed" nudge and another tool-enabled
turn. Adds a regression test for the small-cap duplicate-then-distinct-call flow.
* Studio tools: fix stale Kimi dotted-name regression test
test_pr5624_regressions.py still expected functions.my.tool:0 to resolve to the last
segment (tool). The parser now preserves the full dotted name (my.tool) after removing
only the functions. prefix and :idx suffix, matching current vLLM/SGLang so dotted MCP
names like mcp.server-list survive. Update the assertion, name, and module docstring to
the corrected contract (the raw id is still preserved on the call).
* Studio: render the reasoning block for safetensors and MLX like GGUF
enable_thinking chat templates (Qwen3/Qwen3.5/GLM) prefill an unclosed <think>
into the generation prompt, so the model emits only the closing </think> then
the answer. The safetensors/MLX chat stream emitted that as plain content, so
the reasoning showed inline with no collapsible thinking block, while GGUF
(which surfaces reasoning via reasoning_content) rendered one. This brings
safetensors and MLX to parity.
- _ResponsesReasoningExtractor gains a reasoning_prefilled mode that starts
inside the reasoning block and splits on the first </think>; default False
keeps GGUF and every existing caller byte-identical. It suppresses a stray
re-emitted <think> and holds partial markers back across chunk boundaries.
- _sf_reasoning_prefill_mode gates the mode on reasoning being enabled for the
request, an enable_thinking or enable_thinking_effort style, and the template
actually using the standard <think>/</think> markers. Models with a bespoke
reasoning channel (e.g. gemma's <|think|>/<|channel>) are excluded so their
answer is never swallowed; gpt-oss (Harmony) and thinking-off requests are
excluded too.
- sf_tool_stream and stream_chunks (the latter also serves MLX) feed text
through the extractor, emitting reasoning_content then content deltas, with a
per-turn reset in the tool loop and a flush before each tool_start; only the
visible delta reaches the monitor reply. The two non-streaming drains split
reasoning_content the same way.
- Tests: extractor prefilled mode (streaming and edge cases), the gate matrix
including the gemma-style exclusion, and a route-replay of the tool-loop
reasoning stream.
* Studio: render the reasoning block for safetensors and MLX like GGUF
enable_thinking chat templates (Qwen3/Qwen3.5/GLM) prefill an unclosed <think>
into the generation prompt, so the model emits only the closing </think> then
the answer. The safetensors/MLX chat stream emitted that as plain content, so
the reasoning showed inline with no collapsible thinking block, while GGUF
(which surfaces reasoning via reasoning_content) rendered one. This brings
safetensors and MLX to parity.
- _ResponsesReasoningExtractor gains a reasoning_prefilled mode that starts
inside the reasoning block and splits on the first </think>; default False
keeps GGUF and every existing caller byte-identical. It suppresses a stray
re-emitted <think> and holds partial markers back across chunk boundaries.
- _sf_reasoning_prefill_mode gates the mode on reasoning being enabled for the
request, an enable_thinking or enable_thinking_effort style, and the template
actually using the standard <think>/</think> markers. Models with a bespoke
reasoning channel (e.g. gemma's <|think|>/<|channel>) are excluded so their
answer is never swallowed; gpt-oss (Harmony) and thinking-off requests are
excluded too.
- sf_tool_stream and stream_chunks (the latter also serves MLX) feed text
through the extractor, emitting reasoning_content then content deltas, with a
per-turn reset in the tool loop and a flush before each tool_start; only the
visible delta reaches the monitor reply. The two non-streaming drains split
reasoning_content the same way.
- Tests: extractor prefilled mode (streaming and edge cases), the gate matrix
including the gemma-style exclusion, and a route-replay of the tool-loop
reasoning stream.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: don't force a tool re-prompt on a negated intent (safetensors parity)
The safetensors _INTENT_SIGNAL claimed to mirror GGUF but was missing the
negative lookahead, so a refusal like "I will not search the web for that"
matched the "i will" intent and triggered the plan-without-action re-prompt
(STOP... you MUST call a tool), overriding a valid no-tool answer. GGUF already
excludes not/never. Add the same (?!\s+(?:not|never)\b) lookahead so both
backends agree. Extends the intent parity test with negated refusals.
* studio: parse the outer envelope before DeepSeek/Kimi markers embedded in its args
parse_tool_calls_from_text ran the DeepSeek/Kimi marker pre-pass before the shared
<tool_call>/<function=...> parser. When a Qwen/Hermes call's argument contained
literal Kimi/DeepSeek markup (for example a user asking the model to explain that
syntax), the pre-pass matched the embedded marker and returned it, executing the
wrong tool and dropping the real call. Skip the pre-pass when a <tool_call> or
<function=...> envelope opens before the first DeepSeek/Kimi marker, so the shared
parser takes the outer call; a genuine marker-led call (no leading envelope) still
goes through the pre-pass. Tests for the embedded-marker case and the control.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: trim redundant comments (comment-only, AST-verified)
* Studio: trim redundant comments (comment-only, AST-verified)
* Studio: prevent Gemma tool-parser DoS on stray delimiters
_gemma_parse_value returned the input index unchanged when text[i] was a
stray delimiter (,}]), so the list and mapping caller loops that advance
on the returned index spun forever at 100% CPU on malformed input such as
[},]. Advance past the delimiter so parsing always terminates.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: strip Magistral [THINK] reasoning from final display/history
strip_tool_markup removed [TOOL_CALLS] and <function> markup but left a
leading Magistral [THINK]...[/THINK] block intact, so its bracket-form
reasoning (not the <think> the reasoning channel renders) leaked into the
safetensors display and conversation history while GGUF/llama.cpp routes
it natively. Drop the leading reasoning block at end-of-turn (final=True)
via the existing _strip_mistral_reasoning helper; streaming is untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep times in wrapper-less Gemma tool arguments
The wrapper-less Gemma value scanner used _GEMMA_KEY_RE = [\w.\-]+ for keys,
which also matches a digit-leading token, so a comma followed by a time or
ratio inside a value (call:web_search{query:meet at 10:00, 11:00 tomorrow})
was misread as a new 11: key, truncating the query and injecting a bogus
argument. Require keys to start with a letter or underscore, matching the
identifier-start rule the wrapped path already uses (_GEMMA_NEXT_KEY_RE).
Add a regression test.
* Studio: treat markers/close-tags inside tool-call arguments as data
Four parser correctness fixes where a valid argument string was mistaken for
structure:
- DeepSeek: find the envelope-end token outside JSON strings, so a query/code
argument containing the literal token no longer truncates the body and drops
the whole call.
- GLM: locate the real </arg_value> as the one whose next token is <arg_key> /
</tool_call> / end, so a value containing a literal </arg_value> (or
</tool_call>) is kept instead of executing the tool with corrupted arguments.
- Attribute-form <function name="..."> envelopes now count in the embedded-marker
guard, so a DeepSeek/Kimi marker inside a parameter value does not hijack the
outer call and run the wrong tool.
- Wrapper-less Gemma call:NAME{...} is gated on the enabled tool names (parse and
display strip), mirroring the Llama bare-JSON gate, so a disabled/example name in
prose is not stolen as a call and the real answer is preserved.
Add regression tests for each.
* Gate route Gemma wrapperless strip by enabled tools; make Kimi section-end search string-aware
Route-level display stripping now threads the enabled tool-name set into the
Gemma wrapperless-call strip, so prose that mentions a disabled tool
(call:foo{...}) is preserved while active tool calls are still stripped. This
mirrors the parser-level gate already used in tool_call_parser.
The Kimi section-end lookup now searches outside JSON string literals, so a
section-end marker appearing inside an argument string no longer triggers a
false truncation that drops a valid tool call.
* Run DeepSeek/Kimi pre-pass when a closed tool-call example precedes a real block
The marker pre-pass was skipped whenever any <tool_call>/<function> opener
appeared before the first DeepSeek/Kimi marker, even when that opener was a
CLOSED syntax example in prose that ends before the real block. In that case
parse_tool_calls_from_text skipped the DeepSeek/Kimi parsers and the genuine
tool call was dropped while a phantom tool named in the example ran instead.
Only treat a marker as embedded in a leading envelope when removing the closed
outer <tool_call>/<function> envelopes also removes every marker (the marker
actually sat inside one). A marker left standing is a real call, so the pre-pass
runs. The legitimate case of a marker inside a closed outer envelope's arguments
is preserved.
* Honor reasoning_effort none in safetensors prefill; strip Magistral reasoning while streaming
Two safetensors/MLX reasoning fixes surfaced in review:
_sf_reasoning_prefill_mode only checked enable_thinking, so an
enable_thinking_effort (GLM-5.2) request that disables thinking via
reasoning_effort=none (without enable_thinking=False) still began in
prefilled-<think> mode. A plain answer with no </think> was then swallowed
whole into reasoning_content and the visible response came back empty. Thread
reasoning_effort into the predicate and treat none as disabled, mirroring
_request_reasoning_kwargs.
strip_tool_markup_streaming stripped tool markup but not the leading Magistral
[THINK]...[/THINK] bracket block, so the raw chain-of-thought leaked into the
streamed safetensors content instead of the reasoning drawer (GGUF routes it
natively). Apply _strip_mistral_reasoning first, matching the final strip; an
unclosed [THINK] is held from the marker on so nothing flickers.
* Heal truncated outer tool envelopes and keep quoted Gemma args intact
Two follow-ups from review of the marker pre-pass and Gemma parsing:
The leading-envelope guard only removed CLOSED outer <tool_call>/<function>
envelopes before deciding whether a DeepSeek/Kimi marker was embedded, so a
truncated outer call missing its close tag (whose argument embeds a marker) was
treated as a standalone marker and the embedded sample ran instead of the
intended outer call being Auto-Healed. Decide on the last outer opener before the
marker and whether it closed before the marker instead, so a closed syntax
example still runs the pre-pass while a real closed-or-truncated outer call keeps
it.
The wrapper-less Gemma argument scan tracked bracket depth but not quotes, so a
quoted value containing a comma followed by a key-like token (a search query such
as "weather, location: Boston") was split mid-string, truncating the value and
fabricating an extra argument. Track quote state (with escapes) so the top-level
comma boundary is only taken outside quoted spans.
* Span outer envelopes to their real close when locating embedded markers
Locating the DeepSeek/Kimi marker relative to a leading outer envelope used the
FIRST close tag after the opener, so a literal </function> or </tool_call> inside
an argument value (for example python code that contains the text) was mistaken
for the envelope boundary. The marker after it was then treated as a standalone
call and the embedded sample ran instead of the intended outer call.
Match the closed outer envelopes with the shared patterns that already extend to
the real final close (a literal close inside a value is data), and treat a marker
that survives their removal as embedded only when a still-open (truncated) outer
opener precedes it, so Auto-Heal still repairs a truncated outer call. A closed
syntax example before a genuine block still runs the pre-pass.
* Span the tool_call outer envelope to its real close in the marker guard
The leading-envelope check reused the lazy <tool_call>.*?</tool_call> strip
pattern, so a Qwen/Hermes JSON argument containing a literal </tool_call> ended
the span early. A DeepSeek/Kimi sample later in that same string then survived
the closed-envelope removal, and the pre-pass executed the embedded call instead
of the outer <tool_call>. The <function> arm already spanned to its real close;
give <tool_call> the same real-close pattern (with the negative lookahead that
keeps back-to-back calls separate) so a literal close inside a value is data.
* Preserve no-tool Gemma prose and keep later R1 calls when healing a close
Two review follow-ups:
_gemma_strip_gate returned None when no tools were enabled, and None means
strip every markerless call:NAME{...} block, so a no-tool answer that documents
the syntax (or the Anthropic display path, which passes an empty tool list as
None) had that prose deleted. It is a display/history gate, so return the
enabled-name set instead -- an empty set when no tool is enabled, which strips
nothing because every call:NAME{...} is then prose.
The DeepSeek R1 heal path located the close fence with an unbounded forward
search, so when a first call had balanced JSON but omitted its fence the search
landed on a LATER call's terminator and pos advanced past that valid call,
dropping it. Match the close immediately after the JSON (whitespace-skipped) like
the strict path, and advance by just the JSON when it is absent, so a multi-call
turn keeps its later well-formed calls (heal is now a superset of strict).
* Resume wrapper-less Gemma scan past a consumed call's balanced body
The markerless call:NAME{...} scan used finditer, which resumes right after the
opening call: token, so a nested call:OTHER{...} mentioned inside the first
call's own quoted string argument (for example a web_search query that quotes the
Gemma tool syntax) was re-matched and returned as a spurious second tool call,
executing an unintended tool. Walk with a manual cursor that resumes after the
outer call's balanced body (brace matching already skips quoted braces), so a
call's arguments are never rescanned. Genuinely separate back-to-back calls and
disabled/example prose are unaffected.
* Mistral outer call wins over XML literals; align healer signals with its parser
Two follow-ups on the shared-parser ordering after the healing-passthrough
merge:
- A well-formed [TOOL_CALLS] call whose JSON arguments quote tool XML parsed
the literal instead of the outer call (executing the wrong tool). When the
first XML signal sits inside a leading balanced Mistral body it is argument
data, so the Mistral parser now runs first; an XML signal before the trigger
keeps the normal order, so a [TOOL_CALLS] literal inside an XML call's
arguments still stays data.
- passthrough_healing buffered streams on the parser module's broadened signal
list (now including <|python_tag|> and [TOOL_CALLS]) but promotes with
core.tool_healing, which does not parse those forms: a streamed Mistral or
Llama text call was held until finalization and flushed as prose. The healer
keeps its own signal list limited to the formats it can promote, restoring
immediate streaming for the rest.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: Gemma wrapper-less marker literals and quotes, GLM embedded close pair
- The Gemma fallback deferral now keys on an actual wrapped opener
(_GEMMA_TC_RE), not the wrapper literal anywhere in content: a wrapper-less
call whose argument merely mentions <|tool_call> has nothing tool_healing
can parse, and deferring it lost the call entirely (not executed and
stripped from display).
- New _gemma_body_brace_end boundary scanner honors single- and double-quoted
strings like _gemma_parse_stripped_body, shared by parse and strip, so a
quoted brace in a code argument (code:print('}')) no longer truncates the
executed arguments or the strip span.
- _glm_value_close now requires a structural </arg_value> to sit at balanced
quote state: the full pair </arg_value></tool_call> embedded inside a string
literal is data, not an early close. When no candidate balances, the first
token-valid close wins as before.
* Address review: leading envelopes win over rehearsed literals
- New _first_foreign_tool_signal shared by the leading-envelope guards adds
<|python_tag|> to the protected signal set: the spelled-out literal inside a
Mistral call's arguments (a query about Llama built-in tool syntax) executed
the inner literal instead of the outer call.
- New _xml_signal_inside_leading_bare_json guard, sibling of the Mistral one:
a leading bare-JSON call whose string argument quotes tool XML (a code value
citing <function=...>) had the literal promoted by the shared XML pass
before the bare-JSON parser ran.
- Magistral [THINK]...[/THINK] is dropped once at parse entry instead of only
inside the Mistral parser, so a call rehearsed in the think block in a
foreign format can no longer be promoted while the real call after the
block is lost. Parse now agrees with the display strip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: a disabled leading bare-JSON object keeps its literals as data
When the leading bare-JSON object is ordinary content (name not an enabled
tool), the guard proved the first tool signal sits inside it, so falling
through to the XML/python_tag passes promoted quoted string data as a real
call. Drop the object and parse only the tail: a real call after the object
still parses, nothing inside it can be promoted.
* Address review: apostrophes in raw Gemma values, GLM strict key contract, per-model template token
- Quote openers in the wrapper-less Gemma boundary and body scanners now
require value-start context (after : { [ ( , =): an apostrophe inside an
unquoted value (query:what's the weather) opened quote mode, swallowed the
real closing brace, and lost the whole call on common contraction queries.
Quoted values keep hiding delimiters as before.
- A GLM <arg_key> with no <arg_value> tag now rejects the call in strict
mode, matching the unclosed-value contract, instead of executing the tool
with the argument silently dropped; Auto-Heal keeps the lenient skip.
- The native-template fallback reads the hf_token stored on the model record
instead of the instance-wide last-load token, so a later token-less load
cannot break template fetches for a previously loaded gated model (both
the transformers and MLX backends).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: Mistral literals inside leading JSON, whitespace-tolerant wrapped Gemma opener
- The leading bare-JSON guard now treats the [TOOL_CALLS] trigger as a
foreign signal: the Mistral parser runs before the bare-JSON one, so a
literal quoted inside the leading object's strings was promoted over the
outer call (or over ordinary JSON content).
- tool_healing's wrapped Gemma opener tolerates whitespace around call and
the colon: sampling drift emits call: name{ and call : name{, and
rejecting those lost the call entirely because no fallback re-parses the
wrapped form. Strict mode still requires the closing tag.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: DeepSeek/Kimi markers inside leading JSON and Mistral envelopes stay data
The DeepSeek/Kimi pre-pass runs before the outer-call parsers, and
_marker_inside_leading_envelope only protected XML envelopes: a marker
quoted inside a leading bare-JSON or Mistral call's argument strings was
promoted as a separate no-arg call and the real outer call dropped. The
guard now recognizes those two leading envelopes as well; standalone
DeepSeek/Kimi calls keep parsing.
* Address review: accept dotted Gemma argument keys in the key-quoting scanner
The scanner quoted keys of [alnum_-] only, so a dotted key (user.name:...)
was left unquoted, json.loads failed, and the whole wrapped call was lost
(parse empty, strip wipes the markup). Dots now match the parser's own
key/name charset.
* Address review: a real DeepSeek/Kimi call after a disabled leading JSON object still parses
DeepSeek/Kimi markers are foreign signals for the leading bare-JSON guard
too: a marker literal inside a disabled leading object made the envelope
guard skip the pre-pass for the whole message, so a real DeepSeek/Kimi call
after the object was dropped. Routing the case through the guard's
drop-and-parse-the-tail recursion reaches the real call while the literal
inside the object stays data.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: leading Mistral call owns the turn, dotted keys after bare values
- A LEADING parseable [TOOL_CALLS] call now runs the Mistral parser first
unconditionally: literal XML in trailing prose after the call was promoted
by the earlier shared XML pass, executing the quoted example instead of
the real leading call. XML leading keeps the normal order.
- _GEMMA_NEXT_KEY_RE accepts dots so a dotted key after a bare value
(query:foo,user.name:bob) ends the value at the comma instead of being
swallowed into it, matching the round-earlier key-quoting charset.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: a leading wrapper-less Gemma call owns the turn
A quoted foreign literal inside a leading wrapper-less Gemma call's
argument (a query citing another tool syntax) was promoted by tool_healing
before the Gemma fallback ran, executing the quoted example and dropping
the outer call. New leading guard, sibling of the Mistral and bare-JSON
ones, gated on an enabled name since the form is markerless. Foreign markup
leading keeps the normal order.
* Fix merge resolution: restore both leading-guard test classes intact
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: markup quoted inside a nameless leading JSON answer stays data
The leading bare-JSON guard required a top-level name, so a structured JSON
answer quoting tool markup in its strings (a response_format turn
documenting a tool's syntax) had the literal promoted by the later passes.
A nameless leading object that parses as real JSON now routes through the
same decline-then-parse-the-tail path; non-JSON braced prose keeps the old
behaviour, and a real call after the answer still parses.
* Address review: JSON answers stay data, nested Gemma quotes, earliest envelope, no failure caching
- A whole-content JSON value is a structured answer: the markerless Gemma
scan and its strip no longer promote or strip a quoted example of an
enabled tool's syntax inside it.
- Nested stripped-stream Gemma values now unquote quoted string leaves
recursively, so {loc:{city:"New York"}} hands the tool New York, matching
the top-level coercion.
- The DeepSeek/Kimi pre-pass dispatches by earliest envelope opener, so a
leading real call wins over a trailing example of the sibling format in
either direction.
- A failed native-template fetch is no longer cached as no-template: the
next call retries after the model record's token is fixed or a transient
Hub error clears; only definitive loads are cached.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: closed calls precede the marker pre-pass, truncated Gemma scan stops, quoted nested delimiters
- A closed non-DeepSeek/Kimi call preceding the first DS/Kimi marker owns
the turn: a trailing syntax example, or one quoted inside a wrapped Gemma
argument, was promoted by the pre-pass and dropped the real leading call.
Wrapped Gemma joins the outer-envelope pattern sets.
- An unbalanced wrapper-less Gemma call now stops the scan (mirroring the
strip contract) instead of resuming inside its own argument text, where a
quoted enabled call would be promoted.
- Raw-quoted strings in nested stripped-stream Gemma values hide delimiters,
so {city:"New, York"} is one value instead of a split pair, returned
unquoted like the top-level coercion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: string-marker literals in wrapper-less args, mid-value quoted phrases
- The wrapper-less deferral guard no longer keys on the <|"|> literal: a
real call whose argument merely mentions the string marker was deferred to
tool_healing, which has no wrapped opener to parse, losing the call. The
wrapped-opener check alone owns the deferral.
- Double quotes now also open at the start of a word, so a quoted phrase
mid-value (query:find "weather, location: Boston", limit:3) hides its
delimiters instead of splitting the value into garbage keys; apostrophes
keep the value-start-only rule so contractions stay prose.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: strict GLM refuses in-quote close fallback, Gemma guard covers preambles
- _glm_value_close gains a strict flag: a truncated value whose only close
candidates sit inside a string literal rejects the call in strict mode
(Auto-Heal keeps the lenient partial), restoring the strict contract the
quote-aware fallback had weakened.
- The leading wrapper-less Gemma guard no longer requires the call to open
the response: a visible preamble before call:NAME{...} is the normal
shape, and the quoted foreign literal inside the argument was promoted
again in that shape. An enabled balanced call beginning before the first
foreign signal owns it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: contextual GLM quote openers, disabled Gemma examples stay prose, JSON array answers
- The GLM value-close quote tracker uses the same contextual openers as the
Gemma scanners (single quote after punctuation context, double quote also
at word start), so strict mode accepts a normal apostrophe value again
while still rejecting a truncated value whose only close candidates sit
inside a string literal.
- A disabled wrapper-less Gemma call is prose by design, so a tool literal
quoted inside it no longer promotes: the span is dropped for parsing and
the tail parsed, mirroring the nameless-JSON guard.
- Leading JSON ARRAY answers join the leading-JSON envelope guard, so a
marker quoted inside a structured array response stays data.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align closed-envelope regression test with the document-order contract
The test asserted the pre-round-13 behavior (trailing DeepSeek/Kimi block
wins over a leading closed envelope) while the shipped rule is document
order: the leading closed call owns the turn. Rename the test and assert
the leading call so the suite matches the contract exercised by
test_leading_xml_call_wins_over_trailing_kimi_example.
* Parse a leading Llama-3.2 bare-JSON call before the markerless Gemma scan
The bare-JSON form only ever matches a leading call object, and document
order says that call owns the turn. Running the Gemma wrapper-less scan
first let an enabled call:NAME{...} snippet quoted inside the leading
call's string arguments steal the turn when the JSON was not the whole
content (trailing prose or a second ;-separated call), executing the
quoted tool instead of the real one. Reordering cannot take a leading
Gemma call's turn since that content never starts with an object brace.
* Leading-call ownership: Mistral trigger in Gemma guards, closed bare JSON before markers, depth-aware nested Gemma values
Three parser gaps against the document-order contract:
The wrapperless Gemma leading guards did not count [TOOL_CALLS] as a
foreign signal, so a leading Gemma call quoting a Mistral snippet in its
argument lost the turn to the quoted literal. Both the enabled-call and
disabled-example guards now include the trigger, matching the bare-JSON
guard's local inclusion.
_marker_inside_leading_envelope required the DeepSeek/Kimi marker to sit
inside the first closed bare-JSON or Mistral call. A marker after that
closed call (a trailing example or data in a later ;-chained call's
strings) now also defers to the leading call, the same inside-or-after
rule the closed XML envelope patterns already applied.
The nested Gemma primitive value scan split on every comma, corrupting
arguments like opts:{code:print(1,2),lang:py}. It now applies the same
paren/brace depth, contextual quote openers, and comma-only-before-a-key
mapping rule as the top-level scan.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gemma leading guard: a closed enabled call preceding the signal owns the turn
The wrapperless Gemma guard only claimed the turn when the first foreign
signal sat inside the first enabled balanced call. When that call closed
before the signal (a second call quoting a Mistral or Kimi literal, or a
trailing prose example), the guard forfeited the turn and the foreign
parser promoted the quoted literal, dropping the real Gemma calls. Apply
the same inside-or-after ownership rule as the closed bare-JSON and
Mistral envelopes, gated on an enabled name so the name-agnostic legacy
path is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Marker guard: only an executable leading bare-JSON call owns the turn
The bare-JSON branch of the leading-envelope marker guard claimed the
turn for any NAMED leading object. A disabled-name object is prose by
design (the bare-JSON parser will not execute it), so deferring the
DeepSeek/Kimi pre-pass to it lost the real later call entirely. Gate the
ownership claim on the enabled set (or the name-agnostic None path). A
marker inside the disabled object's own strings stays data, matching the
tail-exclusion contract; a marker after it now falls through so the
pre-pass parses the real call. The Mistral branch stays ungated since
[TOOL_CALLS] parsing is never name-gated.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gemma scan skips leading JSON answers; GLM heal bounds values at structural tags
Two fixes to the document-order data contracts:
The markerless Gemma scan only exempted whole-content JSON, so a leading
JSON answer followed by prose had an enabled call:NAME{...} snippet
inside its strings promoted to a real executed call and stripped from
the displayed answer. Both the parse and strip scans now start after a
balanced json-valid leading value span, keeping parse and strip
mirrored. Real calls after the answer still parse; mid-prose JSON gets
no exemption.
The GLM heal fallback for a missing closing arg_value tag took the
entire remainder as the value, executing markup-contaminated arguments
like city="NYC</tool_call>" and swallowing trailing prose. The healed
value now stops at the next arg_key or tool_call close and the pair walk
resumes there. EOF-truncated values keep the partial heal, strict mode
still rejects, and closed values holding a literal close tag in quotes
are untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Compress docstrings in the multi-format tool parser to their contract essence
* Condense parser guard comments and test narration to contract essentials
* verify_import_hoist: exempt __future__ imports and same-diff relocations
Two false positives fired on this PR's refactor. A from __future__ import
is a compiler directive whose name never appears as a runtime load, so
HOISTED-IMPORT-UNUSED can never see it used, yet the file requires it for
PEP 604 annotations on Python 3.9. TARGET-CHANGED flagged the deliberate
move of the strip-pattern constants into core.inference.tool_call_parser
as a silent re-point even though the old module-level target was removed
and the new one added in the same diff. Both get narrow exemptions; a
re-point to a pre-existing target is still caught, and the self-test
negative controls all pass unchanged.
* Leading bare-JSON calls own the turn; function calls end at the first balanced close
The XML-signal guard for a leading bare-JSON call required the signal
strictly inside the object, so a trailing XML example stole the turn
from the leading call; it now applies the same inside-or-after rule as
the Mistral guard. Function-XML calls also ended at the LAST close tag,
which let prose after a closed call that mentions a literal close tag
get swallowed into the final parameter value; calls now end at the
first close tag that is not inside an open parameter, and the strip
mirrors the same rule so parse and strip agree.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Attribute-form calls end at the first balanced close; bare-JSON strip requires the call shape
The attribute form parser still kept the last close tag in the call
window, folding prose after a closed call into the final parameter
value. It now takes the first close not inside an open parameter, the
same rule the equals form and the strip already use.
The leading bare-JSON strip deleted any closed object whose top-level
name matched an enabled tool, including plain JSON answers the parser
correctly rejects as non-calls. The strip (and the drain gate that
delegates to it) now requires the parser's exact call shape, so answers
like {"name":"web_search","result":...} stream and display intact.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* False-alarm markers keep the answer; the bare-JSON strip consumes the whole chain
The trailing strip arms dropped everything from a bare marker to EOF,
so a normal answer that mentions [TOOL_CALLS] or another marker
literally was truncated (or fully swallowed when it started with the
literal) after the no-call drain fallback. Those arms now require a
call-shaped lookahead or marker-at-EOF before dropping; truncated real
calls still strip.
Chained bare-JSON turns executed both calls but stripped only the first
object, so the second call's raw JSON replayed into the next assistant
history message alongside the structured tool_calls. The strip now
consumes the entire chained run of call-shaped enabled objects while
non-call answers, disabled names, and trailing prose stay intact.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* DeepSeek and Kimi trailing strip arms require a call-shaped lookahead
Same false-alarm rule as the bare-word markers: a prose answer that
mentions a DeepSeek or Kimi marker literally keeps its tail, while
truncated real envelopes and bare end-of-text fragments still drop.
* Attribute-form containment, parameter-close-decides rule, preamble-tolerant Mistral guard, strict strip shape
Four document-order and containment fixes. A leading attribute-form
call now parses before the shared XML pass, so markup quoted in its
parameter stays data. The open-parameter scan lets the parameter's own
close tag decide, so any number of literal function closes inside one
value stay data, restoring the pre-close-scan behavior for multi-close
arguments. The leading-Mistral guard tolerates a visible preamble, with
the leading-bare-JSON guard running first so a trigger quoted inside a
leading JSON object stays data. The bare-JSON strip requires the
parser's top-level name in every mode, so nested-name JSON answers
survive name-agnostic stripping.
* Keep buffering long wrapper-less Gemma tool names instead of leaking the prefix
The streaming buffer stopped holding a call:NAME prefix at a fixed
32-char cap, so a Gemma wrapper-less call to a tool whose name exceeds
that (OpenAI allows 64 chars, MCP names run longer) streamed its raw
call:longname text as visible content before the end-of-turn parser
executed it. Hold the variable-length prefix while it still matches the
call: shape, bounded like the bare-JSON path and self-terminating into
prose, draining once the opening brace arrives.
* Keep prose that only mentions DeepSeek/Kimi markers in the route display strip
The route-level _TOOL_XML_RE DeepSeek/Kimi arms consumed from an opener up to
the end of text whenever the marker appeared, so an answer that merely refers
to a marker (for example "See <|tool_call_begin|> in the docs") had the rest
of the reply truncated. The parser-level _TOOL_ALL_PATS already gates these
arms with a call-shaped lookahead. Mirror it here so a marker is only stripped
when a real call follows it or it is a bare fragment at end of text.
* Tighten tool-calling parser and backend comments
* Pass trust_remote_code when reloading native tokenizers
The native-template fallback re-fetches a model's native chat template from
its repo when an Unsloth override template drops the tools schema. The
secondary AutoTokenizer.from_pretrained threaded hf_token but not
trust_remote_code, so for a model loaded with trust_remote_code=True whose
tokenizer repo carries custom code the reload raised, was swallowed, and the
request silently kept the tool-dropping prompt for a model that supports tools.
Store the loaded trust_remote_code on each backend's per-model info dict and
source it in render_native_template, so the reload re-uses exactly the consent
granted at load. For a LoRA adapter the reload targets the base model, whose
remote code was gated and loaded under the same stored flag, so re-passing it
executes no unconsented code. Falsy stored flag preserves the prior behaviour.
Adds a regression test that fails without the flag (custom-code reload raises,
returns None) and passes with it (tools-advertising native prompt returned).
* Treat <|python_tag|> as an outer marker envelope
A Llama-3 <|python_tag|> tool call (built-in NAME.call(...) or custom
{json} form) whose argument quotes a complete DeepSeek/Kimi example was
hijacked by the DeepSeek/Kimi marker pre-pass: the embedded example (for
example delete_all) executed instead of the real outer call. python_tag
is Llama-3's tool-call envelope, so a marker quoted inside its arguments
is data, the same as for <tool_call>, <function=...>, bare JSON, Mistral
and wrapper-less Gemma, which the guard already covers.
Add <|python_tag|> to _OUTER_ENVELOPE_OPEN_RE with a call-shaped
lookahead (mirroring the _TOOL_ALL_PATS python_tag arm) so the marker
pre-pass is suppressed when a python_tag call opens before the first
marker, while a bare prose <|python_tag|> mention is left untouched.
* Tighten tool-call parser comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: Daniel Han <info@unsloth.ai>
Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com>
* studio: tool calling for Llama-3, Mistral, Gemma 4 on safetensors + MLX (#5615)
Adds tool calling for Llama-3, Mistral (pre-v11 + v11+ + [ARGS]), and Gemma 4 to the safetensors / transformers and MLX backends. Parser patched against llama.cpp / vLLM / SGLang per-family parsers and normalises to OpenAI shape. 96 targeted unit tests + cross-OS staging CI (ubuntu / macos-14 / windows) green on the multi-format probe.
* studio: tool-call healing parity between safetensors / MLX and GGUF
After the multi-format parser landed in #5615, the safetensors / MLX
agentic loop and the GGUF loop still differed on healing behaviour.
This commit closes the gaps in both directions so the two backends
react the same way to identical model output.
Changes:
1. core/inference/llama_cpp.py -- the GGUF BUFFERING state machine
now wakes on every emission marker the shared parser knows. Was
("<tool_call>", "<function="); is now the five-tuple imported
from core.inference.tool_call_parser (Qwen / Qwen3.5 / Llama-3
<|python_tag|> / Mistral [TOOL_CALLS] / Gemma 4 <|tool_call>).
Stream cleanup is delegated to the same shared strip_tool_markup
so leaked markup from any family is removed from assistant
content.
2. core/inference/llama_cpp.py -- per-tool canonical heal key. When
a tool arguments field is a bare string and JSON parsing fails,
the GGUF path now heals to {"code": raw_args} for python,
{"command": raw_args} for terminal, and {"query": raw_args} for
everything else. Was hard-coded to {"query": raw_args}, which
silently routed every python / terminal emission through
web_search. Mirrors safetensors_agentic._CANONICAL_HEAL_ARG.
3. core/inference/safetensors_agentic.py -- re-prompt on plan-
without-action. When the model emits a short forward-looking
intent ("I'll search for that", "Let me check", "First, I
will...") and no tool call, the loop nudges the model to act
instead of silently returning a plan-only answer. Up to
_MAX_REPROMPTS=3 (matches GGUF). The intent regex, character
cap, and instruction text are byte-identical to the GGUF path.
The buffer-end fall-through is unified so a buffered intent
emission that never exits the BUFFERING state still triggers
the re-prompt.
4. core/inference/safetensors_agentic.py -- extra iteration slots
for re-prompts. The loop now budgets max_tool_iterations +
_MAX_REPROMPTS + 1 total iterations and tracks the tool-call
count separately, so a stalling model can be nudged 3x without
eating the caller's tool-call budget. Mirrors the _extra slot
reservation in the GGUF path.
Tests (14 new safetensors-side units; 5 GGUF parity pins):
TestLoopRePrompt -- intent-trigger, plain-answer,
no-tools, cap-at-three, budget
preserved, buffer-end intent.
TestLoopCanonicalHealKey -- python / terminal / unknown.
TestGGUFSafetensorsHealingParity -- shared markers used, shared
strip used, canonical heal keys
identical, intent regex matches
same phrases, _MAX_REPROMPTS
equal on both backends.
All 110 targeted tests pass locally; the broader tool / inference /
model-config / sandbox / anthropic / mlx suites stay green.
Why this matters
Without this parity, Llama-3.2 / Mistral / Gemma 4 emissions on Mac
(MLX) and Linux-safetensors stop the agentic loop as soon as the
model says "Let me...", because the GGUF re-prompt logic never
existed on these backends. The two-marker GGUF BUFFERING tuple also
let non-Qwen tool emissions stream out as plain prose when
llama-server's structured channel did not pick them up. Both paths
now drain the same way, heal the same way, and re-prompt the same
way -- so a tool call that works on GGUF works identically on
safetensors / MLX.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix tool-call parser bugs from gemini review on #5620
Three high-priority gemini findings on the tool-call parsing additions:
1. unicode_escape on UTF-8 bytes corrupts non-ASCII literals
(e.g. ✨ becomes â\x9c¨). Replace with json.loads on a quoted
string -- preserves emoji / CJK / RTL while still handling
\n \t \uXXXX escapes.
2. Llama-3 sentinel stripping is order-dependent. A leading
`<|eot_id|><|begin_of_text|>` left `<|begin_of_text|>` behind
because the loop had already passed that sentinel. Loop until
no sentinel matches at the start.
3. Mistral v11+ `[TOOL_CALLS] name { json }` regex uses non-greedy
`\{.*?\}` which truncates at the first `}` of a nested JSON
argument, leaking the tail (e.g. `}}`) into user-visible
streamed text. Same problem for the v0.3 array pattern with
nested brackets. Strip those with balanced brace/bracket
scanning via a new `_strip_mistral_closed_calls` helper called
from `strip_tool_markup`.
Also fix the inference routes' parallel `_TOOL_XML_RE`:
- Same nested-JSON truncation in the Mistral patterns; route the
strip through the parser's balanced-scan helper via a thin
`_strip_tool_xml` wrapper that all existing callers now use.
- Llama-3 `<|python_tag|>[^\n<]*` stopped at any `<`, leaking the
tail of any tool call whose argument contained a literal `<`
(queries, code snippets). Relax to `[^\n]*` which keeps the
strip confined to the actual end-of-line.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/routes: make python_tag strip multi-line aware
Earlier revisions of _TOOL_XML_RE in studio.backend.routes.inference
oscillated between two bug shapes:
5615 r"<\|python_tag\|>[^\n<]*" -- stopped at any literal "<"
so code='if x < 10: pass'
leaked '< 10: pass)' to the
user.
5620.1 r"<\|python_tag\|>[^\n]*" -- single-line only; the second
line of
python.call(code="a\nb")
leaked.
The full parser (_parse_llama3_python_tag) already handles both via
balanced-brace scanning, so the parsing path was fine; the LEAK was
in the streaming strip path that runs on every cumulative emission
while content is still arriving.
Switch to r"<\|python_tag\|>(?:[^<]|<(?!\|))*" so the strip consumes:
* any character that is not a "<" (newlines, JSON, code, ...),
* a "<" only when it is NOT followed by "|" (i.e. NOT a Llama-3
sentinel start like <|eot_id|>, <|eom_id|>, <|begin_of_text|>).
This means:
* code='if x < 10' stays inside the strip (5615 fix preserved),
* multi-line code stays inside the strip (5620 round 2),
* the strip terminates at the next Llama-3 sentinel so trailing
assistant content survives.
Tests: TestRoutesPythonTagStrip (8 cases)
pytest test_safetensors_tool_loop.py test_safetensors_capability_advertise.py
-> 118 passed in 1.81s (was 110).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten verbose comments in tool-call parser sections
Comments were narrating what the code already says. Cut historical
"earlier revisions used X, then Y" narratives down to one-line WHY
notes where the footgun still matters (canonical heal-key parity,
balanced-brace vs non-greedy regex, ``(?:[^<]|<(?!\|))*`` over
``[^\n<]*``/``[^\n]*``). Drop section-header banners.
No behaviour change. Re-ran:
pytest studio/backend/tests/test_safetensors_tool_loop.py \
studio/backend/tests/test_safetensors_capability_advertise.py -q
-> 118 passed.
Regression replay (parser + _coerce_arguments on the 5 #5615 inputs)
-> 21/21.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: parser robustness fixes for PR #5620
Three surgical extensions to the multi-format tool-call parser, each
covering a real fine-tune / template emission shape that the current
parser silently drops. No path narrows; all changes widen what is
accepted.
1. `_parse_tool_call_json` now accepts both `arguments` and
`parameters` keys. A Hermes / Qwen `<tool_call>{json}</tool_call>`
wrapper around a Llama-3.2 fine-tune that emits the `parameters`
key was extracting the tool name and silently discarding the
args, producing a working-shaped call with an empty payload. The
bare-JSON and python_tag paths already accepted both keys; this
path now matches them.
2. `_TC_FUNC_START_RE`, `_TC_PARAM_START_RE`, and `_TC_PARAM_CLOSE_RE`
now also match the attribute form
`<function name="..."><param name="...">v</param></function>` used
by MiniCPM-5 and MiniMax-M2. Names land in either capture group,
and `</param>` is accepted as a short close.
3. `_parse_llama3_bare_json` sentinel-strip now consumes the role
label inserted between `<|start_header_id|>` and
`<|end_header_id|>` by Meta's official Llama-3.x chat template.
Without this, every assistant turn re-fed through the template
prefix `<|start_header_id|>assistant<|end_header_id|>\n\n{json}`
parsed to zero calls, so any history-with-tool-call round-trip
in production silently dropped.
Tests in `studio/backend/tests/test_safetensors_tool_loop.py`:
* `TestParserRobustness::test_tool_call_json_accepts_parameters_key`
* `TestParserRobustness::test_function_xml_attribute_form`
* `TestParserRobustness::test_function_xml_attribute_form_multi_param`
* `TestParserRobustness::test_function_xml_legacy_equals_form_still_works`
(regression guard for the existing `<function=name>` syntax)
* `TestParserRobustness::test_llama3_chat_template_round_trip`
* `TestParserRobustness::test_llama3_round_trip_all_roles`
* `TestParserRobustness::test_llama3_round_trip_with_eot_prefix`
`pytest studio/backend/tests/test_safetensors_tool_loop.py
studio/backend/tests/test_safetensors_capability_advertise.py -q`
goes from 118 to 125 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: terminate function-XML body at </function>, not just </tool_call>
`_parse_function_xml` was looking for `</tool_call>` (the Hermes
wrapper) as the body terminator. When a model emits a standalone
`<function=NAME><parameter=K>v</parameter></function>` followed by
explanatory prose (which models routinely do), no `</tool_call>` is
present, so the body extended to end-of-string and the trailing
prose leaked into the LAST parameter value.
Pre-existing on main (the legacy `<function=NAME>` form had this
bug too). Same affects PR #5620's new attribute-form
`<function name="NAME"><param name="K">v</param></function>`
emission used by MiniCPM-5 / MiniMax-M2.
Fix: `_TC_END_TAG_RE` now matches either `</tool_call>` OR
`</function>`. The existing `_TC_FUNC_CLOSE_RE` / `_TC_PARAM_CLOSE_RE`
strips are unchanged. Multi-call inputs still bound each function
at the next `<function=` start, so no over-eager consumption.
New tests:
* `test_function_xml_followed_by_prose` (legacy form + prose)
* `test_function_attribute_xml_followed_by_prose` (attribute form + prose)
Existing `test_code_with_embedded_xml` still passes (a parameter
value containing literal `<a></a>` is preserved because the
embedded close tag is `</a>`, not `</function>`).
`pytest studio/backend/tests/test_safetensors_tool_loop.py
studio/backend/tests/test_safetensors_capability_advertise.py -q`
goes from 125 to 127 passed.
* Studio: tighten Llama-3.2 bare-JSON guard
A fuzz pass on PR #5811 turned up that ``_parse_llama3_bare_json``
accepted ``parameters`` as a string, contradicting the docstring's
"parameters or arguments is a dict" guard. Prose JSON like
``{"name":"foo","parameters":"a sentence"}`` would wrongly fire the
parser, which the agentic loop would then heal into a real
``foo(query="a sentence")`` call.
Same code lives on this branch, so the same fix applies here.
Tightened guard:
- ``parameters`` must be a dict (Llama-3 spec).
- ``arguments`` may be a dict, or a JSON-encoded string that
decodes to a dict (OpenAI shape, e.g.
``"arguments":"{\"q\":\"x\"}"``). Plain non-JSON strings or
JSON-strings of lists / scalars / null no longer pass.
Mirrors the fix landed in PR #5811 commit 615b8608. Adds the same
4 regression tests under TestParserMultiFormat.
Existing test suite stays green: 127 -> 131 passing.
* studio: fix safetensors tool-call parser gaps vs llama.cpp (Mistral CALL_ID / THINK, attribute-form signal)
Three GGUF-parity fixes to the safetensors tool-call parser, each matching
llama.cpp's reference behaviour:
- Mistral Small 3.2 emits [TOOL_CALLS]name[CALL_ID]<id>[ARGS]{json}. The
parser stopped after the name on seeing [CALL_ID] (neither [ARGS] nor {),
dropping the call. Skip an optional [CALL_ID]<id> segment in both the
parse and strip paths. llama.cpp parses this (test-chat.cpp:4785).
- Magistral wraps reasoning in [THINK]...[/THINK]. A [TOOL_CALLS] inside the
reasoning was parsed as a real call, producing a phantom call. Strip a
leading [THINK] block before scanning so only the post-reasoning call
counts (test-chat.cpp:2285); a literal [THINK] inside a later argument is
left intact.
- The standalone MiniCPM-5 / MiniMax-M2 <function name="..."> attribute form
parsed correctly but was absent from TOOL_XML_SIGNALS and the markup strip
patterns, so the streaming safety-net parse was gated off (dropping the
call) and markup leaked into displayed text. Add the signal and broaden
the strip regexes.
Adds regression tests for all three.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fire safetensors tool calls for the bare-JSON (Llama-3.2) form
The agentic loop's streaming safety-net parse was gated on
has_tool_signal(), which is False for the Llama-3.1 / 3.2 bare-JSON tool
form {"name":..,"parameters":..} (no XML marker). Real tool calls were
therefore dropped: the loop logged "model planned without calling tools",
re-prompted three times, then gave up with zero tool calls, while GGUF's
llama-server parses the same emission natively.
Run parse_tool_calls_from_text() unconditionally in the safety net. The
parser is strict (only fires on a valid tool-call shape) so plain answers
are unaffected. Reproduced on a real unsloth/Llama-3.1-8B-Instruct run:
the model emits {"name":"web_search","parameters":{...}} which now
executes the tool instead of being re-prompted into a no-op.
Adds a loop regression test for the bare-JSON form.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: complete strict-mode contract and fix parser import paths
Address review findings on the multi-format tool-call parser:
- Honor allow_incomplete=False in the remaining sub-parsers. The Llama-3
<|python_tag|>NAME.call(...) parser, the pre-v11 Mistral [TOOL_CALLS] array
parser, and the Gemma 4 <|tool_call> parser ignored strict mode, so a
truncated call (missing closing paren, ], or <tool_call|>) was still healed
and executed with Auto-Heal disabled. Thread strictness through and reject
the unclosed forms, matching the JSON and function-XML paths.
- Drop the duplicate tool_call_parser import block in llama_cpp.py and the
redundant un-aliased TOOL_XML_SIGNALS; only the _SHARED_TOOL_XML_SIGNALS
alias is used as a value.
- Import _strip_mistral_closed_calls from core.inference.tool_call_parser in
routes/inference.py instead of studio.backend.core... The self-contained
run.py launch mode only puts studio/backend on sys.path, so the absolute
package path raised ModuleNotFoundError on the server-tool strip path.
Add strict-mode regression tests for the truncated Llama-3 dot-call and the
unclosed Mistral array.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: preserve XML param indentation and alias Mistral array parameters
Two parser-correctness fixes found by auditing against the model chat templates
and the SGLang / vLLM reference parsers:
- Qwen3.5 XML parameter values lost their leading indentation. The chat template
emits <parameter=k>\nVALUE\n</parameter>, but the parameter-start regex ate the
wrapping newline AND the value's first-line indentation with a trailing \s*,
then str.strip() removed the rest. Narrow the trailing class to horizontal
whitespace only and trim exactly one wrapping newline (via _trim_param_value),
preserving indentation in code/diff arguments. Matches SGLang's qwen3_coder
detector. Applies to both _parse_function_xml (tool_call_parser.py) and the XML
path in tool_healing.py.
- Mistral pre-v11 array objects keyed on parameters dropped their payload.
_consume_mistral_call read only the arguments key; alias parameters the same way
the JSON/XML paths and SGLang's base detector do.
Add regression tests for preserved multi-line indentation and the array
parameters alias.
* Studio: tighten tool-call parser comments
Make the comments in the multi-format tool-call parser and its callers succinct:
compress verbose docstrings/blocks to one or two lines, drop ones that restate the
code, and trim the tiny balanced-scanner helpers. Correctness rationale and
upstream provenance (SGLang/llama.cpp parity, the strict-mode / Auto-Heal
contract, whitespace-preservation, and the Unicode / full-width-pipe notes) are
kept in compact form.
Comment-only: no code or behavior change (verified with comment_tools.py check
--strip-docstrings; parser suite green).
* Studio: make Llama-3 .call and Mistral-array healing parsing linear
Two more O(n^2) ReDoS paths in the multi-format parser, both reachable from
the agentic loop on a long truncated body with no length cap:
- _LLAMA3_KV_RE.finditer over a .call(...) body retried at every offset of a
long word run / unterminated quote (40K -> 14s). Replace with a hand-scan
that reuses the same key/number/literal sub-regexes via anchored match and
walks the string body by hand, so an unterminated quote is O(n). Verified
byte-identical to the old regex over 200K fuzzed inputs.
- _parse_mistral_array healing ran _balanced_brace_end from every { in the
body (20K -> 17s). Walk top-level objects, advancing past each balanced
{...}; this also drops the phantom call the old scan emitted from a nested
argument object.
Add adversarial-length linearity regressions plus positive .call kwargs and
unclosed-array recovery coverage.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: honor strict mode in safety-net, keep empty Gemma args, strip attribute-form function XML
- safetensors safety-net parser now forwards allow_incomplete=auto_heal_tool_calls,
matching the draining path, so a late incomplete tool call is not healed and
executed when Auto-Heal is off.
- Gemma empty bare value ({k:}) now serialises as "" instead of invalid {"k":},
which previously dropped the whole call.
- Route _TOOL_XML_RE also strips the <function name="..."> attribute form
(MiniCPM-5 / MiniMax-M2) so it no longer leaks to the UI.
* Studio: fix attribute-form function-XML literal close tag and zero-arg strict call
Addresses Codex review of the <function name="..."> attribute form in
_parse_function_xml (MiniCPM-5 / MiniMax-M2):
- End the call body at the LAST </function> / </tool_call> within the call's
window, so a literal close tag inside a code/search argument (e.g.
print("</function>")) is preserved instead of truncating the call.
- Accept a closed call with no parameters as a valid zero-argument call in strict
mode (the function close is already required), instead of rejecting it as a
truncated call.
- Tests for both, mirroring the legacy <function=...> coverage.
* Studio: fix tool-call parser/loop review findings on the multi-format path
Address the live code-review findings on the safetensors/MLX + GGUF tool path:
- routes: include the attribute form <function name="..."> in the safetensors
capability whitelist so MiniCPM-5 / MiniMax-M2 templates keep the tool pill
(parser already handles the form; the post-filter wrongly suppressed it).
- safetensors loop: build the plan-without-action re-prompt from the active
tools instead of a hardcoded web_search/python string, and gate it on
auto_heal_tool_calls, matching the GGUF loop.
- safetensors loop: hold a leading bare-JSON object ({"name":..,"parameters":..})
during BUFFERING until it closes, then drain it as a tool call instead of
streaming the raw JSON to clients. The DRAINING/STREAMING resolvers still
recover a plain JSON answer, so this can never drop content.
- parser: anchor the Llama-3 <|python_tag|>NAME.call(...) scan to the tag and
chain ; -separated calls, so all semicolon-separated built-ins parse and a
literal <|python_tag|>x.call(...) inside a JSON string argument no longer
fires the wrong tool.
- parser: consume the optional trailing </s> after a named Mistral
[TOOL_CALLS]name{json} call, mirroring the array shape.
- GGUF streaming strip: use the shared parser patterns (which know
[TOOL_CALLS] and <|python_tag|>) so a textual tool call entering DRAINING is
stripped instead of leaking the marker to streaming clients.
- routes: hoist the _strip_mistral_closed_calls import to module level.
Adds regression tests covering each fix; existing parser suite stays green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden multi-format tool-call detection from review findings
Apply five targeted fixes from the review pass over the multi-format tool
path:
- routes: route display strip delegates to _strip_tool_xml so Mistral
[TOOL_CALLS] blocks with nested JSON are removed from streamed display
text, not just the XML forms.
- tool_call_parser: skip function/parameter starts that fall inside an
already-open parameter block (_inside_open_parameter) so nested example
payloads are not mis-parsed as new calls; extract
strip_llama3_leading_sentinels so the bare-JSON guard is shared.
- safetensors_agentic: probe bare JSON through strip_llama3_leading_sentinels
before the balanced-brace check so a leaked header sentinel does not defeat
the guard.
- tool_healing: allow dotted tool names in the Gemma wrapped start pattern.
- llama_cpp (GGUF): buffer wrapper-less Llama-3.2 {"name":..} calls that carry
no XML signal, drain a complete object silently and hold an incomplete one,
and run the end-of-stream safety net unconditionally so markerless calls are
detected and never leak the raw JSON (including truncated fragments).
Adds regression tests for the GGUF bare-JSON streaming path and the Mistral
display strip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: stop bare-JSON tool calls leaking at EOF, oversized, and into history
The second review pass flagged that the Llama-3.2 bare-JSON tool-call handling
still leaked raw JSON in several spots; ``strip_tool_markup`` only knows
XML/bracket markup, so the bare-JSON form survived it. Fix them symmetrically
across the safetensors and GGUF loops:
- Safetensors stream-end resolver now routes a held bare-JSON fragment to
DRAINING (mirroring GGUF) so a truncated ``{"name":..`` cut off by the end of
the stream is dropped instead of flushed as assistant content. The 7/10
reviewer finding.
- Both loops now drain (suppress) an oversized still-open bare-JSON call once it
passes ``_MAX_BARE_JSON_BUFFER`` instead of streaming the raw prefix, gated on
a ``"name"`` key so a giant plain JSON answer still streams; a complete
oversized call still executes via the safety net.
- Add a shared ``strip_leading_bare_json_call`` helper and apply it to the
content kept for the assistant turn in both loops, so an executed bare-JSON
call is not replayed as visible text or fed back as next-turn history.
Plain JSON answers without a ``"name"`` key are untouched throughout. Adds
regression tests for the EOF, oversized, and next-turn cases on both backends
plus unit tests for the helper.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: bound the Llama-3 python_tag strip on real control sentinels
The route display strip's <|python_tag|> arm ran to the next <| of any kind.
A tool-call argument carrying a literal <|...|> token (for example <|cite|>
inside a string value) truncated the strip early and leaked the call tail into
the visible response. Narrow the stop condition to the genuine Llama control
sentinels (eot_id, eom_id, python_tag, start/end_header_id, begin_of_text,
finetune_right_pad_id) so embedded markup and JSON are consumed while real
header/turn boundaries still bound the strip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: gate markerless bare JSON on enabled tools and close parser/strip asymmetries
The Llama-3.2 custom_tools bare-JSON form has no marker, so any JSON object with a
name key was read as a tool call. An ordinary JSON answer like
{"name":"Alice","parameters":{"age":30}} was misclassified as a call to a
disabled tool and dropped from the visible response. Gate the markerless form on
the enabled tool names (threaded through parse_tool_calls_from_text and
strip_leading_bare_json_call, supplied by both streaming loops): an object whose
name is not an enabled tool is ordinary content. The marker-based forms keep
their name-agnostic behaviour (an explicit signal is a real call attempt), and
unrestricted mode stays ungated.
Also fix two parser/strip asymmetries the parser already tolerated:
- A literal </function> inside a parameter value (print("</function>")) truncated
both the core and route strips at the first close, leaking the tail. Extend the
strip to the call's real close (last </function> before the next opener),
mirroring the parser, without merging separate calls.
- The single-object Mistral [TOOL_CALLS]{...} shape parsed but _strip_mistral_closed_calls
left it, leaking the raw object into display. Strip the balanced object while
keeping trailing prose, matching the array and name shapes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio tools: gate GGUF bare-JSON suppression on enabled tools and fix python-tag exponent parsing
Pass-4 review follow-ups on the GGUF tool loop and Llama-3 parser:
- The GGUF bare-JSON suppression sites still keyed off a raw "name" substring,
so an ordinary JSON answer whose name is not an enabled tool was dropped when
it was truncated, oversized, or reached the no-tool DRAINING fallback (the
parser, helper, and safetensors paths were already gated). All three sites now
use the shared enabled-name gate, and a held bare-JSON buffer that turns out not
to be an enabled call is shown as the answer instead of dropped at stream end.
- The Llama-3 python-tag numeric kwarg regex matched only the mantissa, so
scientific notation was truncated to its leading digits (1e-3 parsed as 1) and a
tool executed with the wrong value. The regex now accepts exponent and decimal
forms, and the int/float classification keys off the exponent too.
Adds regression tests for the truncated / oversized disabled-name JSON cases (and
a counterpart that a truncated enabled call still does not leak) plus the
scientific-notation kwargs.
* Studio tools: gate safetensors bare-JSON drain, fix nested-name gate and function-XML strip
Pass-4 review follow-ups on the shared parser / safetensors loop:
- The safetensors oversized and end-of-stream bare-JSON drain branches keyed off
a raw "name" substring, so a large or truncated ordinary JSON answer whose name
is not an enabled tool was drained instead of streamed. Both now use the shared
enabled-tool-name gate, matching the GGUF path.
- strip_leading_bare_json_call matched the first "name" anywhere, so a plain JSON
answer with a nested name equal to an enabled tool ({"result":{"name":"web_search"}})
was wrongly suppressed. It now extracts the TOP-LEVEL name only, walking past
nested objects/arrays and keeping the text when a top-level value is truncated.
- The function-XML display strip used a regex negative-lookahead that stopped at a
literal <function=...> opener inside a parameter value and then dropped the rest
of the answer to EOF. A scan-based strip mirrors the parser (ignores openers
inside an open <parameter> via _inside_open_parameter) and closes each call at its
real </function>, so trailing assistant text after such a call survives.
Adds regression tests for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tool parsing: 3.9 import safety, disabled-Auto-Heal contract, capability gate
Round-2 review follow-ups on the multi-format tool-call parser:
- tool_call_parser: add `from __future__ import annotations`. The module
is dependency-light by design (external llama-server wrappers import it
standalone) and the package targets python >=3.9, where its PEP 604
`int | None` return annotations would raise TypeError on import.
- safetensors + GGUF drain fallback: gate the leading bare-JSON strip on
auto_heal_tool_calls. With Auto-Heal off, a truncated enabled-name
fragment that did not parse now stays visible, matching the XML strip
in the same branch and the disabled-Auto-Heal contract. With Auto-Heal
on it is still suppressed.
- safetensors capability gate: match the bare-JSON `{"name":` template
marker with a whitespace/escape-tolerant regex so a pretty-printed
`{ "name" :` or JSON-escaped `{\"name\":` template is not mis-classified
as tool-less. The parser already accepts that whitespace via
raw_decode, so the gate must too.
Regression tests added for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tool parsing: symmetric "function" bare-JSON alias and route strip parity
Round-3 review follow-ups, all parser/strip symmetry fixes.
- Bare-JSON "function" alias: the markerless parser accepts a call name via
obj.get("name") or obj.get("function"), but the strip/gates only knew "name",
so a {"function":<enabled tool>} call executed while its raw JSON leaked. Teach
_top_level_bare_json_name the alias (with "name" precedence and the same nested
and truncated-name guards), and widen the guards in strip_leading_bare_json_call,
the safetensors and GGUF _looks_like_enabled_bare_json gates, and the route
capability marker regex.
- Route display/history cleanup: strip a tail-only </param> alias close (the
parser accepts <param name="...">...</param>), and run the parser's guarded
function-XML scan (_inside_open_parameter) before _TOOL_XML_RE so a literal
nested <function=...></function> inside an argument value does not truncate the
strip and leak the tail.
Regression tests added for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio tools: honor tool budget in GGUF loop and guard function-XML streaming strip
Round 4 review fixes. Both are asymmetric-fix bugs where the final/steady path got a
guard the analogous streaming/loop path did not.
- GGUF tool-call budget: the safetensors loop counts real tool-call turns against
max_tool_iterations (re-prompt stalls excepted), but the GGUF loop only bounded the
turn count by the enlarged range (max_tool_iterations + _MAX_REPROMPTS). Since this
PR raised _MAX_REPROMPTS from 1 to 3, a model that keeps making valid tool calls
could run up to three extra tool rounds (with max_tool_iterations=1, four rounds
instead of one). Add a _tool_iters_done counter that increments only when a tool
actually executed in the turn, and stop once the caller's budget is spent so the
post-loop final-answer nudge fires. A duplicate/disabled no-op turn is a correction
turn (like a plan-without-action re-prompt) and does not consume budget, preserving
the existing "already completed" re-prompt behavior.
- Streaming display strip: the final strip runs the guarded _strip_function_xml_calls
scanner (a literal <function=...> inside a parameter value is data, not a nested
call), but the GGUF and safetensors streaming strips still used only the open-ended
regex arms. When a tool-call argument contained literal function markup, the regex
tail ate everything to end-of-text and dropped the real trailing prose after the
call's true </function>. Run the guarded scanner (and the balanced Mistral strip)
before the regex arms in both streaming paths so streaming and final display agree.
Adds regression tests: GGUF valid tool calls respect max_tool_iterations, and the
streaming strip keeps trailing prose after a function-XML call with a literal marker.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio tools: safetensors tool budget counts only executed turns (GGUF parity)
Follow-up to the GGUF budget fix. The safetensors loop charged max_tool_iterations
per non-re-prompt iteration (iteration + 1 - reprompt_count), so a duplicate/disabled
no-op turn spent a budget slot even though no tool ran. With a small cap this dropped
real work: for max_tool_iterations=2, a model that made a valid call, repeated it (an
internal no-op correction turn), then made a distinct valid call executed only the
first -- the third turn was sent with no tools and the distinct call was ignored.
Track whether a turn actually executed a tool (set on record_result) and count only
those turns against the cap, matching the GGUF loop. A duplicate/disabled no-op is a
correction turn -- like a plan-without-action re-prompt -- and no longer consumes
budget, so the model still gets its "already completed" nudge and another tool-enabled
turn. Adds a regression test for the small-cap duplicate-then-distinct-call flow.
* Studio: render the reasoning block for safetensors and MLX like GGUF
enable_thinking chat templates (Qwen3/Qwen3.5/GLM) prefill an unclosed <think>
into the generation prompt, so the model emits only the closing </think> then
the answer. The safetensors/MLX chat stream emitted that as plain content, so
the reasoning showed inline with no collapsible thinking block, while GGUF
(which surfaces reasoning via reasoning_content) rendered one. This brings
safetensors and MLX to parity.
- _ResponsesReasoningExtractor gains a reasoning_prefilled mode that starts
inside the reasoning block and splits on the first </think>; default False
keeps GGUF and every existing caller byte-identical. It suppresses a stray
re-emitted <think> and holds partial markers back across chunk boundaries.
- _sf_reasoning_prefill_mode gates the mode on reasoning being enabled for the
request, an enable_thinking or enable_thinking_effort style, and the template
actually using the standard <think>/</think> markers. Models with a bespoke
reasoning channel (e.g. gemma's <|think|>/<|channel>) are excluded so their
answer is never swallowed; gpt-oss (Harmony) and thinking-off requests are
excluded too.
- sf_tool_stream and stream_chunks (the latter also serves MLX) feed text
through the extractor, emitting reasoning_content then content deltas, with a
per-turn reset in the tool loop and a flush before each tool_start; only the
visible delta reaches the monitor reply. The two non-streaming drains split
reasoning_content the same way.
- Tests: extractor prefilled mode (streaming and edge cases), the gate matrix
including the gemma-style exclusion, and a route-replay of the tool-loop
reasoning stream.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: don't force a tool re-prompt on a negated intent (safetensors parity)
The safetensors _INTENT_SIGNAL claimed to mirror GGUF but was missing the
negative lookahead, so a refusal like "I will not search the web for that"
matched the "i will" intent and triggered the plan-without-action re-prompt
(STOP... you MUST call a tool), overriding a valid no-tool answer. GGUF already
excludes not/never. Add the same (?!\s+(?:not|never)\b) lookahead so both
backends agree. Extends the intent parity test with negated refusals.
* Studio: trim redundant comments (comment-only, AST-verified)
* Studio: prevent Gemma tool-parser DoS on stray delimiters
_gemma_parse_value returned the input index unchanged when text[i] was a
stray delimiter (,}]), so the list and mapping caller loops that advance
on the returned index spun forever at 100% CPU on malformed input such as
[},]. Advance past the delimiter so parsing always terminates.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: strip Magistral [THINK] reasoning from final display/history
strip_tool_markup removed [TOOL_CALLS] and <function> markup but left a
leading Magistral [THINK]...[/THINK] block intact, so its bracket-form
reasoning (not the <think> the reasoning channel renders) leaked into the
safetensors display and conversation history while GGUF/llama.cpp routes
it natively. Drop the leading reasoning block at end-of-turn (final=True)
via the existing _strip_mistral_reasoning helper; streaming is untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor reasoning_effort none in safetensors prefill; strip Magistral reasoning while streaming
Two safetensors/MLX reasoning fixes surfaced in review:
_sf_reasoning_prefill_mode only checked enable_thinking, so an
enable_thinking_effort (GLM-5.2) request that disables thinking via
reasoning_effort=none (without enable_thinking=False) still began in
prefilled-<think> mode. A plain answer with no </think> was then swallowed
whole into reasoning_content and the visible response came back empty. Thread
reasoning_effort into the predicate and treat none as disabled, mirroring
_request_reasoning_kwargs.
strip_tool_markup_streaming stripped tool markup but not the leading Magistral
[THINK]...[/THINK] bracket block, so the raw chain-of-thought leaked into the
streamed safetensors content instead of the reasoning drawer (GGUF routes it
natively). Apply _strip_mistral_reasoning first, matching the final strip; an
unclosed [THINK] is held from the marker on so nothing flickers.
* Mistral outer call wins over XML literals; align healer signals with its parser
Two follow-ups on the shared-parser ordering after the healing-passthrough
merge:
- A well-formed [TOOL_CALLS] call whose JSON arguments quote tool XML parsed
the literal instead of the outer call (executing the wrong tool). When the
first XML signal sits inside a leading balanced Mistral body it is argument
data, so the Mistral parser now runs first; an XML signal before the trigger
keeps the normal order, so a [TOOL_CALLS] literal inside an XML call's
arguments still stays data.
- passthrough_healing buffered streams on the parser module's broadened signal
list (now including <|python_tag|> and [TOOL_CALLS]) but promotes with
core.tool_healing, which does not parse those forms: a streamed Mistral or
Llama text call was held until finalization and flushed as prose. The healer
keeps its own signal list limited to the formats it can promote, restoring
immediate streaming for the rest.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: leading envelopes win over rehearsed literals
- New _first_foreign_tool_signal shared by the leading-envelope guards adds
<|python_tag|> to the protected signal set: the spelled-out literal inside a
Mistral call's arguments (a query about Llama built-in tool syntax) executed
the inner literal instead of the outer call.
- New _xml_signal_inside_leading_bare_json guard, sibling of the Mistral one:
a leading bare-JSON call whose string argument quotes tool XML (a code value
citing <function=...>) had the literal promoted by the shared XML pass
before the bare-JSON parser ran.
- Magistral [THINK]...[/THINK] is dropped once at parse entry instead of only
inside the Mistral parser, so a call rehearsed in the think block in a
foreign format can no longer be promoted while the real call after the
block is lost. Parse now agrees with the display strip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: a disabled leading bare-JSON object keeps its literals as data
When the leading bare-JSON object is ordinary content (name not an enabled
tool), the guard proved the first tool signal sits inside it, so falling
through to the XML/python_tag passes promoted quoted string data as a real
call. Drop the object and parse only the tail: a real call after the object
still parses, nothing inside it can be promoted.
* Address review: Mistral literals inside leading JSON, whitespace-tolerant wrapped Gemma opener
- The leading bare-JSON guard now treats the [TOOL_CALLS] trigger as a
foreign signal: the Mistral parser runs before the bare-JSON one, so a
literal quoted inside the leading object's strings was promoted over the
outer call (or over ordinary JSON content).
- tool_healing's wrapped Gemma opener tolerates whitespace around call and
the colon: sampling drift emits call: name{ and call : name{, and
rejecting those lost the call entirely because no fallback re-parses the
wrapped form. Strict mode still requires the closing tag.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: accept dotted Gemma argument keys in the key-quoting scanner
The scanner quoted keys of [alnum_-] only, so a dotted key (user.name:...)
was left unquoted, json.loads failed, and the whole wrapped call was lost
(parse empty, strip wipes the markup). Dots now match the parser's own
key/name charset.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: leading Mistral call owns the turn, dotted keys after bare values
- A LEADING parseable [TOOL_CALLS] call now runs the Mistral parser first
unconditionally: literal XML in trailing prose after the call was promoted
by the earlier shared XML pass, executing the quoted example instead of
the real leading call. XML leading keeps the normal order.
- _GEMMA_NEXT_KEY_RE accepts dots so a dotted key after a bare value
(query:foo,user.name:bob) ends the value at the comma instead of being
swallowed into it, matching the round-earlier key-quoting charset.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: markup quoted inside a nameless leading JSON answer stays data
The leading bare-JSON guard required a top-level name, so a structured JSON
answer quoting tool markup in its strings (a response_format turn
documenting a tool's syntax) had the literal promoted by the later passes.
A nameless leading object that parses as real JSON now routes through the
same decline-then-parse-the-tail path; non-JSON braced prose keeps the old
behaviour, and a real call after the answer still parses.
* Compress docstrings in the multi-format tool parser to their contract essence
* verify_import_hoist: exempt __future__ imports and same-diff relocations
Two false positives fired on this PR's refactor. A from __future__ import
is a compiler directive whose name never appears as a runtime load, so
HOISTED-IMPORT-UNUSED can never see it used, yet the file requires it for
PEP 604 annotations on Python 3.9. TARGET-CHANGED flagged the deliberate
move of the strip-pattern constants into core.inference.tool_call_parser
as a silent re-point even though the old module-level target was removed
and the new one added in the same diff. Both get narrow exemptions; a
re-point to a pre-existing target is still caught, and the self-test
negative controls all pass unchanged.
* Leading bare-JSON calls own the turn; function calls end at the first balanced close
The XML-signal guard for a leading bare-JSON call required the signal
strictly inside the object, so a trailing XML example stole the turn
from the leading call; it now applies the same inside-or-after rule as
the Mistral guard. Function-XML calls also ended at the LAST close tag,
which let prose after a closed call that mentions a literal close tag
get swallowed into the final parameter value; calls now end at the
first close tag that is not inside an open parameter, and the strip
mirrors the same rule so parse and strip agree.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Attribute-form calls end at the first balanced close; bare-JSON strip requires the call shape
The attribute form parser still kept the last close tag in the call
window, folding prose after a closed call into the final parameter
value. It now takes the first close not inside an open parameter, the
same rule the equals form and the strip already use.
The leading bare-JSON strip deleted any closed object whose top-level
name matched an enabled tool, including plain JSON answers the parser
correctly rejects as non-calls. The strip (and the drain gate that
delegates to it) now requires the parser's exact call shape, so answers
like {"name":"web_search","result":...} stream and display intact.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* False-alarm markers keep the answer; the bare-JSON strip consumes the whole chain
The trailing strip arms dropped everything from a bare marker to EOF,
so a normal answer that mentions [TOOL_CALLS] or another marker
literally was truncated (or fully swallowed when it started with the
literal) after the no-call drain fallback. Those arms now require a
call-shaped lookahead or marker-at-EOF before dropping; truncated real
calls still strip.
Chained bare-JSON turns executed both calls but stripped only the first
object, so the second call's raw JSON replayed into the next assistant
history message alongside the structured tool_calls. The strip now
consumes the entire chained run of call-shaped enabled objects while
non-call answers, disabled names, and trailing prose stay intact.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Attribute-form containment, parameter-close-decides rule, preamble-tolerant Mistral guard, strict strip shape
Four document-order and containment fixes. A leading attribute-form
call now parses before the shared XML pass, so markup quoted in its
parameter stays data. The open-parameter scan lets the parameter's own
close tag decide, so any number of literal function closes inside one
value stay data, restoring the pre-close-scan behavior for multi-close
arguments. The leading-Mistral guard tolerates a visible preamble, with
the leading-bare-JSON guard running first so a trigger quoted inside a
leading JSON object stays data. The bare-JSON strip requires the
parser's top-level name in every mode, so nested-name JSON answers
survive name-agnostic stripping.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Let a leading <|python_tag|> call own the turn over quoted XML literals
The leading-call ownership contract (a leading executable call owns the turn;
foreign markup quoted in its string arguments or trailing prose stays data) was
enforced for the bare-JSON, Mistral and attribute-form leading calls but not
for the Llama-3 <|python_tag|> form. The shared tool_healing XML pass runs
before _parse_llama3_python_tag and does not recognise <|python_tag|>, so a
<function=...> / <tool_call> / [TOOL_CALLS] literal quoted inside a
<|python_tag|> .call(...) string argument (or its JSON parameters) was promoted
and the wrong tool executed. Well-formed single-format examples:
<|python_tag|>web_search.call(query="... <function=foo> ...") -> foo
<|python_tag|>python.call(code="<function=render_html>..</function>") -> render_html
both returned the phantom inner tool instead of the real leading call.
Add a leading-<|python_tag|> guard mirroring the other leading-call guards:
when the tag is the first tool signal, parse it before tool_healing so quoted
foreign markup stays data. A foreign signal before the tag keeps normal
document order. Added TestPythonTagOuterOverXmlLiteral (7 cases).
* studio: tighten tool-calling comments to be shorter and clearer
* studio: shorten tool-format comments in changed files
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: Daniel Han <info@unsloth.ai>
Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com>
* Studio: honor stream=false on the GGUF agentic tool path (#6570)
* Studio: dedup the #6570 non-streaming tool tests and cover cached_tokens
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cover the cached_tokens metadata fix and clarify the drain comment (#6570)
* Studio: align the GGUF tool drain naming and tighten its comment (#6570)
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* studio: report the true reasoning duration and fix the Stop button for thinking models
For a local GGUF the "Thought for N" label was timed entirely on the client by a
brittle edge-detector, so an always-think model (Qwen3 MTP) that buffers its whole
reasoning and flushes it in one chunk showed "1 second" instead of the real
minute-plus. The client cannot time reasoning it receives atomically, so make the
timing backend-authoritative.
Backend: generate_chat_completion_with_tools measures wall-clock reasoning and
emits a Studio reasoning_summary event (duration_ms) at the moment reasoning ends
-- the first answer token, or end-of-stream for a reasoning-only reply -- for both
the tool-detection pass and the final-answer pass. Timing resets per tool
iteration so the final answer's thinking time wins on the client (which takes the
latest reasoning_summary). routes/inference.py forwards the event in the GGUF tool
stream.
Frontend: parse the reasoning_summary SSE into a _reasoningDurationMs chunk and
use it as the authoritative reasoning duration (last write wins), clamped to >= 0
and guarded to a finite number so a malformed or proxied chunk cannot produce a
NaN label; the persisted value wins for the final "Thought for N" label, with the
previous live timer kept only as a fallback when no metadata arrives.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: show tool-call progress for large GGUF tool arguments
The GGUF agentic tool loop only surfaced an early provisional tool card
for render_html, so any other tool (python, terminal, ...) was invisible
in the UI while its arguments streamed. For a large argument such as a
full HTML or code file this left the chat sitting on "Generating..." with
zero progress for tens of seconds while the model was clearly working.
Generalize the provisional tool_start to any enabled tool once its
streamed arguments grow past a threshold (render_html still surfaces
immediately, small-argument tools are unchanged). The provisional and the
real tool_start share the tool_call_id so the frontend reconciles them
into one card. Close the provisional on no-op, denial, parallel-drop,
post-loop, and on stream errors so a card can never spin forever, surface
each parallel call, and skip the early card while a human confirmation
gate is active. Apply the same confirmation-gate guard to the safetensors
agentic loop.
Additional hardening:
- Only emit a provisional card once a real, non-empty tool_call_id is
known. llama.cpp can stream a tool call with an empty id, and a card
keyed by "" cannot reconcile with the real tool_start (the frontend
mints its own id per event), so it would dangle.
- On a connection drop or other mid-iteration failure, close the dangling
provisional card with an error result instead of an empty success so the
UI renders it as failed rather than completed.
- Mirror the provisional cleanup in the safetensors loop: close a
provisional render_html card if the model generator raises mid-stream or
the controller turns the call into an internal no-op.
Adds regression tests for the empty-id guard, the error-result on a
dropped connection, and the safetensors mid-stream exception cleanup.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* fix: extend llama.cpp first-token timeout
* fix: timeout label pluralization
* studio: distinguish llama stream timeout phases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust timeout handling for PR #5841
* Fix lint failure for PR #5841
* Fix/adjust stream timeout handling for PR #5841
* Fix/adjust first token timeout for PR #5841
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust passthrough timeouts for PR #5841
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust preheader stream cancellation for PR #5841
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust timeout PR diff for PR #5841
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust Python 3.9 stream iteration for PR #5841
* Fix first body timeout for PR #5841
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix first token timeout deadlines for PR #5841
---------
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix race in tool-call confirmation gate
* Studio: gate built-in tool calls and harden the confirmation handshake
The Allow / Always allow / Deny controls only lived in the fallback tool
card, but the built-in tools (web search, python, terminal, code
execution, image generation) render with their own components and so
never showed the buttons. Those calls paused after tool_start with no way
to approve them, hanging until the 1 hour timeout. Only MCP tools, which
use the fallback renderer, actually worked.
Render the controls for every tool card by wrapping each registered tool
component (and the fallback) in thread.tsx with a shared
ToolConfirmationControls, so the gate applies uniformly.
Also make the handshake robust:
- The gate keys on a per-call approval_id minted by the backend and
echoed in tool_start, instead of session_id alone, so a stale or
concurrent confirmation can no longer resolve the wrong call.
- The approval slot is registered before tool_start is yielded, closing
the race where a fast click or an auto "Always allow" could reach the
backend before the waiter existed.
- The frontend resolves with the same session id the request was sent
with (plus the approval_id), fixing the new-thread mismatch where the
confirmation targeted a different session than the blocked stream.
- The confirm endpoint returns {resolved}; the UI keeps the buttons and
shows a retry hint until the backend confirms a match, instead of
hiding them on a failed or mistargeted post.
- The gate runs after the disabled-tool and duplicate-call checks, so a
call that will not execute is not put up for approval. A denied call is
still excluded from duplicate detection, so re-issuing and approving it
works.
- "Always allow" is scoped per session to match the backend gate.
Add backend tests for the approval registry, the SSE no-deadlock
handshake, and the loop integration (allow, deny, disabled, duplicate,
re-issue after deny).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move "Confirm tool calls" to the Tools section
* Studio: Keep tool group open while a tool call awaits confirmation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix tool confirmation session scope for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix confirmation follow-ups for PR #5869
* Apply pre-commit formatting for PR #5869
* Fix confirmation cleanup for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden confirmation lookups for PR #5869
* Studio: make the tool-call confirmation decision immutable
resolve_tool_decision accepted a second confirmation for the same approval_id
and overwrote slot["decision"] in the window before the waiter reads it and
pops the slot, so a duplicate or out-of-order POST could flip an Allow to Deny
(and returned a misleading resolved:true). Reject once the slot's event is
already set so the first decision wins. Adds a regression test.
* Fix/adjust tool confirmations for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Unify the Studio local tool-call loop (GGUF + safetensors) behind a shared ToolLoopController: ordered preface-then-tool-card rendering, duplicate-call de-looping with a forced final answer, XML-leak containment, and a parser fix that accepts closed <function=...> calls followed by trailing prose. Includes backend tests for the controller, strict parser, and GGUF route cursor reset.
Michael Han