ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Guard against late tool_calls after visible content, filter incomplete fragments

Browse source 
1. If visible content was already emitted (_last_emitted is non-empty)
   when delta.tool_calls arrives, ignore the tool_calls instead of
   reclassifying the turn as a tool call. llama-server never
   interleaves content and tool_calls (they are mutually exclusive),
   but this guard is defensive for other OpenAI-compatible backends.
   [9/10 reviewers]

2. Filter out incomplete structured tool_calls fragments before
   execution. Entries with empty function.name (from truncation by
   max_tokens, disconnect, or interruption) are skipped instead of
   being passed to execute_tool(). [2/10 reviewers]
This commit is contained in:
Daniel Han 2026-03-27 10:30:37 +00:00
commit 4967f52b9a
1 changed files with 13 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

14
studio/backend/core/inference/llama_cpp.py
View file

@ -1867,6 +1867,11 @@ class LlamaCppBackend:
# ── Structured tool_calls ──
tc_deltas = delta.get("tool_calls")
if tc_deltas:
# Once visible content has been
# emitted, do not reclassify this
# turn as a tool call.
if _last_emitted:
continue
has_structured_tc = True
detect_state = _S_DRAINING
for tc_d in tc_deltas:
@ -2094,7 +2099,14 @@ class LlamaCppBackend:
tool_calls = None
content_text = content_accum
if has_structured_tc:
tool_calls = [tool_calls_acc[i] for i in sorted(tool_calls_acc)]
# Filter out incomplete fragments (e.g. from
# truncation by max_tokens or disconnect).
tool_calls = [
tool_calls_acc[i]
for i in sorted(tool_calls_acc)
if (tool_calls_acc[i].get("function", {})
.get("name", "").strip())
] or None
if (
not tool_calls
and auto_heal_tool_calls