ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix: only strip closed tool-call XML pairs in SSE output

Browse source 
The open-ended <tool_call>.*$ regex was stripping everything from
<tool_call> to end of string, including legitimate model text that
followed a tool block.  This caused the stream to appear stuck since
the cumulative text never grew past the strip point.

Switch to only stripping closed <tool_call>...</tool_call> pairs so
text after a tool block is preserved.
This commit is contained in:
Daniel Han 2026-03-27 14:15:54 +00:00
commit e451d2dae4
1 changed files with 12 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

17
studio/backend/routes/inference.py
View file

@ -1200,11 +1200,18 @@ async def openai_chat_completions(
# "content" type -- cumulative text
cumulative = event.get("text", "")
# Strip tool-call XML that may have leaked
# through the backend's content stream.
for pat in _tool_xml_strip:
cumulative = pat.sub("", cumulative)
cumulative = cumulative.rstrip()
# Strip closed tool-call XML pairs that may
# have leaked through the backend stream.
# Only strip closed pairs here (not open-ended)
# so legitimate text after a tool block is kept.
cumulative = _re.sub(
r"<tool_call>.*?</tool_call>", "",
cumulative, flags = _re.DOTALL,
)
cumulative = _re.sub(
r"<function=\w+>.*?</function>", "",
cumulative, flags = _re.DOTALL,
)
new_text = cumulative[len(prev_text) :]
prev_text = cumulative
if not new_text: