Studio chat: tool-call nudging on by default (API stays opt-in) (#6883)

* Studio chat: tool-call nudging on by default (API stays opt-in)

Healing is already default-on everywhere and the nudge retry from the
client-tool passthrough is opt-in on the API. Studio chat had neither
signal: the frontend never sent nudge_tool_calls, and the safetensors
and MLX server-side loop lacked the GGUF loop's plan-without-action
re-prompt entirely.

Backend: the re-prompt helpers move from llama_cpp.py into
tool_call_parser.py (shared, cycle-free; the GGUF loop imports them
under its old names with zero behavior change) and
run_safetensors_tool_loop now re-prompts once at the streaming
no-tool-call exit, gated on Auto-Heal, active tools, nothing executed
yet, and short forward-looking text. Re-prompts do not consume tool
iterations.

Frontend: the chat adapter sends nudge_tool_calls from a new
nudgeToolCalls runtime setting (default true) with the same
persistence, hydration, and settings toggle plumbing as Auto-Heal.
Request-model defaults are untouched, so raw API callers stay opt-in.

* Address review: persist the nudge setting, consume the flag in the loops, skip the re-prompt after RAG autoinject

ChatSettingsPayload uses extra forbid, so a settings patch containing
nudgeToolCalls failed to persist any settings; the field is now typed
and round-trips. nudge_tool_calls now plumbs into both server-side tool
loops and gates the plan-without-action re-prompt with None meaning on,
so API callers keep today's behavior, explicit false disables it, and
Studio's default-on flag actually controls the path Studio chat runs.
The safetensors loop no longer re-prompts after RAG autoinject: the
injected retrieval bypasses the tool controller, so the nothing-executed
gate saw an empty history and re-asked after a successful retrieval.

* Safetensors loop: the plan-without-action retry requires an explicit nudge flag

The retry is new on this loop, so an omitted nudge_tool_calls must not
change existing API behavior; Studio opts in explicitly. The GGUF loop
keeps None as on because its re-prompt predates the flag.

* Suppress the plan-without-action re-prompt after a denied tool confirmation

A denial appends TOOL_REJECTED_MESSAGE but records nothing in the tool
controller history, so the nothing-executed gate re-prompted the model
to call the tool the user had just rejected, producing another
confirmation prompt. A denial now suppresses the re-prompt for the rest
of the request, mirroring the RAG autoinject handling.

* Tighten plan-without-action re-prompt comments

* Tighten plan-without-action re-prompt comments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: match unified plan-without-action nudge cap to GGUF default of 3

The shared MAX_ACT_REPROMPTS was set to 1, but GGUF's established default
(llama_cpp.py) has re-prompted a stalling model up to 3 times since #5620.
Restore the GGUF-matched cap so safetensors and MLX inherit the same
behavior, and update the safetensors cap test to assert the cap dynamically.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Daniel Han 2026-07-06 19:41:19 -07:00 committed by GitHub
commit 9dabe96786
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
18 changed files with 506 additions and 82 deletions

View file

@ -1168,6 +1168,48 @@ def test_internal_reprompt_disabled_when_auto_heal_disabled(monkeypatch):
assert len(payloads) == 1
def test_internal_reprompt_disabled_when_nudge_tool_calls_false(monkeypatch):
# Explicit nudge_tool_calls=False disables the plan-without-action
# re-prompt even with Auto-Heal on (None keeps the default-on behavior).
streams = [[_sse({"content": "I will use render_html now."}), _done()]]
payloads: list[dict] = []
backend = _make_backend(monkeypatch, streams, payloads)
def fake_execute_tool(name, arguments, **_kwargs):
raise AssertionError(f"unexpected tool execution: {name} {arguments}")
monkeypatch.setattr("core.inference.tools.execute_tool", fake_execute_tool)
tools = [
{
"type": "function",
"function": {
"name": "render_html",
"description": "Render HTML.",
"parameters": {
"type": "object",
"properties": {"code": {"type": "string"}},
"required": ["code"],
},
},
}
]
events = list(
backend.generate_chat_completion_with_tools(
messages = [{"role": "user", "content": "Make a red square."}],
tools = tools,
max_tool_iterations = 1,
auto_heal_tool_calls = True,
nudge_tool_calls = False,
)
)
content_texts = [event.get("text", "") for event in events if event.get("type") == "content"]
assert content_texts == ["I will use render_html now."]
assert len(payloads) == 1
def test_auto_heal_disabled_parses_well_formed_xml_when_tools_enabled(monkeypatch):
streams = [
[