Studio chat: tool-call nudging on by default (API stays opt-in) (#6883)
* Studio chat: tool-call nudging on by default (API stays opt-in) Healing is already default-on everywhere and the nudge retry from the client-tool passthrough is opt-in on the API. Studio chat had neither signal: the frontend never sent nudge_tool_calls, and the safetensors and MLX server-side loop lacked the GGUF loop's plan-without-action re-prompt entirely. Backend: the re-prompt helpers move from llama_cpp.py into tool_call_parser.py (shared, cycle-free; the GGUF loop imports them under its old names with zero behavior change) and run_safetensors_tool_loop now re-prompts once at the streaming no-tool-call exit, gated on Auto-Heal, active tools, nothing executed yet, and short forward-looking text. Re-prompts do not consume tool iterations. Frontend: the chat adapter sends nudge_tool_calls from a new nudgeToolCalls runtime setting (default true) with the same persistence, hydration, and settings toggle plumbing as Auto-Heal. Request-model defaults are untouched, so raw API callers stay opt-in. * Address review: persist the nudge setting, consume the flag in the loops, skip the re-prompt after RAG autoinject ChatSettingsPayload uses extra forbid, so a settings patch containing nudgeToolCalls failed to persist any settings; the field is now typed and round-trips. nudge_tool_calls now plumbs into both server-side tool loops and gates the plan-without-action re-prompt with None meaning on, so API callers keep today's behavior, explicit false disables it, and Studio's default-on flag actually controls the path Studio chat runs. The safetensors loop no longer re-prompts after RAG autoinject: the injected retrieval bypasses the tool controller, so the nothing-executed gate saw an empty history and re-asked after a successful retrieval. * Safetensors loop: the plan-without-action retry requires an explicit nudge flag The retry is new on this loop, so an omitted nudge_tool_calls must not change existing API behavior; Studio opts in explicitly. The GGUF loop keeps None as on because its re-prompt predates the flag. * Suppress the plan-without-action re-prompt after a denied tool confirmation A denial appends TOOL_REJECTED_MESSAGE but records nothing in the tool controller history, so the nothing-executed gate re-prompted the model to call the tool the user had just rejected, producing another confirmation prompt. A denial now suppresses the re-prompt for the rest of the request, mirroring the RAG autoinject handling. * Tighten plan-without-action re-prompt comments * Tighten plan-without-action re-prompt comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: match unified plan-without-action nudge cap to GGUF default of 3 The shared MAX_ACT_REPROMPTS was set to 1, but GGUF's established default (llama_cpp.py) has re-prompted a stalling model up to 3 times since #5620. Restore the GGUF-matched cap so safetensors and MLX inherit the same behavior, and update the safetensors cap test to assert the cap dynamically. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
parent
c2a7b78f6b
commit
9dabe96786
18 changed files with 506 additions and 82 deletions
|
|
@ -1168,6 +1168,48 @@ def test_internal_reprompt_disabled_when_auto_heal_disabled(monkeypatch):
|
|||
assert len(payloads) == 1
|
||||
|
||||
|
||||
def test_internal_reprompt_disabled_when_nudge_tool_calls_false(monkeypatch):
|
||||
# Explicit nudge_tool_calls=False disables the plan-without-action
|
||||
# re-prompt even with Auto-Heal on (None keeps the default-on behavior).
|
||||
streams = [[_sse({"content": "I will use render_html now."}), _done()]]
|
||||
payloads: list[dict] = []
|
||||
backend = _make_backend(monkeypatch, streams, payloads)
|
||||
|
||||
def fake_execute_tool(name, arguments, **_kwargs):
|
||||
raise AssertionError(f"unexpected tool execution: {name} {arguments}")
|
||||
|
||||
monkeypatch.setattr("core.inference.tools.execute_tool", fake_execute_tool)
|
||||
|
||||
tools = [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "render_html",
|
||||
"description": "Render HTML.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {"code": {"type": "string"}},
|
||||
"required": ["code"],
|
||||
},
|
||||
},
|
||||
}
|
||||
]
|
||||
|
||||
events = list(
|
||||
backend.generate_chat_completion_with_tools(
|
||||
messages = [{"role": "user", "content": "Make a red square."}],
|
||||
tools = tools,
|
||||
max_tool_iterations = 1,
|
||||
auto_heal_tool_calls = True,
|
||||
nudge_tool_calls = False,
|
||||
)
|
||||
)
|
||||
|
||||
content_texts = [event.get("text", "") for event in events if event.get("type") == "content"]
|
||||
assert content_texts == ["I will use render_html now."]
|
||||
assert len(payloads) == 1
|
||||
|
||||
|
||||
def test_auto_heal_disabled_parses_well_formed_xml_when_tools_enabled(monkeypatch):
|
||||
streams = [
|
||||
[
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue