Commit graph

2 commits

Author SHA1 Message Date
pre-commit-ci[bot]
699da0decd [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2026-05-22 14:43:02 +00:00
Daniel Han
09ca2a1777 Studio: retry agentic loop on validation failure
Adds a pre-dispatch validation pass inside both agentic tool loops
(generate_chat_completion_with_tools in core/inference/llama_cpp.py and
run_safetensors_tool_loop in core/inference/safetensors_agentic.py).

The pass catches two failure modes between parser and dispatch:

* Unknown tool name (not in the request's tools[] array).
* Arguments that cannot decode to a JSON object when auto_heal is off.

On a caught call the loop appends a corrective tool-result message
tied to the hallucinated tool_call_id (not a fabricated id, so the
OpenAI chat template stays valid) and re-enters the model. When the
call has no usable id we fall back to a user-role correction since
tool-role messages require a matching prior call id.

The retry pass is bounded by max_validation_retries (default 2, new
ChatCompletionRequest field, threaded through the route layer). On
budget exhaustion the call falls through to the existing per-tool
error path so today's behavior is preserved.

When auto_heal_tool_calls is on the heal path still runs in the
dispatch loop unchanged; F3 only catches the strict-shape failures
the coercer cannot fix.
2026-05-22 14:42:19 +00:00