Enforce parallel_tool_calls=False client-side on local GGUF (PR #5711)
Two related findings from round 12 reviewers: 1. The local GGUF tool loop in `generate_chat_completion_with_tools` iterates every entry of `tool_calls` returned by llama-server, even when the caller explicitly opted out of parallel tool calls. The `parallel_tool_calls` flag is forwarded to llama-server, but llama .cpp does not enforce it on every jinja template (https://github.com/ggml-org/llama.cpp/issues/22043), so a model that ignores the flag still ran multiple tools per turn. Cap `tool_calls` to the first entry when the flag is False so the client-side contract holds regardless of upstream behavior. 2. llama-server documents `parallel_tool_calls` as defaulting to FALSE (https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md), so the previous chat-adapter shape (forward only on explicit false) meant the UI's default-on state could never enable parallel tool calls there. Always forward the user's preference on the local path so the toggle actually does what it says. External providers default to true everywhere, so the external branch is unchanged. Test pins the GGUF tool-loop cap by source-level assertion (the loop itself is integration-only).
This commit is contained in:
parent
737c5ad0e0
commit
67e371934c
3 changed files with 37 additions and 4 deletions
|
|
@ -5015,6 +5015,15 @@ class LlamaCppBackend:
|
|||
_accumulated_predicted_ms += _it.get("predicted_ms", 0)
|
||||
_accumulated_predicted_n += _it.get("predicted_n", 0)
|
||||
|
||||
# When the caller opted out of parallel tool calls
|
||||
# (parallel_tool_calls=False), enforce at most one call
|
||||
# per assistant turn even if llama-server emitted more.
|
||||
# llama.cpp's parallel_tool_calls flag isn't enforced by
|
||||
# every jinja template (see ggml-org/llama.cpp#22043),
|
||||
# so this client-side cap is the only guarantee.
|
||||
if parallel_tool_calls is False and tool_calls:
|
||||
tool_calls = tool_calls[:1]
|
||||
|
||||
assistant_msg = {"role": "assistant", "content": content_text}
|
||||
if tool_calls:
|
||||
assistant_msg["tool_calls"] = tool_calls
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue