1915ca98db* fetch bare hostnames as https instead of refusing them * normalize host:port URLs and route schemeless github repos to the readme API * only rewrite dotted host:port URLs with in-range ports * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * reject relative paths and oversized ports in url normalization * Match web-fetch ports as ASCII digits so a unicode digit cannot raise str.isdigit() is True for digit-class characters int() refuses (superscript two, circled digit one), so _normalize_url_scheme reached int(port) and raised ValueError out of _fetch_url_raw, which runs before its try block. A web_search url of "example.com:<superscript two>" surfaced a generic tool exception instead of the Blocked: message it returned before this branch. Match the port against an anchored [0-9]{1,5} instead; the five-digit cap that kept the range check from converting an unbounded integer is now in the pattern. * Apply the invalid-port guard to redirect targets too _fetch_url_raw wraps the initial parsed.port in try/except ValueError, but the redirect hop reads rp.port unguarded, so a server answering Location: https://example.org:99999/next fell through to the broad handler as "Failed to fetch URL: Port out of range 0-65535" rather than a deliberate block. No request is dispatched either way; this just makes the two paths report the same way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep the redirect-port test compact The formatter expands a signature carrying a spaced kwarg default, which put the stub opener on eleven lines. **kw absorbs the timeout the fetch loop passes and leaves the whole stub on four. * Never let a malformed URL escape _fetch_url_raw as an exception The URL is model-supplied, so every bad form should come back as one of the documented (error, body, content_type) strings. Three gaps remained: urlparse itself raises on an unmatched IPv6 bracket and on a netloc that NFKC-decomposes into a delimiter (//exam(fullwidth-solidus)ple.com), and both calls sat outside a guard. getaddrinfo raises UnicodeError, which is a ValueError and not the OSError _validate_and_resolve_host catches, when IDNA encoding rejects a hostname. Over a 3158 URL corpus that injects tabs, newlines, C0 controls, delimiters and NFKC confusables at every position, main raises 42 times and this raises none. Also strip surrounding whitespace in _normalize_url_scheme. _web_search already stripped, but normalization moved down to the fetch layer, so a direct _fetch_page_text caller did not get it. * Name the host in the status badge and tool card for bare URLs status_for_tool and the web-search tool card both required an explicit scheme before reading the hostname, so every URL this branch newly makes fetchable showed the generic "Reading page..." and "Read page" instead of the host. Under permission_mode=ask that means the approval card named no destination for exactly the inputs the branch enables. The backend reuses _normalize_url_scheme. The frontend cannot, since new URL() throws on a bare host, so RE_BARE_HOST mirrors the same grammar: only a dotted host with an optional in-range port gets the https prefix, leaving /login, javascript: and userinfo forms to render generically as before. Also mention bare hostnames in the url parameter description, since they are part of the accepted interface now. * Do not let a malformed URL in the status badge kill the tool turn status_for_tool runs inside prepare_call, before the fetch and outside the handler that wraps tool execution, so a ValueError from urlparse ends the whole turn instead of letting _fetch_url_raw return its blocked message. _normalize_url_scheme catches its own parse error and hands back the original string, so the parse here still has to be guarded. Reachable with https://[::1 or a host that NFKC-decomposes into a delimiter. This predates the branch, main raises identically, but the badge is one of the lines this branch touches and the rest of it already promises no malformed URL escapes as an exception. * Tighten the comments added by this branch * Revert the web_search url description change The premise of this branch is that models already emit bare hostnames unprompted, which is why the fetch layer had to stop refusing them. Advertising the bare form in the tool schema does not enable anything, it just steers models toward it, and that is the form carrying every edge case: ambiguous with dotted custom schemes, and unlike an explicit scheme it does not cover IPv6 literals, IDN or trailing-dot FQDNs. The fetch layer tolerates bare hosts. The schema should keep recommending a full URL. This also drops the one change here with no regression test. * Match the backend port rule in the tool card host The card's bare-host pattern required at least one digit after the colon, but the backend fetches an empty port (example.com: and example.com:/path go to the default HTTPS port), so a successful fetch rendered as "Read page" with no host. Allowing an empty port alone would have swung it the other way: example.com:0 is refused by the backend but new URL() accepts it, so the card would have named a host that is never fetched. That mismatch was there before this change too. Mirror the backend rule instead, an empty port or one in 1-65535, checked against every case in the normalizer's own matrix. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>
263 lines
10 KiB
Python
263 lines
10 KiB
Python
# SPDX-License-Identifier: AGPL-3.0-only
|
||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||
|
||
from __future__ import annotations
|
||
|
||
import json
|
||
import sys
|
||
from pathlib import Path
|
||
|
||
import pytest
|
||
|
||
_BACKEND_DIR = str(Path(__file__).resolve().parent.parent)
|
||
if _BACKEND_DIR not in sys.path:
|
||
sys.path.insert(0, _BACKEND_DIR)
|
||
|
||
from core.inference.tool_loop_controller import (
|
||
ToolLoopController,
|
||
append_deferred_nudges,
|
||
canonical_tool_call_key,
|
||
coerce_tool_arguments,
|
||
status_for_tool,
|
||
strip_result_for_model,
|
||
tool_event_provenance,
|
||
)
|
||
|
||
|
||
def test_append_deferred_nudges_merges_deduped_into_one_message():
|
||
conversation = [{"role": "assistant", "tool_calls": [1]}, {"role": "tool", "content": "r"}]
|
||
nudges = [
|
||
{"role": "user", "content": "duplicate"},
|
||
{"role": "user", "content": "duplicate"}, # dropped: same content
|
||
{"role": "user", "content": "disabled foo"},
|
||
]
|
||
append_deferred_nudges(conversation, nudges)
|
||
# One user message, after the results, with distinct contents joined.
|
||
assert conversation[2:] == [{"role": "user", "content": "duplicate\n\ndisabled foo"}]
|
||
# Empty is a no-op.
|
||
before = list(conversation)
|
||
append_deferred_nudges(conversation, [])
|
||
assert conversation == before
|
||
|
||
|
||
def _tool(name: str) -> dict:
|
||
return {"type": "function", "function": {"name": name}}
|
||
|
||
|
||
def _call(
|
||
name: str,
|
||
args,
|
||
call_id: str = "call_0",
|
||
) -> dict:
|
||
return {
|
||
"id": call_id,
|
||
"type": "function",
|
||
"function": {
|
||
"name": name,
|
||
"arguments": json.dumps(args) if isinstance(args, dict) else args,
|
||
},
|
||
}
|
||
|
||
|
||
def test_canonical_tool_call_key_sorts_arguments():
|
||
a = canonical_tool_call_key("web_search", {"query": "gpu", "limit": 5})
|
||
b = canonical_tool_call_key("web_search", {"limit": 5, "query": "gpu"})
|
||
c = canonical_tool_call_key("python", {"limit": 5, "query": "gpu"})
|
||
|
||
assert a == b
|
||
assert a != c
|
||
assert a == 'web_search:{"limit":5,"query":"gpu"}'
|
||
|
||
|
||
def test_coerce_tool_arguments_parses_json_and_heals_raw_strings():
|
||
parsed = coerce_tool_arguments('{"query":"gpu prices"}', heal = True)
|
||
healed = coerce_tool_arguments("print(1)", heal = True, tool_name = "python")
|
||
raw = coerce_tool_arguments("not-json", heal = False, tool_name = "python")
|
||
|
||
assert parsed.arguments == {"query": "gpu prices"}
|
||
assert not parsed.healed
|
||
assert healed.arguments == {"code": "print(1)"}
|
||
assert healed.healed
|
||
assert raw.arguments == {"raw": "not-json"}
|
||
assert not raw.healed
|
||
|
||
|
||
def test_status_and_provenance_match_local_event_conventions():
|
||
assert status_for_tool("web_search", {"query": "gpus"}) == "Searching: gpus"
|
||
assert (
|
||
status_for_tool("web_search", {"url": "https://www.example.com/a"})
|
||
== "Reading: example.com"
|
||
)
|
||
assert status_for_tool("python", {"code": "print(1)\nprint(2)"}) == "Running Python: print(1)"
|
||
assert tool_event_provenance(healed = True, forced = False, provisional = None) == {
|
||
"source": "local",
|
||
"healed": True,
|
||
}
|
||
|
||
|
||
@pytest.mark.parametrize(
|
||
"url, expected",
|
||
[
|
||
# bare hosts are fetched, so the badge must name them
|
||
("google.com", "Reading: google.com"),
|
||
("www.google.com/x", "Reading: google.com"),
|
||
("//google.com", "Reading: google.com"),
|
||
("example.com:8443/path", "Reading: example.com"),
|
||
("github.com/unslothai/unsloth", "Reading: github.com"),
|
||
# still generic for what the fetch layer refuses
|
||
("/login", "Reading page..."),
|
||
("javascript:alert(1)", "Reading page..."),
|
||
# urlparse raises on these, outside the fetch's handler: degrade, not raise
|
||
("https://[::1", "Reading page..."),
|
||
("https://::1]", "Reading page..."),
|
||
("//exam／ple.com", "Reading page..."),
|
||
("//example.com＠", "Reading page..."),
|
||
],
|
||
)
|
||
def test_status_names_the_host_for_schemeless_urls(url, expected):
|
||
assert status_for_tool("web_search", {"url": url}) == expected
|
||
|
||
|
||
def test_prepare_execute_builds_visible_events_and_model_tool_message():
|
||
controller = ToolLoopController(tools = [_tool("web_search")])
|
||
decision = controller.prepare_call(_call("web_search", {"query": "gpu prices"}))
|
||
|
||
assert decision.should_execute
|
||
assert decision.emit_visible_events
|
||
assert decision.status_text == "Searching: gpu prices"
|
||
assert decision.tool_start_payload()["arguments"] == {"query": "gpu prices"}
|
||
assert decision.tool_start_event()["type"] == "tool_start"
|
||
assert decision.as_assistant_tool_call()["function"]["arguments"] == '{"query":"gpu prices"}'
|
||
|
||
completion = controller.record_result(decision, "Search result\n__IMAGES__:{...}")
|
||
|
||
assert completion.tool_end_payload()["result"] == "Search result\n__IMAGES__:{...}"
|
||
assert completion.tool_end_event()["type"] == "tool_end"
|
||
assert completion.tool_message() == {
|
||
"role": "tool",
|
||
"name": "web_search",
|
||
"content": "Search result",
|
||
"tool_call_id": "call_0",
|
||
}
|
||
|
||
|
||
def test_successful_duplicate_is_internal_noop_and_keeps_remaining_tools():
|
||
controller = ToolLoopController(tools = [_tool("web_search"), _tool("python")])
|
||
first = controller.prepare_call(_call("web_search", {"query": "gpu prices"}, "call_a"))
|
||
controller.record_result(first, "ok")
|
||
|
||
duplicate = controller.prepare_call(_call("web_search", {"query": "gpu prices"}, "call_b"))
|
||
completion = controller.record_noop(duplicate)
|
||
|
||
assert duplicate.action == "duplicate"
|
||
assert not duplicate.should_execute
|
||
assert not duplicate.emit_visible_events
|
||
duplicate_nudge = completion.model_message()["content"]
|
||
assert duplicate_nudge.startswith(
|
||
"One earlier request to call tool 'web_search' in this batch was not executed"
|
||
)
|
||
assert "previous tool request" not in duplicate_nudge.lower()
|
||
assert "already completed successfully" in duplicate_nudge
|
||
assert "different enabled tool" in duplicate_nudge
|
||
assert completion.model_message()["role"] == "user"
|
||
assert not controller.force_final_answer
|
||
assert [tool["function"]["name"] for tool in controller.active_tools()] == [
|
||
"web_search",
|
||
"python",
|
||
]
|
||
|
||
|
||
def test_repeated_successful_duplicate_becomes_terminal_after_one_recovery_nudge():
|
||
controller = ToolLoopController(tools = [_tool("web_search"), _tool("python")])
|
||
first = controller.prepare_call(_call("web_search", {"query": "gpu prices"}, "call_a"))
|
||
controller.record_result(first, "ok")
|
||
|
||
duplicate_one = controller.prepare_call(_call("web_search", {"query": "gpu prices"}, "call_b"))
|
||
completion_one = controller.record_noop(duplicate_one)
|
||
|
||
assert duplicate_one.action == "duplicate"
|
||
assert "already completed successfully" in completion_one.model_message()["content"]
|
||
assert not controller.force_final_answer
|
||
assert [tool["function"]["name"] for tool in controller.active_tools()] == [
|
||
"web_search",
|
||
"python",
|
||
]
|
||
|
||
duplicate_two = controller.prepare_call(_call("web_search", {"query": "gpu prices"}, "call_c"))
|
||
completion_two = controller.record_noop(duplicate_two)
|
||
|
||
assert duplicate_two.action == "duplicate"
|
||
assert "already completed successfully" in completion_two.model_message()["content"]
|
||
assert controller.force_final_answer
|
||
assert controller.active_tools() == []
|
||
|
||
|
||
def test_failed_call_does_not_block_retry():
|
||
controller = ToolLoopController(tools = [_tool("web_search")])
|
||
first = controller.prepare_call(_call("web_search", {"query": "gpu prices"}))
|
||
controller.record_result(first, "Error: temporary failure")
|
||
|
||
retry = controller.prepare_call(_call("web_search", {"query": "gpu prices"}))
|
||
|
||
assert retry.should_execute
|
||
assert retry.action == "execute"
|
||
|
||
|
||
def test_empty_enabled_tool_list_blocks_all_tool_calls():
|
||
controller = ToolLoopController(tools = [])
|
||
decision = controller.prepare_call(_call("web_search", {"query": "gpu prices"}))
|
||
completion = controller.record_noop(decision)
|
||
|
||
assert decision.action == "disabled"
|
||
assert not decision.emit_visible_events
|
||
assert completion.model_message()["role"] == "user"
|
||
disabled_nudge = completion.model_message()["content"]
|
||
assert disabled_nudge.startswith(
|
||
"One earlier request to call tool 'web_search' in this batch was not executed"
|
||
)
|
||
assert "previous tool request" not in disabled_nudge.lower()
|
||
assert "not enabled" in disabled_nudge
|
||
assert controller.force_final_answer
|
||
assert controller.active_tools() == []
|
||
|
||
|
||
def test_disabled_tool_is_internal_noop_not_visible_tool_error():
|
||
controller = ToolLoopController(tools = [_tool("web_search")])
|
||
decision = controller.prepare_call(_call("python", {"code": "print(1)"}))
|
||
completion = controller.record_noop(decision)
|
||
|
||
assert decision.action == "disabled"
|
||
assert not decision.emit_visible_events
|
||
assert completion.model_message()["role"] == "user"
|
||
assert "not enabled" in completion.model_message()["content"]
|
||
assert controller.force_final_answer
|
||
assert controller.active_tools() == []
|
||
|
||
|
||
def test_render_html_success_filters_active_tools_and_repeat_is_internal():
|
||
controller = ToolLoopController(tools = [_tool("render_html"), _tool("web_search")])
|
||
assert [t["function"]["name"] for t in controller.active_tools()] == [
|
||
"render_html",
|
||
"web_search",
|
||
]
|
||
|
||
first = controller.prepare_call(_call("render_html", {"code": "<html></html>"}, "call_html_1"))
|
||
controller.record_result(first, "Rendered HTML canvas: Demo")
|
||
|
||
assert [t["function"]["name"] for t in controller.active_tools()] == ["web_search"]
|
||
|
||
repeat = controller.prepare_call(_call("render_html", {"code": "<html></html>"}, "call_html_2"))
|
||
completion = controller.record_noop(repeat)
|
||
|
||
assert repeat.action == "render_html_repeat"
|
||
assert not repeat.emit_visible_events
|
||
assert completion.model_message()["role"] == "user"
|
||
assert "Do not call render_html again" in completion.model_message()["content"]
|
||
assert controller.force_final_answer
|
||
assert controller.active_tools() == []
|
||
|
||
|
||
def test_strip_result_for_model_removes_frontend_image_sentinel():
|
||
assert strip_result_for_model('text\n__IMAGES__:{"paths":[]}') == "text"
|
||
assert strip_result_for_model("text __IMAGES__:payload") == "text"
|
||
assert strip_result_for_model("plain text") == "plain text"
|