Do not overstate the python host allowlist, and keep the blocked-command fallback tool-neutral
Two accuracy fixes to the sandbox tool notes:
- The sandbox keeps networking on (no CLONE_NEWNET) and the AST host check only
inspects literal URL/host arguments, so a dynamically built request to a
private host (host = '10.0.0.5'; requests.get('http://' + host + '/x')) runs
with no network_calls violation. The note claimed the python tool can fetch
only from a fixed allowlist and not arbitrary addresses, advertising a boundary
the code does not enforce. Reframe it as intent (the python tool is intended to
fetch from public sources such as github.com, huggingface.co, and pypi.org
rather than private hosts) without the false only/arbitrary absolute.
- The blocked-network-command message told the model to fetch from Python code
even when only the terminal is enabled that turn (python absent from the
schema), which can induce an invalid tool call. _bash_exec has no per-turn
tool signal, so make the fallback tool-neutral and gated: if a code-execution
tool is enabled this turn, public files can be fetched from code instead.
Update the note tests to lock in both changes; the bypass note variant already
omits the network sentence and stays consistent.
This commit is contained in:
parent
10f6db4f3e
commit
fda0c11682
2 changed files with 34 additions and 24 deletions
|
|
@ -2968,11 +2968,10 @@ _SANDBOX_PATHS_NOTE_INTRO = (
|
|||
"host are already here."
|
||||
)
|
||||
_SANDBOX_PATHS_NOTE_NETWORK = (
|
||||
" Internet access is limited: the python tool can fetch "
|
||||
"only from a fixed allowlist of public sites (such as github.com, "
|
||||
"huggingface.co, and pypi.org), not the user's own machines, private hosts, "
|
||||
"or arbitrary addresses, and the terminal blocks direct download commands "
|
||||
"like curl and wget."
|
||||
" Internet access is limited: the python tool is intended to fetch "
|
||||
"from public sources such as github.com, huggingface.co, and pypi.org "
|
||||
"rather than the user's own machines or private hosts, and the terminal "
|
||||
"blocks direct download commands like curl and wget."
|
||||
)
|
||||
_SANDBOX_PATHS_NOTE_TAIL = (
|
||||
" Documents the user attaches to the chat are retrieved "
|
||||
|
|
@ -2985,7 +2984,8 @@ _SANDBOX_PATHS_NOTE_TAIL = (
|
|||
"you need are not here, ask the user to provide them or an exact path "
|
||||
"instead of guessing one."
|
||||
)
|
||||
# Default (sandboxed) note: keeps the precise allowlist + curl/wget restriction.
|
||||
# Default (sandboxed) note: steers the python tool at public sources and keeps
|
||||
# the curl/wget restriction (without overstating the host check as a hard wall).
|
||||
_SANDBOX_PATHS_NOTE = (
|
||||
_SANDBOX_PATHS_NOTE_INTRO + _SANDBOX_PATHS_NOTE_NETWORK + _SANDBOX_PATHS_NOTE_TAIL
|
||||
)
|
||||
|
|
@ -5914,11 +5914,12 @@ def _bash_exec(
|
|||
if blocked & _NETWORK_BLOCKED_COMMANDS:
|
||||
return (
|
||||
base + " These download commands are blocked by name in "
|
||||
"the terminal. Public files on allowlisted sites (such as "
|
||||
"github.com, huggingface.co, or pypi.org) can be fetched "
|
||||
"from Python code instead. If you need files from another "
|
||||
"location, ask the user to place them in the working "
|
||||
"directory or give a path the sandbox can read."
|
||||
"the terminal. If a code-execution tool is enabled this "
|
||||
"turn, public files on sites such as github.com, "
|
||||
"huggingface.co, or pypi.org can be fetched from code "
|
||||
"instead. If you need files from another location, ask the "
|
||||
"user to place them in the working directory or give a path "
|
||||
"the sandbox can read."
|
||||
)
|
||||
return base
|
||||
elif not _harden_parent_against_proc_env_leak():
|
||||
|
|
|
|||
|
|
@ -5,9 +5,10 @@
|
|||
tool descriptions (#7242).
|
||||
|
||||
The note must not misdirect the model with claims that are false for the real
|
||||
sandbox: (1) the sandbox is not network-isolated -- the python tool can reach an
|
||||
allowlist of public hosts (github.com / huggingface.co / pypi.org), so it must
|
||||
not claim it "cannot reach ... remote hosts" at all; (2) a project sandbox is a
|
||||
sandbox: (1) the sandbox is not network-isolated -- the python tool should fetch
|
||||
from public hosts (github.com / huggingface.co / pypi.org), so it must not claim
|
||||
it "cannot reach ... remote hosts" at all, nor overstate the host check as a hard
|
||||
wall that makes private/arbitrary hosts impossible; (2) a project sandbox is a
|
||||
shared per-project directory, so a new thread can inherit files from prior
|
||||
threads -- the note must not claim the workdir "starts empty" or "persists only
|
||||
for this conversation".
|
||||
|
|
@ -36,13 +37,17 @@ from core.inference.tools import (
|
|||
|
||||
def test_note_does_not_claim_full_network_isolation():
|
||||
lowered = _SANDBOX_PATHS_NOTE.lower()
|
||||
# The old absolute claim is false for the python tool (allowlisted egress).
|
||||
# The old absolute claim is false for the python tool (it does reach egress).
|
||||
assert "cannot reach other machines or remote hosts" not in lowered
|
||||
# It should instead scope the block to private/arbitrary hosts and name the
|
||||
# allowlist so the model still fetches valid public URLs.
|
||||
assert "allowlist" in lowered
|
||||
# It must not overstate the host check as an enforced hard boundary:
|
||||
# _sandbox_preexec leaves networking on and the AST check only inspects
|
||||
# literal hosts, so a dynamically built request to a private host still runs.
|
||||
assert "only from a fixed allowlist" not in lowered
|
||||
assert "arbitrary addresses" not in lowered
|
||||
# It should still steer to public sources so the model fetches valid URLs.
|
||||
assert "public sources" in lowered
|
||||
assert "github.com" in lowered and "huggingface.co" in lowered
|
||||
assert "arbitrary" in lowered or "private" in lowered
|
||||
assert "private" in lowered
|
||||
|
||||
|
||||
def test_note_does_not_claim_project_sandbox_starts_empty():
|
||||
|
|
@ -78,7 +83,7 @@ def test_note_scopes_network_block_to_the_terminal_not_all_shell_commands():
|
|||
lowered = _SANDBOX_PATHS_NOTE.lower()
|
||||
assert "shell network commands are blocked" not in lowered
|
||||
assert "curl" in lowered and "wget" in lowered
|
||||
assert "python tool can fetch" in lowered
|
||||
assert "python tool is intended to fetch" in lowered
|
||||
|
||||
|
||||
def test_note_distinguishes_attachments_from_sandbox_uploads():
|
||||
|
|
@ -91,13 +96,17 @@ def test_note_distinguishes_attachments_from_sandbox_uploads():
|
|||
assert "attach" in lowered
|
||||
|
||||
|
||||
def test_blocked_network_command_message_points_to_python_egress():
|
||||
# A blocked curl/wget must not tell the model the whole sandbox is offline: the
|
||||
# python tool can still fetch allowlisted public hosts.
|
||||
def test_blocked_network_command_message_gates_code_fallback_on_tool_availability():
|
||||
# A blocked curl/wget must not tell the model the whole sandbox is offline, and
|
||||
# must not name a specific tool (e.g. python) as the remedy: when only the
|
||||
# terminal is enabled the python tool is absent from the schema, so an
|
||||
# unconditional "fetch it from Python code" instruction invites an invalid tool
|
||||
# call. The fallback is gated on a code-execution tool being enabled this turn.
|
||||
msg = _bash_exec("curl https://raw.githubusercontent.com/foo/bar/main/x.py").lower()
|
||||
assert "blocked command" in msg
|
||||
assert "cannot reach other machines or remote hosts" not in msg
|
||||
assert "python" in msg
|
||||
assert "from python code instead" not in msg
|
||||
assert "if a code-execution tool is enabled this turn" in msg
|
||||
assert "github.com" in msg
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue