* Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix race in tool-call confirmation gate * Studio: gate built-in tool calls and harden the confirmation handshake The Allow / Always allow / Deny controls only lived in the fallback tool card, but the built-in tools (web search, python, terminal, code execution, image generation) render with their own components and so never showed the buttons. Those calls paused after tool_start with no way to approve them, hanging until the 1 hour timeout. Only MCP tools, which use the fallback renderer, actually worked. Render the controls for every tool card by wrapping each registered tool component (and the fallback) in thread.tsx with a shared ToolConfirmationControls, so the gate applies uniformly. Also make the handshake robust: - The gate keys on a per-call approval_id minted by the backend and echoed in tool_start, instead of session_id alone, so a stale or concurrent confirmation can no longer resolve the wrong call. - The approval slot is registered before tool_start is yielded, closing the race where a fast click or an auto "Always allow" could reach the backend before the waiter existed. - The frontend resolves with the same session id the request was sent with (plus the approval_id), fixing the new-thread mismatch where the confirmation targeted a different session than the blocked stream. - The confirm endpoint returns {resolved}; the UI keeps the buttons and shows a retry hint until the backend confirms a match, instead of hiding them on a failed or mistargeted post. - The gate runs after the disabled-tool and duplicate-call checks, so a call that will not execute is not put up for approval. A denied call is still excluded from duplicate detection, so re-issuing and approving it works. - "Always allow" is scoped per session to match the backend gate. Add backend tests for the approval registry, the SSE no-deadlock handshake, and the loop integration (allow, deny, disabled, duplicate, re-issue after deny). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Move "Confirm tool calls" to the Tools section * Studio: add Bypass Permissions (skip confirmation, disable tool sandbox) Adds an opt-in Bypass Permissions toggle next to Confirm tool calls. When on, no tool call shows a confirmation prompt and the python/terminal sandbox is disabled: safety checks, command blocklist, and resource limits are skipped. Secret env vars are still stripped and HOME stays repointed at the session workdir. Default off keeps current behavior, and it takes precedence over Confirm tool calls. Enabling it requires accepting a warning each time. * Studio: harden Bypass Permissions secret handling and fix Anthropic tool path Follow-up to the Bypass Permissions feature. Addresses the review findings: - Anthropic /v1/messages 500: declare bypass_permissions on AnthropicMessagesRequest so tool requests that omit the field default to False instead of raising AttributeError (extra='allow' does not set absent attributes). - /proc parent-env leak: stripping the child env did not stop a same-uid bypassed child from reading /proc/<parent>/environ to recover the tool-executing process's unfiltered secrets. Clear PR_SET_DUMPABLE on that process before the first bypass exec so its /proc entries become root-owned. Hardening is fail-closed: if prctl is denied, bypass execution is refused rather than run with the parent environ still readable. Mitigation, not a full boundary; documented in the code. - Broker/capability vars: strip SSH_AUTH_SOCK, SSH_AGENT_PID, GPG_AGENT_INFO, GNUPGHOME, KUBECONFIG, DOCKER_HOST so a bypassed tool cannot use the operator's live agents. - Credential-bearing URL values: drop any env var whose value embeds URL userinfo (scheme://user:pass@ and token-only scheme://token@) regardless of the variable name. Benign proxy/index URLs without credentials are kept, so proxy-only and internal-index setups still work in bypass mode. - Windows temp isolation: repoint TEMP and TMP (not just TMPDIR) at the per-session sandbox dir. - Frontend: stop persisting bypassPermissions; a reload now starts with the sandbox/confirmation bypass off and requires re-accepting the warning dialog. Adds regression tests for each finding in test_bypass_permissions.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: strip cred-location env vars (HF_HOME etc.) in Bypass Permissions Repointing HOME did not stop SDKs auto-reading cached creds via vars that point at the real home/cache/config: HF_HOME (startup always sets it; token lives under $HF_HOME/token), HF/XDG cache roots, NETRC/BOTO_CONFIG/ PIP_CONFIG_FILE, and Windows HOMEDRIVE/HOMEPATH. Drop those, and repoint USERPROFILE/APPDATA/LOCALAPPDATA at the per-session workdir. Adds regression tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: lock in bypass HF token resolution with an end-to-end test The drop-based fix relies on the whole HF_HOME/XDG fallback chain being removed so huggingface_hub resolves under the repointed HOME. Add a test that sets HF_HOME and XDG_CACHE_HOME at a real cache and asserts the resolved token path lands under the workdir, not the operator's cache. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: strip npm _auth, MYSQL_PWD, and BASH_ENV from bypass env Three more credential vectors dodged the bypass scrubber: NPM_CONFIG__AUTH (npm _auth, base64 so no URL userinfo and no AUTH marker), MYSQL_PWD (markers use PASSWD, not PWD, since PWD is the cwd var), and BASH_ENV (bash -c sources it for non-interactive shells, so a startup file can re-export stripped secrets). Add an AUTH marker, the exact MYSQL_PWD name, and drop BASH_ENV plus PGPASSFILE. Adds regression tests incl. an end-to-end check that a bypass terminal call does not source BASH_ENV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: extend bypass env scrubber and enforce confirm precedence in loops From a parallel review pass over the bypass changes: - Drop more credential-location vars in _build_bypass_env: npm/yarn/git/cargo/ rclone config pointers (NPM_CONFIG_USERCONFIG, NPM_CONFIG_GLOBALCONFIG, YARN_RC_FILENAME, GIT_CONFIG_GLOBAL, GIT_CONFIG_SYSTEM, CARGO_HOME, RCLONE_CONFIG) and the GIT_ASKPASS/SSH_ASKPASS auth helpers. - Enforce confirm_tool_calls AND NOT bypass_permissions inside the safetensors and GGUF tool loops, not just at the route, so a direct internal caller passing both flags never prompts. - Soften the toggle hint: environment secrets are stripped, but bypassed code can still read files and credentials on the machine (no overclaim that keys stay hidden). Adds regression tests for the new names and the loop-level precedence. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: add GGUF loop test for bypass-over-confirm precedence The safetensors loop precedence is covered behaviorally; the GGUF loop needs a live llama-server so add an AST guard asserting its _needs_confirm gate references both confirm_tool_calls and bypass_permissions, matching the other llama_cpp source-inspection tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: add red Bypass Permissions badge in the composer When Bypass Permissions is on, show a persistent red pill in the composer tool-pill row (like the Search/Code pills), matching Claude Code's always- visible bypass indicator. Clicking it turns bypass off, mirroring the other composer toggles. Enabling still goes through the settings toggle + warning dialog. Adds a data-variant=danger style for the destructive-colored pill. * Studio: show Bypass Permissions badge in the Thread composer too The empty-state and active Thread render their own composer (thread.tsx), not shared-composer, so the badge only appeared in the split layout. Mirror the red dismissible pill in ComposerAction so it shows in every composer. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep the Bypass Permissions badge visible when the composer is collapsed The Thread composer only renders the pill row when expanded, so the active-mode badge vanished on the default (collapsed) empty state. Render it before the expand gate (it returns null when bypass is off) so the red indicator always shows while bypass is on. * Studio: make the Bypass Permissions confirm button a solid red button The destructive button variant is a subtle 10% tint that read as bare red text next to the outlined Cancel. Force the solid destructive fill (the variant's class loses to the tint through AlertDialogAction's Slot merge, so use the ! override the codebase already uses for this case) and shorten the label to 'I understand' so it fits the small dialog's two-column footer. * Studio: add Bypass Permissions to the composer + More menu Adds a 'Bypass Permissions' entry to the composer plus-menu (under More by default) in both composers, so it can be toggled without opening Run settings. Enabling routes through the same danger warning dialog; disabling is immediate. A shared BypassPermissionsMenuItem keeps the two composers in sync. * Studio: harden bypass env scrubber for IMDS opt-out and connection strings Two gaps in the Bypass Permissions secret scrubber: - The broad AWS_ prefix also dropped AWS_EC2_METADATA_DISABLED, a non-secret opt-out. Removing it re-opens the IMDS instance-role credential path that the operator explicitly disabled, so a bypassed boto/AWS-CLI call could recover cloud creds. Keep that flag (and AWS_EC2_METADATA_V1_DISABLED) via a keep-list while still stripping the real AWS credential vars. - Azure App Service connection strings (SQLCONNSTR_/CUSTOMCONNSTR_/..., WEBSITE_CONTENTAZUREFILECONNECTIONSTRING) and values like Password=/AccountKey= /SharedAccessKey= slipped past the name and URL-only value classifiers. Add CONNSTR/CONNECTIONSTRING name markers and a connection-string value matcher. * Studio: let Bypass Permissions suppress the confirm-tool-calls guards The confirm-vs-bypass precedence (confirm and not bypass) was applied at the loop call sites but not at the earlier request guards, so a client sending confirm_tool_calls + bypass_permissions together was rejected (stream=true required / unsupported for external or Anthropic tools) before the precedence took effect. Gate all four confirm guards on not bypass_permissions so both flags together proceed with the gate suppressed, matching the documented rule. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
732 lines
27 KiB
Python
732 lines
27 KiB
Python
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
|
|
|
"""Tests for Bypass Permissions (skip confirmation + disable sandbox).
|
|
|
|
Covers the secret-name classifier, the two env builders, the
|
|
``disable_sandbox`` branch of ``_python_exec`` / ``_bash_exec`` (which env is
|
|
used, which pre-exec is used, and that safety checks / the blocklist are
|
|
skipped), the request-model default, the confirm-vs-bypass precedence rule the
|
|
route enforces, and that the agentic loop forwards ``disable_sandbox`` while
|
|
never gating under bypass.
|
|
|
|
Run with: ``PYTHONPATH=studio/backend python -m pytest studio/backend/tests/test_bypass_permissions.py -q``
|
|
"""
|
|
|
|
import os
|
|
import sys
|
|
|
|
import pytest
|
|
|
|
import core.inference.tools as tools
|
|
from core.inference.tools import (
|
|
_bash_exec,
|
|
_build_bypass_env,
|
|
_build_safe_env,
|
|
_is_cred_location_env_name,
|
|
_is_secret_env_name,
|
|
_is_secret_env_value,
|
|
_python_exec,
|
|
)
|
|
from core.inference.safetensors_agentic import run_safetensors_tool_loop
|
|
|
|
_POSIX_ONLY = pytest.mark.skipif(
|
|
sys.platform == "win32", reason = "preexec_fn / setsid are POSIX-only"
|
|
)
|
|
|
|
|
|
# ── secret-name classifier ──────────────────────────────────────────
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"name",
|
|
[
|
|
"HF_TOKEN",
|
|
"HUGGING_FACE_HUB_TOKEN",
|
|
"WANDB_API_KEY",
|
|
"GH_TOKEN",
|
|
"GITHUB_TOKEN",
|
|
"OPENAI_API_KEY",
|
|
"ANTHROPIC_API_KEY",
|
|
"AWS_SECRET_ACCESS_KEY",
|
|
"AWS_ACCESS_KEY_ID",
|
|
"AZURE_CLIENT_SECRET",
|
|
"GOOGLE_APPLICATION_CREDENTIALS",
|
|
"MY_DB_PASSWORD",
|
|
"x_api_key",
|
|
"SOME_PRIVATE_KEY",
|
|
"LD_PRELOAD",
|
|
],
|
|
)
|
|
def test_secret_names_are_flagged(name):
|
|
assert _is_secret_env_name(name) is True
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"name", ["PATH", "HOME", "LANG", "TERM", "PWD", "SHELL", "HOSTVAR", "MY_VAR"]
|
|
)
|
|
def test_benign_names_are_not_flagged(name):
|
|
assert _is_secret_env_name(name) is False
|
|
|
|
|
|
# ── env builders ────────────────────────────────────────────────────
|
|
|
|
|
|
def test_bypass_env_keeps_benign_strips_secret_repoints_home(monkeypatch, tmp_path):
|
|
monkeypatch.setenv("HOSTVAR", "benign-123")
|
|
monkeypatch.setenv("HF_TOKEN", "secret-abc")
|
|
env = _build_bypass_env(str(tmp_path))
|
|
assert env.get("HOSTVAR") == "benign-123" # full host env inherited
|
|
assert "HF_TOKEN" not in env # ...minus secrets
|
|
assert env["HOME"] == str(tmp_path) # $HOME-based cred lookups defused
|
|
assert env["TMPDIR"] == str(tmp_path)
|
|
|
|
|
|
def test_safe_env_excludes_host_and_secret(monkeypatch, tmp_path):
|
|
monkeypatch.setenv("HOSTVAR", "benign-123")
|
|
monkeypatch.setenv("HF_TOKEN", "secret-abc")
|
|
env = _build_safe_env(str(tmp_path))
|
|
assert "HOSTVAR" not in env # whitelist build -> host vars never reach child
|
|
assert "HF_TOKEN" not in env
|
|
|
|
|
|
# ── Popen kwargs capture (no real execution) ────────────────────────
|
|
|
|
|
|
class _FakeProc:
|
|
returncode = 0
|
|
|
|
def communicate(self, timeout = None):
|
|
return ("FAKEOUT", None)
|
|
|
|
def poll(self):
|
|
return 0
|
|
|
|
def kill(self):
|
|
pass
|
|
|
|
|
|
@pytest.fixture
|
|
def captured_popen(monkeypatch):
|
|
cap = {}
|
|
|
|
def fake_popen(cmd, **kwargs):
|
|
cap["cmd"] = cmd
|
|
cap["kwargs"] = kwargs
|
|
return _FakeProc()
|
|
|
|
monkeypatch.setattr(tools.subprocess, "Popen", fake_popen)
|
|
return cap
|
|
|
|
|
|
@_POSIX_ONLY
|
|
def test_python_sandboxed_uses_sandbox_preexec_and_safe_env(captured_popen, monkeypatch):
|
|
monkeypatch.setenv("HF_TOKEN", "secret-abc")
|
|
_python_exec("print(1)", None, 5, "t", disable_sandbox = False)
|
|
assert captured_popen["kwargs"]["preexec_fn"] is tools._sandbox_preexec
|
|
assert "HF_TOKEN" not in captured_popen["kwargs"]["env"]
|
|
|
|
|
|
@_POSIX_ONLY
|
|
def test_python_bypass_uses_bypass_preexec_and_bypass_env(captured_popen, monkeypatch):
|
|
monkeypatch.setenv("HOSTVAR", "benign-xyz")
|
|
monkeypatch.setenv("HF_TOKEN", "secret-abc")
|
|
_python_exec("print(1)", None, 5, "t", disable_sandbox = True)
|
|
assert captured_popen["kwargs"]["preexec_fn"] is tools._bypass_preexec
|
|
env = captured_popen["kwargs"]["env"]
|
|
assert env.get("HOSTVAR") == "benign-xyz"
|
|
assert "HF_TOKEN" not in env
|
|
|
|
|
|
def test_bash_blocklist_enforced_when_sandboxed(captured_popen):
|
|
out = _bash_exec("rm -rf /", None, 5, "t", disable_sandbox = False)
|
|
assert "Blocked" in out
|
|
assert "cmd" not in captured_popen # never reached Popen
|
|
|
|
|
|
def test_bash_blocklist_skipped_when_bypassed(captured_popen):
|
|
out = _bash_exec("rm -rf /", None, 5, "t", disable_sandbox = True)
|
|
assert out == "FAKEOUT" # blocklist skipped -> reached (faked) execution
|
|
assert captured_popen["cmd"][0] in ("bash", "cmd")
|
|
|
|
|
|
@_POSIX_ONLY
|
|
def test_bash_bypass_uses_bypass_preexec(captured_popen):
|
|
_bash_exec("echo hi", None, 5, "t", disable_sandbox = True)
|
|
assert captured_popen["kwargs"]["preexec_fn"] is tools._bypass_preexec
|
|
|
|
|
|
# ── real end-to-end python execution under bypass ───────────────────
|
|
|
|
|
|
@_POSIX_ONLY
|
|
def test_python_bypass_real_exec_sees_host_env_but_not_secret(monkeypatch):
|
|
monkeypatch.setenv("HOSTVAR", "benign-xyz")
|
|
monkeypatch.setenv("HF_TOKEN", "secret-pqr")
|
|
code = (
|
|
"import os;"
|
|
"print('H=' + str(os.environ.get('HOSTVAR')),"
|
|
" 'T=' + str(os.environ.get('HF_TOKEN')))"
|
|
)
|
|
out = _python_exec(code, None, 30, "test-bypass", disable_sandbox = True)
|
|
assert "H=benign-xyz" in out # unrestricted: real host var visible
|
|
assert "T=None" in out # ...but the secret was stripped
|
|
assert "secret-pqr" not in out
|
|
|
|
|
|
# ── _bypass_preexec is setsid-only (no rlimits) ─────────────────────
|
|
|
|
|
|
@_POSIX_ONLY
|
|
def test_bypass_preexec_only_sets_session(monkeypatch):
|
|
calls = {"setsid": 0}
|
|
monkeypatch.setattr(
|
|
tools.os, "setsid", lambda: calls.__setitem__("setsid", calls["setsid"] + 1)
|
|
)
|
|
# _resource must not be touched by the bypass pre-exec.
|
|
if tools._resource is not None:
|
|
monkeypatch.setattr(
|
|
tools._resource,
|
|
"setrlimit",
|
|
lambda *a, **k: pytest.fail("bypass pre-exec must not set rlimits"),
|
|
)
|
|
tools._bypass_preexec()
|
|
assert calls["setsid"] == 1
|
|
|
|
|
|
# ── request model default ───────────────────────────────────────────
|
|
|
|
|
|
def test_request_model_bypass_default_false():
|
|
from models.inference import ChatCompletionRequest
|
|
assert ChatCompletionRequest.model_fields["bypass_permissions"].default is False
|
|
|
|
|
|
# ── confirm-vs-bypass precedence (mirrors the route rule) ───────────
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"confirm,bypass,effective_confirm",
|
|
[
|
|
(False, False, False),
|
|
(True, False, True),
|
|
(False, True, False),
|
|
(True, True, False),
|
|
],
|
|
)
|
|
def test_confirm_precedence_rule(confirm, bypass, effective_confirm):
|
|
# The route computes: confirm_tool_calls = confirm and not bypass.
|
|
assert (bool(confirm) and not bool(bypass)) is effective_confirm
|
|
|
|
|
|
# ── agentic loop forwards disable_sandbox, never gates under bypass ──
|
|
|
|
_DEFAULT_TOOLS = [
|
|
{"type": "function", "function": {"name": "python"}},
|
|
{"type": "function", "function": {"name": "web_search"}},
|
|
]
|
|
|
|
|
|
def _tool_call(name, args_json):
|
|
return f'<tool_call>{{"name": "{name}", "arguments": {args_json}}}</tool_call>'
|
|
|
|
|
|
def _multi_turn(turns):
|
|
it = iter(turns)
|
|
|
|
def _gen(_messages):
|
|
try:
|
|
yield next(it)
|
|
except StopIteration:
|
|
return
|
|
|
|
return _gen
|
|
|
|
|
|
def test_loop_forwards_disable_sandbox_and_does_not_gate():
|
|
seen = []
|
|
|
|
def fake_exec(
|
|
name,
|
|
arguments,
|
|
*,
|
|
cancel_event = None,
|
|
timeout = None,
|
|
session_id = None,
|
|
rag_scope = None,
|
|
disable_sandbox = False,
|
|
):
|
|
seen.append(disable_sandbox)
|
|
return f"RAN[{name}]"
|
|
|
|
events = list(
|
|
run_safetensors_tool_loop(
|
|
single_turn = _multi_turn([_tool_call("python", '{"code": "x"}'), "done"]),
|
|
messages = [{"role": "user", "content": "hi"}],
|
|
tools = _DEFAULT_TOOLS,
|
|
execute_tool = fake_exec,
|
|
session_id = "s",
|
|
confirm_tool_calls = False, # route forces this off under bypass
|
|
bypass_permissions = True,
|
|
)
|
|
)
|
|
assert seen == [True] # disable_sandbox threaded through
|
|
starts = [e for e in events if e["type"] == "tool_start"]
|
|
assert starts and starts[0]["awaiting_confirmation"] is False
|
|
assert starts[0]["approval_id"] == ""
|
|
|
|
|
|
def test_loop_bypass_overrides_confirm_for_direct_callers():
|
|
# Even if a direct internal caller passes confirm_tool_calls=True, bypass
|
|
# must suppress the confirm gate at the loop level (not only at the route).
|
|
def fake_exec(
|
|
name,
|
|
arguments,
|
|
*,
|
|
cancel_event = None,
|
|
timeout = None,
|
|
session_id = None,
|
|
rag_scope = None,
|
|
disable_sandbox = False,
|
|
):
|
|
return f"RAN[{name}]"
|
|
|
|
events = list(
|
|
run_safetensors_tool_loop(
|
|
single_turn = _multi_turn([_tool_call("python", '{"code": "x"}'), "done"]),
|
|
messages = [{"role": "user", "content": "hi"}],
|
|
tools = _DEFAULT_TOOLS,
|
|
execute_tool = fake_exec,
|
|
session_id = "s",
|
|
confirm_tool_calls = True, # raw caller leaves this on...
|
|
bypass_permissions = True, # ...but bypass must still win
|
|
)
|
|
)
|
|
starts = [e for e in events if e["type"] == "tool_start"]
|
|
assert starts and starts[0]["awaiting_confirmation"] is False
|
|
assert starts[0]["approval_id"] == ""
|
|
|
|
|
|
def test_gguf_loop_confirm_gate_respects_bypass():
|
|
# The GGUF loop needs a live llama-server, so (per the other llama_cpp
|
|
# tests) assert via AST that its _needs_confirm gate applies the bypass
|
|
# precedence, mirroring the safetensors behavioral test above.
|
|
import ast
|
|
import inspect
|
|
import textwrap
|
|
|
|
llama_cpp = pytest.importorskip("core.inference.llama_cpp")
|
|
src = textwrap.dedent(
|
|
inspect.getsource(llama_cpp.LlamaCppBackend.generate_chat_completion_with_tools)
|
|
)
|
|
gates = [
|
|
node
|
|
for node in ast.walk(ast.parse(src))
|
|
if isinstance(node, ast.Assign)
|
|
and any(getattr(t, "id", None) == "needs_confirm" for t in node.targets)
|
|
]
|
|
assert gates, "could not find the needs_confirm gate in the GGUF loop"
|
|
names = {n.id for g in gates for n in ast.walk(g.value) if isinstance(n, ast.Name)}
|
|
assert "confirm_tool_calls" in names
|
|
assert "bypass_permissions" in names # bypass must suppress the GGUF gate
|
|
|
|
|
|
# ── broker / capability env vars are stripped (regression) ──────────
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"name",
|
|
["SSH_AUTH_SOCK", "SSH_AGENT_PID", "GPG_AGENT_INFO", "GNUPGHOME", "KUBECONFIG"],
|
|
)
|
|
def test_broker_capability_names_are_flagged(name):
|
|
# Not secrets by value, but they hand the child the operator's live agent
|
|
# (ssh/gpg) or kube credentials, so bypass mode must drop them.
|
|
assert _is_secret_env_name(name) is True
|
|
|
|
|
|
# ── credential-bearing URL values stripped regardless of name ───────
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"value",
|
|
[
|
|
"https://user:s3cr3t@feed.example.invalid/simple", # user:pass@
|
|
"https://ghp_deadbeef@github.com/org/private.git", # token-only@
|
|
"https://__token__@pypi.example.invalid/simple",
|
|
"https://ghp_1234:@npm.pkg.github.com/simple", # empty password
|
|
"postgres://dbuser:dbpass@db.example.invalid/app",
|
|
],
|
|
)
|
|
def test_url_userinfo_values_are_flagged(value):
|
|
assert _is_secret_env_value(value) is True
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"value",
|
|
[
|
|
"https://example.invalid/simple", # no userinfo
|
|
"http://proxy.corp.example:8080", # benign proxy
|
|
"https://pypi.corp.example/simple", # benign internal index
|
|
"redis://localhost:6379/0", # no creds
|
|
"https://example.invalid/path?ref=a@b", # '@' only in query, not userinfo
|
|
],
|
|
)
|
|
def test_non_credential_url_values_are_not_flagged(value):
|
|
assert _is_secret_env_value(value) is False
|
|
|
|
|
|
def test_url_userinfo_value_is_stripped_even_with_benign_name(monkeypatch, tmp_path):
|
|
# NAME dodges the classifier, but the VALUE embeds userinfo -> must go.
|
|
monkeypatch.setenv("MY_FEED", "https://user:s3cr3t@feed.example.invalid/simple")
|
|
monkeypatch.setenv("REPO_URL", "https://ghp_deadbeef@github.com/org/private.git")
|
|
# A URL without credentials is harmless and should be kept.
|
|
monkeypatch.setenv("PLAIN_URL", "https://example.invalid/simple")
|
|
env = _build_bypass_env(str(tmp_path))
|
|
assert "MY_FEED" not in env
|
|
assert "REPO_URL" not in env
|
|
assert env.get("PLAIN_URL") == "https://example.invalid/simple"
|
|
|
|
|
|
def test_bypass_env_keeps_noncredential_proxy_and_index_urls(monkeypatch, tmp_path):
|
|
# Benign routing/config vars must survive bypass mode (proxy-only or
|
|
# internal-index networks); only credentialed values are dropped.
|
|
monkeypatch.setenv("HTTP_PROXY", "http://proxy.corp.example:8080")
|
|
monkeypatch.setenv("PIP_INDEX_URL", "https://pypi.corp.example/simple")
|
|
monkeypatch.setenv("PIP_EXTRA_INDEX_URL", "https://user:token@pypi.example.invalid/simple")
|
|
env = _build_bypass_env(str(tmp_path))
|
|
assert env["HTTP_PROXY"] == "http://proxy.corp.example:8080"
|
|
assert env["PIP_INDEX_URL"] == "https://pypi.corp.example/simple"
|
|
assert "PIP_EXTRA_INDEX_URL" not in env # this one carries credentials
|
|
|
|
|
|
# ── AWS IMDS-disable hardening flag is kept (regression) ────────────
|
|
|
|
|
|
def test_aws_imds_disable_flag_is_kept_but_creds_stripped(monkeypatch, tmp_path):
|
|
# AWS_EC2_METADATA_DISABLED is a non-secret opt-out: dropping it would let a
|
|
# bypassed boto/AWS-CLI call fall back to the instance role via IMDS even
|
|
# though the operator disabled that path. Keep it; drop the real creds.
|
|
monkeypatch.setenv("AWS_EC2_METADATA_DISABLED", "true")
|
|
monkeypatch.setenv("AWS_ACCESS_KEY_ID", "AKIAEXAMPLE")
|
|
monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "shhh")
|
|
assert _is_secret_env_name("AWS_EC2_METADATA_DISABLED") is False
|
|
assert _is_secret_env_name("AWS_ACCESS_KEY_ID") is True
|
|
env = _build_bypass_env(str(tmp_path))
|
|
assert env.get("AWS_EC2_METADATA_DISABLED") == "true"
|
|
assert "AWS_ACCESS_KEY_ID" not in env
|
|
assert "AWS_SECRET_ACCESS_KEY" not in env
|
|
|
|
|
|
# ── connection-string env vars are stripped (regression) ────────────
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"name",
|
|
[
|
|
"SQLCONNSTR_DB", # Azure App Service injected connection strings
|
|
"MYSQLCONNSTR_DB",
|
|
"SQLAZURECONNSTR_DB",
|
|
"POSTGRESQLCONNSTR_DB",
|
|
"CUSTOMCONNSTR_CACHE",
|
|
"WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
|
|
],
|
|
)
|
|
def test_connection_string_names_are_flagged(name):
|
|
assert _is_secret_env_name(name) is True
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"value",
|
|
[
|
|
"Server=tcp:db;Database=app;User ID=u;Password=p@ss;", # ADO.NET
|
|
"DefaultEndpointsProtocol=https;AccountName=x;AccountKey=abc123==;", # storage
|
|
"Endpoint=sb://x;SharedAccessKeyName=n;SharedAccessKey=zzz=", # Service Bus
|
|
],
|
|
)
|
|
def test_connection_string_values_are_flagged(value):
|
|
assert _is_secret_env_value(value) is True
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"value",
|
|
[
|
|
"Server=tcp:db;Database=app;User ID=u;", # no password field
|
|
"Endpoint=sb://x;SharedAccessKeyName=n", # key NAME only, no secret
|
|
"AccountName=x;EndpointSuffix=core.windows.net", # no AccountKey
|
|
],
|
|
)
|
|
def test_connection_string_noncredential_values_are_not_flagged(value):
|
|
assert _is_secret_env_value(value) is False
|
|
|
|
|
|
def test_connection_string_value_stripped_even_with_benign_name(monkeypatch, tmp_path):
|
|
# NAME dodges the classifier, but the VALUE is a credentialed conn string.
|
|
monkeypatch.setenv("APP_DB", "Server=tcp:db;Database=app;User ID=u;Password=p@ss;")
|
|
monkeypatch.setenv("SQLCONNSTR_DB", "DefaultEndpointsProtocol=https;AccountKey=abc==")
|
|
env = _build_bypass_env(str(tmp_path))
|
|
assert "APP_DB" not in env # value-based catch
|
|
assert "SQLCONNSTR_DB" not in env # name-based catch
|
|
|
|
|
|
# ── temp dirs repointed on every platform (regression) ──────────────
|
|
|
|
|
|
def test_bypass_env_repoints_all_temp_vars(monkeypatch, tmp_path):
|
|
# Windows tempfile honours TEMP/TMP, not TMPDIR; all three must repoint.
|
|
monkeypatch.setenv("TEMP", "/host/tmp")
|
|
monkeypatch.setenv("TMP", "/host/tmp")
|
|
env = _build_bypass_env(str(tmp_path))
|
|
assert env["TMPDIR"] == str(tmp_path)
|
|
assert env["TEMP"] == str(tmp_path)
|
|
assert env["TMP"] == str(tmp_path)
|
|
|
|
|
|
# ── credential-location redirect vars are dropped (regression) ──────────
|
|
# Vars that point SDKs at the real home/cache/config (cached tokens), e.g.
|
|
# HF_HOME which startup always sets -> the live leak the HOME repoint missed.
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"name",
|
|
[
|
|
"HF_HOME",
|
|
"HF_HUB_CACHE",
|
|
"HUGGINGFACE_HUB_CACHE",
|
|
"HF_XET_CACHE",
|
|
"TRANSFORMERS_CACHE",
|
|
"HF_DATASETS_CACHE",
|
|
"XDG_CONFIG_HOME",
|
|
"XDG_CACHE_HOME",
|
|
"XDG_DATA_HOME",
|
|
"NETRC",
|
|
"BOTO_CONFIG",
|
|
"PIP_CONFIG_FILE",
|
|
"CLOUDSDK_CONFIG",
|
|
"KAGGLE_CONFIG_DIR",
|
|
"DOCKER_CONFIG",
|
|
"WANDB_DIR",
|
|
"WANDB_CONFIG_DIR",
|
|
"NPM_CONFIG_USERCONFIG",
|
|
"NPM_CONFIG_GLOBALCONFIG",
|
|
"YARN_RC_FILENAME",
|
|
"GIT_CONFIG_GLOBAL",
|
|
"GIT_CONFIG_SYSTEM",
|
|
"CARGO_HOME",
|
|
"RCLONE_CONFIG",
|
|
"GIT_ASKPASS",
|
|
"SSH_ASKPASS",
|
|
"BASH_ENV",
|
|
"HOMEDRIVE",
|
|
"HOMEPATH",
|
|
],
|
|
)
|
|
def test_cred_location_names_are_flagged(name):
|
|
assert _is_cred_location_env_name(name) is True
|
|
|
|
|
|
@pytest.mark.parametrize("name", ["PATH", "HOME", "LANG", "PWD", "MY_VAR"])
|
|
def test_benign_names_not_flagged_as_cred_location(name):
|
|
assert _is_cred_location_env_name(name) is False
|
|
|
|
|
|
def test_bypass_env_drops_hf_home_so_cached_token_unreachable(monkeypatch, tmp_path):
|
|
# The live leak: startup sets HF_HOME at the real cache, whose $HF_HOME/token
|
|
# holds the operator's token. Repointing HOME does not stop huggingface_hub
|
|
# from reading $HF_HOME/token, so HF_HOME must be dropped in bypass mode.
|
|
real_cache = tmp_path / "real_hf_cache"
|
|
real_cache.mkdir()
|
|
(real_cache / "token").write_text("hf_cachedOperatorToken")
|
|
monkeypatch.setenv("HF_HOME", str(real_cache))
|
|
monkeypatch.setenv("HF_HUB_CACHE", str(real_cache / "hub"))
|
|
env = _build_bypass_env(str(tmp_path))
|
|
assert "HF_HOME" not in env # dropped -> HF falls back to $HOME/.cache (empty)
|
|
assert "HF_HUB_CACHE" not in env
|
|
|
|
|
|
def test_bypass_env_hf_token_resolves_outside_real_cache(monkeypatch, tmp_path):
|
|
# End-to-end: even when HF_HOME and XDG_CACHE_HOME both point at the real
|
|
# cache, the bypass env must make huggingface_hub resolve the token under the
|
|
# workdir (guards the XDG fallback chain, not just "HF_HOME absent").
|
|
pytest.importorskip("huggingface_hub")
|
|
import subprocess
|
|
|
|
real_cache = tmp_path / "real_hf"
|
|
real_cache.mkdir()
|
|
workdir = tmp_path / "sandbox"
|
|
workdir.mkdir()
|
|
monkeypatch.setenv("HF_HOME", str(real_cache))
|
|
monkeypatch.setenv("XDG_CACHE_HOME", str(real_cache))
|
|
monkeypatch.setenv("XDG_CONFIG_HOME", str(real_cache))
|
|
env = _build_bypass_env(str(workdir))
|
|
token_path = subprocess.run(
|
|
[
|
|
sys.executable,
|
|
"-c",
|
|
"import huggingface_hub.constants as c; print(c.HF_TOKEN_PATH)",
|
|
],
|
|
env = env,
|
|
capture_output = True,
|
|
text = True,
|
|
).stdout.strip()
|
|
assert str(real_cache) not in token_path # never the operator's cache
|
|
assert token_path.startswith(str(workdir)) # resolved under the sandbox
|
|
|
|
|
|
def test_bypass_env_drops_credential_config_path_vars(monkeypatch, tmp_path):
|
|
# NETRC / BOTO_CONFIG / PIP_CONFIG_FILE point clients at real credential
|
|
# files before $HOME, so they must not survive into the bypassed child.
|
|
monkeypatch.setenv("NETRC", "/home/op/.netrc")
|
|
monkeypatch.setenv("PGPASSFILE", "/home/op/.pgpass")
|
|
monkeypatch.setenv("BOTO_CONFIG", "/home/op/.boto")
|
|
monkeypatch.setenv("PIP_CONFIG_FILE", "/home/op/.pip/pip.conf")
|
|
env = _build_bypass_env(str(tmp_path))
|
|
assert "NETRC" not in env
|
|
assert "PGPASSFILE" not in env
|
|
assert "BOTO_CONFIG" not in env
|
|
assert "PIP_CONFIG_FILE" not in env
|
|
|
|
|
|
def test_bypass_env_strips_npm_auth_and_mysql_pwd(monkeypatch, tmp_path):
|
|
# NPM_CONFIG__AUTH (npm _auth, base64) and MYSQL_PWD dodge the URL-value
|
|
# check and the PASSWD marker, but must still be dropped.
|
|
monkeypatch.setenv("NPM_CONFIG__AUTH", "aGVsbG86c2VjcmV0")
|
|
monkeypatch.setenv("MYSQL_PWD", "db-password")
|
|
assert _is_secret_env_name("NPM_CONFIG__AUTH") is True
|
|
assert _is_secret_env_name("MYSQL_PWD") is True
|
|
env = _build_bypass_env(str(tmp_path))
|
|
assert "NPM_CONFIG__AUTH" not in env
|
|
assert "MYSQL_PWD" not in env
|
|
|
|
|
|
@_POSIX_ONLY
|
|
def test_bash_bypass_does_not_source_bash_env(monkeypatch, tmp_path):
|
|
# bash -c sources $BASH_ENV for non-interactive shells; an operator startup
|
|
# file could re-export stripped secrets, so a real bypass call must not see it.
|
|
startup = tmp_path / "startup.sh"
|
|
startup.write_text("export RECOVERED=leaked\n")
|
|
monkeypatch.setenv("BASH_ENV", str(startup))
|
|
out = _bash_exec("echo R=$RECOVERED", None, 30, "bash-env-test", disable_sandbox = True)
|
|
assert "R=leaked" not in out # BASH_ENV dropped -> startup not sourced
|
|
assert "R=" in out
|
|
|
|
|
|
def test_bypass_env_repoints_windows_profile_vars(monkeypatch, tmp_path):
|
|
# On Windows, SDKs read cached creds under USERPROFILE/APPDATA/LOCALAPPDATA,
|
|
# not $HOME. Set ones are repointed at the workdir; HOMEDRIVE/HOMEPATH drop.
|
|
monkeypatch.setenv("USERPROFILE", "/host/profile")
|
|
monkeypatch.setenv("APPDATA", "/host/profile/AppData/Roaming")
|
|
monkeypatch.setenv("LOCALAPPDATA", "/host/profile/AppData/Local")
|
|
monkeypatch.setenv("HOMEDRIVE", "C:")
|
|
monkeypatch.setenv("HOMEPATH", "\\Users\\op")
|
|
env = _build_bypass_env(str(tmp_path))
|
|
assert env["USERPROFILE"] == str(tmp_path)
|
|
assert env["APPDATA"] == str(tmp_path)
|
|
assert env["LOCALAPPDATA"] == str(tmp_path)
|
|
assert "HOMEDRIVE" not in env
|
|
assert "HOMEPATH" not in env
|
|
|
|
|
|
def test_bypass_env_does_not_add_unset_windows_profile_vars(monkeypatch, tmp_path):
|
|
# Only repoint Windows profile vars that were actually set (no pollution on
|
|
# Linux/macOS where they are absent).
|
|
monkeypatch.delenv("USERPROFILE", raising = False)
|
|
monkeypatch.delenv("APPDATA", raising = False)
|
|
monkeypatch.delenv("LOCALAPPDATA", raising = False)
|
|
env = _build_bypass_env(str(tmp_path))
|
|
assert "USERPROFILE" not in env
|
|
assert "APPDATA" not in env
|
|
assert "LOCALAPPDATA" not in env
|
|
|
|
|
|
# ── parent /proc env-leak hardening (regression) ────────────────────
|
|
|
|
|
|
@_POSIX_ONLY
|
|
def test_bypass_exec_hardens_parent_proc_env(monkeypatch, captured_popen):
|
|
# Stripping the child env is not enough: a same-UID child can read the
|
|
# parent's /proc environ. The exec paths must invoke the parent hardening
|
|
# when (and only when) the sandbox is disabled.
|
|
calls = {"n": 0}
|
|
|
|
def fake_harden():
|
|
calls["n"] += 1
|
|
return True
|
|
|
|
monkeypatch.setattr(tools, "_harden_parent_against_proc_env_leak", fake_harden)
|
|
_python_exec("print(1)", None, 5, "t", disable_sandbox = True)
|
|
_bash_exec("echo hi", None, 5, "t", disable_sandbox = True)
|
|
assert calls["n"] == 2
|
|
|
|
calls["n"] = 0
|
|
_python_exec("print(1)", None, 5, "t", disable_sandbox = False)
|
|
_bash_exec("echo hi", None, 5, "t", disable_sandbox = False)
|
|
assert calls["n"] == 0 # never hardened on the sandboxed path
|
|
|
|
|
|
def test_bypass_exec_fails_closed_when_hardening_fails(monkeypatch, captured_popen):
|
|
# If the parent cannot be hardened (e.g. prctl denied), the unsandboxed
|
|
# child must NOT run - otherwise the parent environ stays readable.
|
|
monkeypatch.setattr(tools, "_harden_parent_against_proc_env_leak", lambda: False)
|
|
out_py = _python_exec("print(1)", None, 5, "t", disable_sandbox = True)
|
|
out_sh = _bash_exec("echo hi", None, 5, "t", disable_sandbox = True)
|
|
assert "refusing bypass execution" in out_py
|
|
assert "refusing bypass execution" in out_sh
|
|
assert "cmd" not in captured_popen # never reached Popen
|
|
|
|
|
|
@_POSIX_ONLY
|
|
def test_proc_env_unreadable_after_hardening():
|
|
# Mechanism check: after hardening, a same-UID child can no longer read the
|
|
# parent process /proc environ. Restores the dumpable flag afterwards so the
|
|
# process-global state does not leak into later tests.
|
|
import subprocess
|
|
|
|
if tools._libc is None:
|
|
pytest.skip("no libc/prctl available")
|
|
pid = os.getpid()
|
|
probe = (
|
|
"try:\n"
|
|
f" open('/proc/{pid}/environ', 'rb').read()\n"
|
|
" print('READABLE')\n"
|
|
"except PermissionError:\n"
|
|
" print('DENIED')\n"
|
|
)
|
|
prev_dumpable = tools._libc.prctl(3, 0, 0, 0, 0) # PR_GET_DUMPABLE
|
|
prev_guard = tools._parent_proc_hardened
|
|
try:
|
|
# Establish a clean readable baseline: another test may have already
|
|
# cleared the dumpable flag on this process.
|
|
tools._libc.prctl(4, 1, 0, 0, 0) # PR_SET_DUMPABLE = 1
|
|
before = subprocess.run(
|
|
[sys.executable, "-c", probe], capture_output = True, text = True
|
|
).stdout.strip()
|
|
if before != "READABLE":
|
|
pytest.skip("/proc already restricted in this environment")
|
|
|
|
tools._parent_proc_hardened = False
|
|
assert tools._harden_parent_against_proc_env_leak() is True
|
|
|
|
after = subprocess.run(
|
|
[sys.executable, "-c", probe], capture_output = True, text = True
|
|
).stdout.strip()
|
|
assert after == "DENIED"
|
|
finally:
|
|
if prev_dumpable in (0, 1):
|
|
try:
|
|
tools._libc.prctl(4, prev_dumpable, 0, 0, 0)
|
|
except (OSError, AttributeError):
|
|
pass
|
|
tools._parent_proc_hardened = prev_guard
|
|
|
|
|
|
# ── Anthropic request model declares the field (regression) ─────────
|
|
|
|
|
|
def test_anthropic_request_model_bypass_default_false():
|
|
# Omitting the field on the Anthropic path must default to False rather than
|
|
# raising AttributeError (extra='allow' does not set absent attributes).
|
|
from models.inference import AnthropicMessagesRequest
|
|
|
|
assert AnthropicMessagesRequest.model_fields["bypass_permissions"].default is False
|
|
req = AnthropicMessagesRequest(model = "x", messages = [], max_tokens = 8)
|
|
assert bool(req.bypass_permissions) is False
|