unsloth/studio/backend/tests/test_bypass_permissions.py
Daniel Han ca0528d1f8
Studio: Bypass Permissions (skip confirmation, disable tool sandbox) (#5895)
* Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix race in tool-call confirmation gate

* Studio: gate built-in tool calls and harden the confirmation handshake

The Allow / Always allow / Deny controls only lived in the fallback tool
card, but the built-in tools (web search, python, terminal, code
execution, image generation) render with their own components and so
never showed the buttons. Those calls paused after tool_start with no way
to approve them, hanging until the 1 hour timeout. Only MCP tools, which
use the fallback renderer, actually worked.

Render the controls for every tool card by wrapping each registered tool
component (and the fallback) in thread.tsx with a shared
ToolConfirmationControls, so the gate applies uniformly.

Also make the handshake robust:
- The gate keys on a per-call approval_id minted by the backend and
  echoed in tool_start, instead of session_id alone, so a stale or
  concurrent confirmation can no longer resolve the wrong call.
- The approval slot is registered before tool_start is yielded, closing
  the race where a fast click or an auto "Always allow" could reach the
  backend before the waiter existed.
- The frontend resolves with the same session id the request was sent
  with (plus the approval_id), fixing the new-thread mismatch where the
  confirmation targeted a different session than the blocked stream.
- The confirm endpoint returns {resolved}; the UI keeps the buttons and
  shows a retry hint until the backend confirms a match, instead of
  hiding them on a failed or mistargeted post.
- The gate runs after the disabled-tool and duplicate-call checks, so a
  call that will not execute is not put up for approval. A denied call is
  still excluded from duplicate detection, so re-issuing and approving it
  works.
- "Always allow" is scoped per session to match the backend gate.

Add backend tests for the approval registry, the SSE no-deadlock
handshake, and the loop integration (allow, deny, disabled, duplicate,
re-issue after deny).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Move "Confirm tool calls" to the Tools section

* Studio: add Bypass Permissions (skip confirmation, disable tool sandbox)

Adds an opt-in Bypass Permissions toggle next to Confirm tool calls. When on,
no tool call shows a confirmation prompt and the python/terminal sandbox is
disabled: safety checks, command blocklist, and resource limits are skipped.
Secret env vars are still stripped and HOME stays repointed at the session
workdir. Default off keeps current behavior, and it takes precedence over
Confirm tool calls. Enabling it requires accepting a warning each time.

* Studio: harden Bypass Permissions secret handling and fix Anthropic tool path

Follow-up to the Bypass Permissions feature. Addresses the review findings:

- Anthropic /v1/messages 500: declare bypass_permissions on
  AnthropicMessagesRequest so tool requests that omit the field default to
  False instead of raising AttributeError (extra='allow' does not set absent
  attributes).
- /proc parent-env leak: stripping the child env did not stop a same-uid
  bypassed child from reading /proc/<parent>/environ to recover the
  tool-executing process's unfiltered secrets. Clear PR_SET_DUMPABLE on that
  process before the first bypass exec so its /proc entries become root-owned.
  Hardening is fail-closed: if prctl is denied, bypass execution is refused
  rather than run with the parent environ still readable. Mitigation, not a
  full boundary; documented in the code.
- Broker/capability vars: strip SSH_AUTH_SOCK, SSH_AGENT_PID, GPG_AGENT_INFO,
  GNUPGHOME, KUBECONFIG, DOCKER_HOST so a bypassed tool cannot use the
  operator's live agents.
- Credential-bearing URL values: drop any env var whose value embeds URL
  userinfo (scheme://user:pass@ and token-only scheme://token@) regardless of
  the variable name. Benign proxy/index URLs without credentials are kept, so
  proxy-only and internal-index setups still work in bypass mode.
- Windows temp isolation: repoint TEMP and TMP (not just TMPDIR) at the
  per-session sandbox dir.
- Frontend: stop persisting bypassPermissions; a reload now starts with the
  sandbox/confirmation bypass off and requires re-accepting the warning dialog.

Adds regression tests for each finding in test_bypass_permissions.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: strip cred-location env vars (HF_HOME etc.) in Bypass Permissions

Repointing HOME did not stop SDKs auto-reading cached creds via vars that
point at the real home/cache/config: HF_HOME (startup always sets it; token
lives under $HF_HOME/token), HF/XDG cache roots, NETRC/BOTO_CONFIG/
PIP_CONFIG_FILE, and Windows HOMEDRIVE/HOMEPATH. Drop those, and repoint
USERPROFILE/APPDATA/LOCALAPPDATA at the per-session workdir. Adds regression
tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: lock in bypass HF token resolution with an end-to-end test

The drop-based fix relies on the whole HF_HOME/XDG fallback chain being
removed so huggingface_hub resolves under the repointed HOME. Add a test
that sets HF_HOME and XDG_CACHE_HOME at a real cache and asserts the
resolved token path lands under the workdir, not the operator's cache.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: strip npm _auth, MYSQL_PWD, and BASH_ENV from bypass env

Three more credential vectors dodged the bypass scrubber: NPM_CONFIG__AUTH
(npm _auth, base64 so no URL userinfo and no AUTH marker), MYSQL_PWD (markers
use PASSWD, not PWD, since PWD is the cwd var), and BASH_ENV (bash -c sources
it for non-interactive shells, so a startup file can re-export stripped
secrets). Add an AUTH marker, the exact MYSQL_PWD name, and drop BASH_ENV plus
PGPASSFILE. Adds regression tests incl. an end-to-end check that a bypass
terminal call does not source BASH_ENV.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: extend bypass env scrubber and enforce confirm precedence in loops

From a parallel review pass over the bypass changes:
- Drop more credential-location vars in _build_bypass_env: npm/yarn/git/cargo/
  rclone config pointers (NPM_CONFIG_USERCONFIG, NPM_CONFIG_GLOBALCONFIG,
  YARN_RC_FILENAME, GIT_CONFIG_GLOBAL, GIT_CONFIG_SYSTEM, CARGO_HOME,
  RCLONE_CONFIG) and the GIT_ASKPASS/SSH_ASKPASS auth helpers.
- Enforce confirm_tool_calls AND NOT bypass_permissions inside the safetensors
  and GGUF tool loops, not just at the route, so a direct internal caller
  passing both flags never prompts.
- Soften the toggle hint: environment secrets are stripped, but bypassed code
  can still read files and credentials on the machine (no overclaim that keys
  stay hidden).
Adds regression tests for the new names and the loop-level precedence.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: add GGUF loop test for bypass-over-confirm precedence

The safetensors loop precedence is covered behaviorally; the GGUF loop needs a
live llama-server so add an AST guard asserting its _needs_confirm gate
references both confirm_tool_calls and bypass_permissions, matching the other
llama_cpp source-inspection tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: add red Bypass Permissions badge in the composer

When Bypass Permissions is on, show a persistent red pill in the composer
tool-pill row (like the Search/Code pills), matching Claude Code's always-
visible bypass indicator. Clicking it turns bypass off, mirroring the other
composer toggles. Enabling still goes through the settings toggle + warning
dialog. Adds a data-variant=danger style for the destructive-colored pill.

* Studio: show Bypass Permissions badge in the Thread composer too

The empty-state and active Thread render their own composer (thread.tsx),
not shared-composer, so the badge only appeared in the split layout. Mirror
the red dismissible pill in ComposerAction so it shows in every composer.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep the Bypass Permissions badge visible when the composer is collapsed

The Thread composer only renders the pill row when expanded, so the active-mode
badge vanished on the default (collapsed) empty state. Render it before the
expand gate (it returns null when bypass is off) so the red indicator always
shows while bypass is on.

* Studio: make the Bypass Permissions confirm button a solid red button

The destructive button variant is a subtle 10% tint that read as bare red text
next to the outlined Cancel. Force the solid destructive fill (the variant's
class loses to the tint through AlertDialogAction's Slot merge, so use the !
override the codebase already uses for this case) and shorten the label to
'I understand' so it fits the small dialog's two-column footer.

* Studio: add Bypass Permissions to the composer + More menu

Adds a 'Bypass Permissions' entry to the composer plus-menu (under More by
default) in both composers, so it can be toggled without opening Run settings.
Enabling routes through the same danger warning dialog; disabling is immediate.
A shared BypassPermissionsMenuItem keeps the two composers in sync.

* Studio: harden bypass env scrubber for IMDS opt-out and connection strings

Two gaps in the Bypass Permissions secret scrubber:

- The broad AWS_ prefix also dropped AWS_EC2_METADATA_DISABLED, a non-secret
  opt-out. Removing it re-opens the IMDS instance-role credential path that the
  operator explicitly disabled, so a bypassed boto/AWS-CLI call could recover
  cloud creds. Keep that flag (and AWS_EC2_METADATA_V1_DISABLED) via a keep-list
  while still stripping the real AWS credential vars.
- Azure App Service connection strings (SQLCONNSTR_/CUSTOMCONNSTR_/...,
  WEBSITE_CONTENTAZUREFILECONNECTIONSTRING) and values like Password=/AccountKey=
  /SharedAccessKey= slipped past the name and URL-only value classifiers. Add
  CONNSTR/CONNECTIONSTRING name markers and a connection-string value matcher.

* Studio: let Bypass Permissions suppress the confirm-tool-calls guards

The confirm-vs-bypass precedence (confirm and not bypass) was applied at the
loop call sites but not at the earlier request guards, so a client sending
confirm_tool_calls + bypass_permissions together was rejected (stream=true
required / unsupported for external or Anthropic tools) before the precedence
took effect. Gate all four confirm guards on not bypass_permissions so both
flags together proceed with the gate suppressed, matching the documented rule.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-15 04:04:22 -07:00

732 lines
27 KiB
Python

# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Tests for Bypass Permissions (skip confirmation + disable sandbox).
Covers the secret-name classifier, the two env builders, the
``disable_sandbox`` branch of ``_python_exec`` / ``_bash_exec`` (which env is
used, which pre-exec is used, and that safety checks / the blocklist are
skipped), the request-model default, the confirm-vs-bypass precedence rule the
route enforces, and that the agentic loop forwards ``disable_sandbox`` while
never gating under bypass.
Run with: ``PYTHONPATH=studio/backend python -m pytest studio/backend/tests/test_bypass_permissions.py -q``
"""
import os
import sys
import pytest
import core.inference.tools as tools
from core.inference.tools import (
_bash_exec,
_build_bypass_env,
_build_safe_env,
_is_cred_location_env_name,
_is_secret_env_name,
_is_secret_env_value,
_python_exec,
)
from core.inference.safetensors_agentic import run_safetensors_tool_loop
_POSIX_ONLY = pytest.mark.skipif(
sys.platform == "win32", reason = "preexec_fn / setsid are POSIX-only"
)
# ── secret-name classifier ──────────────────────────────────────────
@pytest.mark.parametrize(
"name",
[
"HF_TOKEN",
"HUGGING_FACE_HUB_TOKEN",
"WANDB_API_KEY",
"GH_TOKEN",
"GITHUB_TOKEN",
"OPENAI_API_KEY",
"ANTHROPIC_API_KEY",
"AWS_SECRET_ACCESS_KEY",
"AWS_ACCESS_KEY_ID",
"AZURE_CLIENT_SECRET",
"GOOGLE_APPLICATION_CREDENTIALS",
"MY_DB_PASSWORD",
"x_api_key",
"SOME_PRIVATE_KEY",
"LD_PRELOAD",
],
)
def test_secret_names_are_flagged(name):
assert _is_secret_env_name(name) is True
@pytest.mark.parametrize(
"name", ["PATH", "HOME", "LANG", "TERM", "PWD", "SHELL", "HOSTVAR", "MY_VAR"]
)
def test_benign_names_are_not_flagged(name):
assert _is_secret_env_name(name) is False
# ── env builders ────────────────────────────────────────────────────
def test_bypass_env_keeps_benign_strips_secret_repoints_home(monkeypatch, tmp_path):
monkeypatch.setenv("HOSTVAR", "benign-123")
monkeypatch.setenv("HF_TOKEN", "secret-abc")
env = _build_bypass_env(str(tmp_path))
assert env.get("HOSTVAR") == "benign-123" # full host env inherited
assert "HF_TOKEN" not in env # ...minus secrets
assert env["HOME"] == str(tmp_path) # $HOME-based cred lookups defused
assert env["TMPDIR"] == str(tmp_path)
def test_safe_env_excludes_host_and_secret(monkeypatch, tmp_path):
monkeypatch.setenv("HOSTVAR", "benign-123")
monkeypatch.setenv("HF_TOKEN", "secret-abc")
env = _build_safe_env(str(tmp_path))
assert "HOSTVAR" not in env # whitelist build -> host vars never reach child
assert "HF_TOKEN" not in env
# ── Popen kwargs capture (no real execution) ────────────────────────
class _FakeProc:
returncode = 0
def communicate(self, timeout = None):
return ("FAKEOUT", None)
def poll(self):
return 0
def kill(self):
pass
@pytest.fixture
def captured_popen(monkeypatch):
cap = {}
def fake_popen(cmd, **kwargs):
cap["cmd"] = cmd
cap["kwargs"] = kwargs
return _FakeProc()
monkeypatch.setattr(tools.subprocess, "Popen", fake_popen)
return cap
@_POSIX_ONLY
def test_python_sandboxed_uses_sandbox_preexec_and_safe_env(captured_popen, monkeypatch):
monkeypatch.setenv("HF_TOKEN", "secret-abc")
_python_exec("print(1)", None, 5, "t", disable_sandbox = False)
assert captured_popen["kwargs"]["preexec_fn"] is tools._sandbox_preexec
assert "HF_TOKEN" not in captured_popen["kwargs"]["env"]
@_POSIX_ONLY
def test_python_bypass_uses_bypass_preexec_and_bypass_env(captured_popen, monkeypatch):
monkeypatch.setenv("HOSTVAR", "benign-xyz")
monkeypatch.setenv("HF_TOKEN", "secret-abc")
_python_exec("print(1)", None, 5, "t", disable_sandbox = True)
assert captured_popen["kwargs"]["preexec_fn"] is tools._bypass_preexec
env = captured_popen["kwargs"]["env"]
assert env.get("HOSTVAR") == "benign-xyz"
assert "HF_TOKEN" not in env
def test_bash_blocklist_enforced_when_sandboxed(captured_popen):
out = _bash_exec("rm -rf /", None, 5, "t", disable_sandbox = False)
assert "Blocked" in out
assert "cmd" not in captured_popen # never reached Popen
def test_bash_blocklist_skipped_when_bypassed(captured_popen):
out = _bash_exec("rm -rf /", None, 5, "t", disable_sandbox = True)
assert out == "FAKEOUT" # blocklist skipped -> reached (faked) execution
assert captured_popen["cmd"][0] in ("bash", "cmd")
@_POSIX_ONLY
def test_bash_bypass_uses_bypass_preexec(captured_popen):
_bash_exec("echo hi", None, 5, "t", disable_sandbox = True)
assert captured_popen["kwargs"]["preexec_fn"] is tools._bypass_preexec
# ── real end-to-end python execution under bypass ───────────────────
@_POSIX_ONLY
def test_python_bypass_real_exec_sees_host_env_but_not_secret(monkeypatch):
monkeypatch.setenv("HOSTVAR", "benign-xyz")
monkeypatch.setenv("HF_TOKEN", "secret-pqr")
code = (
"import os;"
"print('H=' + str(os.environ.get('HOSTVAR')),"
" 'T=' + str(os.environ.get('HF_TOKEN')))"
)
out = _python_exec(code, None, 30, "test-bypass", disable_sandbox = True)
assert "H=benign-xyz" in out # unrestricted: real host var visible
assert "T=None" in out # ...but the secret was stripped
assert "secret-pqr" not in out
# ── _bypass_preexec is setsid-only (no rlimits) ─────────────────────
@_POSIX_ONLY
def test_bypass_preexec_only_sets_session(monkeypatch):
calls = {"setsid": 0}
monkeypatch.setattr(
tools.os, "setsid", lambda: calls.__setitem__("setsid", calls["setsid"] + 1)
)
# _resource must not be touched by the bypass pre-exec.
if tools._resource is not None:
monkeypatch.setattr(
tools._resource,
"setrlimit",
lambda *a, **k: pytest.fail("bypass pre-exec must not set rlimits"),
)
tools._bypass_preexec()
assert calls["setsid"] == 1
# ── request model default ───────────────────────────────────────────
def test_request_model_bypass_default_false():
from models.inference import ChatCompletionRequest
assert ChatCompletionRequest.model_fields["bypass_permissions"].default is False
# ── confirm-vs-bypass precedence (mirrors the route rule) ───────────
@pytest.mark.parametrize(
"confirm,bypass,effective_confirm",
[
(False, False, False),
(True, False, True),
(False, True, False),
(True, True, False),
],
)
def test_confirm_precedence_rule(confirm, bypass, effective_confirm):
# The route computes: confirm_tool_calls = confirm and not bypass.
assert (bool(confirm) and not bool(bypass)) is effective_confirm
# ── agentic loop forwards disable_sandbox, never gates under bypass ──
_DEFAULT_TOOLS = [
{"type": "function", "function": {"name": "python"}},
{"type": "function", "function": {"name": "web_search"}},
]
def _tool_call(name, args_json):
return f'<tool_call>{{"name": "{name}", "arguments": {args_json}}}</tool_call>'
def _multi_turn(turns):
it = iter(turns)
def _gen(_messages):
try:
yield next(it)
except StopIteration:
return
return _gen
def test_loop_forwards_disable_sandbox_and_does_not_gate():
seen = []
def fake_exec(
name,
arguments,
*,
cancel_event = None,
timeout = None,
session_id = None,
rag_scope = None,
disable_sandbox = False,
):
seen.append(disable_sandbox)
return f"RAN[{name}]"
events = list(
run_safetensors_tool_loop(
single_turn = _multi_turn([_tool_call("python", '{"code": "x"}'), "done"]),
messages = [{"role": "user", "content": "hi"}],
tools = _DEFAULT_TOOLS,
execute_tool = fake_exec,
session_id = "s",
confirm_tool_calls = False, # route forces this off under bypass
bypass_permissions = True,
)
)
assert seen == [True] # disable_sandbox threaded through
starts = [e for e in events if e["type"] == "tool_start"]
assert starts and starts[0]["awaiting_confirmation"] is False
assert starts[0]["approval_id"] == ""
def test_loop_bypass_overrides_confirm_for_direct_callers():
# Even if a direct internal caller passes confirm_tool_calls=True, bypass
# must suppress the confirm gate at the loop level (not only at the route).
def fake_exec(
name,
arguments,
*,
cancel_event = None,
timeout = None,
session_id = None,
rag_scope = None,
disable_sandbox = False,
):
return f"RAN[{name}]"
events = list(
run_safetensors_tool_loop(
single_turn = _multi_turn([_tool_call("python", '{"code": "x"}'), "done"]),
messages = [{"role": "user", "content": "hi"}],
tools = _DEFAULT_TOOLS,
execute_tool = fake_exec,
session_id = "s",
confirm_tool_calls = True, # raw caller leaves this on...
bypass_permissions = True, # ...but bypass must still win
)
)
starts = [e for e in events if e["type"] == "tool_start"]
assert starts and starts[0]["awaiting_confirmation"] is False
assert starts[0]["approval_id"] == ""
def test_gguf_loop_confirm_gate_respects_bypass():
# The GGUF loop needs a live llama-server, so (per the other llama_cpp
# tests) assert via AST that its _needs_confirm gate applies the bypass
# precedence, mirroring the safetensors behavioral test above.
import ast
import inspect
import textwrap
llama_cpp = pytest.importorskip("core.inference.llama_cpp")
src = textwrap.dedent(
inspect.getsource(llama_cpp.LlamaCppBackend.generate_chat_completion_with_tools)
)
gates = [
node
for node in ast.walk(ast.parse(src))
if isinstance(node, ast.Assign)
and any(getattr(t, "id", None) == "needs_confirm" for t in node.targets)
]
assert gates, "could not find the needs_confirm gate in the GGUF loop"
names = {n.id for g in gates for n in ast.walk(g.value) if isinstance(n, ast.Name)}
assert "confirm_tool_calls" in names
assert "bypass_permissions" in names # bypass must suppress the GGUF gate
# ── broker / capability env vars are stripped (regression) ──────────
@pytest.mark.parametrize(
"name",
["SSH_AUTH_SOCK", "SSH_AGENT_PID", "GPG_AGENT_INFO", "GNUPGHOME", "KUBECONFIG"],
)
def test_broker_capability_names_are_flagged(name):
# Not secrets by value, but they hand the child the operator's live agent
# (ssh/gpg) or kube credentials, so bypass mode must drop them.
assert _is_secret_env_name(name) is True
# ── credential-bearing URL values stripped regardless of name ───────
@pytest.mark.parametrize(
"value",
[
"https://user:s3cr3t@feed.example.invalid/simple", # user:pass@
"https://ghp_deadbeef@github.com/org/private.git", # token-only@
"https://__token__@pypi.example.invalid/simple",
"https://ghp_1234:@npm.pkg.github.com/simple", # empty password
"postgres://dbuser:dbpass@db.example.invalid/app",
],
)
def test_url_userinfo_values_are_flagged(value):
assert _is_secret_env_value(value) is True
@pytest.mark.parametrize(
"value",
[
"https://example.invalid/simple", # no userinfo
"http://proxy.corp.example:8080", # benign proxy
"https://pypi.corp.example/simple", # benign internal index
"redis://localhost:6379/0", # no creds
"https://example.invalid/path?ref=a@b", # '@' only in query, not userinfo
],
)
def test_non_credential_url_values_are_not_flagged(value):
assert _is_secret_env_value(value) is False
def test_url_userinfo_value_is_stripped_even_with_benign_name(monkeypatch, tmp_path):
# NAME dodges the classifier, but the VALUE embeds userinfo -> must go.
monkeypatch.setenv("MY_FEED", "https://user:s3cr3t@feed.example.invalid/simple")
monkeypatch.setenv("REPO_URL", "https://ghp_deadbeef@github.com/org/private.git")
# A URL without credentials is harmless and should be kept.
monkeypatch.setenv("PLAIN_URL", "https://example.invalid/simple")
env = _build_bypass_env(str(tmp_path))
assert "MY_FEED" not in env
assert "REPO_URL" not in env
assert env.get("PLAIN_URL") == "https://example.invalid/simple"
def test_bypass_env_keeps_noncredential_proxy_and_index_urls(monkeypatch, tmp_path):
# Benign routing/config vars must survive bypass mode (proxy-only or
# internal-index networks); only credentialed values are dropped.
monkeypatch.setenv("HTTP_PROXY", "http://proxy.corp.example:8080")
monkeypatch.setenv("PIP_INDEX_URL", "https://pypi.corp.example/simple")
monkeypatch.setenv("PIP_EXTRA_INDEX_URL", "https://user:token@pypi.example.invalid/simple")
env = _build_bypass_env(str(tmp_path))
assert env["HTTP_PROXY"] == "http://proxy.corp.example:8080"
assert env["PIP_INDEX_URL"] == "https://pypi.corp.example/simple"
assert "PIP_EXTRA_INDEX_URL" not in env # this one carries credentials
# ── AWS IMDS-disable hardening flag is kept (regression) ────────────
def test_aws_imds_disable_flag_is_kept_but_creds_stripped(monkeypatch, tmp_path):
# AWS_EC2_METADATA_DISABLED is a non-secret opt-out: dropping it would let a
# bypassed boto/AWS-CLI call fall back to the instance role via IMDS even
# though the operator disabled that path. Keep it; drop the real creds.
monkeypatch.setenv("AWS_EC2_METADATA_DISABLED", "true")
monkeypatch.setenv("AWS_ACCESS_KEY_ID", "AKIAEXAMPLE")
monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "shhh")
assert _is_secret_env_name("AWS_EC2_METADATA_DISABLED") is False
assert _is_secret_env_name("AWS_ACCESS_KEY_ID") is True
env = _build_bypass_env(str(tmp_path))
assert env.get("AWS_EC2_METADATA_DISABLED") == "true"
assert "AWS_ACCESS_KEY_ID" not in env
assert "AWS_SECRET_ACCESS_KEY" not in env
# ── connection-string env vars are stripped (regression) ────────────
@pytest.mark.parametrize(
"name",
[
"SQLCONNSTR_DB", # Azure App Service injected connection strings
"MYSQLCONNSTR_DB",
"SQLAZURECONNSTR_DB",
"POSTGRESQLCONNSTR_DB",
"CUSTOMCONNSTR_CACHE",
"WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
],
)
def test_connection_string_names_are_flagged(name):
assert _is_secret_env_name(name) is True
@pytest.mark.parametrize(
"value",
[
"Server=tcp:db;Database=app;User ID=u;Password=p@ss;", # ADO.NET
"DefaultEndpointsProtocol=https;AccountName=x;AccountKey=abc123==;", # storage
"Endpoint=sb://x;SharedAccessKeyName=n;SharedAccessKey=zzz=", # Service Bus
],
)
def test_connection_string_values_are_flagged(value):
assert _is_secret_env_value(value) is True
@pytest.mark.parametrize(
"value",
[
"Server=tcp:db;Database=app;User ID=u;", # no password field
"Endpoint=sb://x;SharedAccessKeyName=n", # key NAME only, no secret
"AccountName=x;EndpointSuffix=core.windows.net", # no AccountKey
],
)
def test_connection_string_noncredential_values_are_not_flagged(value):
assert _is_secret_env_value(value) is False
def test_connection_string_value_stripped_even_with_benign_name(monkeypatch, tmp_path):
# NAME dodges the classifier, but the VALUE is a credentialed conn string.
monkeypatch.setenv("APP_DB", "Server=tcp:db;Database=app;User ID=u;Password=p@ss;")
monkeypatch.setenv("SQLCONNSTR_DB", "DefaultEndpointsProtocol=https;AccountKey=abc==")
env = _build_bypass_env(str(tmp_path))
assert "APP_DB" not in env # value-based catch
assert "SQLCONNSTR_DB" not in env # name-based catch
# ── temp dirs repointed on every platform (regression) ──────────────
def test_bypass_env_repoints_all_temp_vars(monkeypatch, tmp_path):
# Windows tempfile honours TEMP/TMP, not TMPDIR; all three must repoint.
monkeypatch.setenv("TEMP", "/host/tmp")
monkeypatch.setenv("TMP", "/host/tmp")
env = _build_bypass_env(str(tmp_path))
assert env["TMPDIR"] == str(tmp_path)
assert env["TEMP"] == str(tmp_path)
assert env["TMP"] == str(tmp_path)
# ── credential-location redirect vars are dropped (regression) ──────────
# Vars that point SDKs at the real home/cache/config (cached tokens), e.g.
# HF_HOME which startup always sets -> the live leak the HOME repoint missed.
@pytest.mark.parametrize(
"name",
[
"HF_HOME",
"HF_HUB_CACHE",
"HUGGINGFACE_HUB_CACHE",
"HF_XET_CACHE",
"TRANSFORMERS_CACHE",
"HF_DATASETS_CACHE",
"XDG_CONFIG_HOME",
"XDG_CACHE_HOME",
"XDG_DATA_HOME",
"NETRC",
"BOTO_CONFIG",
"PIP_CONFIG_FILE",
"CLOUDSDK_CONFIG",
"KAGGLE_CONFIG_DIR",
"DOCKER_CONFIG",
"WANDB_DIR",
"WANDB_CONFIG_DIR",
"NPM_CONFIG_USERCONFIG",
"NPM_CONFIG_GLOBALCONFIG",
"YARN_RC_FILENAME",
"GIT_CONFIG_GLOBAL",
"GIT_CONFIG_SYSTEM",
"CARGO_HOME",
"RCLONE_CONFIG",
"GIT_ASKPASS",
"SSH_ASKPASS",
"BASH_ENV",
"HOMEDRIVE",
"HOMEPATH",
],
)
def test_cred_location_names_are_flagged(name):
assert _is_cred_location_env_name(name) is True
@pytest.mark.parametrize("name", ["PATH", "HOME", "LANG", "PWD", "MY_VAR"])
def test_benign_names_not_flagged_as_cred_location(name):
assert _is_cred_location_env_name(name) is False
def test_bypass_env_drops_hf_home_so_cached_token_unreachable(monkeypatch, tmp_path):
# The live leak: startup sets HF_HOME at the real cache, whose $HF_HOME/token
# holds the operator's token. Repointing HOME does not stop huggingface_hub
# from reading $HF_HOME/token, so HF_HOME must be dropped in bypass mode.
real_cache = tmp_path / "real_hf_cache"
real_cache.mkdir()
(real_cache / "token").write_text("hf_cachedOperatorToken")
monkeypatch.setenv("HF_HOME", str(real_cache))
monkeypatch.setenv("HF_HUB_CACHE", str(real_cache / "hub"))
env = _build_bypass_env(str(tmp_path))
assert "HF_HOME" not in env # dropped -> HF falls back to $HOME/.cache (empty)
assert "HF_HUB_CACHE" not in env
def test_bypass_env_hf_token_resolves_outside_real_cache(monkeypatch, tmp_path):
# End-to-end: even when HF_HOME and XDG_CACHE_HOME both point at the real
# cache, the bypass env must make huggingface_hub resolve the token under the
# workdir (guards the XDG fallback chain, not just "HF_HOME absent").
pytest.importorskip("huggingface_hub")
import subprocess
real_cache = tmp_path / "real_hf"
real_cache.mkdir()
workdir = tmp_path / "sandbox"
workdir.mkdir()
monkeypatch.setenv("HF_HOME", str(real_cache))
monkeypatch.setenv("XDG_CACHE_HOME", str(real_cache))
monkeypatch.setenv("XDG_CONFIG_HOME", str(real_cache))
env = _build_bypass_env(str(workdir))
token_path = subprocess.run(
[
sys.executable,
"-c",
"import huggingface_hub.constants as c; print(c.HF_TOKEN_PATH)",
],
env = env,
capture_output = True,
text = True,
).stdout.strip()
assert str(real_cache) not in token_path # never the operator's cache
assert token_path.startswith(str(workdir)) # resolved under the sandbox
def test_bypass_env_drops_credential_config_path_vars(monkeypatch, tmp_path):
# NETRC / BOTO_CONFIG / PIP_CONFIG_FILE point clients at real credential
# files before $HOME, so they must not survive into the bypassed child.
monkeypatch.setenv("NETRC", "/home/op/.netrc")
monkeypatch.setenv("PGPASSFILE", "/home/op/.pgpass")
monkeypatch.setenv("BOTO_CONFIG", "/home/op/.boto")
monkeypatch.setenv("PIP_CONFIG_FILE", "/home/op/.pip/pip.conf")
env = _build_bypass_env(str(tmp_path))
assert "NETRC" not in env
assert "PGPASSFILE" not in env
assert "BOTO_CONFIG" not in env
assert "PIP_CONFIG_FILE" not in env
def test_bypass_env_strips_npm_auth_and_mysql_pwd(monkeypatch, tmp_path):
# NPM_CONFIG__AUTH (npm _auth, base64) and MYSQL_PWD dodge the URL-value
# check and the PASSWD marker, but must still be dropped.
monkeypatch.setenv("NPM_CONFIG__AUTH", "aGVsbG86c2VjcmV0")
monkeypatch.setenv("MYSQL_PWD", "db-password")
assert _is_secret_env_name("NPM_CONFIG__AUTH") is True
assert _is_secret_env_name("MYSQL_PWD") is True
env = _build_bypass_env(str(tmp_path))
assert "NPM_CONFIG__AUTH" not in env
assert "MYSQL_PWD" not in env
@_POSIX_ONLY
def test_bash_bypass_does_not_source_bash_env(monkeypatch, tmp_path):
# bash -c sources $BASH_ENV for non-interactive shells; an operator startup
# file could re-export stripped secrets, so a real bypass call must not see it.
startup = tmp_path / "startup.sh"
startup.write_text("export RECOVERED=leaked\n")
monkeypatch.setenv("BASH_ENV", str(startup))
out = _bash_exec("echo R=$RECOVERED", None, 30, "bash-env-test", disable_sandbox = True)
assert "R=leaked" not in out # BASH_ENV dropped -> startup not sourced
assert "R=" in out
def test_bypass_env_repoints_windows_profile_vars(monkeypatch, tmp_path):
# On Windows, SDKs read cached creds under USERPROFILE/APPDATA/LOCALAPPDATA,
# not $HOME. Set ones are repointed at the workdir; HOMEDRIVE/HOMEPATH drop.
monkeypatch.setenv("USERPROFILE", "/host/profile")
monkeypatch.setenv("APPDATA", "/host/profile/AppData/Roaming")
monkeypatch.setenv("LOCALAPPDATA", "/host/profile/AppData/Local")
monkeypatch.setenv("HOMEDRIVE", "C:")
monkeypatch.setenv("HOMEPATH", "\\Users\\op")
env = _build_bypass_env(str(tmp_path))
assert env["USERPROFILE"] == str(tmp_path)
assert env["APPDATA"] == str(tmp_path)
assert env["LOCALAPPDATA"] == str(tmp_path)
assert "HOMEDRIVE" not in env
assert "HOMEPATH" not in env
def test_bypass_env_does_not_add_unset_windows_profile_vars(monkeypatch, tmp_path):
# Only repoint Windows profile vars that were actually set (no pollution on
# Linux/macOS where they are absent).
monkeypatch.delenv("USERPROFILE", raising = False)
monkeypatch.delenv("APPDATA", raising = False)
monkeypatch.delenv("LOCALAPPDATA", raising = False)
env = _build_bypass_env(str(tmp_path))
assert "USERPROFILE" not in env
assert "APPDATA" not in env
assert "LOCALAPPDATA" not in env
# ── parent /proc env-leak hardening (regression) ────────────────────
@_POSIX_ONLY
def test_bypass_exec_hardens_parent_proc_env(monkeypatch, captured_popen):
# Stripping the child env is not enough: a same-UID child can read the
# parent's /proc environ. The exec paths must invoke the parent hardening
# when (and only when) the sandbox is disabled.
calls = {"n": 0}
def fake_harden():
calls["n"] += 1
return True
monkeypatch.setattr(tools, "_harden_parent_against_proc_env_leak", fake_harden)
_python_exec("print(1)", None, 5, "t", disable_sandbox = True)
_bash_exec("echo hi", None, 5, "t", disable_sandbox = True)
assert calls["n"] == 2
calls["n"] = 0
_python_exec("print(1)", None, 5, "t", disable_sandbox = False)
_bash_exec("echo hi", None, 5, "t", disable_sandbox = False)
assert calls["n"] == 0 # never hardened on the sandboxed path
def test_bypass_exec_fails_closed_when_hardening_fails(monkeypatch, captured_popen):
# If the parent cannot be hardened (e.g. prctl denied), the unsandboxed
# child must NOT run - otherwise the parent environ stays readable.
monkeypatch.setattr(tools, "_harden_parent_against_proc_env_leak", lambda: False)
out_py = _python_exec("print(1)", None, 5, "t", disable_sandbox = True)
out_sh = _bash_exec("echo hi", None, 5, "t", disable_sandbox = True)
assert "refusing bypass execution" in out_py
assert "refusing bypass execution" in out_sh
assert "cmd" not in captured_popen # never reached Popen
@_POSIX_ONLY
def test_proc_env_unreadable_after_hardening():
# Mechanism check: after hardening, a same-UID child can no longer read the
# parent process /proc environ. Restores the dumpable flag afterwards so the
# process-global state does not leak into later tests.
import subprocess
if tools._libc is None:
pytest.skip("no libc/prctl available")
pid = os.getpid()
probe = (
"try:\n"
f" open('/proc/{pid}/environ', 'rb').read()\n"
" print('READABLE')\n"
"except PermissionError:\n"
" print('DENIED')\n"
)
prev_dumpable = tools._libc.prctl(3, 0, 0, 0, 0) # PR_GET_DUMPABLE
prev_guard = tools._parent_proc_hardened
try:
# Establish a clean readable baseline: another test may have already
# cleared the dumpable flag on this process.
tools._libc.prctl(4, 1, 0, 0, 0) # PR_SET_DUMPABLE = 1
before = subprocess.run(
[sys.executable, "-c", probe], capture_output = True, text = True
).stdout.strip()
if before != "READABLE":
pytest.skip("/proc already restricted in this environment")
tools._parent_proc_hardened = False
assert tools._harden_parent_against_proc_env_leak() is True
after = subprocess.run(
[sys.executable, "-c", probe], capture_output = True, text = True
).stdout.strip()
assert after == "DENIED"
finally:
if prev_dumpable in (0, 1):
try:
tools._libc.prctl(4, prev_dumpable, 0, 0, 0)
except (OSError, AttributeError):
pass
tools._parent_proc_hardened = prev_guard
# ── Anthropic request model declares the field (regression) ─────────
def test_anthropic_request_model_bypass_default_false():
# Omitting the field on the Anthropic path must default to False rather than
# raising AttributeError (extra='allow' does not set absent attributes).
from models.inference import AnthropicMessagesRequest
assert AnthropicMessagesRequest.model_fields["bypass_permissions"].default is False
req = AnthropicMessagesRequest(model = "x", messages = [], max_tokens = 8)
assert bool(req.bypass_permissions) is False