- Scan command substitutions regardless of the outer command and quoting: bash runs $(...) /
backtick payloads even inside double quotes, so echo "$(head -1 /etc/passwd)" read the file
while the scanner only inspected reader-command arguments. Extract every substitution payload
(new _extract_command_subs) and recurse the read scanner into it.
- Normalize .. before the system-bin exemption: _is_local_executable_path trusted any /usr/bin/
prefix, so /usr/bin/../../<workdir>/evil escaped the local-exec check and ran an unguarded
shebang. normpath the command path first (the leading-slash check on the original token keeps
./evil local).
- Block flock as a command-executing wrapper: flock [opts] <file>|<fd> <command> (or flock -c
<command>) runs an arbitrary command in an unguarded child; its file/fd operand + -c forms make
the command word hard to resolve, so block the wrapper outright.
- Treat coproc as a command-position keyword: coproc [NAME] command runs COMMAND asynchronously,
so coproc touch /tmp/escape must resolve touch as the command; add coproc to the keyword set.
- Scan trap handler payloads: trap 'CMD' SIGSPEC runs CMD in the unguarded shell on EXIT / a
signal, so recurse the blocked-command and sensitive-read scanners into the handler operand; a
reset (trap - EXIT) / ignore (trap '' EXIT) has nothing to run.
- Guard the low-level posix / nt directory readers: posix.listdir / posix.scandir re-export the
ORIGINAL enumerators, so an opaque sensitive path (posix.listdir('/root')) slipped past the
os.* dir guard; apply the same sensitive-read confinement to the low-level modules (via a
module-parametrized _guard_dir_reader).
- Reapply fd deniers + dir-reader guards to a freshly created posix / nt module: _reguard_created
only rewrapped open + path mutators, so a fresh module's fchmod / fchown (host-metadata mutation
on a read-only outside fd) and listdir / scandir were unguarded; reapply them too.
Adds TestRound33Bypasses plus runtime posix dir-reader / fresh-module fd-denier tests.
- Preserve command position after shell compound-statement keywords: if / while / until (and
the existing then / do / else / elif) run their CONDITION command, so `if touch /tmp/escape;
then :; fi` executed the child writer while the scanner mistook `if` for the command and
skipped `touch`. Add if/while/until to the command-position keyword set (fixing both the
blocked-command scanner and the wrapper-aware command-word resolver) and to the sensitive-read
scanner's separators so a reader in a condition body is scanned too.
- Add chrt to the command-prefix wrappers: its arity was declared but chrt was not resolved as a
prefix, so `chrt -o 0 touch /tmp/x` treated chrt as the command and never inspected touch.
- Block mktemp as a child writer: mktemp creates a file/dir at a caller-chosen template path
(mktemp /tmp/x.XXXXXX, mktemp -d) outside the workdir in an unguarded child.
- Make the loader-table / namespace-dict / builtins recovery checks alias-aware:
- sys.modules subscript and .get() now use the alias-aware _is_sys_modules, so
`m = sys.modules; m['os'].system(...)` / `m.get('os')...` is caught like the direct form.
- namespace-dict subscript resolves a single-assignment alias
(`g = globals(); g['__builtins__'].__import__('os')`) via a new _is_namespace_dict_expr.
- the dynamic-import check resolves a builtins alias
(`b = __builtins__; b.__import__('os')`) via a new _is_builtins_ref.
Adds TestRound32Bypasses.
- Block frame / traceback introspection that recovers a runtime guard's original callable: the
open()/os.* guard wrappers hold the unguarded callable as a free variable (real), so a snippet
that triggers a denied open() could read it back via a trace hook or the caught exception's
traceback (frame.f_locals['real'], tb.tb_frame.f_locals) and call it directly. __closure__ /
cell_contents were already blocked, so the frame path was the remaining channel; add the frame
acquisition + value-read attributes (f_locals, f_globals, f_back, f_builtins, tb_frame,
tb_next, gi_frame, cr_frame, ag_frame, settrace, setprofile, _getframe, _current_frames,
currentframe) to the introspection-gadget set, flagged for any receiver in both the attribute
and getattr-string forms.
- Treat an opaque compile() source as executable: compile() does not itself run, but its code
object can be executed WITHOUT exec / eval (fn.__code__ = compile(src, '<p>', 'exec'); fn()),
so a non-literal compile source is as unverifiable as an opaque exec / eval payload and is now
blocked too. A literal compile source is still analyzed recursively and stays allowed.
- Treat a leading shell ! as command-position syntax: in bash ! negates the pipeline exit status
but the following word is still the executed command, so ! touch /tmp/escape / ! python3 -c ...
slipped past the child-writer / interpreter blocklist. Skip a command-position ! in the
command scanner and the wrapper-aware command-word resolver so the real command is scanned; a !
in argument position ([ ! -f x ], find . ! -name ...) is unaffected.
Adds TestRound31Bypasses.
- Track env -C in a subprocess argv read scan: subprocess.run(['env', '-C', '/etc', 'cat',
'passwd']) chdirs the child to /etc before the reader runs, so the relative reader arg reads
/etc/passwd. Extract an argv env -C / --chdir dir (new _argv_env_chdir) and fold it into the
cwd used to resolve relative argv reads, mirroring the shell-string env -C handling.
- Resolve from-imported yaml.load / load_all aliases: the safe-loader check only ran for the
yaml.load attribute form, so from yaml import load; load(payload) bypassed it. Track the
bare-name yaml load aliases and apply the same safe-loader check to the direct-call form.
- Fail closed on a non-literal shell startup env: the BASH_ENV / ENV check only inspected an
inline env={...} dict, so env=e, dict(BASH_ENV='env.sh'), and a computed-key dict still set a
startup script bash / sh sources before the scanned -c payload. Fold the dict(...) form and,
for a shell child (shell=True or an argv resolving to bash / sh), flag an opaque / non-literal
env mapping.
- Block a BASH_ENV / ENV assignment prefix before a shell in the shell-string scanner:
BASH_ENV=env.sh bash -c '...' (and the env BASH_ENV=env.sh bash -c form) sources the workdir
script before the -c payload; scan the command segment before each shell command word for a
non-empty startup-env assignment.
- Match a sensitive directory named without a trailing slash: the directory markers carry a
trailing slash to match descendants, so an unguarded ls /root / find /etc/ssh enumerating the
dir itself was accepted. Append a slash to the candidate before the marker check so the dir
itself matches without loosening the component boundary.
- Treat pickle.Unpickler(f).load() as a deserialization sink: the sink-name list caught
pickle.load but not the equivalent Unpickler(file).load() API (incl. dill / _pickle /
cloudpickle and a from-imported ctor). Detect the Unpickler-constructor receiver of a .load()
/ .load_all() method call.
- Recurse into find -exec nested shell reads: find . -exec sh -c 'cat /etc/passwd' ; runs the
quoted -c payload in an unguarded child; the read scanner only recursed into a shell that was
the command word. Scan each -exec segment through the read scanner (mirrors the blocked-command
find -exec handling).
Adds TestRound30Bypasses.
- Resolve a wrapper-prefixed shell argv before scanning: subprocess.run(['env', 'bash', '-c',
'cat /etc/passwd']) hid the nested shell behind argv[0]=env, so only argv[0] was checked for a
shell binary and the -c payload was never scanned. Resolve the executed command word past
wrapper prefixes (via _blocked_in_argv) so env / timeout / nice wrapped bash -c is scanned.
- Skip a wrapper's numeric duration in the shell-string read scanner: timeout 1 bash -c
'cat /etc/passwd' treated the operand 1 as the command word, so the nested bash -c was not
reached. Add the same _is_wrapper_numeric_arg skip the blocklist path already uses.
- Honor args= when failing closed on a dynamic cwd: the fail-closed only inspected positional
argv, so subprocess.run(args=['cat', 'passwd'], cwd=P) slipped. Resolve the argv from the
public args= keyword too.
- Resolve a relative env -C against the ambient subprocess cwd: env -C . cat passwd under
cwd=/etc chdirs to /etc, not the bare fragment, so the relative reader still reads /etc/passwd.
Join a relative env -C / --chdir= operand onto the current child cwd instead of replacing it.
- Treat diff-style utilities as file readers: diff / sdiff / diff3 / colordiff / cmp print file
contents, so an escaping glob (diff /etc/pass* /dev/null) exfiltrated a secret. Add them to
the shell-read command allowlist.
- Expand a literal star-star dict unpack for the shell / cwd decisions: a shell= or cwd= smuggled
through subprocess.run(cmd, **{'shell': True}) was invisible to the kwarg loop (kw.arg is None).
Iterate keywords through a helper that also expands a literal dict unpack.
- Materialize a device-sink path via the base str.replace before trusting it: a str subclass
could override replace() to return '/dev/null' while its real value escaped the workdir. Call
the genuine str.replace on the underlying buffer so the real path is checked.
Adds TestRound29Bypasses plus a runtime device-sink str-subclass escape test.
- Recognize an aliased open-module receiver (import builtins as b; b.open(...), import io as i;
i.open(...), import os as o; o.open(...)) as a read callee via a new _open_mod_aliases set, so
an aliased traversal / sensitive read is caught like the literal builtins/io/os.open forms.
- Record os shell aliases from `from os import system as s` / popen: the import-walk elif that
consumed the os module only recorded `open`, so the later shell-alias branch never saw it and
the read scan skipped s('cat /etc/passwd'). Handle os shell functions in that branch and split
the subprocess from-import handling into its own branch.
- Combine a subprocess cwd= with a shell payload's relative reads: subprocess.run('cat passwd',
shell=True, cwd='/etc') is resolved to /etc/passwd (the shared read scanner now takes a cwd
seed, overridable per-command by env -C), and a NON-literal cwd fails closed for a relative
reader.
- Allow a Python write to a standard device sink (/dev/null, /dev/stdout, ...) in the runtime
guard, checked on the requested path (not its realpath, so /dev/stdout is not followed to a
redirected outside file); benign output-suppression patterns are no longer denied.
Adds TestRound28Bypasses plus runtime device-sink write tests.
- Block running an explicit LOCAL executable path at command position (./evil, subdir/tool) in
both the argv scanner and the shell command scanner: a sandboxed snippet can create + chmod a
local script with an interpreter shebang and run it, starting an unguarded child the basename
scan never sees. Absolute system-bin paths (/bin, /usr/bin, ...) stay allowed and are still
interpreter-checked by basename.
- Fail closed on a child file-reader (cat / head / ...) with a relative argv path under a
NON-literal subprocess cwd= (cwd=P that could evaluate to /etc), which cannot be proven
sandbox-local; a literal benign cwd and a non-reader program stay allowed.
- Treat a shell startup variable (BASH_ENV / ENV) in an explicit subprocess env= dict as a shell
escape: bash / sh sources it before the -c payload runs.
- Runtime backstop: treat the exact /root path (not only /root/*) as sensitive so a directory
reader over the root home is denied, and wrap Path.glob / Path.rglob like Path.iterdir so a
dynamically built receiver pointing at a sensitive directory is screened.
- Make the newline -> ; command-separator rewrite quote-aware, and neutralize quoted separators
before the command-boundary regex, so a quoted multiline string (echo "ok\nrm") is not
mis-blocked; unquoted separators and command substitution ($(...) / backticks, including
inside double quotes) still block.
Adds TestRound27Bypasses plus runtime tests for exact /root and pathlib glob / rglob.
- Block bash's history builtin when it reads/writes a file (history -w / -a / -r / -n): it can
create or overwrite an arbitrary host path (or read a file into the buffer) in the unguarded
shell child. Bare history / -c / -d / -p / -s stay allowed.
- Combine a subprocess cwd= with relative argv paths in the sensitive-read scan, so
subprocess.run(['cat', 'passwd'], cwd='/etc') is seen as a /etc/passwd read.
- Track env -C DIR / --chdir DIR in the shell-string read scan so a later relative reader
argument (env -C /etc cat passwd) resolves against DIR.
- Record aliases created by tuple/list unpacking assignments ((s,) = (os.system,); a, b =
os.system, 1; [e] = [exec]) in the scope alias index, pairing a literal target with a literal
RHS element-wise, so the shell/exec/deserializer sink checks see them.
- Pin the builtins the runtime path guard consults (isinstance / int / bytes / str / any) into
the guard namespace, so sandboxed code cannot reassign builtins.isinstance to make
isinstance(path, int) treat an outside path as an fd and approve an absolute write.
- Re-pin os.path.stat + S_ISLNK before each realpath resolution in the guard, so a
os.path.stat.S_ISLNK = lambda mode: False (stopping realpath from following an in-workdir
symlink that escapes) cannot approve a write the real open() then routes outside.
- Restore the /root/ protection in the runtime sensitive-read backstop for an opaque path,
carving out package / library trees (site-packages, dist-packages, the stdlib) so imports
under a root home are not broken.
Adds TestRound26Bypasses plus runtime tests for the pinned builtins / stat and root-home reads.
- Model bash brace expansion (comma lists) before the block and read scans, so a payload such
as {touch,/tmp/x} or {python3,-c} '...' is seen as the writer / interpreter bash actually
runs. Only unquoted groups with a top-level comma expand; {} (find -exec), ${VAR} parameter
expansion, numeric {1..5} sequences and quoted braces are left intact, and expansion is
bounded.
- Resolve the reader / command word in the shell-string sensitive-read scan past leading
VAR=value assignments and command wrappers (env / nice / timeout / ...), flag a VAR=value
whose value is a sensitive path, and recursively scan a nested bash -c '<payload>' shell, so
a read hidden behind a normal command-prefix form is caught. The classifier and terminal
scanners now share one _scan_command_string_for_reads with a strict_traversal knob (strict
for os.system shell strings, lenient .. for benign in-tree terminal navigation).
- Treat unbound MRO / getattribute access on a guarded file class as the same recovery gadget
as io.FileIO.__mro__: type.mro(io.FileIO), type.__getattribute__(io.FileIO, '__mro__') /
object.__getattribute__(..., 'mro'), and getattr(io.FileIO, '__mro__') are blocked.
Adds TestRound25Bypasses plus terminal brace / prefixed-read regression tests.
- Block sed w-path writes without a separating space (w followed by a slash,
tilde or tab) and sed e / s///e scripts that execute a shell command (new
_SED_WRITE_RE / _SED_EXEC_RE / _SED_SFLAG_RE checks in the mutating-util scan).
- Add rmdir to the POSIX child-writer denylist.
- Split a glued input redirection (sh here-string payload) before shell
detection by adding the input-redirect operator to the shlex punctuation_chars.
- Deny PyYAML unsafe deserialization: yaml.unsafe_load / full_load(_all) are
unconditional sinks, and yaml.load / load_all are flagged unless given an
explicit safe Loader (SafeLoader / CSafeLoader / BaseLoader).
- Rewrite operator.methodcaller('system', ...)(os) to the direct os.system(...)
call in both the signal-escape visitor and the sensitive-read scanner so a
methodcaller-hidden shell / read sink is analyzed.
- Fix chained single-assignment alias resolution (s = os.system; t = s; t(...)):
the scope walk yielded assignments out of order, so process them in source
order before propagating alias identity through smap / emap / dmap.
- Apply the sensitive-read scan to direct terminal (bash) commands, which run in
an unguarded shell child that the Python-tool open() backstop does not cover;
block reads of host identity / credential files, sensitive-target directory
traversal, and escaping-glob / expansion reads while allowing benign in-tree
relative navigation.
Adds TestRound24Bypasses plus terminal sensitive-read regression tests.
Round 19 review follow-ups on the Studio code-exec sandbox classifier and runtime guard:
- Record a global/nonlocal sink alias in its TARGET scope (module for global, enclosing for
nonlocal): global s; s = os.system; s('...') now resolves s to the shell sink instead of
being skipped as a rebound name (and a global name no longer shadows an outer alias).
- Fail closed on any expansion in COMMAND POSITION, not just command substitution: a
variable-expanded command word (p=python3; $p -c ...) / ${VAR} is unprovable. Argument
position ($HOME, echo $(date)) stays allowed.
- Honor wrapper/command-position logic in the mutating-utility and shell-script scans via a
shared command-word index, so env sed -i / timeout 5 bash s.sh no longer hide behind a
wrapper prefix.
- Use the subprocess-exec callee resolver for the shell=True read scan, so
from subprocess import run as r; r('head /etc/passwd', shell=True) and r = subprocess.run
aliases are tokenized.
- Resolve sys.modules aliases (m = sys.modules; m.pop('_io')) for the loader-table mutation
checks (bound and unbound forms).
- Treat a __dict__ subscript keyed by a gadget dunder (type(open).__dict__['__closure__'].
__get__(open)) as gadget access, closing the descriptor-lookup route around the attribute
gadget scan.
- Unwrap inline literal containers in the higher-order sink check, so
list(map([eval][0], [...])) / map({'e': exec}['e'], ...) are flagged.
- Runtime: the mutator guard now accepts the path via its public keyword (os.makedirs(name=),
os.mkdir(path=)) instead of raising TypeError, while still confining the write.
Adds TestRound19Bypasses and keyword-path runtime tests; full sandbox suite green.
Round 18 review follow-ups on the Studio code-exec sandbox classifier and runtime guard:
- Fail closed on a command-position command substitution ($(printf touch) / `printf touch`
as the command word): the expansion becomes the command name and cannot be proven safe. An
argument-position substitution (echo $(date), x=$(cmd)) stays allowed.
- Model direct imports of the os C backend (posix/nt -> os aliases) and pty: posix.system(...)
resolves as an os shell sink and pty.spawn(...)/pty.fork() is flagged as an unguarded child.
- Reject unbound sys.modules mutation (dict.pop(sys.modules, '_io'),
type(sys.modules).__delitem__(sys.modules, ...)) alongside the bound sys.modules.pop form.
- Treat mutating flags of normally read-only utilities as child writers: sed -i, sort -o FILE,
find ... -delete, dd of=FILE, tee FILE, truncate. Non-mutating uses stay allowed.
- Detect os / subprocess re-exported through a stdlib module (pathlib.os.system,
tempfile.os.system, subprocess.os.system): the .os attribute IS the os module.
- Resolve instance-attribute sink aliases (c.e = exec; c.e(payload) / obj.s = os.system;
obj.s('rm -rf /')) tree-wide, alongside the existing class-attribute aliases.
- Import the guard's remaining pure-Python dep (shutil) with the workdir still stripped from
sys.path; the restore now runs at the very end of the prelude, so no workdir/shutil.py can
shadow it.
- Network policy: resolve import aliases (import requests as r; r.get(...)) and inspect the
url= / address= keyword arguments so aliased or keyword-host calls to a metadata / untrusted
host are no longer skipped.
Adds TestRound18Bypasses and extends the workdir-shadowing runtime test to shutil; full
sandbox suite green.
Round 17 review follow-ups on the Studio code-exec sandbox classifier and runtime guard:
- Track a code object built through a LOCAL compile alias (cfn = compile; co = cfn(src);
types.FunctionType(co, {})()) by resolving the callee through the scope exec-builtin map,
so the FunctionType execution gadget still gets the recursive payload analysis.
- Recognize type(lambda: None) as the function constructor: type(lambda: None) IS
types.FunctionType, so type(lambda: None)(code, {})() executed a compile() code object
without the eval/exec gate.
- Include lambda and comprehension scopes in the alias index: (lambda e=exec: e(payload))()
and [e(payload) for e in [exec]] now resolve e back to the exec sink. Lambdas/comprehensions
become their own alias scopes, and a one-element comprehension generator binds its target.
- Record annotated single-assignment aliases (e: object = exec; e(payload)) alongside plain
assignments, so the AnnAssign RHS is analyzed.
- Expand ${IFS} / $IFS to whitespace before shell command matching, so a separator-obfuscated
writer/reader (rm${IFS}-rf${IFS}/, cat${IFS}/etc/shadow) is tokenized as bash runs it.
- Import the child-guard's stdlib deps (os/io/pathlib/re) with the workdir stripped from
sys.path, then restore it, so a malicious workdir/os.py or pathlib.py cannot shadow a guard
import and run unguarded at import time.
- Fail closed on shell redirection to any real-file target (the unguarded child follows a
pre-existing symlink); only fd duplications (>&2) and the standard device sinks
(/dev/null, ...) are allowed. This also fixes a pre-existing false positive where a benign
redirect to /dev/null was blocked.
Adds TestRound17Bypasses and a workdir-shadowing runtime test; full sandbox suite green.
Round 16 review follow-ups on the Studio code-exec sandbox classifier and runtime guard:
- Apply the runtime sensitive-read backstop to Path.open reads (not just write
modes), so a dynamically assembled pathlib receiver (Path(globals()['P']).read_text())
cannot exfiltrate a host secret. On Python <= 3.11 pathlib holds the original io.open,
so confining at the public Path.open level is the version-robust fix. read_text /
read_bytes route through the same self.open() and are covered.
- Scan the -c payload of a subprocess shell argv for sensitive reads. subprocess.run(
['sh', '-c', 'head -1 /etc/passwd']) has no blocked command, but the unguarded child
prints the secret, so the payload is now tokenized and read-scanned like a string sink.
- Normalize bash ANSI-C ($'...') and locale ($"...") quoting before command matching.
shlex leaves $'touch' as the literal $touch, so a writer / interpreter hidden behind
ANSI-C quoting ($'touch' x, $'\x74ouch' x) previously evaded the command blocklist;
the escapes bash resolves (\n, \xHH, octal, \uHHHH) are decoded first.
- Recognize from-imported subprocess exec names as read sinks: from subprocess import
run as r; r(['cat', '../../etc/shadow']) now hits the traversal / sensitive-read check.
- Treat a shell glob that can expand outside the workdir (absolute / ~ rooted, e.g.
head /etc/shad*) as an escaping read expansion and fail closed, for reader arguments
and input redirects. A relative in-workdir glob (grep foo *.txt) stays allowed.
Adds TestRound16Bypasses and pathlib runtime backstop tests; full sandbox suite green.
Round 15 review follow-ups on the Studio code-exec sandbox classifier and runtime guard:
- Resolve single-assignment aliases for shutil.copy* and subprocess exec read
sinks (c = shutil.copy; c('../../etc/passwd', ...); r = subprocess.run; r([...]))
so the traversal / sensitive-path check fires on the aliased callee.
- Block gc.get_referents / get_referrers / get_objects (and from-import aliases):
they walk the object graph to a guarded wrapper's closure cell to recover the
original unguarded open / os.* callable.
- Cap list / tuple concatenation during constant folding so a doubling chain
(a + a + a + ...) cannot materialize an oversized sequence in the parent process
before the child rlimits apply.
- Add a runtime sensitive-read backstop in the child prelude: deny a read whose
realpath resolves to a known host secret (SSH / cloud / kube / netrc / HF-token /
/etc/passwd family / /proc) outside the workdir. This covers opaque read paths the
static scanner cannot fold (open(globals()['x'])) and pre-existing in-workdir
symlinks to secrets, while leaving benign outside reads and library imports intact.
- Fail closed on relative multi-component shell redirect targets (echo x > sub/out.txt)
whose subdirectory component could be a symlink traversing outside the workdir; a
bare single-component target stays allowed.
Adds TestRound15Bypasses and runtime backstop tests; full sandbox suite green.
Static classifier:
- Flag mro().pop(i) / __mro__.pop(i) base-class extraction alongside the subscript and __getitem__ forms.
- Normalize operator.attrgetter('name')(obj) as attribute access whether or not the result is immediately invoked, so attrgetter('__closure__')(open)[0] gadget recovery is caught.
- Resolve a container-hidden open alias (o = [open][0]; o('../../etc/passwd').read()) as a read callee.
- Fail closed on an opaque read path assembled from obfuscation primitives (open(''.join(map(chr, ...))).read()), matching the exec-payload obfuscation policy.
- Treat cd behind the command / builtin shell wrappers as a cwd escape before allowing a relative redirect.
- Block importlib file loaders as execution sinks (SourceFileLoader(...).load_module(), spec.loader.exec_module(...)).
- Add the in-cluster Kubernetes service-account credential path to the sensitive-read list.
Adds TestRound12Bypasses covering each vector plus benign controls.
Static classifier:
- Flag __getattribute__/__getattr__ with a non-foldable (runtime-assembled) attribute name as an obfuscated gadget access.
- Recognize POSIX noclobber redirect targets (>| path) in the shell redirect scanner.
- Detect a container-wrapped compile() code object passed to types.FunctionType.
- Flag mro().__getitem__(i) / __mro__.__getitem__(i) base-class extraction alongside the subscript form.
- Scan shell command strings (os.system / subprocess shell=True / getoutput) for embedded sensitive-file reads.
- Treat a default-parameter value that is a dangerous callable (def f(e=exec)) as a sink alias.
- Normalize a trailing .__call__ for shell/import/deserializer sinks (os.system.__call__, __import__.__call__, pickle.loads.__call__).
- Resolve a container-unwrapped sink assigned first (s = [os.system][0]; s(...)).
- Look through no-op pathlib methods (resolve/absolute/expanduser) when resolving a read receiver.
- Recognize from-imported shutil copy sinks (from shutil import copy as c).
- Extend the higher-order first-class-value check to shell and deserializer sinks (map(os.system, ...), partial(subprocess.getoutput, ...), map(pickle.loads, ...)).
- Normalize operator.attrgetter('name')(obj) as attribute-access obfuscation.
Adds TestRound11Bypasses covering each vector plus benign controls.
Static classifier:
- Flag mro()[i] base-class extraction alongside subscripted __mro__ (FileIO C base recovery).
- Fail closed on non-literal shell redirect targets and cd to an outside directory.
- Block sys.modules mutating methods (pop/popitem/clear/setdefault/update) that drop a guarded module for reimport.
- Detect indirect eval/exec: <builtin>.__call__(payload) and eval/exec/compile passed by reference to a higher-order call (map/reduce/partial), including starred literals.
- Block inspect.getclosurevars() closure recovery of a guarded wrapper.
- Track runpy run_path/run_module from-import aliases.
- Expand starred literal path arguments in the sensitive-read scanner.
- Resolve container-hidden deserializers (([pickle.loads][0])(payload)).
Runtime backstop:
- Guard the low-level posix/nt chdir (cwd escape) and fchmod/fchown fd metadata mutators, matching the os.* deniers.
Adds TestRound10Bypasses and low-level posix runtime tests.
Static classifier:
- deny sys.modules Store/Del: del sys.modules['posix']; import posix drops the guard-patched module for a fresh unwrapped C module
- flag code.InteractiveInterpreter().runcode / InteractiveConsole().runsource as code-object execution sinks (opaque compile results run un-analyzed)
- resolve indirect read-only open callees: from os/io/builtins import open as X (os.open read-only is deliberately allowed outside the workdir, so traversal must be caught statically)
- fold os.path.normpath / abspath on literals so a traversal that only emerges after normalization is scanned
- resolve inline-container-hidden exec/eval: ({'e': exec}['e'])(...) / [exec][0](...)
- normalize the bound one-arg __getattribute__ form obj.__getattribute__('name') (builtins.open.__getattribute__('__closure__'))
- descend into a literal list/tuple argv so subprocess.run(['cat', '/etc/passwd']) is caught
- enforce the analyzer node budget: charge each tree's node count against _MAX_ANALYZER_NODES and fail closed above it (parent-process DoS guard)
Runtime backstop:
- capture and re-pin os.lstat / os.readlink / os.getcwd / os.stat before realpath, since it consults the live symlink helpers -- monkeypatching os.lstat to fail could stop realpath following an in-workdir symlink that points outside
Static classifier:
- flag cell_contents (the only closure-cell reader) as a gadget so recovering a guarded wrapper's original callable via __closure__ fails closed even when the __closure__ name is built at runtime
- treat a class body as its own alias scope (class C: e = eval; e(...) now recognized) while keeping methods lexically skipping the class scope, so a same-named class attr does not shadow the module-level sink a method reaches
- block output redirection (> / >> / &> / N>) to an absolute / ~ / .. target: a child shell runs unguarded; relative in-workdir redirects stay allowed
- resolve pathlib constructor import aliases (from pathlib import Path as P) before traversal reads
- flag an integer-indexed __mro__ (io.FileIO.__mro__[1]) that extracts the unguarded FileIO C base class; plain iteration / slicing stays allowed
- resolve aliased read callees before traversal checks: o = open and import shutil as sh; sh.copy(...)
- recurse into env -S / --split-string operands so env -S 'python3 -c ...' still detects the interpreter
Static classifier:
- fix scope-local walker so a nested def/class reassigning an alias name no longer inflates the outer single-assignment count and drops a real module-level sink alias
- track non-bare compile aliases (builtins.compile, from builtins import compile as comp) for the types.FunctionType(c) code-object gadget
- block child-process file writers at shell command position (touch/tee/cp/mv/mkdir/install/truncate/mkfifo/mknod/shred/unlink): a spawned child runs without the in-process write guard
- expand a literal **{...} unpack in the read scanner so open(**{'file': '../../etc/passwd'}) is resolved
- resolve a pathlib expression bound to a single-assignment name before read methods (p = Path('..')/'etc'/'passwd'; p.read_text())
- keep a wrapper's separated option argument in command position so stdbuf -o L python -c ... still detects the interpreter (env -i rm still caught; no FP on grep patterns)
- treat object.__getattribute__ / type.__getattribute__ as attribute obfuscation, covering gadget dunders and sensitive-module attrs (also closes __closure__ recovery of a guarded wrapper's original callable)
- block runpy.run_path / runpy.run_module execution sinks
- treat shutil.copy*/move SOURCE as a read callee so a .. traversal source is caught
Runtime backstop:
- normalize a bytes realpath (fsdecode) before the workdir prefix compare so a legitimate in-workdir bytes write is not denied by a TypeError; outside bytes writes still denied
Static classifier:
- resolve pathlib expressions passed to open()/read callees so open(Path('/etc') / 'passwd') blocks like open('/etc/passwd')
- flag getattr()/setattr() of an introspection gadget dunder (__globals__, __subclasses__, ...) regardless of receiver
- add the .get() twin of the globals()/locals()/vars() namespace-dict subscript guard
- constant-fold sys.modules[...] and sys.modules.get(...) keys so a concatenated key is caught
- track 'from builtins import __import__ as imp' as a dynamic import alias
- treat deserializer module aliases (pickle, dill, ...) as sensitive targets for getattr/vars/__dict__
- flag code objects executed through types.FunctionType(compile(src, ...), ...), including the c = compile(src); FunctionType(c) two-step
- block language interpreters (python/perl/ruby/node/...) at shell command position: a spawned child runs without the in-process write guard
Runtime backstop:
- pin os.fspath/os.path.realpath to captured originals inside _within so a sandboxed reassignment of os.fspath cannot make realpath resolve an outside write target to an in-workdir path
Scope-aware assignment aliases (extends the per-scope index):
- Resolve single-assignment aliases of dangerous callables in the call's own scope:
e = builtins.eval, im = importlib.import_module (and imp = __import__), and
l = pickle.loads (incl. aliased modules). Previously only bare-name and from-import
aliases were recognized.
- Count function parameters as local bindings so a parameter lexically shadows an
outer sink alias of the same name (fixes a false positive where def f(s): s(...)
with a module-level s = os.system flagged the parameter call).
Sensitive-read scanner:
- Resolve pathlib join receivers -- (Path('/etc') / 'passwd').read_text() and
Path('/etc').joinpath('passwd') -- not just a bare Path(...) constructor.
- Normalize path spellings (collapse redundant separators / '.' and resolve '..')
before the exact / dir checks, so /etc//passwd, /etc/./passwd and
/tmp/../etc/passwd are matched.
- Fold function-local single-assignment path constants (def f(): p = '/etc/passwd';
open(p)), not only module-level constants.
- Flag sys.modules.get('os') as the method-call twin of sys.modules['os'].
Runtime realpath backstop:
- Path.open coerces a str-subclass mode through the base str (matching the other open
wrappers) so a lying __contains__ cannot skip the write check.
- Path.rename/replace/link materialize the target once so a stateful __fspath__ cannot
return an in-workdir path for the check and an outside one for the real call (the
pre-3.11 accessor path where this wrapper is the only confinement).
Adds regression tests across the classifier, aliasing and runtime-backstop suites.
Scope-aware alias resolution (replaces the flat, module-wide alias maps):
- A new per-scope index resolves shell-sink, exec-builtin and compiled-code aliases
with Python lexical scoping. This fixes two problems the flat maps had: a safe
`c = compile('1+1')` in one function no longer shadows a dynamic `exec(c)` in
another (a real bypass), and a `s = os.system` in one function no longer makes a
benign `s = print` call in another look like a shell sink (a false positive), while
still catching a genuine function-local sink and honoring local shadowing of a
module-level alias.
Runtime realpath backstop:
- io.FileIO now passes the MATERIALIZED fspath to the real constructor (a stateful
__fspath__ could otherwise return an outside path to the C constructor).
- Deny an integer fd path for the mutating single-path wrappers (os.chmod(fd) etc.):
a read-only fd opened on an outside file could otherwise mutate host metadata.
Constant-folder allocation DoS:
- Refuse dynamic printf widths/precisions ('%*s', '%.*f') that draw their size from a
runtime argument.
- Bound str.replace / str.join output before it allocates (a long replacement over
many occurrences, or joining many long parts, can build a multi-gigabyte string).
Static classifier:
- Flag builtins / a sensitive module reached through the namespace dict:
globals()['__builtins__'], locals()[...] and globals()['os'].
Adds regression tests across the aliasing, runtime-backstop, const-fold and classifier
suites for every item above.
Runtime realpath backstop:
- Guard the low-level posix / nt module mutators (os re-exports from them, so
posix.open / posix.rename / ... stayed reachable with the originals).
- Guard io.FileIO / _io.FileIO constructors for write modes (a C constructor that
opens a file without routing through open()).
- Add os.mkfifo / os.utime / os.setxattr / os.removexattr (and lchflags) to the
guarded single-path mutators.
- Materialize fspath ONCE per call so a stateful __fspath__ cannot return a workdir
path for the check and an outside path for the syscall (TOCTOU).
- Coerce open() mode through the base str and os.open flags through the base int, so
a str-subclass __contains__ or an int-subclass __and__ cannot lie to the guard.
Constant-folder allocation DoS:
- Refuse str.format templates with a nested width field ({:{}}) driven by an
oversized numeric argument before format() allocates.
Static classifier:
- Reconstruct the full pathlib receiver path (all constructor args, joined) and
accept module-qualified pathlib.Path so Path('/etc', 'passwd').read_text() and
pathlib.Path(...) reads are inspected, not just single-arg bare Path(...).
- Treat builtins.__import__ / __builtins__.__import__ as a dynamic import.
- Count alias single-assignment per function scope instead of tree-wide, so two
functions binding the same local name no longer cancel out and miss a real sink.
Adds regression tests across the runtime-backstop, const-fold, aliasing and
classifier suites for every item above.
Fixes a further batch of P1 bypasses and analysis-time DoS vectors found in review.
Static classifier:
- Decode exec/compile bytes payloads the way CPython does (PEP 263 coding cookie
via tokenize.detect_encoding), then analyze the real source. A bytes payload whose
UTF-8 view is pure comments but whose utf-7 decode runs hidden code no longer slips
through; a payload decoding to a blocked op blocks, a benign one stays allowed.
- Resolve exec-builtin aliases assigned in nested scopes (def f(): e = exec; e(...)),
matching the shell-sink aliasing (stored-once guard keeps it low false-positive).
- Treat deserializer modules (pickle/marshal/dill/...) as dangerous dynamic-import
targets so __import__('pickle').loads(blob) is caught.
- Flag vars(os) / vars(__builtins__) as a module-__dict__ obfuscation, like os.__dict__.
- Inspect the pathlib receiver path for read methods: Path('../../.ssh/id_rsa').read_text()
/ read_bytes() / open() now check the constructor path, not only call args.
- Fold literal os.path.join / posixpath.join so open(os.path.join('/etc','passwd')).read()
is seen by the sensitive-read scanner instead of treated as opaque.
Constant-folder allocation DoS (folding runs in-process, before subprocess rlimits):
- Reject oversized f-string / str.format / %-format widths and precisions before
format() allocates the padded string.
- Reject oversized str padding-method widths (ljust/rjust/center/zfill).
- Cap list/tuple repetition (seq * n) as str/bytes repetition already was.
Runtime realpath backstop:
- Do not publish __wrapped__ on the guard wrappers (functools.wraps would expose the
original unguarded callable, e.g. open.__wrapped__(outside, 'w')).
- Guard the low-level _io.open entry point (io.open / builtins.open originate there).
- Confine os.chdir to the workdir and deny os.fchdir so a cwd escape cannot turn a
later relative read/write into a host-path access.
- Deny fd-based metadata mutators (os.fchmod / os.fchown) that could reuse a read-only
descriptor opened on an outside file.
Adds regression tests across the const-fold, aliasing, exec-recursion and runtime-
backstop suites for every item above.
Harden the code-exec sandbox against bypasses raised in review, keeping the
static gate a pure classifier (it never executes the tool call):
- Shell scan: `timeout` duration/float args (5m, 0.5, 2h) no longer drop the
following command out of command position, and `find -exec CMD ... ;` rescans
the whole slice so a wrapped `env`/`timeout`/`sh -c` target is still caught.
- exec/eval/compile of a bytes payload that is not valid UTF-8 Python now blocks:
those sinks honor PEP 263 coding cookies (e.g. utf-7) that the static UTF-8 view
cannot see; plain ASCII bytes payloads stay allowed.
- Resolve aliased/indirect reaches to exec, dynamic import, sensitive modules and
deserialization sinks: builtins.eval / __builtins__.exec, from builtins import
exec as e, importlib aliases, sys.modules[...] (and the getattr form), os.__dict__,
posix/nt, and pickle/marshal module-and-symbol aliases plus the *.load variants.
- Shell-sink aliasing walks the whole tree, so a function-local `s = os.system`
alias is resolved (the stored-once guard keeps it low false-positive).
- Drop __mro__ / __code__ from the introspection-gadget dunders: on their own they
do not reach an execution primitive and are read by ordinary ML/debug code.
- Refuse folding oversized `bytes(n)` / `bytearray(n)` so static analysis cannot OOM.
Runtime realpath backstop:
- Fail closed on a mutating dir_fd / src_dir_fd / dst_dir_fd (an fd-relative path
cannot be confined by a string realpath) for os and shutil mutators.
- Confine Path.rename/replace/symlink_to/hardlink_to when the destination is passed
as the `target=` keyword, not only positionally.
- Splice the guard after a leading docstring and `from __future__` imports instead
of prepending it, so future-import programs no longer raise SyntaxError while the
sandbox is still established before the first real statement.
Adds regression tests for each gap across the shell, const-fold, exec-recursion,
aliasing and runtime-backstop suites.
On Python <= 3.11, pathlib._NormalAccessor captures io.open (and os.* mutators)
into class attributes at pathlib import time. A C builtin captured there does not
bind on instance access, but a Python wrapper does: self shifts into the next
positional, so Path.open / Path.write_text raised 'open() argument mode must be
str, not PosixPath' once the guard had replaced io.open with a Python wrapper
before pathlib was imported. (3.12+ dropped the accessor, which is why it only
failed on the 3.10 CI leg.)
Import io and pathlib at the top of the guard, before any patching, so the
accessor captures the original builtins, and confine Path.open by wrapping the
public method directly (mode-aware) rather than relying on the io.open patch to
reach it. Direct io.open() writers are still guarded for the zipfile-based cases.
The static filesystem write-confinement (the LOCAL/ESCAPE/UNKNOWN path resolver
_resolve_path / _resolve_path_call plus the _FS_* mutating-op inventory) was the
largest and most complex part of the classifier, and for writes it duplicated the
runtime realpath backstop, which is strictly more robust: it resolves the true
realpath at the syscall boundary, so it also catches dynamic paths, pre-existing
symlinks, and library writers the static pass could not prove.
Make the runtime backstop the single filesystem-write boundary and delete the
static resolver:
- Harden the backstop to close the gaps the static layer used to cover: guard the
low-level os.open (any mutating flag confines the target; a mutating dir_fd fails
closed) and io.open (which also carries pathlib.Path.open('w')), and add
os.mknod / lchmod / lchown / chflags and shutil.chown / copymode / copystat to
the wrapped set. Native-C writers (cv2.imwrite) and the realpath TOCTOU window
remain documented residuals that only OS-level isolation can close.
- Remove _resolve_path, _resolve_path_call, _resolve_join, _classify_path_string,
_is_pathlib_expr and the _FS_* / _PATHLIB_CTORS / _PATH_DEPTH_CAP constants, and
the write half of the filesystem visitor plus the FS_READ_STRICT knob.
- Reads are not confined by the backstop, so keep a small static sensitive-read
scanner (_is_sensitive_abs_path) that still blocks host-secret reads via a
sensitive absolute / ~ literal in any call arg (covers open, os.open, and library
loaders such as pandas.read_csv('/etc/passwd')) and .. / ~ traversal on the
dedicated open/read callees.
Net: about 300 fewer lines in tools.py and one fewer concept to audit; static
analysis now scopes to exec, shell, network and sensitive-reads while writes are
confined at runtime. Rework the filesystem tests around the new contract and add
os.open / io.open / Path.open / dir_fd escape cases to the backstop suite.
* Studio: add Vulkan llama.cpp support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address gemini's feedback
* Studio: move the Vulkan VRAM probe into a standalone script
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Improve Vulkan probe error reporting
* Resolve llama-server symlink so Vulkan build is detected
* Drop unreachable Vulkan fallback in GPU free-memory dispatcher
* Skip the Intel GPU probe when NVIDIA or ROCm is present
* Reserve host RAM headroom for Vulkan integrated GPUs
* Add a `UNSLOTH_FORCE_VULKAN` environment variable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor GGML_VK_VISIBLE_DEVICES, reserve discrete Vulkan VRAM headroom, and clear Intel GPU on --cpu-fallback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Route Intel and forced-Vulkan hosts to the upstream Vulkan prebuilt, add arm64 Vulkan, keep Vulkan out of RAG auto-detect
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Clear the fork release pin when routing a Vulkan host to the upstream repo
* Gate auto-Vulkan routing on no physical NVIDIA so hidden CUDA devices aren't used
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Pin Vulkan launches with --device Vulkan<i> instead of the raw GGML_VK_VISIBLE_DEVICES index space
* Let user --device override the Vulkan pin, and gate direct Vulkan asset picks on no physical NVIDIA
* Update RAG auto-backend test mocks for the _resolve_auto binary and Vulkan probes
* Keep the add_dll_directory handle alive through the Vulkan probe DLL loads
* Revert RAG auto Vulkan guard, guard multi-backend Vulkan detection, and preserve forced Vulkan across updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use getattr for RTLD_GLOBAL in the Vulkan probe CDLL mode
* Skip CUDA/ROCm APU and datacenter GPU tuning on Vulkan builds
On a Vulkan llama.cpp build gpu_indices are ggml compact ordinals, not
CUDA/ROCm physical ids, so _amd_apu_wants_unified_memory and
_apply_datacenter_env were reading the wrong device. On a mixed AMD APU
plus discrete GPU host that could raise a spurious system-RAM shortfall
and block a valid discrete-GPU load. Gate all three call sites on
not is_vulkan_backend; the Vulkan path already reserves iGPU host
headroom and the backend ignores GGML_CUDA_* anyway.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten Vulkan-guard comment in load_model
* Reduce comments in Vulkan support to be more succinct
* Resolve shell-wrapper llama-server entrypoint to the real lib dir
create_exec_entrypoint falls back to a #!/bin/sh wrapper at the install
root when it cannot symlink into build/bin. _find_llama_server_binary
returns that root entrypoint, but Path.resolve() does not follow a shell
wrapper, so _llama_lib_dir returned the install root and _is_vulkan_backend
missed libggml-vulkan.so -- silently skipping the Vulkan probe and --device
pin on an otherwise valid Vulkan install. Follow the wrapper's exec target
to build/bin. Regression test: test_shell_wrapper_entrypoint_resolves_to_real_lib_dir.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio /v1/messages: accept thinking and unknown content blocks
The Anthropic-compatible /v1/messages endpoint modeled a message's content as
Union[str, list[{text|image|tool_use|tool_result}]], so any other block type
made Pydantic reject the whole request with
`messages.N.content.str: Input should be a valid string`. Resuming a Claude
session commonly replays assistant turns that carry `thinking` (extended
thinking) blocks, and sometimes a null content for a tool-only turn, both of
which tripped this and returned a 400.
Accept them:
- Add a permissive AnthropicUnknownBlock fallback (any block whose type is not
one of the four known ones), so thinking/redacted_thinking/provider-specific/
future blocks validate. A validator keeps known types on their typed models,
so a malformed known block (e.g. a tool_use without id) still fails cleanly.
- Coerce a null message (and tool_result) content to "" so the converter's
`for block in content` stays safe.
The converter already drops block types it does not translate, so a thinking
block is not forwarded to the model.
* Studio /v1/messages: keep user content validation strict
Make the thinking/null leniency role-aware so it never silently drops real
user input. Assistant turns (replayed history) still accept unknown/thinking
blocks and coerce a null tool-only turn to empty. User turns keep the strict
boundary: a null user content is rejected, and a content block the converter
cannot translate is rejected instead of being dropped into an empty prompt.
Also remove an empty file committed by accident.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio /v1/messages: coalesce resumed user turns and tighten content checks
- The /v1/messages count and generation paths now coalesce the adjacent user
turns that dropping an empty or null assistant turn can leave behind, so a
strict GGUF chat template no longer 400s on non-alternating roles.
- A user content block with a non-string type (list / dict) is rejected as a
clean 400 instead of raising TypeError and escaping as a 500.
- The assistant null-to-empty coercion only applies to an explicit null; an
assistant turn that omits content entirely still fails required-field
validation instead of being silently coerced to an empty string.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio /v1/messages: tighten comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Tighten the eval/exec/compile dynamic policy so an executing sink (eval/exec/
runpy) applied to a payload that cannot be statically recovered is refused
unconditionally, not only when an RCE-core module happens to be imported in the
snippet. An un-analyzable executing payload can synthesize any shell, network,
or filesystem escape at runtime, so the prior in-scope-import heuristic left
exec(input()) and eval(user_var) allowed whenever no such import was present.
compile() of the same payload stays allowed since it does not run.
A payload that is fully recovered as a constant but is invalid Python for the
sink's mode (for example eval("data = 1") or eval("not python !!")) is now
allowed: its exact source is known and it raises SyntaxError at runtime, so it
is not an execution vector. Only genuinely opaque, non-recoverable payloads
(for example eval of a runtime-computed f-string) reach the block.
Remove the now-unused _RCE_CORE_MODULES set and _scope_imported_roots helper,
and move the opaque-f-string case in the tests to the blocked set.
Add ln to the bash command denylist so a symlink escape cannot be created from
the terminal tool. In the sandboxed (non-bypass) _python_exec path, prepend a
one-line guard to the generated temp module that monkeypatches only MUTATING file
ops (builtins.open in write/append/x/+ modes, os remove/unlink/rmdir/removedirs/
rename/renames/replace/truncate/chmod/chown/mkdir/makedirs/symlink/link, shutil
rmtree/move/copy/copy2/copyfile/copytree, pathlib write_text/write_bytes/unlink/
rename/replace/mkdir/rmdir/chmod/symlink_to/hardlink_to/touch) to resolve the true
os.path.realpath of the target and raise PermissionError unless it lands inside
the injected session workdir. Reads are left unpatched. The guard runs in its own
namespace so helper names never leak into user globals, and it is skipped
entirely under disable_sandbox. This catches what the static gate cannot prove:
pre-existing symlink escapes and dynamic library-writer paths that funnel through
builtins.open. Benign in-workdir relative writes and library imports are
unaffected (importlib swallows out-of-workdir bytecode-cache write failures).
Add pragmatic aliasing so an aliased shell sink with a dangerous argument is
caught: a name stored exactly once and bound to a resolved os/subprocess sink
(s = os.system; s('rm -rf /')) and inline literal-container indexing
([os.system][0](...), (os.system,)[0](...), {'k': os.system}['k'](...)) both feed
the existing _find_blocked_commands argument check. Resolution is deliberately
low-false-positive: only unambiguous single assignments and inline literal
containers, never a flow-insensitive union, so s = os.system; s = print; s('hi')
is not aliased. The shell-sink set is lifted to module scope (_SHELL_SINK_FUNCS)
so the alias pre-pass and the visitor share one definition. Interprocedural and
flow-sensitive taint remain out of scope (deferred to a full fixpoint).
Add a filesystem_violations category backed by _resolve_path, a LOCAL / ESCAPE /
UNKNOWN classifier that constant-folds strings and understands os.path.join,
pathlib Path()/'/'/joinpath, and f-strings with real join plus absolute-reset
semantics. expanduser / expandvars / os.environ / getcwd / dynamic parts collapse
to UNKNOWN. A new _FilesystemPolicyVisitor inventories destructive and mutating
ops (open write/append/x/+, os remove/unlink/rmdir/rename/replace/truncate/chmod/
chown/mkdir/makedirs/mknod/symlink/link/chdir, shutil rmtree/move/copy*, pathlib
write_text/write_bytes/unlink/rename/replace/mkdir/rmdir/chmod/symlink_to/touch,
tempfile dir=, and a curated numpy/pandas/torch/joblib/PIL/matplotlib/cv2 writer
set) and applies prove-or-block: mutating LOCAL allows, UNKNOWN/ESCAPE blocks.
rename/move check src and dst; symlink/link check both target and link path;
chdir must be LOCAL; tempfile dir= must be LOCAL. Reads block only on a provable
escape (sensitive absolute path or ..'/~ traversal), with an FS_READ_STRICT knob
for prove-or-block reads. A callee-independent literal-sensitive-path scan blocks
loaders like pandas.read_csv('/etc/shadow'). Library writers block only on a
provable escape so in-memory buffers are not over-blocked; the Stage 5 runtime
backstop covers the dynamic residual.
Replace the blanket dynamic_exec block with a recursive payload analyzer gated by
UNSLOTH_STUDIO_SINK_ANALYZER (default on; =0 restores the legacy ban). For eval /
exec / compile (and single-assignment aliases like e = exec), constant-fold the
first argument; a recovered source string is bracket-depth pre-scanned, size and
recursion-depth bounded, then re-classified through the full analyzer. An inner
sink blocks and surfaces the inner reason; a clean inner payload allows; a bound
or budget breach fails closed. Non-foldable payloads follow a low-false-positive
dynamic policy: block when assembled from decode / fetch / runtime-assembly
primitives (including nested exec and large string repetition) or when an
RCE-core module is imported in scope, else allow.
Keep the gadget-dunder and dynamic-import blocks but constant-fold import names
so __import__('hugging'+'face_hub') resolves to a real module. Refine getattr /
setattr on a sensitive module so a benign constant attribute (getattr(os,
'getpid')) is allowed while a dynamic or dangerous constant attribute blocks. Add
pickle/marshal/dill.loads as unverifiable code-deserialization sinks. eval('2+2'),
compile('a+b','<s>','eval') and ast.literal_eval now pass; base64/hex/rot13/chr
and gadget-obfuscated escapes still block. Wire a filesystem_violations category
through is_safe, the info dict, and the reason assembly (populated in a later
stage). The three legacy tests that asserted the blanket ban are updated to the
new recurse-the-payload behavior.
Introduce _const_fold, a whitelist-only, bounded, side-effect-free partial
evaluator plus a single-assignment const-prop environment builder. It recomputes
pure transforms on literals only (concat, repeat, join, format, f-strings,
slice/reverse, chr/ord, base64/hex/rot13/zlib decode, pure builtins and string
methods) and never executes, imports, or reflects on user code. Depth, op, size,
and sequence caps guarantee it can only fail to recover a value, never crash or
hang. This is the foundation the later eval/exec unwrapping and filesystem path
resolver build on.
* unstructured block removal
* Enhance unstructured block handling
* Restrict block cleanup to upload UIDs
* cleanup for seed block uploads
* upload cleanup queue for unstructured blocks in recipe studio
* Fix unstructured upload cleanup edge cases
* Fix unstructured upload import ownership
* Fix-unstructured-import-path-ownership
* Guard failed-delete restore against stale block in unstructured drop zone
* Drain queued upload cleanups when autosave is skipped
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: render thinking blocks for safetensors inference with prefilled <think> templates
Reasoning templates like Qwen3.6 end the generation prompt with an open
<think> tag. skip_prompt streaming drops it, so the frontend never sees
the opening tag and shows reasoning as plain text. Detect the prefill
and re-emit it at the start of the stream on the transformers and MLX
paths. Also stop stripping think tags in _clean_generated_text when a
tokenizer marks them special.
* Studio: guard think re-emit for special close tags, yield prefill early
Address review feedback:
- Guard: skip re-emitting the open <think> when the tokenizer marks </think>
as a special token, since skip_special_tokens would strip the model's close
tag and leave an unclosed block that swallows the answer. Falls back to
plain text (pre-fix behaviour) for those tokenizers.
- Yield the prefilled <think> before the first token so the thinking block
renders during prompt prefill instead of after the first generated token.
- Drop the now-unnecessary _clean_generated_text think-tag exemption; the
guard handles the special-token case at the source.
No mainstream reasoning model (Qwen3.6, Qwen3, DeepSeek-R1, QwQ, GLM-4.6)
marks think tags special, so behaviour is unchanged for them.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lyxot <longyixing331@gmail.com>
