The latest huggingface-hub release added the Sandboxes feature. Its
bootstrap (_sandbox.py) fetches the static sbx-server binary into /tmp with
an Authorization header and marks it executable, which is exactly the
staged-dropper pattern the scanner hunts, and three while True polling loops
in _sandbox.py / hf_api.py / utils/_http.py match the beaconing heuristic.
All four verified against the official huggingface/huggingface_hub
repository: the snippet is the documented sandbox server injection and the
loops are deadline-style job and sandbox polling. Entries generated with
--write-baseline and reviewed line by line; scan_packages.py huggingface-hub
now exits 0 with the four findings suppressed.
The pip scan-packages job keys its baseline on the matched-code hash (main's
scanner). This branch's dependency set resolves a newer fastapi (its routing
while-True loop hashes differently than the baselined one) and adds gguf,
whose HF_TOKEN-authenticated download helper trips the env+network check.
Both reviewed benign: the fastapi hit is its own websocket routing loop and
the gguf hit is the official package's Hub download path. Sync the scanner
and baseline from main and add the two reviewed entries.
check_js_file put the bundle's KB size inside the finding's check label, which is
part of the baseline match key (package, file, check). When tensorboard's
projector_binary.js grew from 1918 KB to 1933 KB, the reviewed baseline entry stopped
matching and the benign HIGH resurfaced, red-failing the studio and extras
scan-packages shards. Move the size into the evidence field (shown for review, not
matched) and keep the check label constant, then update the one tensorboard baseline
entry to the size-agnostic label. The finding is suppressed again and will not
re-break when the bundle grows by a few KB. Scanner self-tests pass unchanged.
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: drop OpenEnv and unused ExecuTorch/open_spiel install deps
* Studio: drop 8 more unused install deps from extras
* Studio: restore tomli<3.11 for kernels; tidy dep-cleanup comments and tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: refresh scan-packages baseline for scipy _external + unsloth-zoo tests
scipy moved its vendored array_api_compat from scipy/_lib to
scipy/_external, so the four allowlisted array_api_compat __init__.py
entries stopped matching and resurfaced as unsuppressed CRITICAL
"Downloads and executes remote code" findings on all three pip
scan-packages shards (extras, hf-stack, studio). Add the _external
paths next to the existing _lib ones so both scipy layouts stay covered.
Allowlist two unsloth-zoo test-file false positives now present in the
hf-stack shard: tests/test_mlx_save_export_regressions.py (writes to
/tmp dropper) and tests/test_mlx_trainer_internals.py (obfuscation plus
exec/eval).
Drop nine stale entries for packages removed from the Studio
requirements and no longer in any shard closure (evaluate, pytest,
hypothesis, kgb, langid), confirmed absent via with-deps resolution of
all three shards.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Package scanners: cut false positives and make the CI gate blocking
scan_packages.py and scan_npm_packages.py red-failed on legitimate
library code, so the security-audit steps were left advisory. Reduce
the false positives at the source and flip both gates to blocking.
scan_packages.py:
- Scan code only: blank comments and bare docstrings/doctests before
matching (line numbers preserved), so prose and >>> examples cannot
trip a finding.
- Drop the platform.system() branch from the anti-analysis regex (under
DOTALL it matched across the whole file, so every cross-platform
library tripped it) and fix the dead /proc/self/status alternative.
- Add a reviewed baseline allowlist (scan_packages_baseline.json) keyed
on (package, basename, check): only non-baselined CRITICAL/HIGH exit
1, and a new kind of finding in a listed file still fails.
- sdist fallback: when --with-deps cannot resolve a shard (a sdist-only
package or a version conflict), drop to per-spec and fetch the raw
sdist from the PyPI JSON API (no pip build, no setup.py), so every
package is still scanned and no shard exits 2.
scan_npm_packages.py:
- Mirror the code-only JS/TS scanning (blank // and /* */ comments,
string/template/regex aware) and the baseline allowlist. The npm
corpus is clean today, so the baseline is empty.
security-audit.yml:
- Flip both scan steps to blocking (SCAN_ENFORCE=1), capturing the
scanner exit via PIPESTATUS so tee does not mask it.
tests/security: add coverage for the strip, baseline and sdist paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review feedback on the package scanners
- Do not blank f-strings during code-only scanning (they evaluate at
import); and when a file uses exec/eval, rescan the original for
payload carriers hidden in a docstring/string so exec(__doc__) style
payloads stay visible.
- sdist fallback: recover transitive deps with their version specifier
(fetch the pinned version, not latest), and recover deps in the
--no-deps branch too so a sdist-only transitive dependency is still
scanned instead of silently skipped.
- Baseline: key by package-relative path, not basename, so a future
same-named file in another directory is not auto-suppressed.
Regenerated the baseline accordingly.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
