ac765d2efb* studio/ci: pre-install lockfile supply-chain audit (npm + cargo)
The Mini Shai-Hulud wave that hit @tanstack/* on 2026-05-11 19:20-19:26
UTC (GHSA-g7cv-rxg3-hmpx) pushed 84 malicious versions across 42
packages. Each compromised tarball carried an `optionalDependencies`
entry pointing at a GitHub-hosted prepare script that exfiltrated
GitHub / npm / AWS / Vault / SSH credentials on `npm install` / `npm
ci`. Our current lockfile pins ALL @tanstack/* at pre-malicious
versions so we were not exposed, but the only defense layer between
"dependabot opens a security-update PR during a malicious window" and
"a compromised package's postinstall runs on the CI runner" is the
advisory-DB latency. `npm audit` and OSV-Scanner are reactive: there
is a window between malicious publication and GHSA landing.
Add a pre-install lockfile audit that fires on the injection pattern
itself, BEFORE `npm ci` gets a chance to execute lifecycle scripts:
scripts/lockfile_supply_chain_audit.py
npm side (studio/frontend/package-lock.json, lockfileVersion 2/3):
1. every `resolved` URL must point to registry.npmjs.org;
direct GitHub / git+ / file: refs are the Shai-Hulud vector
2. every non-bundled entry must carry an `integrity` SHA
3. raw-text scan for known IOC strings (router_init.js,
tanstack_runner.js, router_runtime.js, @tanstack/setup,
the specific TanStack worm commit hash, getsession.org
exfiltration host, "A Mini Shai-Hulud has Appeared" marker)
4. nested `node_modules/.../node_modules/` fold-ins are
transparent -- they ride on the parent tarball's integrity
cargo side (studio/src-tauri/Cargo.lock):
5. every `source` must be the crates.io registry
6. registry crates must have a `checksum`
7. one allowlist entry: fix-path-env from
tauri-apps/fix-path-env-rs at pinned SHA c4c45d5. Any other
non-registry source -- or a bump of that pinned SHA --
re-fires the audit until reviewed + appended
Wire into four workflows:
.github/workflows/security-audit.yml -- new step inside the
advisory-audit job, immediately before `npm audit` so the
structural pass and the advisory-DB pass appear together in
the GitHub step summary.
.github/workflows/studio-frontend-ci.yml,
.github/workflows/wheel-smoke.yml,
.github/workflows/studio-tauri-smoke.yml -- new step immediately
BEFORE `npm ci`. If a future malicious bump lands in our lockfile,
the audit refuses and `npm ci` never runs, so no `prepare` /
`postinstall` from a compromised tarball can execute on the
runner.
Note on --ignore-scripts: every npm ci in our CI is followed directly
by `npm run build` or `tauri build`, both of which depend on package
install scripts (esbuild's native-binary postinstall, etc.). Blanket
--ignore-scripts breaks the build, so the pre-install structural
audit is the practical mitigation. The audit reads lockfiles only;
it never executes anything from them.
Verified:
- Clean state: 0 findings on the current tree (npm + cargo).
- Fault injection: synthetic `@tanstack/setup` IOC + non-registry
`resolved` URL both fire with exit code 1.
- YAML parses cleanly for all four modified workflows.
Refs:
- https://tanstack.com/blog/npm-supply-chain-compromise-postmortem
- https://github.com/TanStack/router/issues/7383
- https://github.com/TanStack/router/security/advisories/GHSA-g7cv-rxg3-hmpx
- https://www.aikido.dev/blog/mini-shai-hulud-is-back-tanstack-compromised
- https://www.stepsecurity.io/blog/mini-shai-hulud-is-back-a-self-spreading-supply-chain-attack-hits-the-npm-ecosystem
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
817 lines
40 KiB
YAML
817 lines
40 KiB
YAML
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
|
|
|
|
# Multi-language supply-chain audit. Triggers:
|
|
# - PRs touching any dependency manifest (Python / npm / Cargo) or
|
|
# this workflow file,
|
|
# - push to main / pip,
|
|
# - nightly @ 04:13 UTC so newly-published advisories surface even
|
|
# when no PR opens,
|
|
# - workflow_dispatch for ad-hoc invocations.
|
|
#
|
|
# Two jobs:
|
|
# - advisory-audit: one runner that runs pip-audit + npm audit +
|
|
# cargo audit back-to-back. All three are
|
|
# advisory-DB lookups -- fast, lockfile-driven,
|
|
# no archive download. Setting up the python /
|
|
# node / rust toolchains on one runner and
|
|
# running the three commands serially is
|
|
# cheaper than spinning up three runners.
|
|
# - pip-scan-packages: 3-shard matrix that downloads + pattern-scans
|
|
# every PyPI archive in the transitive closure.
|
|
# This is the expensive job (~6 min/shard,
|
|
# running in parallel) and it must stay
|
|
# independent so a CVE-DB hit in advisory-audit
|
|
# does not block the supply-chain pattern scan
|
|
# (or vice versa).
|
|
#
|
|
# All steps are non-blocking initially. The default branch already
|
|
# carries a known-vuln backlog (the dependabot banner shows 17 today,
|
|
# pip-audit catches 2 more, npm/cargo will catch their own); a hard
|
|
# gate now would block every PR on a baseline we have not triaged.
|
|
# As each baseline closes, drop continue-on-error per step.
|
|
#
|
|
# Dependency coverage:
|
|
# - unsloth core (pyproject.toml [project.dependencies])
|
|
# - unsloth `huggingfacenotorch` extras (the canonical install path
|
|
# for fine-tuning users; pulls transformers / peft / accelerate /
|
|
# trl / datasets / diffusers / sentence-transformers / etc.)
|
|
# - all six Studio backend requirements files
|
|
# - Studio frontend (npm) and Tauri shell (cargo)
|
|
# Each Python step builds a filtered dep list from pyproject.toml +
|
|
# requirements/*.txt before auditing. We do NOT install any of these
|
|
# -- pip-audit resolves through PyPI metadata, scan_packages.py
|
|
# downloads sdist/wheel archives and inspects them without running
|
|
# install hooks, so an attacker who has compromised a transitive dep
|
|
# cannot execute code in this workflow.
|
|
|
|
name: Security audit
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- 'studio/backend/requirements/**'
|
|
- 'studio/frontend/package.json'
|
|
- 'studio/frontend/package-lock.json'
|
|
- 'studio/src-tauri/Cargo.toml'
|
|
- 'studio/src-tauri/Cargo.lock'
|
|
- 'pyproject.toml'
|
|
- 'scripts/scan_packages.py'
|
|
- '.github/workflows/security-audit.yml'
|
|
push:
|
|
branches: [main, pip]
|
|
schedule:
|
|
- cron: '13 4 * * *' # 04:13 UTC daily, off the cron rush
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
# ─────────────────────────────────────────────────────────────────────
|
|
# Combined advisory-DB audit: pip-audit + npm audit + cargo audit
|
|
# all on one runner. Each step is continue-on-error so a finding in
|
|
# one toolchain does not suppress the others.
|
|
# ─────────────────────────────────────────────────────────────────────
|
|
advisory-audit:
|
|
name: advisory audit (pip + npm + cargo)
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 25
|
|
steps:
|
|
# step-security/harden-runner installs an eBPF-based egress
|
|
# firewall on the runner. In `audit` mode it logs every outbound
|
|
# connection without blocking; in `block` mode it rejects
|
|
# anything outside `allowed-endpoints`. We run audit-only
|
|
# initially: the next time this job hits a real PyPI advisory or
|
|
# an attacker-funded archive in pip-scan-packages, the audit log
|
|
# tells us exactly which hosts were dialed and we promote the
|
|
# allowlist to block. Would have *contained* the litellm exfil
|
|
# even if scan_packages had missed the .pth payload.
|
|
# SHA-pinned (not @v2): the litellm 1.82.7 attack chain hijacked
|
|
# mutable tags on aquasecurity/trivy-action and would have hit
|
|
# anyone using @v0 / @v2 / @latest references. Pinning to a 40-
|
|
# char SHA freezes this action at known-good code; Dependabot's
|
|
# github-actions ecosystem will auto-bump the SHA.
|
|
# v2.19.1 commit:
|
|
- name: Harden runner (egress audit)
|
|
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
|
|
with:
|
|
egress-policy: audit
|
|
disable-sudo: true
|
|
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
# Full history so TruffleHog can diff base..head; without
|
|
# this it sees only the latest commit and reports nothing.
|
|
fetch-depth: 0
|
|
|
|
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
|
with:
|
|
python-version: '3.12'
|
|
cache: 'pip'
|
|
|
|
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
|
|
with:
|
|
node-version: '22'
|
|
cache: 'npm'
|
|
cache-dependency-path: studio/frontend/package-lock.json
|
|
|
|
- uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable @ 2026-03-27
|
|
|
|
- uses: swatinem/rust-cache@23869a5bd66c73db3c0ac40331f3206eb23791dc # v2.9.1
|
|
with:
|
|
workspaces: studio/src-tauri -> target
|
|
|
|
- name: Install pip-audit + cargo-audit
|
|
# cargo-audit pulls advisories from the RustSec advisory-db on
|
|
# first run and caches them under ~/.cargo/advisory-db. Pin
|
|
# --locked so the version we install matches Cargo.lock
|
|
# determinism. cargo-audit 0.22 supports the CVSS 4.0 schema
|
|
# used in 2026 advisories (e.g. RUSTSEC-2026-0073); 0.21
|
|
# crashes with a TOML parse error on that file.
|
|
# npm audit is bundled with the node toolchain, no install.
|
|
run: |
|
|
python -m pip install --upgrade pip 'pip-audit>=2.7'
|
|
cargo install --locked --version '^0.22' cargo-audit
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# Python: pip-audit
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: Build filtered Python requirements set
|
|
# Two transforms:
|
|
# (1) Generate audit-reqs/unsloth-deps.txt from pyproject.toml
|
|
# so pip-audit sees the unsloth pip package's own dep set
|
|
# (core + huggingfacenotorch extras: transformers / peft /
|
|
# accelerate / trl / datasets / diffusers /
|
|
# sentence-transformers / huggingface_hub / hf_transfer /
|
|
# etc.).
|
|
# (2) Copy each studio/backend/requirements/*.txt into
|
|
# audit-reqs/ with `git+` lines stripped. pip-audit's `-r`
|
|
# mode does a dry-run resolve against PyPI metadata; a
|
|
# `git+https://...` spec forces it to clone, which is
|
|
# both slow and outside the threat model (we audit
|
|
# PyPI-served archives; a git ref is whatever HEAD says
|
|
# on the runner). A comment line is left in place so the
|
|
# skipped specs are obvious in the artifact.
|
|
# The `huggingface` extra is `huggingfacenotorch` plus torch /
|
|
# torchvision / triton, deliberately skipped: Studio backend
|
|
# already pins a torch and the +cu* / +cpu local-version tags
|
|
# trip up the PyPI resolver in `-r` mode.
|
|
run: |
|
|
mkdir -p audit-reqs
|
|
python <<'PY' > audit-reqs/unsloth-deps.txt
|
|
import tomllib
|
|
with open("pyproject.toml", "rb") as f:
|
|
d = tomllib.load(f)
|
|
core = d["project"]["dependencies"]
|
|
extras = d["project"]["optional-dependencies"]["huggingfacenotorch"]
|
|
print("# Auto-generated from pyproject.toml by security-audit.yml.")
|
|
print("# core deps + huggingfacenotorch extras.")
|
|
for spec in core + extras:
|
|
print(spec)
|
|
PY
|
|
for f in studio.txt extras.txt extras-no-deps.txt \
|
|
no-torch-runtime.txt overrides.txt triton-kernels.txt; do
|
|
python <<PY > "audit-reqs/$f"
|
|
src = "studio/backend/requirements/$f"
|
|
with open(src) as fh:
|
|
for line in fh:
|
|
stripped = line.strip()
|
|
before_comment = stripped.split("#", 1)[0]
|
|
if "git+" in before_comment:
|
|
print(f"# [security-audit] skipped git+ spec: {stripped}")
|
|
continue
|
|
print(line.rstrip("\n"))
|
|
PY
|
|
done
|
|
|
|
- name: pip-audit (declared Python deps, no install)
|
|
# `-r requirements.txt` resolves the requirements through pip's
|
|
# dependency resolver against PyPI metadata and audits the
|
|
# resolved tree without ever executing setup.py / install
|
|
# hooks. Way faster than installing the full Studio runtime
|
|
# and -- critically -- safer: an attacker who has compromised
|
|
# a transitive dep cannot run code in this job.
|
|
#
|
|
# extras.txt + extras-no-deps.txt have legacy setup.py
|
|
# packages (notably openai-whisper) whose setup.py imports
|
|
# `pkg_resources`, which the isolated build env's current
|
|
# setuptools no longer ships. PIP_CONSTRAINT pins an older
|
|
# setuptools into the build env so those builds resolve.
|
|
# Per-file loop so one bad file doesn't take out the whole
|
|
# audit.
|
|
continue-on-error: true
|
|
env:
|
|
PIP_CONSTRAINT: ${{ github.workspace }}/audit-reqs/build-constraints.txt
|
|
run: |
|
|
set +e
|
|
cat > audit-reqs/build-constraints.txt <<'CONSTRAINTS'
|
|
setuptools<78
|
|
wheel
|
|
CONSTRAINTS
|
|
: > logs-pip-audit.txt
|
|
for f in unsloth-deps studio extras extras-no-deps \
|
|
no-torch-runtime overrides triton-kernels; do
|
|
if ! grep -qE '^[^#[:space:]]' "audit-reqs/$f.txt"; then
|
|
echo "[security-audit] $f.txt has no PyPI specs after git+ filter, skipping" \
|
|
| tee -a logs-pip-audit.txt
|
|
continue
|
|
fi
|
|
echo "::group::pip-audit -r audit-reqs/$f.txt"
|
|
{
|
|
echo
|
|
echo "=== $f ==="
|
|
pip-audit -r "audit-reqs/$f.txt" --format=columns
|
|
echo "=== end $f (rc=$?) ==="
|
|
} 2>&1 | tee -a logs-pip-audit.txt
|
|
echo "::endgroup::"
|
|
done
|
|
{
|
|
echo "## pip-audit (Python)"
|
|
echo
|
|
echo '### Coverage'
|
|
echo '- unsloth core + `huggingfacenotorch` extras (pyproject.toml)'
|
|
echo '- studio/backend/requirements/{studio,extras,extras-no-deps,no-torch-runtime,overrides,triton-kernels}.txt'
|
|
echo '- `git+` specs are stripped before audit (out of scope: we audit PyPI archives)'
|
|
echo
|
|
echo '### Findings'
|
|
echo '```'
|
|
cat logs-pip-audit.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# Pre-install lockfile supply-chain audit (npm + cargo).
|
|
# Catches structural anomalies (non-registry resolved URLs,
|
|
# missing integrity hashes, known IOC strings) BEFORE `npm
|
|
# audit` or OSV-Scanner consult the advisory DB. The advisory
|
|
# path is reactive -- there is a window between a malicious
|
|
# publication and the GHSA landing. This step fires on the
|
|
# injection pattern itself so it catches the same class of
|
|
# attack the moment the lockfile shape becomes wrong.
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: Lockfile supply-chain audit (pre-install scan)
|
|
run: |
|
|
python3 scripts/lockfile_supply_chain_audit.py
|
|
{
|
|
echo "## Lockfile supply-chain audit"
|
|
echo
|
|
echo "Scanned: studio/frontend/package-lock.json + studio/src-tauri/Cargo.lock"
|
|
echo
|
|
echo "No structural anomalies or known IOC strings."
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# npm: Studio frontend
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: npm audit (Studio frontend)
|
|
# `npm audit` resolves the lockfile through the npmjs.com
|
|
# advisory DB. `--audit-level=high` filters the noise floor
|
|
# to only HIGH and CRITICAL. We do NOT pass --omit=dev: a
|
|
# malicious dev-only dep can still steal secrets from a CI
|
|
# runner, so dev deps need to be in the audit surface.
|
|
continue-on-error: true
|
|
working-directory: studio/frontend
|
|
run: |
|
|
set +e
|
|
npm audit --audit-level=high | tee ../../logs-npm-audit.txt
|
|
# Always also write the full JSON for grep-ability.
|
|
npm audit --json > ../../logs-npm-audit.json || true
|
|
{
|
|
echo "## npm audit (Studio frontend)"
|
|
echo
|
|
echo '```'
|
|
tail -200 ../../logs-npm-audit.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# cargo: Studio Tauri shell
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: cargo audit (Studio Tauri)
|
|
# `--deny warnings` would make the job fail on any advisory.
|
|
# Keep non-blocking initially; drop continue-on-error after
|
|
# the baseline closes.
|
|
continue-on-error: true
|
|
working-directory: studio/src-tauri
|
|
run: |
|
|
set +e
|
|
cargo audit | tee ../../logs-cargo-audit.txt
|
|
{
|
|
echo "## cargo audit (Studio Tauri)"
|
|
echo
|
|
echo '```'
|
|
tail -200 ../../logs-cargo-audit.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# OSV-Scanner: cross-ecosystem advisory DB (PyPI + npm + cargo)
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: OSV-Scanner (PyPI + npm + cargo, cross-ecosystem advisories)
|
|
# OSV's advisory feed is a superset of GitHub-Advisory + RustSec
|
|
# + npm advisories; running it alongside the per-ecosystem audit
|
|
# tools catches CVEs that haven't propagated to the per-ecosystem
|
|
# DBs yet (e.g. langchain-core CVE-2025-68664 was on OSV before
|
|
# GitHub Advisory). Single binary, one transitive resolver, all
|
|
# three lockfile types in one pass. Non-blocking until baselines
|
|
# close.
|
|
continue-on-error: true
|
|
run: |
|
|
set +e
|
|
# OSV-Scanner ships a raw binary (no tarball) in v2.x.
|
|
curl -fsSL -o /tmp/osv-scanner \
|
|
https://github.com/google/osv-scanner/releases/download/v2.0.2/osv-scanner_linux_amd64
|
|
chmod +x /tmp/osv-scanner
|
|
/tmp/osv-scanner --version
|
|
/tmp/osv-scanner scan source \
|
|
--lockfile=studio/frontend/package-lock.json \
|
|
--lockfile=studio/src-tauri/Cargo.lock \
|
|
--lockfile=requirements.txt:audit-reqs/unsloth-deps.txt \
|
|
--lockfile=requirements.txt:audit-reqs/studio.txt \
|
|
--lockfile=requirements.txt:audit-reqs/no-torch-runtime.txt \
|
|
--lockfile=requirements.txt:audit-reqs/overrides.txt \
|
|
--lockfile=requirements.txt:audit-reqs/extras.txt \
|
|
--lockfile=requirements.txt:audit-reqs/extras-no-deps.txt \
|
|
--format=table 2>&1 | tee logs-osv-scanner.txt
|
|
{
|
|
echo "## OSV-Scanner (cross-ecosystem)"
|
|
echo
|
|
echo '```'
|
|
tail -200 logs-osv-scanner.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# Semgrep: design-flaw detection (catches what regex-pattern
|
|
# scanning of malicious authors cannot — first-party logic bugs
|
|
# like langchain-core CVE-2025-68664 dumps/dumpd injection,
|
|
# n8n CVE-2025-68668 _pyodide.eval_code sandbox escape, marimo
|
|
# CVE-2026-39987 unauth WebSocket).
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: Semgrep (supply-chain + python rule packs)
|
|
continue-on-error: true
|
|
run: |
|
|
set +e
|
|
python -m pip install --quiet 'semgrep>=1.95'
|
|
semgrep --version
|
|
semgrep scan \
|
|
--config p/supply-chain \
|
|
--config p/python \
|
|
--config p/javascript \
|
|
--config p/security-audit \
|
|
--severity ERROR --severity WARNING \
|
|
--metrics off \
|
|
--timeout 120 \
|
|
studio/backend unsloth scripts \
|
|
2>&1 | tee logs-semgrep.txt
|
|
{
|
|
echo "## Semgrep (supply-chain + python + javascript rules)"
|
|
echo
|
|
echo '```'
|
|
tail -200 logs-semgrep.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# Lockfile pin verifier. The litellm 1.82.7 attack window was
|
|
# ~40 minutes; anyone resolving with `>=` got the malicious
|
|
# version automatically. Flag every spec in the requirements
|
|
# files that does not pin to an exact `==` (or `@` for git
|
|
# refs, or `===` for arbitrary equality). Warning-only for now;
|
|
# graduate to blocking once the baseline is clean.
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: Lockfile pin verifier (Python requirements)
|
|
continue-on-error: true
|
|
run: |
|
|
python <<'PY' | tee logs-pin-verifier.txt
|
|
import re
|
|
from pathlib import Path
|
|
|
|
# Specs that look like `pkg==1.2.3` or `pkg @ git+...` or
|
|
# bare comments / -r lines are pinned-or-not-applicable.
|
|
PINNED = re.compile(r"^\s*[A-Za-z0-9_.\-]+\s*(?:===|==)\s*[^,;]+\s*$")
|
|
GIT_OR_URL = re.compile(r"^\s*[A-Za-z0-9_.\-]+\s*@\s*(?:git\+|https?://)")
|
|
|
|
unpinned = []
|
|
for f in sorted(Path("studio/backend/requirements").glob("*.txt")):
|
|
for i, raw in enumerate(f.read_text().splitlines(), 1):
|
|
line = raw.strip()
|
|
if not line or line.startswith("#") or line.startswith("-"):
|
|
continue
|
|
spec = line.split("#", 1)[0].strip().split(";", 1)[0].strip()
|
|
if not spec:
|
|
continue
|
|
if "git+" in spec or PINNED.match(spec) or GIT_OR_URL.match(spec):
|
|
continue
|
|
unpinned.append((str(f), i, line))
|
|
|
|
print(f"::group::Lockfile pin status")
|
|
if unpinned:
|
|
print(f"WARN: {len(unpinned)} non-`==` specs across requirements/*.txt")
|
|
print("(litellm 1.82.7 wave hit anyone on `>=`; tighten when feasible.)")
|
|
for f, i, line in unpinned[:80]:
|
|
print(f" {f}:{i}: {line}")
|
|
if len(unpinned) > 80:
|
|
print(f" ... and {len(unpinned) - 80} more")
|
|
else:
|
|
print("OK: every spec is exact-pinned.")
|
|
print("::endgroup::")
|
|
PY
|
|
{
|
|
echo "## Lockfile pin verifier"
|
|
echo
|
|
echo '```'
|
|
cat logs-pin-verifier.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# Trivy is deliberately NOT installed here. Trivy was the entry
|
|
# point for the litellm 1.82.7 supply-chain compromise (March
|
|
# 2026): attackers force-rewrote 76 of 77 tags in
|
|
# aquasecurity/trivy-action to point at malicious commits;
|
|
# anyone running the action with a tag ref auto-pulled a
|
|
# credential-harvesting payload. By design a security scanner
|
|
# has broad read access to runner secrets, which is exactly
|
|
# what made it the ideal pivot. We pick up Trivy's CVE coverage
|
|
# from OSV-Scanner (NVD + GHSA + GitLab) and its secret
|
|
# detection from TruffleHog. IaC misconfig detection (Trivy's
|
|
# one unique value-add) is unfilled for now -- revisit with
|
|
# checkov / kics when we ship a Dockerfile or k8s manifests.
|
|
# See https://docs.litellm.ai/blog/security-update-march-2026
|
|
# and the Microsoft / Trend Micro / Snyk incident write-ups.
|
|
# ─────────────────────────────────────────────────────────────
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# TruffleHog secret-leak scan on the PR diff. Catches API keys
|
|
# / tokens / cred files committed accidentally. --only-verified
|
|
# filters out probabilistic findings, so we only flag tokens
|
|
# that the source provider confirmed are live. On push to main
|
|
# / pip we scan the full repo; on PR we scan base..head.
|
|
# SHA-pinned for the same reason as harden-runner above.
|
|
# v3.95.2 commit:
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: TruffleHog (secrets in diff)
|
|
continue-on-error: true
|
|
uses: trufflesecurity/trufflehog@17456f8c7d042d8c82c9a8ca9e937231f9f42e26 # v3.95.2
|
|
with:
|
|
path: ./
|
|
base: ${{ github.event.pull_request.base.sha || '' }}
|
|
head: ${{ github.event.pull_request.head.sha || github.sha }}
|
|
# The action passes --no-update internally; passing it here
|
|
# too triggers `flag 'no-update' cannot be repeated`. Stick
|
|
# with --only-verified so we only flag tokens the source
|
|
# provider confirmed are live (no probabilistic findings).
|
|
extra_args: --only-verified
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# CycloneDX SBOM. Lets downstream consumers audit what's
|
|
# actually shipped in unsloth wheels and the Studio backend
|
|
# runtime. Generates one JSON file per requirements input plus
|
|
# a combined SBOM keyed off pyproject.toml; uploads as a build
|
|
# artifact (and a future step can attest it via SLSA).
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: Generate CycloneDX SBOM
|
|
continue-on-error: true
|
|
run: |
|
|
set +e
|
|
python -m pip install --quiet 'cyclonedx-bom>=4.6'
|
|
mkdir -p sbom
|
|
# Per-requirements-file SBOM (the audit-reqs/ files are the
|
|
# filtered, git+-stripped views built earlier in this job).
|
|
# cyclonedx-py 4.x uses `--sv` for spec version and `-o` for
|
|
# the output file; the older `--schema-version`/`--outfile`
|
|
# spellings are not accepted.
|
|
for f in audit-reqs/*.txt; do
|
|
base=$(basename "$f" .txt)
|
|
if grep -qE '^[^#[:space:]]' "$f"; then
|
|
cyclonedx-py requirements "$f" \
|
|
--sv 1.6 \
|
|
--of JSON \
|
|
-o "sbom/sbom-$base.json" 2>&1 | tail -5 || true
|
|
fi
|
|
done
|
|
# Project-level SBOM from pyproject.toml.
|
|
cyclonedx-py environment \
|
|
--sv 1.6 \
|
|
--of JSON \
|
|
-o sbom/sbom-environment.json 2>&1 | tail -5 || true
|
|
ls -la sbom/
|
|
{
|
|
echo "## CycloneDX SBOM"
|
|
echo
|
|
echo "Generated SBOM files:"
|
|
ls sbom/ | sed 's/^/- sbom\//'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# GitHub Actions pinning verifier. tj-actions/changed-files
|
|
# was compromised in March 2025; anyone using `@v4` (a mutable
|
|
# ref) auto-shipped the malicious version. Catch every
|
|
# non-SHA-pinned `uses:` across the workflows tree. Warn-only
|
|
# initially so the existing baseline doesn't block PRs.
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: GitHub Actions pinning verifier
|
|
continue-on-error: true
|
|
run: |
|
|
python <<'PY' | tee logs-actions-pinning.txt
|
|
import re
|
|
from pathlib import Path
|
|
# SHA pin = 40 hex chars after @
|
|
SHA_PIN = re.compile(r"@[0-9a-f]{40}\b")
|
|
# First-party / GitHub-published actions get a softer pass
|
|
# (still recommended to pin; not a security gate).
|
|
FIRST_PARTY = re.compile(r"^\s*-\s*uses:\s*(actions|github)/[^@]+@")
|
|
USES = re.compile(r"^\s*-\s*uses:\s*([^@\s]+)@(\S+)")
|
|
unpinned_third = []
|
|
unpinned_first = []
|
|
for f in sorted(Path(".github/workflows").glob("*.yml")):
|
|
for i, line in enumerate(f.read_text().splitlines(), 1):
|
|
m = USES.match(line)
|
|
if not m:
|
|
continue
|
|
name, ref = m.group(1), m.group(2)
|
|
if SHA_PIN.search(line):
|
|
continue
|
|
bucket = unpinned_first if FIRST_PARTY.match(line) else unpinned_third
|
|
bucket.append((str(f), i, name, ref))
|
|
print("::group::Action pinning status")
|
|
print(f"third-party actions on mutable refs: {len(unpinned_third)}")
|
|
for f, i, n, r in unpinned_third:
|
|
print(f" HIGH {f}:{i}: {n}@{r}")
|
|
print()
|
|
print(f"first-party (actions/* | github/*) on mutable refs: {len(unpinned_first)}")
|
|
for f, i, n, r in unpinned_first[:30]:
|
|
print(f" WARN {f}:{i}: {n}@{r}")
|
|
if len(unpinned_first) > 30:
|
|
print(f" ... and {len(unpinned_first) - 30} more")
|
|
print()
|
|
print("Recommendation: pin third-party actions to a 40-char SHA.")
|
|
print("Dependabot's github-actions ecosystem will auto-bump them.")
|
|
print("::endgroup::")
|
|
PY
|
|
{
|
|
echo "## GitHub Actions pinning verifier"
|
|
echo
|
|
echo '```'
|
|
cat logs-actions-pinning.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# Hash-pin verifier. `==` pinning protects against version
|
|
# drift but not against a re-uploaded malicious wheel at the
|
|
# same version (PyPI lets a yanked release be re-published with
|
|
# different bytes for ~5 minutes via `--filename` collision).
|
|
# `pip install --require-hashes` rejects any download whose
|
|
# SHA-256 doesn't match. Inspector step that reports how many
|
|
# specs would gain from a hash pin -- conversion is a roadmap
|
|
# item (needs pip-tools / uv pip compile --generate-hashes).
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: Hash-pin verifier (Python requirements)
|
|
continue-on-error: true
|
|
run: |
|
|
python <<'PY' | tee logs-hash-verifier.txt
|
|
import re
|
|
from pathlib import Path
|
|
PINNED = re.compile(r"^\s*[A-Za-z0-9_.\-]+\s*==\s*[^,;]+\s*$")
|
|
HASH_LINE = re.compile(r"--hash=sha256:[0-9a-f]{64}")
|
|
total_pinned = 0
|
|
with_hash = 0
|
|
for f in sorted(Path("studio/backend/requirements").glob("*.txt")):
|
|
text = f.read_text()
|
|
for raw in text.splitlines():
|
|
line = raw.strip()
|
|
if not line or line.startswith("#") or line.startswith("-"):
|
|
continue
|
|
spec = line.split("#", 1)[0].strip().split(";", 1)[0]
|
|
if PINNED.match(spec):
|
|
total_pinned += 1
|
|
if HASH_LINE.search(raw):
|
|
with_hash += 1
|
|
print(f"::group::Hash-pin status")
|
|
print(f" exact == pins: {total_pinned}")
|
|
print(f" with --hash=sha256: {with_hash}")
|
|
print(f" without --hash: {total_pinned - with_hash}")
|
|
print()
|
|
print("Roadmap: convert to hash-locked installs via")
|
|
print("`uv pip compile --generate-hashes` and `pip install --require-hashes`.")
|
|
print("Hash-locked installs would have refused a republished")
|
|
print("malicious litellm 1.82.7 wheel even at the same version.")
|
|
print("::endgroup::")
|
|
PY
|
|
{
|
|
echo "## Hash-pin verifier"
|
|
echo
|
|
echo '```'
|
|
cat logs-hash-verifier.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
|
if: always()
|
|
with:
|
|
name: advisory-audit-logs
|
|
path: |
|
|
logs-pip-audit.txt
|
|
logs-npm-audit.txt
|
|
logs-npm-audit.json
|
|
logs-cargo-audit.txt
|
|
logs-osv-scanner.txt
|
|
logs-semgrep.txt
|
|
logs-pin-verifier.txt
|
|
logs-actions-pinning.txt
|
|
logs-hash-verifier.txt
|
|
audit-reqs/
|
|
sbom/
|
|
retention-days: 30
|
|
|
|
# ─────────────────────────────────────────────────────────────────────
|
|
# Python: pre-install package scan (no install, no execution)
|
|
# ─────────────────────────────────────────────────────────────────────
|
|
pip-scan-packages:
|
|
# Downloads each declared dep WITHOUT installing it and inspects
|
|
# the archive contents for known malicious patterns: weaponized
|
|
# .pth files, credential stealers, obfuscated payloads,
|
|
# install-time droppers, suspicious subprocess / network /
|
|
# base64-blob combinations.
|
|
#
|
|
# This is the kind of check that would have caught:
|
|
# - litellm 1.82.7 / 1.82.8 (March 2026, supply-chain compromise)
|
|
# - the typo-squat campaign against PyTorch Lightning
|
|
# before either landed in the install path. pip-audit only knows
|
|
# about CVE-published vulnerabilities, so it does NOT see novel
|
|
# malicious uploads. scan_packages.py runs deterministic regex
|
|
# pattern matching, no LLM calls.
|
|
#
|
|
# `--with-deps` makes the scan transitive: every package the
|
|
# declared set resolves to gets fetched and pattern-scanned, not
|
|
# just the top-level pins. Resolving the full transitive closure
|
|
# of the unsloth + Studio dep tree downloads several hundred
|
|
# archives, hence the longer timeout.
|
|
#
|
|
# Sharded across runners for wall-clock parallelism. Each shard
|
|
# runs scan_packages.py once with --with-deps so its own slice
|
|
# benefits from pip's deduped transitive resolve. Shard
|
|
# composition tries to balance load:
|
|
# - hf-stack: pyproject extras + no-torch-runtime
|
|
# (~150 archives, transformers/peft/accelerate/...)
|
|
# - studio: FastAPI/Studio backend + overrides + extras-no-deps
|
|
# (~150 archives, smaller scientific stack)
|
|
# - extras: the heavy openai-whisper / scikit-learn / librosa
|
|
# stack (~250 archives, dominant cost)
|
|
# triton-kernels.txt is git+-only, fully skipped.
|
|
name: ${{ matrix.shard.name }}
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 25
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
shard:
|
|
- name: 'pip scan-packages :: hf-stack'
|
|
id: hf-stack
|
|
files: 'unsloth-deps no-torch-runtime'
|
|
- name: 'pip scan-packages :: studio'
|
|
id: studio
|
|
files: 'studio overrides extras-no-deps'
|
|
- name: 'pip scan-packages :: extras'
|
|
id: extras
|
|
files: 'extras'
|
|
steps:
|
|
# Egress audit on every shard. Each shard pulls hundreds of
|
|
# PyPI archives -- if a malicious wheel ever phones home from
|
|
# within the scanner sandbox (it shouldn't; we never execute
|
|
# the archive), harden-runner's audit log records the host.
|
|
- name: Harden runner (egress audit)
|
|
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
|
|
with:
|
|
egress-policy: audit
|
|
disable-sudo: true
|
|
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
|
|
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
|
with:
|
|
python-version: '3.12'
|
|
cache: 'pip'
|
|
|
|
- name: Install scan_packages.py runtime deps
|
|
# scan_packages.py imports requests + packaging at runtime to
|
|
# talk to PyPI's JSON API and to parse version specifiers. We
|
|
# do not install the packages it scans -- those are downloaded
|
|
# raw and inspected without ever touching `pip install`.
|
|
run: python -m pip install --upgrade pip requests packaging
|
|
|
|
- name: Build filtered requirements set
|
|
# Mirrors the advisory-audit job's input transform: pyproject.toml
|
|
# extraction + git+ stripping. scan_packages.py downloads
|
|
# PyPI archives without building, so it tolerates legacy
|
|
# setup.py packages (no resolver dry-run); but `--with-deps`
|
|
# delegates resolution to a single `pip download` call that
|
|
# cannot satisfy `git+` specs without git operations, so we
|
|
# strip them here too.
|
|
run: |
|
|
mkdir -p audit-reqs
|
|
python <<'PY' > audit-reqs/unsloth-deps.txt
|
|
import tomllib
|
|
with open("pyproject.toml", "rb") as f:
|
|
d = tomllib.load(f)
|
|
core = d["project"]["dependencies"]
|
|
extras = d["project"]["optional-dependencies"]["huggingfacenotorch"]
|
|
print("# Auto-generated from pyproject.toml by security-audit.yml.")
|
|
print("# core deps + huggingfacenotorch extras.")
|
|
for spec in core + extras:
|
|
print(spec)
|
|
PY
|
|
for f in studio.txt extras.txt extras-no-deps.txt \
|
|
no-torch-runtime.txt overrides.txt triton-kernels.txt; do
|
|
python <<PY > "audit-reqs/$f"
|
|
src = "studio/backend/requirements/$f"
|
|
with open(src) as fh:
|
|
for line in fh:
|
|
stripped = line.strip()
|
|
before_comment = stripped.split("#", 1)[0]
|
|
if "git+" in before_comment:
|
|
print(f"# [security-audit] skipped git+ spec: {stripped}")
|
|
continue
|
|
print(line.rstrip("\n"))
|
|
PY
|
|
done
|
|
|
|
- name: Sanity-check scan_packages.py
|
|
# The scanner lives at scripts/scan_packages.py in this repo
|
|
# so we don't depend on a network fetch at job time.
|
|
run: |
|
|
test -f scripts/scan_packages.py
|
|
head -3 scripts/scan_packages.py
|
|
grep -q "Standalone pre-install package scanner" scripts/scan_packages.py
|
|
|
|
- name: Scan declared + transitive Python deps
|
|
# scan_packages.py exits 1 on CRITICAL/HIGH findings, 0 on
|
|
# clean. We swallow the exit because the baseline isn't
|
|
# triaged yet; surface the findings in the workflow summary.
|
|
# Drop continue-on-error after the first clean run on main.
|
|
#
|
|
# `--with-deps` walks PyPI metadata to enumerate every
|
|
# transitive dep the declared set would install, then scans
|
|
# them all. Without this flag, we'd only catch a malicious
|
|
# *direct* dep -- and supply-chain attacks usually land
|
|
# several hops down (litellm 1.82.7 was a dep of a dep for
|
|
# most users).
|
|
#
|
|
# This step runs once per matrix shard. Within a shard, every
|
|
# -r file is fed to a single `pip download` call so pip
|
|
# intersects version constraints and yields a deduped
|
|
# transitive set (no point fetching the same transformers
|
|
# wheel five times). Across shards we accept some redundant
|
|
# downloads in exchange for wall-clock parallelism.
|
|
continue-on-error: true
|
|
env:
|
|
SHARD_FILES: ${{ matrix.shard.files }}
|
|
run: |
|
|
set +e
|
|
mkdir -p logs
|
|
LOG="logs-scan-packages-${{ matrix.shard.id }}.txt"
|
|
echo "::group::shard ${{ matrix.shard.id }} input files"
|
|
REQ_ARGS=()
|
|
for f in $SHARD_FILES; do
|
|
if grep -qE '^[^#[:space:]]' "audit-reqs/$f.txt"; then
|
|
echo " + audit-reqs/$f.txt"
|
|
REQ_ARGS+=( -r "audit-reqs/$f.txt" )
|
|
else
|
|
echo " - audit-reqs/$f.txt (empty after git+ filter, skipping)"
|
|
fi
|
|
done
|
|
echo "::endgroup::"
|
|
if [ ${#REQ_ARGS[@]} -eq 0 ]; then
|
|
echo "[security-audit] shard ${{ matrix.shard.id }}: no PyPI specs, nothing to scan" \
|
|
| tee "$LOG"
|
|
else
|
|
python scripts/scan_packages.py --with-deps "${REQ_ARGS[@]}" \
|
|
2>&1 | tee "$LOG"
|
|
fi
|
|
{
|
|
echo "## scan_packages :: shard ${{ matrix.shard.id }}"
|
|
echo
|
|
echo "### Files in this shard"
|
|
for f in $SHARD_FILES; do echo "- audit-reqs/$f.txt"; done
|
|
echo
|
|
echo '### Findings (tail)'
|
|
echo '```'
|
|
tail -200 "$LOG"
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
|
if: always()
|
|
with:
|
|
name: scan-packages-log-${{ matrix.shard.id }}
|
|
path: |
|
|
logs-scan-packages-${{ matrix.shard.id }}.txt
|
|
audit-reqs/
|
|
retention-days: 30
|