ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/.github/workflows/security-audit.yml
Daniel Han fa2cd359b3 CI(security): shard scan_packages across 3 runners + dedupe per-shard 
Previous run took ~10+ minutes because each requirements file ran
its own --with-deps resolve serially, and the six files all share
~70% of their transitive set (transformers, peft, accelerate land
in three of them). Net effect: the same 200+ archives downloaded and
pattern-scanned three times in series.

Two changes:
  1. Within a shard, feed every -r file to ONE scan_packages call so
     pip's resolver intersects version constraints once and yields
     a single deduped transitive set.
  2. Across shards, run three matrix jobs in parallel:
       - hf-stack: unsloth-deps + no-torch-runtime  (pyproject extras)
       - studio:   studio + overrides + extras-no-deps
       - extras:   extras (heavy openai-whisper / scikit-learn stack)
     Wall clock now bounded by the slowest shard rather than the
     sum, dropping ~10 min to ~3-5 min.

Each shard uploads its own artifact (scan-packages-log-<id>) so log
correlation stays clean. fail-fast: false so one shard's findings
don't suppress the others.
2026-05-06 23:08:55 +00:00

472 lines
20 KiB
YAML
Raw Blame History

# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# Multi-language supply-chain audit. Each job is independent and runs
# in parallel. Triggers:
# - PRs touching any dependency manifest (Python / npm / Cargo) or
# this workflow file,
# - push to main / pip,
# - nightly @ 04:13 UTC so newly-published advisories surface even
# when no PR opens,
# - workflow_dispatch for ad-hoc invocations.
#
# All four jobs are non-blocking initially. The default branch already
# carries a known-vuln backlog (the dependabot banner shows 17 today,
# pip-audit catches 2 more, npm/cargo will catch their own); a hard
# gate now would block every PR on a baseline we have not triaged.
# As each baseline closes, drop continue-on-error.
#
# Dependency coverage:
# - unsloth core (pyproject.toml [project.dependencies])
# - unsloth `huggingfacenotorch` extras (the canonical install path
# for fine-tuning users; pulls transformers / peft / accelerate /
# trl / datasets / diffusers / sentence-transformers / etc.)
# - all six Studio backend requirements files
# - Studio frontend (npm) and Tauri shell (cargo)
# Each Python job builds a combined dep list from pyproject.toml +
# requirements/*.txt before auditing. We do NOT install any of these
# -- pip-audit resolves through PyPI metadata, scan_packages.py
# downloads sdist/wheel archives and inspects them without running
# install hooks, so an attacker who has compromised a transitive dep
# cannot execute code in this workflow.
name: Security audit
on:
pull_request:
paths:
- 'studio/backend/requirements/**'
- 'studio/frontend/package.json'
- 'studio/frontend/package-lock.json'
- 'studio/src-tauri/Cargo.toml'
- 'studio/src-tauri/Cargo.lock'
- 'pyproject.toml'
- 'scripts/scan_packages.py'
- '.github/workflows/security-audit.yml'
push:
branches: [main, pip]
schedule:
- cron: '13 4 * * *' # 04:13 UTC daily, off the cron rush
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
# ─────────────────────────────────────────────────────────────────────
# Python: declared-deps audit (resolve from PyPI, do NOT install)
# ─────────────────────────────────────────────────────────────────────
pip-audit:
name: pip-audit (declared Python deps, no install)
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'pip'
- name: Install pip-audit (only)
run: python -m pip install --upgrade pip 'pip-audit>=2.7'
- name: Build filtered requirements set
# Two transforms:
# (1) Generate audit-reqs/unsloth-deps.txt from pyproject.toml
# so pip-audit sees the unsloth pip package's own dep set
# (core + huggingfacenotorch extras: transformers / peft /
# accelerate / trl / datasets / diffusers /
# sentence-transformers / huggingface_hub / hf_transfer /
# etc.).
# (2) Copy each studio/backend/requirements/*.txt into
# audit-reqs/ with `git+` lines stripped. pip-audit's `-r`
# mode does a dry-run resolve against PyPI metadata; a
# `git+https://...` spec forces it to clone, which is
# both slow and outside the threat model (we audit
# PyPI-served archives; a git ref is whatever HEAD says
# on the runner). A comment line is left in place so the
# skipped specs are obvious in the artifact.
# The `huggingface` extra is `huggingfacenotorch` plus torch /
# torchvision / triton, deliberately skipped: Studio backend
# already pins a torch and the +cu* / +cpu local-version tags
# trip up the PyPI resolver in `-r` mode.
run: |
mkdir -p audit-reqs
python <<'PY' > audit-reqs/unsloth-deps.txt
import tomllib
with open("pyproject.toml", "rb") as f:
d = tomllib.load(f)
core = d["project"]["dependencies"]
extras = d["project"]["optional-dependencies"]["huggingfacenotorch"]
print("# Auto-generated from pyproject.toml by security-audit.yml.")
print("# core deps + huggingfacenotorch extras.")
for spec in core + extras:
print(spec)
PY
for f in studio.txt extras.txt extras-no-deps.txt \
no-torch-runtime.txt overrides.txt triton-kernels.txt; do
python <<PY > "audit-reqs/$f"
import re
src = "studio/backend/requirements/$f"
with open(src) as fh:
for line in fh:
stripped = line.strip()
# Skip pure git+ specs but leave a marker so the
# exclusion is auditable.
before_comment = stripped.split("#", 1)[0]
if "git+" in before_comment:
print(f"# [security-audit] skipped git+ spec: {stripped}")
continue
print(line.rstrip("\n"))
PY
done
echo "::group::audit-reqs/unsloth-deps.txt"
cat audit-reqs/unsloth-deps.txt
echo "::endgroup::"
for f in audit-reqs/*.txt; do
echo "::group::$f"
cat "$f"
echo "::endgroup::"
done
- name: Audit declared deps via -r (no install)
# `-r requirements.txt` resolves the requirements through pip's
# dependency resolver against PyPI metadata and audits the
# resolved tree without ever executing setup.py / install
# hooks. Way faster than installing the full Studio runtime
# and -- critically -- safer: an attacker who has compromised
# a transitive dep cannot run code in this job.
#
# extras.txt + extras-no-deps.txt are audited separately
# (`continue-on-error` per step) because some of their members
# ship legacy setup.py scripts that the resolver tries to
# build at metadata-collection time. openai-whisper's setup.py
# imports `pkg_resources`, which the isolated build env's
# current setuptools no longer ships. PIP_CONSTRAINT pins an
# older setuptools into the build env so those builds resolve;
# if that still fails, the step continues so the rest of the
# audit completes.
continue-on-error: true
env:
# Pin setuptools into pip's isolated build envs. This fixes
# the "ModuleNotFoundError: No module named 'pkg_resources'"
# raised when openai-whisper's setup.py imports it.
PIP_CONSTRAINT: ${{ github.workspace }}/audit-reqs/build-constraints.txt
run: |
set +e
cat > audit-reqs/build-constraints.txt <<'CONSTRAINTS'
setuptools<78
wheel
CONSTRAINTS
: > logs-pip-audit.txt
for f in unsloth-deps studio extras extras-no-deps \
no-torch-runtime overrides triton-kernels; do
if ! grep -qE '^[^#[:space:]]' "audit-reqs/$f.txt"; then
echo "[security-audit] $f.txt has no PyPI specs after git+ filter, skipping" \
| tee -a logs-pip-audit.txt
continue
fi
echo "::group::pip-audit -r audit-reqs/$f.txt"
{
echo
echo "=== $f ==="
pip-audit -r "audit-reqs/$f.txt" --format=columns
echo "=== end $f (rc=$?) ==="
} 2>&1 | tee -a logs-pip-audit.txt
echo "::endgroup::"
done
{
echo "## pip-audit"
echo
echo '### Coverage'
echo '- unsloth core + `huggingfacenotorch` extras (pyproject.toml)'
echo '- studio/backend/requirements/{studio,extras,extras-no-deps,no-torch-runtime,overrides,triton-kernels}.txt'
echo '- `git+` specs are stripped before audit (out of scope: we audit PyPI archives)'
echo
echo '### Findings'
echo '```'
cat logs-pip-audit.txt
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@v4
if: always()
with:
name: pip-audit-log
path: |
logs-pip-audit.txt
audit-reqs/
retention-days: 30
# ─────────────────────────────────────────────────────────────────────
# Python: pre-install package scan (no install, no execution)
# ─────────────────────────────────────────────────────────────────────
pip-scan-packages:
# Downloads each declared dep WITHOUT installing it and inspects
# the archive contents for known malicious patterns: weaponized
# .pth files, credential stealers, obfuscated payloads,
# install-time droppers, suspicious subprocess / network /
# base64-blob combinations.
#
# This is the kind of check that would have caught:
# - litellm 1.82.7 / 1.82.8 (March 2026, supply-chain compromise)
# - the typo-squat campaign against PyTorch Lightning
# before either landed in the install path. pip-audit only knows
# about CVE-published vulnerabilities, so it does NOT see novel
# malicious uploads. scan_packages.py runs deterministic regex
# pattern matching, no LLM calls.
#
# `--with-deps` makes the scan transitive: every package the
# declared set resolves to gets fetched and pattern-scanned, not
# just the top-level pins. Resolving the full transitive closure
# of the unsloth + Studio dep tree downloads several hundred
# archives, hence the longer timeout.
#
# Sharded across runners for wall-clock parallelism. Each shard
# runs scan_packages.py once with --with-deps so its own slice
# benefits from pip's deduped transitive resolve. Shard
# composition tries to balance load:
# - hf-stack: pyproject extras + no-torch-runtime
# (~150 archives, transformers/peft/accelerate/...)
# - studio: FastAPI/Studio backend + overrides + extras-no-deps
# (~150 archives, smaller scientific stack)
# - extras: the heavy openai-whisper / scikit-learn / librosa
# stack (~250 archives, dominant cost)
# triton-kernels.txt is git+-only, fully skipped.
name: ${{ matrix.shard.name }}
runs-on: ubuntu-latest
timeout-minutes: 25
strategy:
fail-fast: false
matrix:
shard:
- name: 'pip scan-packages :: hf-stack'
id: hf-stack
files: 'unsloth-deps no-torch-runtime'
- name: 'pip scan-packages :: studio'
id: studio
files: 'studio overrides extras-no-deps'
- name: 'pip scan-packages :: extras'
id: extras
files: 'extras'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'pip'
- name: Install scan_packages.py runtime deps
# scan_packages.py imports requests + packaging at runtime to
# talk to PyPI's JSON API and to parse version specifiers. We
# do not install the packages it scans -- those are downloaded
# raw and inspected without ever touching `pip install`.
run: python -m pip install --upgrade pip requests packaging
- name: Build filtered requirements set
# Mirrors the pip-audit job's input transform: pyproject.toml
# extraction + git+ stripping. scan_packages.py downloads
# PyPI archives without building, so it tolerates legacy
# setup.py packages (no resolver dry-run); but `--with-deps`
# delegates resolution to a single `pip download` call that
# cannot satisfy `git+` specs without git operations, so we
# strip them here too.
run: |
mkdir -p audit-reqs
python <<'PY' > audit-reqs/unsloth-deps.txt
import tomllib
with open("pyproject.toml", "rb") as f:
d = tomllib.load(f)
core = d["project"]["dependencies"]
extras = d["project"]["optional-dependencies"]["huggingfacenotorch"]
print("# Auto-generated from pyproject.toml by security-audit.yml.")
print("# core deps + huggingfacenotorch extras.")
for spec in core + extras:
print(spec)
PY
for f in studio.txt extras.txt extras-no-deps.txt \
no-torch-runtime.txt overrides.txt triton-kernels.txt; do
python <<PY > "audit-reqs/$f"
src = "studio/backend/requirements/$f"
with open(src) as fh:
for line in fh:
stripped = line.strip()
before_comment = stripped.split("#", 1)[0]
if "git+" in before_comment:
print(f"# [security-audit] skipped git+ spec: {stripped}")
continue
print(line.rstrip("\n"))
PY
done
- name: Sanity-check scan_packages.py
# The scanner lives at scripts/scan_packages.py in this repo
# so we don't depend on a network fetch at job time.
run: |
test -f scripts/scan_packages.py
head -3 scripts/scan_packages.py
grep -q "Standalone pre-install package scanner" scripts/scan_packages.py
- name: Scan declared + transitive Python deps
# scan_packages.py exits 1 on CRITICAL/HIGH findings, 0 on
# clean. We swallow the exit because the baseline isn't
# triaged yet; surface the findings in the workflow summary.
# Drop continue-on-error after the first clean run on main.
#
# `--with-deps` walks PyPI metadata to enumerate every
# transitive dep the declared set would install, then scans
# them all. Without this flag, we'd only catch a malicious
# *direct* dep -- and supply-chain attacks usually land
# several hops down (litellm 1.82.7 was a dep of a dep for
# most users).
#
# This step runs once per matrix shard. Within a shard, every
# -r file is fed to a single `pip download` call so pip
# intersects version constraints and yields a deduped
# transitive set (no point fetching the same transformers
# wheel five times). Across shards we accept some redundant
# downloads in exchange for wall-clock parallelism.
continue-on-error: true
env:
SHARD_FILES: ${{ matrix.shard.files }}
run: |
set +e
mkdir -p logs
LOG="logs-scan-packages-${{ matrix.shard.id }}.txt"
echo "::group::shard ${{ matrix.shard.id }} input files"
REQ_ARGS=()
for f in $SHARD_FILES; do
if grep -qE '^[^#[:space:]]' "audit-reqs/$f.txt"; then
echo " + audit-reqs/$f.txt"
REQ_ARGS+=( -r "audit-reqs/$f.txt" )
else
echo " - audit-reqs/$f.txt (empty after git+ filter, skipping)"
fi
done
echo "::endgroup::"
if [ ${#REQ_ARGS[@]} -eq 0 ]; then
echo "[security-audit] shard ${{ matrix.shard.id }}: no PyPI specs, nothing to scan" \
| tee "$LOG"
else
python scripts/scan_packages.py --with-deps "${REQ_ARGS[@]}" \
2>&1 | tee "$LOG"
fi
{
echo "## scan_packages :: shard ${{ matrix.shard.id }}"
echo
echo "### Files in this shard"
for f in $SHARD_FILES; do echo "- audit-reqs/$f.txt"; done
echo
echo '### Findings (tail)'
echo '```'
tail -200 "$LOG"
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@v4
if: always()
with:
name: scan-packages-log-${{ matrix.shard.id }}
path: |
logs-scan-packages-${{ matrix.shard.id }}.txt
audit-reqs/
retention-days: 30
# ─────────────────────────────────────────────────────────────────────
# npm: frontend dep audit
# ─────────────────────────────────────────────────────────────────────
npm-audit:
name: npm audit (Studio frontend)
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: studio/frontend/package-lock.json
- name: npm audit (high+ severity, declared + transitive)
# `npm audit` resolves the lockfile through the npmjs.com
# advisory DB. `--audit-level=high` filters the noise floor
# to only HIGH and CRITICAL. We do NOT pass --omit=dev: a
# malicious dev-only dep can still steal secrets from a CI
# runner, so dev deps need to be in the audit surface.
continue-on-error: true
working-directory: studio/frontend
run: |
set +e
npm audit --audit-level=high | tee ../../logs-npm-audit.txt
# Always also write the full JSON for grep-ability.
npm audit --json > ../../logs-npm-audit.json || true
{
echo "## npm audit (Studio frontend)"
echo
echo '```'
tail -200 ../../logs-npm-audit.txt
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@v4
if: always()
with:
name: npm-audit-log
path: |
logs-npm-audit.txt
logs-npm-audit.json
retention-days: 30
# ─────────────────────────────────────────────────────────────────────
# cargo: Tauri Rust dep audit
# ─────────────────────────────────────────────────────────────────────
cargo-audit:
name: cargo audit (Studio Tauri)
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: swatinem/rust-cache@v2
with:
workspaces: studio/src-tauri -> target
- name: Install cargo-audit
# cargo-audit pulls advisories from
# https://github.com/rustsec/advisory-db on first run and
# caches them under ~/.cargo/advisory-db. Pin --locked so the
# version we install matches Cargo.lock determinism.
run: cargo install --locked --version '^0.21' cargo-audit
- name: cargo audit (RustSec advisory DB)
# `--deny warnings` would make the job fail on any advisory.
# Keep non-blocking initially; drop continue-on-error after
# the baseline closes.
continue-on-error: true
working-directory: studio/src-tauri
run: |
set +e
cargo audit | tee ../../logs-cargo-audit.txt
{
echo "## cargo audit (Studio Tauri)"
echo
echo '```'
tail -200 ../../logs-cargo-audit.txt
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@v4
if: always()
with:
name: cargo-audit-log
path: logs-cargo-audit.txt
retention-days: 30
Reference in a new issue View git blame Copy permalink