fadf48f562
Consolidates eight of the ten open dependabot PRs into a single review surface. Two are intentionally left out because they bundle breaking-change deps that need source migration outside the scope of a combine PR; dependabot recreates them on the next weekly tick: - #5364 (cargo-tauri group, 13 updates) bumps hmac 0.12 -> 0.13 which removes Hmac::new_from_slice from the inherent impl, so studio/src-tauri/src/native_backend_lease.rs:152 fails to compile (error E0599). Also bumps sha2 0.10 -> 0.11 and reqwest 0.12 -> 0.13 which have their own migration surfaces. Needs a focused PR after the source migration. - #5365 (bun-frontend group, 14 updates) bumps TypeScript 5.9 -> 6.0 promoting the baseUrl deprecation to a hard tsc error, plus @assistant-ui/react 0.12 -> 0.14 renaming the unstable_* exports, react-day-picker 9 -> 10 dropping the table classname, and recharts 3.7 -> 3.8.1 tightening the Key prop type. Same shape: needs a focused frontend migration PR. The remaining eight land cleanly: - #4916 oxc-parser 0.123.0 -> 0.129.0 in /studio/backend/core/ data_recipe/oxc-validator (npm-oxc-validator group) - #5343 hono 4.12.17 -> 4.12.18 and ip-address 10.1.0 -> 10.2.0 in /studio/frontend (security advisory: GHSA-p77w-8qqv-26rm, GHSA-qp7p-654g-cw7p, GHSA-hm8q-7f3q-5f36) - #5362 python group (3 updates) in pyproject.toml - #5363 actions group (3 updates) in release-desktop / security-audit / studio-tauri-smoke workflows - #5366 openssl 0.10.76 -> 0.10.79 (security) - #5367 rand 0.10.0 -> 0.10.1 (security) - #5368 tauri 2.10.3 -> 2.11.1 (security; transitive bumps for tray-icon and wry come along) - #5369 rustls-webpki 0.103.10 -> 0.103.13 (security) studio/src-tauri/Cargo.toml stays at main's values (hmac 0.12, sha2 0.10, reqwest 0.12, rand 0.10.0, windows-sys 0.59); Cargo.lock is regenerated by running cargo update -p <pkg> --precise <ver> for each of the four security advisories on top of main, so only the four pinned packages move and the rest of the dep graph stays identical to main. Validated locally: cargo check passes through the Rust source; the only error is the build-time frontendDist check unrelated to source.
796 lines
39 KiB
YAML
796 lines
39 KiB
YAML
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
|
|
|
|
# Multi-language supply-chain audit. Triggers:
|
|
# - PRs touching any dependency manifest (Python / npm / Cargo) or
|
|
# this workflow file,
|
|
# - push to main / pip,
|
|
# - nightly @ 04:13 UTC so newly-published advisories surface even
|
|
# when no PR opens,
|
|
# - workflow_dispatch for ad-hoc invocations.
|
|
#
|
|
# Two jobs:
|
|
# - advisory-audit: one runner that runs pip-audit + npm audit +
|
|
# cargo audit back-to-back. All three are
|
|
# advisory-DB lookups -- fast, lockfile-driven,
|
|
# no archive download. Setting up the python /
|
|
# node / rust toolchains on one runner and
|
|
# running the three commands serially is
|
|
# cheaper than spinning up three runners.
|
|
# - pip-scan-packages: 3-shard matrix that downloads + pattern-scans
|
|
# every PyPI archive in the transitive closure.
|
|
# This is the expensive job (~6 min/shard,
|
|
# running in parallel) and it must stay
|
|
# independent so a CVE-DB hit in advisory-audit
|
|
# does not block the supply-chain pattern scan
|
|
# (or vice versa).
|
|
#
|
|
# All steps are non-blocking initially. The default branch already
|
|
# carries a known-vuln backlog (the dependabot banner shows 17 today,
|
|
# pip-audit catches 2 more, npm/cargo will catch their own); a hard
|
|
# gate now would block every PR on a baseline we have not triaged.
|
|
# As each baseline closes, drop continue-on-error per step.
|
|
#
|
|
# Dependency coverage:
|
|
# - unsloth core (pyproject.toml [project.dependencies])
|
|
# - unsloth `huggingfacenotorch` extras (the canonical install path
|
|
# for fine-tuning users; pulls transformers / peft / accelerate /
|
|
# trl / datasets / diffusers / sentence-transformers / etc.)
|
|
# - all six Studio backend requirements files
|
|
# - Studio frontend (npm) and Tauri shell (cargo)
|
|
# Each Python step builds a filtered dep list from pyproject.toml +
|
|
# requirements/*.txt before auditing. We do NOT install any of these
|
|
# -- pip-audit resolves through PyPI metadata, scan_packages.py
|
|
# downloads sdist/wheel archives and inspects them without running
|
|
# install hooks, so an attacker who has compromised a transitive dep
|
|
# cannot execute code in this workflow.
|
|
|
|
name: Security audit
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- 'studio/backend/requirements/**'
|
|
- 'studio/frontend/package.json'
|
|
- 'studio/frontend/package-lock.json'
|
|
- 'studio/src-tauri/Cargo.toml'
|
|
- 'studio/src-tauri/Cargo.lock'
|
|
- 'pyproject.toml'
|
|
- 'scripts/scan_packages.py'
|
|
- '.github/workflows/security-audit.yml'
|
|
push:
|
|
branches: [main, pip]
|
|
schedule:
|
|
- cron: '13 4 * * *' # 04:13 UTC daily, off the cron rush
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
# ─────────────────────────────────────────────────────────────────────
|
|
# Combined advisory-DB audit: pip-audit + npm audit + cargo audit
|
|
# all on one runner. Each step is continue-on-error so a finding in
|
|
# one toolchain does not suppress the others.
|
|
# ─────────────────────────────────────────────────────────────────────
|
|
advisory-audit:
|
|
name: advisory audit (pip + npm + cargo)
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 25
|
|
steps:
|
|
# step-security/harden-runner installs an eBPF-based egress
|
|
# firewall on the runner. In `audit` mode it logs every outbound
|
|
# connection without blocking; in `block` mode it rejects
|
|
# anything outside `allowed-endpoints`. We run audit-only
|
|
# initially: the next time this job hits a real PyPI advisory or
|
|
# an attacker-funded archive in pip-scan-packages, the audit log
|
|
# tells us exactly which hosts were dialed and we promote the
|
|
# allowlist to block. Would have *contained* the litellm exfil
|
|
# even if scan_packages had missed the .pth payload.
|
|
# SHA-pinned (not @v2): the litellm 1.82.7 attack chain hijacked
|
|
# mutable tags on aquasecurity/trivy-action and would have hit
|
|
# anyone using @v0 / @v2 / @latest references. Pinning to a 40-
|
|
# char SHA freezes this action at known-good code; Dependabot's
|
|
# github-actions ecosystem will auto-bump the SHA.
|
|
# v2.19.1 commit:
|
|
- name: Harden runner (egress audit)
|
|
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
|
|
with:
|
|
egress-policy: audit
|
|
disable-sudo: true
|
|
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
# Full history so TruffleHog can diff base..head; without
|
|
# this it sees only the latest commit and reports nothing.
|
|
fetch-depth: 0
|
|
|
|
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
|
with:
|
|
python-version: '3.12'
|
|
cache: 'pip'
|
|
|
|
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
|
|
with:
|
|
node-version: '22'
|
|
cache: 'npm'
|
|
cache-dependency-path: studio/frontend/package-lock.json
|
|
|
|
- uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable @ 2026-03-27
|
|
|
|
- uses: swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2.9.1
|
|
with:
|
|
workspaces: studio/src-tauri -> target
|
|
|
|
- name: Install pip-audit + cargo-audit
|
|
# cargo-audit pulls advisories from the RustSec advisory-db on
|
|
# first run and caches them under ~/.cargo/advisory-db. Pin
|
|
# --locked so the version we install matches Cargo.lock
|
|
# determinism. cargo-audit 0.22 supports the CVSS 4.0 schema
|
|
# used in 2026 advisories (e.g. RUSTSEC-2026-0073); 0.21
|
|
# crashes with a TOML parse error on that file.
|
|
# npm audit is bundled with the node toolchain, no install.
|
|
run: |
|
|
python -m pip install --upgrade pip 'pip-audit>=2.7'
|
|
cargo install --locked --version '^0.22' cargo-audit
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# Python: pip-audit
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: Build filtered Python requirements set
|
|
# Two transforms:
|
|
# (1) Generate audit-reqs/unsloth-deps.txt from pyproject.toml
|
|
# so pip-audit sees the unsloth pip package's own dep set
|
|
# (core + huggingfacenotorch extras: transformers / peft /
|
|
# accelerate / trl / datasets / diffusers /
|
|
# sentence-transformers / huggingface_hub / hf_transfer /
|
|
# etc.).
|
|
# (2) Copy each studio/backend/requirements/*.txt into
|
|
# audit-reqs/ with `git+` lines stripped. pip-audit's `-r`
|
|
# mode does a dry-run resolve against PyPI metadata; a
|
|
# `git+https://...` spec forces it to clone, which is
|
|
# both slow and outside the threat model (we audit
|
|
# PyPI-served archives; a git ref is whatever HEAD says
|
|
# on the runner). A comment line is left in place so the
|
|
# skipped specs are obvious in the artifact.
|
|
# The `huggingface` extra is `huggingfacenotorch` plus torch /
|
|
# torchvision / triton, deliberately skipped: Studio backend
|
|
# already pins a torch and the +cu* / +cpu local-version tags
|
|
# trip up the PyPI resolver in `-r` mode.
|
|
run: |
|
|
mkdir -p audit-reqs
|
|
python <<'PY' > audit-reqs/unsloth-deps.txt
|
|
import tomllib
|
|
with open("pyproject.toml", "rb") as f:
|
|
d = tomllib.load(f)
|
|
core = d["project"]["dependencies"]
|
|
extras = d["project"]["optional-dependencies"]["huggingfacenotorch"]
|
|
print("# Auto-generated from pyproject.toml by security-audit.yml.")
|
|
print("# core deps + huggingfacenotorch extras.")
|
|
for spec in core + extras:
|
|
print(spec)
|
|
PY
|
|
for f in studio.txt extras.txt extras-no-deps.txt \
|
|
no-torch-runtime.txt overrides.txt triton-kernels.txt; do
|
|
python <<PY > "audit-reqs/$f"
|
|
src = "studio/backend/requirements/$f"
|
|
with open(src) as fh:
|
|
for line in fh:
|
|
stripped = line.strip()
|
|
before_comment = stripped.split("#", 1)[0]
|
|
if "git+" in before_comment:
|
|
print(f"# [security-audit] skipped git+ spec: {stripped}")
|
|
continue
|
|
print(line.rstrip("\n"))
|
|
PY
|
|
done
|
|
|
|
- name: pip-audit (declared Python deps, no install)
|
|
# `-r requirements.txt` resolves the requirements through pip's
|
|
# dependency resolver against PyPI metadata and audits the
|
|
# resolved tree without ever executing setup.py / install
|
|
# hooks. Way faster than installing the full Studio runtime
|
|
# and -- critically -- safer: an attacker who has compromised
|
|
# a transitive dep cannot run code in this job.
|
|
#
|
|
# extras.txt + extras-no-deps.txt have legacy setup.py
|
|
# packages (notably openai-whisper) whose setup.py imports
|
|
# `pkg_resources`, which the isolated build env's current
|
|
# setuptools no longer ships. PIP_CONSTRAINT pins an older
|
|
# setuptools into the build env so those builds resolve.
|
|
# Per-file loop so one bad file doesn't take out the whole
|
|
# audit.
|
|
continue-on-error: true
|
|
env:
|
|
PIP_CONSTRAINT: ${{ github.workspace }}/audit-reqs/build-constraints.txt
|
|
run: |
|
|
set +e
|
|
cat > audit-reqs/build-constraints.txt <<'CONSTRAINTS'
|
|
setuptools<78
|
|
wheel
|
|
CONSTRAINTS
|
|
: > logs-pip-audit.txt
|
|
for f in unsloth-deps studio extras extras-no-deps \
|
|
no-torch-runtime overrides triton-kernels; do
|
|
if ! grep -qE '^[^#[:space:]]' "audit-reqs/$f.txt"; then
|
|
echo "[security-audit] $f.txt has no PyPI specs after git+ filter, skipping" \
|
|
| tee -a logs-pip-audit.txt
|
|
continue
|
|
fi
|
|
echo "::group::pip-audit -r audit-reqs/$f.txt"
|
|
{
|
|
echo
|
|
echo "=== $f ==="
|
|
pip-audit -r "audit-reqs/$f.txt" --format=columns
|
|
echo "=== end $f (rc=$?) ==="
|
|
} 2>&1 | tee -a logs-pip-audit.txt
|
|
echo "::endgroup::"
|
|
done
|
|
{
|
|
echo "## pip-audit (Python)"
|
|
echo
|
|
echo '### Coverage'
|
|
echo '- unsloth core + `huggingfacenotorch` extras (pyproject.toml)'
|
|
echo '- studio/backend/requirements/{studio,extras,extras-no-deps,no-torch-runtime,overrides,triton-kernels}.txt'
|
|
echo '- `git+` specs are stripped before audit (out of scope: we audit PyPI archives)'
|
|
echo
|
|
echo '### Findings'
|
|
echo '```'
|
|
cat logs-pip-audit.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# npm: Studio frontend
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: npm audit (Studio frontend)
|
|
# `npm audit` resolves the lockfile through the npmjs.com
|
|
# advisory DB. `--audit-level=high` filters the noise floor
|
|
# to only HIGH and CRITICAL. We do NOT pass --omit=dev: a
|
|
# malicious dev-only dep can still steal secrets from a CI
|
|
# runner, so dev deps need to be in the audit surface.
|
|
continue-on-error: true
|
|
working-directory: studio/frontend
|
|
run: |
|
|
set +e
|
|
npm audit --audit-level=high | tee ../../logs-npm-audit.txt
|
|
# Always also write the full JSON for grep-ability.
|
|
npm audit --json > ../../logs-npm-audit.json || true
|
|
{
|
|
echo "## npm audit (Studio frontend)"
|
|
echo
|
|
echo '```'
|
|
tail -200 ../../logs-npm-audit.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# cargo: Studio Tauri shell
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: cargo audit (Studio Tauri)
|
|
# `--deny warnings` would make the job fail on any advisory.
|
|
# Keep non-blocking initially; drop continue-on-error after
|
|
# the baseline closes.
|
|
continue-on-error: true
|
|
working-directory: studio/src-tauri
|
|
run: |
|
|
set +e
|
|
cargo audit | tee ../../logs-cargo-audit.txt
|
|
{
|
|
echo "## cargo audit (Studio Tauri)"
|
|
echo
|
|
echo '```'
|
|
tail -200 ../../logs-cargo-audit.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# OSV-Scanner: cross-ecosystem advisory DB (PyPI + npm + cargo)
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: OSV-Scanner (PyPI + npm + cargo, cross-ecosystem advisories)
|
|
# OSV's advisory feed is a superset of GitHub-Advisory + RustSec
|
|
# + npm advisories; running it alongside the per-ecosystem audit
|
|
# tools catches CVEs that haven't propagated to the per-ecosystem
|
|
# DBs yet (e.g. langchain-core CVE-2025-68664 was on OSV before
|
|
# GitHub Advisory). Single binary, one transitive resolver, all
|
|
# three lockfile types in one pass. Non-blocking until baselines
|
|
# close.
|
|
continue-on-error: true
|
|
run: |
|
|
set +e
|
|
# OSV-Scanner ships a raw binary (no tarball) in v2.x.
|
|
curl -fsSL -o /tmp/osv-scanner \
|
|
https://github.com/google/osv-scanner/releases/download/v2.0.2/osv-scanner_linux_amd64
|
|
chmod +x /tmp/osv-scanner
|
|
/tmp/osv-scanner --version
|
|
/tmp/osv-scanner scan source \
|
|
--lockfile=studio/frontend/package-lock.json \
|
|
--lockfile=studio/src-tauri/Cargo.lock \
|
|
--lockfile=requirements.txt:audit-reqs/unsloth-deps.txt \
|
|
--lockfile=requirements.txt:audit-reqs/studio.txt \
|
|
--lockfile=requirements.txt:audit-reqs/no-torch-runtime.txt \
|
|
--lockfile=requirements.txt:audit-reqs/overrides.txt \
|
|
--lockfile=requirements.txt:audit-reqs/extras.txt \
|
|
--lockfile=requirements.txt:audit-reqs/extras-no-deps.txt \
|
|
--format=table 2>&1 | tee logs-osv-scanner.txt
|
|
{
|
|
echo "## OSV-Scanner (cross-ecosystem)"
|
|
echo
|
|
echo '```'
|
|
tail -200 logs-osv-scanner.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# Semgrep: design-flaw detection (catches what regex-pattern
|
|
# scanning of malicious authors cannot — first-party logic bugs
|
|
# like langchain-core CVE-2025-68664 dumps/dumpd injection,
|
|
# n8n CVE-2025-68668 _pyodide.eval_code sandbox escape, marimo
|
|
# CVE-2026-39987 unauth WebSocket).
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: Semgrep (supply-chain + python rule packs)
|
|
continue-on-error: true
|
|
run: |
|
|
set +e
|
|
python -m pip install --quiet 'semgrep>=1.95'
|
|
semgrep --version
|
|
semgrep scan \
|
|
--config p/supply-chain \
|
|
--config p/python \
|
|
--config p/javascript \
|
|
--config p/security-audit \
|
|
--severity ERROR --severity WARNING \
|
|
--metrics off \
|
|
--timeout 120 \
|
|
studio/backend unsloth scripts \
|
|
2>&1 | tee logs-semgrep.txt
|
|
{
|
|
echo "## Semgrep (supply-chain + python + javascript rules)"
|
|
echo
|
|
echo '```'
|
|
tail -200 logs-semgrep.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# Lockfile pin verifier. The litellm 1.82.7 attack window was
|
|
# ~40 minutes; anyone resolving with `>=` got the malicious
|
|
# version automatically. Flag every spec in the requirements
|
|
# files that does not pin to an exact `==` (or `@` for git
|
|
# refs, or `===` for arbitrary equality). Warning-only for now;
|
|
# graduate to blocking once the baseline is clean.
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: Lockfile pin verifier (Python requirements)
|
|
continue-on-error: true
|
|
run: |
|
|
python <<'PY' | tee logs-pin-verifier.txt
|
|
import re
|
|
from pathlib import Path
|
|
|
|
# Specs that look like `pkg==1.2.3` or `pkg @ git+...` or
|
|
# bare comments / -r lines are pinned-or-not-applicable.
|
|
PINNED = re.compile(r"^\s*[A-Za-z0-9_.\-]+\s*(?:===|==)\s*[^,;]+\s*$")
|
|
GIT_OR_URL = re.compile(r"^\s*[A-Za-z0-9_.\-]+\s*@\s*(?:git\+|https?://)")
|
|
|
|
unpinned = []
|
|
for f in sorted(Path("studio/backend/requirements").glob("*.txt")):
|
|
for i, raw in enumerate(f.read_text().splitlines(), 1):
|
|
line = raw.strip()
|
|
if not line or line.startswith("#") or line.startswith("-"):
|
|
continue
|
|
spec = line.split("#", 1)[0].strip().split(";", 1)[0].strip()
|
|
if not spec:
|
|
continue
|
|
if "git+" in spec or PINNED.match(spec) or GIT_OR_URL.match(spec):
|
|
continue
|
|
unpinned.append((str(f), i, line))
|
|
|
|
print(f"::group::Lockfile pin status")
|
|
if unpinned:
|
|
print(f"WARN: {len(unpinned)} non-`==` specs across requirements/*.txt")
|
|
print("(litellm 1.82.7 wave hit anyone on `>=`; tighten when feasible.)")
|
|
for f, i, line in unpinned[:80]:
|
|
print(f" {f}:{i}: {line}")
|
|
if len(unpinned) > 80:
|
|
print(f" ... and {len(unpinned) - 80} more")
|
|
else:
|
|
print("OK: every spec is exact-pinned.")
|
|
print("::endgroup::")
|
|
PY
|
|
{
|
|
echo "## Lockfile pin verifier"
|
|
echo
|
|
echo '```'
|
|
cat logs-pin-verifier.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# Trivy is deliberately NOT installed here. Trivy was the entry
|
|
# point for the litellm 1.82.7 supply-chain compromise (March
|
|
# 2026): attackers force-rewrote 76 of 77 tags in
|
|
# aquasecurity/trivy-action to point at malicious commits;
|
|
# anyone running the action with a tag ref auto-pulled a
|
|
# credential-harvesting payload. By design a security scanner
|
|
# has broad read access to runner secrets, which is exactly
|
|
# what made it the ideal pivot. We pick up Trivy's CVE coverage
|
|
# from OSV-Scanner (NVD + GHSA + GitLab) and its secret
|
|
# detection from TruffleHog. IaC misconfig detection (Trivy's
|
|
# one unique value-add) is unfilled for now -- revisit with
|
|
# checkov / kics when we ship a Dockerfile or k8s manifests.
|
|
# See https://docs.litellm.ai/blog/security-update-march-2026
|
|
# and the Microsoft / Trend Micro / Snyk incident write-ups.
|
|
# ─────────────────────────────────────────────────────────────
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# TruffleHog secret-leak scan on the PR diff. Catches API keys
|
|
# / tokens / cred files committed accidentally. --only-verified
|
|
# filters out probabilistic findings, so we only flag tokens
|
|
# that the source provider confirmed are live. On push to main
|
|
# / pip we scan the full repo; on PR we scan base..head.
|
|
# SHA-pinned for the same reason as harden-runner above.
|
|
# v3.95.2 commit:
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: TruffleHog (secrets in diff)
|
|
continue-on-error: true
|
|
uses: trufflesecurity/trufflehog@17456f8c7d042d8c82c9a8ca9e937231f9f42e26 # v3.95.2
|
|
with:
|
|
path: ./
|
|
base: ${{ github.event.pull_request.base.sha || '' }}
|
|
head: ${{ github.event.pull_request.head.sha || github.sha }}
|
|
# The action passes --no-update internally; passing it here
|
|
# too triggers `flag 'no-update' cannot be repeated`. Stick
|
|
# with --only-verified so we only flag tokens the source
|
|
# provider confirmed are live (no probabilistic findings).
|
|
extra_args: --only-verified
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# CycloneDX SBOM. Lets downstream consumers audit what's
|
|
# actually shipped in unsloth wheels and the Studio backend
|
|
# runtime. Generates one JSON file per requirements input plus
|
|
# a combined SBOM keyed off pyproject.toml; uploads as a build
|
|
# artifact (and a future step can attest it via SLSA).
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: Generate CycloneDX SBOM
|
|
continue-on-error: true
|
|
run: |
|
|
set +e
|
|
python -m pip install --quiet 'cyclonedx-bom>=4.6'
|
|
mkdir -p sbom
|
|
# Per-requirements-file SBOM (the audit-reqs/ files are the
|
|
# filtered, git+-stripped views built earlier in this job).
|
|
# cyclonedx-py 4.x uses `--sv` for spec version and `-o` for
|
|
# the output file; the older `--schema-version`/`--outfile`
|
|
# spellings are not accepted.
|
|
for f in audit-reqs/*.txt; do
|
|
base=$(basename "$f" .txt)
|
|
if grep -qE '^[^#[:space:]]' "$f"; then
|
|
cyclonedx-py requirements "$f" \
|
|
--sv 1.6 \
|
|
--of JSON \
|
|
-o "sbom/sbom-$base.json" 2>&1 | tail -5 || true
|
|
fi
|
|
done
|
|
# Project-level SBOM from pyproject.toml.
|
|
cyclonedx-py environment \
|
|
--sv 1.6 \
|
|
--of JSON \
|
|
-o sbom/sbom-environment.json 2>&1 | tail -5 || true
|
|
ls -la sbom/
|
|
{
|
|
echo "## CycloneDX SBOM"
|
|
echo
|
|
echo "Generated SBOM files:"
|
|
ls sbom/ | sed 's/^/- sbom\//'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# GitHub Actions pinning verifier. tj-actions/changed-files
|
|
# was compromised in March 2025; anyone using `@v4` (a mutable
|
|
# ref) auto-shipped the malicious version. Catch every
|
|
# non-SHA-pinned `uses:` across the workflows tree. Warn-only
|
|
# initially so the existing baseline doesn't block PRs.
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: GitHub Actions pinning verifier
|
|
continue-on-error: true
|
|
run: |
|
|
python <<'PY' | tee logs-actions-pinning.txt
|
|
import re
|
|
from pathlib import Path
|
|
# SHA pin = 40 hex chars after @
|
|
SHA_PIN = re.compile(r"@[0-9a-f]{40}\b")
|
|
# First-party / GitHub-published actions get a softer pass
|
|
# (still recommended to pin; not a security gate).
|
|
FIRST_PARTY = re.compile(r"^\s*-\s*uses:\s*(actions|github)/[^@]+@")
|
|
USES = re.compile(r"^\s*-\s*uses:\s*([^@\s]+)@(\S+)")
|
|
unpinned_third = []
|
|
unpinned_first = []
|
|
for f in sorted(Path(".github/workflows").glob("*.yml")):
|
|
for i, line in enumerate(f.read_text().splitlines(), 1):
|
|
m = USES.match(line)
|
|
if not m:
|
|
continue
|
|
name, ref = m.group(1), m.group(2)
|
|
if SHA_PIN.search(line):
|
|
continue
|
|
bucket = unpinned_first if FIRST_PARTY.match(line) else unpinned_third
|
|
bucket.append((str(f), i, name, ref))
|
|
print("::group::Action pinning status")
|
|
print(f"third-party actions on mutable refs: {len(unpinned_third)}")
|
|
for f, i, n, r in unpinned_third:
|
|
print(f" HIGH {f}:{i}: {n}@{r}")
|
|
print()
|
|
print(f"first-party (actions/* | github/*) on mutable refs: {len(unpinned_first)}")
|
|
for f, i, n, r in unpinned_first[:30]:
|
|
print(f" WARN {f}:{i}: {n}@{r}")
|
|
if len(unpinned_first) > 30:
|
|
print(f" ... and {len(unpinned_first) - 30} more")
|
|
print()
|
|
print("Recommendation: pin third-party actions to a 40-char SHA.")
|
|
print("Dependabot's github-actions ecosystem will auto-bump them.")
|
|
print("::endgroup::")
|
|
PY
|
|
{
|
|
echo "## GitHub Actions pinning verifier"
|
|
echo
|
|
echo '```'
|
|
cat logs-actions-pinning.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# ─────────────────────────────────────────────────────────────
|
|
# Hash-pin verifier. `==` pinning protects against version
|
|
# drift but not against a re-uploaded malicious wheel at the
|
|
# same version (PyPI lets a yanked release be re-published with
|
|
# different bytes for ~5 minutes via `--filename` collision).
|
|
# `pip install --require-hashes` rejects any download whose
|
|
# SHA-256 doesn't match. Inspector step that reports how many
|
|
# specs would gain from a hash pin -- conversion is a roadmap
|
|
# item (needs pip-tools / uv pip compile --generate-hashes).
|
|
# ─────────────────────────────────────────────────────────────
|
|
- name: Hash-pin verifier (Python requirements)
|
|
continue-on-error: true
|
|
run: |
|
|
python <<'PY' | tee logs-hash-verifier.txt
|
|
import re
|
|
from pathlib import Path
|
|
PINNED = re.compile(r"^\s*[A-Za-z0-9_.\-]+\s*==\s*[^,;]+\s*$")
|
|
HASH_LINE = re.compile(r"--hash=sha256:[0-9a-f]{64}")
|
|
total_pinned = 0
|
|
with_hash = 0
|
|
for f in sorted(Path("studio/backend/requirements").glob("*.txt")):
|
|
text = f.read_text()
|
|
for raw in text.splitlines():
|
|
line = raw.strip()
|
|
if not line or line.startswith("#") or line.startswith("-"):
|
|
continue
|
|
spec = line.split("#", 1)[0].strip().split(";", 1)[0]
|
|
if PINNED.match(spec):
|
|
total_pinned += 1
|
|
if HASH_LINE.search(raw):
|
|
with_hash += 1
|
|
print(f"::group::Hash-pin status")
|
|
print(f" exact == pins: {total_pinned}")
|
|
print(f" with --hash=sha256: {with_hash}")
|
|
print(f" without --hash: {total_pinned - with_hash}")
|
|
print()
|
|
print("Roadmap: convert to hash-locked installs via")
|
|
print("`uv pip compile --generate-hashes` and `pip install --require-hashes`.")
|
|
print("Hash-locked installs would have refused a republished")
|
|
print("malicious litellm 1.82.7 wheel even at the same version.")
|
|
print("::endgroup::")
|
|
PY
|
|
{
|
|
echo "## Hash-pin verifier"
|
|
echo
|
|
echo '```'
|
|
cat logs-hash-verifier.txt
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
|
if: always()
|
|
with:
|
|
name: advisory-audit-logs
|
|
path: |
|
|
logs-pip-audit.txt
|
|
logs-npm-audit.txt
|
|
logs-npm-audit.json
|
|
logs-cargo-audit.txt
|
|
logs-osv-scanner.txt
|
|
logs-semgrep.txt
|
|
logs-pin-verifier.txt
|
|
logs-actions-pinning.txt
|
|
logs-hash-verifier.txt
|
|
audit-reqs/
|
|
sbom/
|
|
retention-days: 30
|
|
|
|
# ─────────────────────────────────────────────────────────────────────
|
|
# Python: pre-install package scan (no install, no execution)
|
|
# ─────────────────────────────────────────────────────────────────────
|
|
pip-scan-packages:
|
|
# Downloads each declared dep WITHOUT installing it and inspects
|
|
# the archive contents for known malicious patterns: weaponized
|
|
# .pth files, credential stealers, obfuscated payloads,
|
|
# install-time droppers, suspicious subprocess / network /
|
|
# base64-blob combinations.
|
|
#
|
|
# This is the kind of check that would have caught:
|
|
# - litellm 1.82.7 / 1.82.8 (March 2026, supply-chain compromise)
|
|
# - the typo-squat campaign against PyTorch Lightning
|
|
# before either landed in the install path. pip-audit only knows
|
|
# about CVE-published vulnerabilities, so it does NOT see novel
|
|
# malicious uploads. scan_packages.py runs deterministic regex
|
|
# pattern matching, no LLM calls.
|
|
#
|
|
# `--with-deps` makes the scan transitive: every package the
|
|
# declared set resolves to gets fetched and pattern-scanned, not
|
|
# just the top-level pins. Resolving the full transitive closure
|
|
# of the unsloth + Studio dep tree downloads several hundred
|
|
# archives, hence the longer timeout.
|
|
#
|
|
# Sharded across runners for wall-clock parallelism. Each shard
|
|
# runs scan_packages.py once with --with-deps so its own slice
|
|
# benefits from pip's deduped transitive resolve. Shard
|
|
# composition tries to balance load:
|
|
# - hf-stack: pyproject extras + no-torch-runtime
|
|
# (~150 archives, transformers/peft/accelerate/...)
|
|
# - studio: FastAPI/Studio backend + overrides + extras-no-deps
|
|
# (~150 archives, smaller scientific stack)
|
|
# - extras: the heavy openai-whisper / scikit-learn / librosa
|
|
# stack (~250 archives, dominant cost)
|
|
# triton-kernels.txt is git+-only, fully skipped.
|
|
name: ${{ matrix.shard.name }}
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 25
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
shard:
|
|
- name: 'pip scan-packages :: hf-stack'
|
|
id: hf-stack
|
|
files: 'unsloth-deps no-torch-runtime'
|
|
- name: 'pip scan-packages :: studio'
|
|
id: studio
|
|
files: 'studio overrides extras-no-deps'
|
|
- name: 'pip scan-packages :: extras'
|
|
id: extras
|
|
files: 'extras'
|
|
steps:
|
|
# Egress audit on every shard. Each shard pulls hundreds of
|
|
# PyPI archives -- if a malicious wheel ever phones home from
|
|
# within the scanner sandbox (it shouldn't; we never execute
|
|
# the archive), harden-runner's audit log records the host.
|
|
- name: Harden runner (egress audit)
|
|
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
|
|
with:
|
|
egress-policy: audit
|
|
disable-sudo: true
|
|
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
|
|
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
|
with:
|
|
python-version: '3.12'
|
|
cache: 'pip'
|
|
|
|
- name: Install scan_packages.py runtime deps
|
|
# scan_packages.py imports requests + packaging at runtime to
|
|
# talk to PyPI's JSON API and to parse version specifiers. We
|
|
# do not install the packages it scans -- those are downloaded
|
|
# raw and inspected without ever touching `pip install`.
|
|
run: python -m pip install --upgrade pip requests packaging
|
|
|
|
- name: Build filtered requirements set
|
|
# Mirrors the advisory-audit job's input transform: pyproject.toml
|
|
# extraction + git+ stripping. scan_packages.py downloads
|
|
# PyPI archives without building, so it tolerates legacy
|
|
# setup.py packages (no resolver dry-run); but `--with-deps`
|
|
# delegates resolution to a single `pip download` call that
|
|
# cannot satisfy `git+` specs without git operations, so we
|
|
# strip them here too.
|
|
run: |
|
|
mkdir -p audit-reqs
|
|
python <<'PY' > audit-reqs/unsloth-deps.txt
|
|
import tomllib
|
|
with open("pyproject.toml", "rb") as f:
|
|
d = tomllib.load(f)
|
|
core = d["project"]["dependencies"]
|
|
extras = d["project"]["optional-dependencies"]["huggingfacenotorch"]
|
|
print("# Auto-generated from pyproject.toml by security-audit.yml.")
|
|
print("# core deps + huggingfacenotorch extras.")
|
|
for spec in core + extras:
|
|
print(spec)
|
|
PY
|
|
for f in studio.txt extras.txt extras-no-deps.txt \
|
|
no-torch-runtime.txt overrides.txt triton-kernels.txt; do
|
|
python <<PY > "audit-reqs/$f"
|
|
src = "studio/backend/requirements/$f"
|
|
with open(src) as fh:
|
|
for line in fh:
|
|
stripped = line.strip()
|
|
before_comment = stripped.split("#", 1)[0]
|
|
if "git+" in before_comment:
|
|
print(f"# [security-audit] skipped git+ spec: {stripped}")
|
|
continue
|
|
print(line.rstrip("\n"))
|
|
PY
|
|
done
|
|
|
|
- name: Sanity-check scan_packages.py
|
|
# The scanner lives at scripts/scan_packages.py in this repo
|
|
# so we don't depend on a network fetch at job time.
|
|
run: |
|
|
test -f scripts/scan_packages.py
|
|
head -3 scripts/scan_packages.py
|
|
grep -q "Standalone pre-install package scanner" scripts/scan_packages.py
|
|
|
|
- name: Scan declared + transitive Python deps
|
|
# scan_packages.py exits 1 on CRITICAL/HIGH findings, 0 on
|
|
# clean. We swallow the exit because the baseline isn't
|
|
# triaged yet; surface the findings in the workflow summary.
|
|
# Drop continue-on-error after the first clean run on main.
|
|
#
|
|
# `--with-deps` walks PyPI metadata to enumerate every
|
|
# transitive dep the declared set would install, then scans
|
|
# them all. Without this flag, we'd only catch a malicious
|
|
# *direct* dep -- and supply-chain attacks usually land
|
|
# several hops down (litellm 1.82.7 was a dep of a dep for
|
|
# most users).
|
|
#
|
|
# This step runs once per matrix shard. Within a shard, every
|
|
# -r file is fed to a single `pip download` call so pip
|
|
# intersects version constraints and yields a deduped
|
|
# transitive set (no point fetching the same transformers
|
|
# wheel five times). Across shards we accept some redundant
|
|
# downloads in exchange for wall-clock parallelism.
|
|
continue-on-error: true
|
|
env:
|
|
SHARD_FILES: ${{ matrix.shard.files }}
|
|
run: |
|
|
set +e
|
|
mkdir -p logs
|
|
LOG="logs-scan-packages-${{ matrix.shard.id }}.txt"
|
|
echo "::group::shard ${{ matrix.shard.id }} input files"
|
|
REQ_ARGS=()
|
|
for f in $SHARD_FILES; do
|
|
if grep -qE '^[^#[:space:]]' "audit-reqs/$f.txt"; then
|
|
echo " + audit-reqs/$f.txt"
|
|
REQ_ARGS+=( -r "audit-reqs/$f.txt" )
|
|
else
|
|
echo " - audit-reqs/$f.txt (empty after git+ filter, skipping)"
|
|
fi
|
|
done
|
|
echo "::endgroup::"
|
|
if [ ${#REQ_ARGS[@]} -eq 0 ]; then
|
|
echo "[security-audit] shard ${{ matrix.shard.id }}: no PyPI specs, nothing to scan" \
|
|
| tee "$LOG"
|
|
else
|
|
python scripts/scan_packages.py --with-deps "${REQ_ARGS[@]}" \
|
|
2>&1 | tee "$LOG"
|
|
fi
|
|
{
|
|
echo "## scan_packages :: shard ${{ matrix.shard.id }}"
|
|
echo
|
|
echo "### Files in this shard"
|
|
for f in $SHARD_FILES; do echo "- audit-reqs/$f.txt"; done
|
|
echo
|
|
echo '### Findings (tail)'
|
|
echo '```'
|
|
tail -200 "$LOG"
|
|
echo '```'
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
|
if: always()
|
|
with:
|
|
name: scan-packages-log-${{ matrix.shard.id }}
|
|
path: |
|
|
logs-scan-packages-${{ matrix.shard.id }}.txt
|
|
audit-reqs/
|
|
retention-days: 30
|