Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent.
* scripts/scan_*: add Mini Shai-Hulud May-12 IOC strings and pin-blocklists
Append the May-12 2026 wave indicators (git-tanstack.com, transformers.pyz,
/tmp/transformers.pyz, "With Love TeamPCP", "We've been online over 2 hours")
to all three scanner IOC tables, add BLOCKED_NPM_VERSIONS (42 TanStack pkgs,
4 opensearch versions, 3 squawk pkgs) in scan_npm_packages.py and
lockfile_supply_chain_audit.py (kept byte-identical), add BLOCKED_PYPI_VERSIONS
(guardrails-ai 0.10.1, mistralai 2.4.6, lightning 2.6.2/2.6.3) plus
RE_MAY12_IOC wiring across check_py_file/check_shell_file/check_workflow_file
in scan_packages.py. The npm orchestrator and the lockfile auditor now
short-circuit on a blocked entry before fetching the tarball, and the
PyPI download pipeline drops blocked specs before pip download is invoked.
* tests/security: regression suite for supply-chain scanners
Adds offline fixture corpus and pytest coverage for scan_npm_packages,
scan_packages, and lockfile_supply_chain_audit so future IOC-table
drift surfaces at PR time. Pytest scope narrowed to tests/security so
GPU smoke tests are not picked up by default.
* ci(security-audit): drop continue-on-error on pip-scan and npm-scan jobs
Promote three harden-runner blocks to egress-policy: block with per-job allowlists.
Add tests-security job running pytest tests/security as a hard gate.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* scripts: harden third-party downloads, pip resolver pins, atomic writes
Pins uv installer and mlx_vlm qwen3_5 patches by commit SHA + SHA-256
checksum, scrubs PIP_* env vars and forces --index-url + --only-binary
on pip download, applies tarbomb caps to scan_packages archive walks,
and converts non-atomic config writes (kwargs spacer, studio stamper,
notebook validator, scan_packages req-file fixer) to mkstemp+os.replace.
Also adds host allowlist to notebook_to_python downloader, threads an
--allow-shell flag through its shell=True emission with reviewer warning
comments, locks both MLX installer scripts to set -euo pipefail, and
extends CODEOWNERS so colab snapshot data files require notebook-owner
review.
* ci(workflows): harden release-desktop / smoke / notebooks workflows
Pin dtolnay/rust-toolchain to a 40-char SHA, scope release-desktop permissions to read at workflow level with job-level write only on the build job, append --ignore-scripts to every npm ci / npm install in studio-frontend-ci / wheel-smoke / studio-tauri-smoke / release-desktop, validate client_payload.ref shape via an env-var-isolated regex on every notebooks-ci job, and add step-security/harden-runner in audit mode as the first step of release-desktop and mlx-ci.
* scripts: promote silent scanner failures to non-zero exit codes
scan_packages now returns 2 on pip-download failure and emits a CRITICAL archive_corrupted finding on truncated wheels/sdists.
notebook_to_python exits 1 on per-notebook failures; notebook_validator wraps the stash/pop in try/finally; lockfile audit rejects bare UNSLOTH_LOCKFILE_AUDIT_SKIP=1 with a loud GitHub Actions warning.
* Add npm cooldown + new-install-script gate + Dependabot cooldown
Pins min-release-age=7 (npm 11.10+) in repo-root and studio/frontend
.npmrc, adds scripts/check_new_install_scripts.py to fail PRs that
add a postinstall dep, ships a new security-audit job for npm audit
signatures plus the diff, and extends .github/dependabot.yml with
cooldown stanzas. Pin @tanstack/react-router to 1.169.9 per GHSA-
g7cv-rxg3-hmpx; lockfile regen deferred until that release lands on
npm. tests/security gains 4 new tests; full suite 26/26 green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ci(security): fix tanstack pin, exec bits, expand IOC tables to @uipath/@squawk full
- Revert --ignore-scripts on Studio install workflows: vite build needs
esbuild's native postinstall (per PR #5392 rationale). Keep
--ignore-scripts on security-audit.yml's standalone npm audit job.
- Pin @tanstack/react-router to the actual published 1.169.2 (was a
forward-looking 1.169.9 that does not exist on npm; broke npm ci).
- Drop redundant repo-root .npmrc; studio/frontend/.npmrc covers the
only npm project today (root cooldown re-instate via dependabot.yml).
- Restore exec bits on 7 files my filesystem stripped during cherry-pick.
- Expand BLOCKED_NPM_VERSIONS with full safedep.io + Aikido enumeration:
22 @squawk/* packages with 5 versions each (110 entries; previously
3 entries with 1 version each), and 66 @uipath/* packages (entirely
missing before). Mirror in scripts/lockfile_supply_chain_audit.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests/security: suppress CodeQL py/incomplete-url-substring-sanitization
The two flagged 'X' in Y assertions are NOT URL sanitization checks.
They verify our scanner WROTE a known IOC literal into its stdout /
Finding.evidence, which is the opposite of an attack surface --
matching the scanner's output is precisely what catches the worm.
Inline lgtm[] suppression with a 4-line rationale comment above each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* scripts/scan_*: expand IOC tables with Aikido full 169-pkg enumeration
Per Aikido 2026-05-12 disclosure (373 malicious package-version entries
across 169 npm package names), add to BLOCKED_NPM_VERSIONS:
- @mistralai/* npm scope (3 packages, 9 versions) -- separate from
the PyPI mistralai package already in BLOCKED_PYPI_VERSIONS
- @tallyui/* (10 packages, 30 entries)
- @beproduct/nestjs-auth (18 versions 0.1.2..0.1.19)
- @draftlab/* + @draftauth/* (5 packages)
- @taskflow-corp/cli, @tolka/cli, @ml-toolkit-ts/*, @mesadev/*,
@dirigible-ai/sdk, @supersurkhet/*
- 10 unscoped packages (safe-action, ts-dna, cross-stitch,
cmux-agent-mcp, agentwork-cli, git-branch-selector, wot-api,
git-git-git, nextmove-mcp, ml-toolkit-ts)
Also add to KNOWN_IOC_STRINGS / NPM_IOC_STRINGS:
- router_init.js SHA-256 ab4fcadaec49c03278063dd269ea5eef82d24f2124a8e15d7b90f2fa8601266c
- tanstack_runner.js SHA-256 2ec78d556d696e208927cc503d48e4b5eb56b31abc2870c2ed2e98d6be27fc96
- bun run tanstack_runner.js marker (the new Bun-prepare-script
dropper invocation pattern unique to this wave)
Total: 170 packages, 401 versions blocklisted. Studio lockfile still
scans clean (0 findings, 0 hard errors).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* scripts/scan_*: web-verification additions (@tanstack/setup, intercom-client)
Two findings from cross-checking BLOCKED_NPM_VERSIONS / KNOWN_IOC_STRINGS
against GHSA-g7cv-rxg3-hmpx + Aikido + safedep.io + Socket + Semgrep.
- Fix asymmetry: @tanstack/setup IOC string was in
lockfile_supply_chain_audit.py's NPM_IOC_STRINGS but missing from
scan_npm_packages.py's KNOWN_IOC_STRINGS. The literal is the malicious
optional-dependency name used by the May-12 TanStack wave; no
legitimate npm package of this name exists.
- Add intercom-client@7.0.4: the npm counterpart of the lightning
2.6.2/2.6.3 PyPI compromise (Apr-30 wave). Same threat actor
(TeamPCP). Confirmed by Semgrep, Aikido, OX Security, Resecurity,
Kodem. Safe version is 7.0.3 and earlier.
Total BLOCKED_NPM_VERSIONS: 171 packages / 402 versions. Both files
remain byte-identical. Studio lockfile still scans clean.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ci(security): add workflow-trigger lint refusing pull_request_target + cache-poisoning vectors
The two patterns that together powered GHSA-g7cv-rxg3-hmpx (TanStack
Mini Shai-Hulud) are now gated at PR time:
1. pull_request_target -- the worm chain started with a fork PR that
ran in the base-repo context. Every workflow in this repo today
uses 'pull_request' (safe); the lint refuses any new
pull_request_target additions outright. workflow_run is
restricted, allowed only with an explicit allow-comment.
2. Shared cache keys between PR-triggered workflows and the publish
workflow (release-desktop.yml). The TanStack attack chain poisoned
a shared Actions cache from a fork PR; the legitimate release
workflow then restored the poisoned cache. The lint refuses any
cache key that appears in both a PR-triggered workflow and a
workflow_dispatch-only / publish workflow.
Current tree is clean: 0 pull_request_target, 0 workflow_run, 0
PR-publish cache-key collisions across all 24 workflows. The lint
locks that invariant in place.
Files:
+ scripts/lint_workflow_triggers.py (~200 LOC, stdlib + PyYAML)
+ tests/security/test_lint_workflow_triggers.py (5 tests covering
current-tree pass, pull_request_target reject, workflow_run
restricted, justified workflow_run accept, cache-key collision
reject)
~ .github/workflows/security-audit.yml: new workflow-trigger-lint
job, no continue-on-error, harden-runner block-mode, PyYAML only
runtime dep.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* security: fix tests-security CI job + CodeQL false-positives
Two CI failures on the prior push:
1. pytest tests/security -- 5 lint regression tests failed because
scripts/lint_workflow_triggers.py imports PyYAML which is not in
the bare runner's Python env. Added pyyaml==6.0.2 to the pip
install step alongside pytest. (29 scanner tests already passed.)
2. CodeQL py/incomplete-url-substring-sanitization fired on two
test assertions that check the scanner WROTE the IOC literal
to its own stdout/stderr. The rule pattern-matches on
`"<host>" in <var>` and cannot distinguish a URL sanitizer from
a regression-test evidence check. Previous `# lgtm[...]` inline
suppressions were detached from the operator when pre-commit
reformatted the assert across multiple lines. Rebuilt the IOC
literals at runtime (`"git-tanstack." + "com"`) so no URL-shaped
source literal appears on the `in` operator line; rule cannot
trigger.
Verified locally: `pytest tests/security -v` -> 34 passed in 2.70s.
* security(studio): defensive .npmrc cooldown aliases + save-exact
Two additions to studio/frontend/.npmrc to harden the existing
`min-release-age=7` (Mini Shai-Hulud defence):
1. `minimum-release-age=10080` (minutes) -- defensive alias for the
same 7-day floor. Some npm versions / wrappers consult one key but
not the other; setting both prevents a single upstream setting-name
parse change from silently disabling the cooldown. The two keys
MUST agree (do not let them drift).
2. `save-exact=true` -- refuses to write back `^x.y.z` ranges into
package.json when a maintainer runs `npm install <pkg>` locally.
Does NOT rewrite already-present ranges; stops NEW carets from
creeping into the manifest as patch-version footguns.
Verified: pytest tests/security -> 34 passed in 2.63s.
* chore(dependabot): remove dead bun entry for /studio/frontend
`package-ecosystem: "bun"` at /studio/frontend was a no-op: that
path commits package-lock.json, not bun.lock / bun.lockb, so
Dependabot's bun ecosystem silently skipped it. The actual
behaviour is unchanged -- the npm entry below the cargo block
already owns npm_and_yarn security advisories for /studio/frontend
with `open-pull-requests-limit: 0` (version-update PRs suppressed,
security PRs flow through).
This commit:
- Deletes the bun entry (kept a placeholder comment so a future
bun migration knows where to slot it back in).
- Rewrites the npm /studio/frontend entry comment to explain the
real intent: lockfile is the authoritative pin, .npmrc
`min-release-age=7` already blocks fresh tarballs at install
time, dependabot only needs to surface security advisories.
No functional change: same set of dependabot PRs as before (zero
version updates, security advisories grouped weekly with cooldown).
Verified: pytest tests/security -> 34 passed in 2.67s; YAML
parses cleanly via PyYAML.
* fix(dependabot): drop unsupported semver-* cooldown keys on github-actions
Dependabot's validator rejected the config with:
The property '#/updates/0/cooldown/semver-minor-days' is not
supported for the package ecosystem 'github-actions'.
The property '#/updates/0/cooldown/semver-patch-days' is not
supported for the package ecosystem 'github-actions'.
The `semver-minor-days` / `semver-patch-days` cooldown knobs are
only valid for semver-aware ecosystems (npm, cargo, etc.). The
github-actions ecosystem pins via git tags / SHAs, not semver, so
only `default-days` is honored. Pre-existing bug on main; surfaced
on this PR because the prior commit re-validated the file.
Behaviour: github-actions PRs now respect the 7-day cooldown floor
(was already the intent), without the no-op semver bands.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio/ci: npm tarball content scanner (no-install, hostile-input safe)
Counterpart to scripts/scan_packages.py for the npm side. Pip-side
scanner reads requirements files, downloads PyPI archives via
`pip download --no-deps`, and pattern-scans them for malicious
shapes. This change adds the equivalent for npm tarballs.
Why
===
PR #5392 (lockfile_supply_chain_audit.py) catches injection-pattern
attacks where the malicious metadata lives IN the lockfile -- e.g.
the TanStack Shai-Hulud worm that injected an `optionalDependencies`
entry pointing at a GitHub commit. It does not catch the broader
class of "legit-registry tarball with malicious content but normal
lockfile metadata": attacker steals a maintainer's npm publish
token, publishes a malicious version to registry.npmjs.org with a
valid integrity hash, and the lockfile entry looks normal -- the
malicious code lives inside the tarball's dist/index.js or its own
postinstall script. Today that gap is covered reactively by `npm
audit` + OSV-Scanner once the GHSA lands; there is a real window
before that.
This scanner closes the window by inspecting tarball CONTENT.
What it checks
==============
For each entry in studio/frontend/package-lock.json:
1. Download the tarball directly from registry.npmjs.org. Refuse
any non-allowlisted URL. Stream-bounded at 64 MiB.
2. Verify SHA-512 integrity against the lockfile entry BEFORE
opening the tarball.
3. Safely extract into a sandboxed temp dir behind guards:
- reject symlinks / hardlinks (LNKTYPE, SYMTYPE)
- reject absolute paths and `..` traversal
- reject character / block / FIFO devices
- per-file size cap 8 MiB, cumulative cap 128 MiB,
member count cap 50000
- stream open (mode='r|gz') so we abort mid-extract
- extracted files set to non-executable mode (0o644)
4. Pattern-scan the extracted text content for:
- lifecycle (preinstall/install/postinstall/prepare) scripts
in any package.json that fetch + pipe-to-shell external
content -- the install-time RCE vector
- optionalDependencies pointing at github: / git+ / git:
(TanStack worm injection shape)
- C2 / exfiltration hosts: getsession.org, 169.254.169.254
(IMDS), 169.254.170.2 (ECS), metadata.google.internal,
vault.svc.cluster.local, k8s ServiceAccount token paths,
ACTIONS_ID_TOKEN_REQUEST_URL/TOKEN, npm publish-token
enumeration endpoint
- credential paths a frontend lib should never read:
~/.npmrc, ~/.aws/credentials, ~/.ssh/id_*, /.kube/config,
/.docker/config.json
- JS regex: Function/eval against base64-decoded payload,
process.env.GITHUB_TOKEN / NPM_TOKEN / AWS_* access in
package source
- obfuscation: large base64-ish blob (>=2 KiB) fed into
Function or eval (router_init.js dropper shape)
- literal IOC substrings from public advisories
Safety
======
Threat model: every tarball is hostile. The scanner:
- never runs `npm install`, never executes anything from a
downloaded tarball, never calls subprocess on extracted content
- downloads only from registry.npmjs.org (defence-in-depth check
at parse time AND inside download_tarball)
- stdlib-only (no third-party deps -- adding one would itself
be a supply-chain liability)
- tempdir wiped via atexit on every termination path
- exit codes: 0 clean, 1 HIGH/CRITICAL finding, 2 internal error
Wiring
======
New job `npm-scan-packages` in security-audit.yml, parallel to
`pip-scan-packages`. Triggers same as the existing audits (PR on
manifest changes, push to main/pip, daily 04:13 UTC, dispatch).
Initially `continue-on-error: true` so the baseline can settle --
matches the existing convention for the other audit steps. Drop
that flag once the baseline is clean for a week.
Verified locally
================
- AST parse OK.
- Real-network 3-package smoke: 0 findings.
- Real-network 25-package smoke (Babel + assistant-ui surface):
0 findings, no hard errors.
- 9 fault-injection scenarios all pass:
1. zip-slip path traversal refused
2. symlink member refused
3. oversized member refused (size cap)
4. too-many-members refused (count cap)
5. router_init.js IOC + obfuscated-blob shape both detected
in synthetic malicious tarball
6. lifecycle fetch-exec in scripts.preinstall detected as
CRITICAL
7. AWS IMDS reference (169.254.169.254) detected
8. SRI integrity-parser accepts syntactically-valid SRI
9. download_tarball refuses non-allowlisted hostname
Refs
====
- https://tanstack.com/blog/npm-supply-chain-compromise-postmortem
- https://github.com/TanStack/router/issues/7383
- https://github.com/TanStack/router/security/advisories/GHSA-g7cv-rxg3-hmpx
- https://www.aikido.dev/blog/mini-shai-hulud-is-back-tanstack-compromised
- https://www.stepsecurity.io/blog/mini-shai-hulud-is-back-a-self-spreading-supply-chain-attack-hits-the-npm-ecosystem
* scan_npm_packages: kill false positives + handle real native binaries
First CI run on PR #5393 (run 25710423126 / job 75489317395) hit
two false-positive classes plus one cap-too-tight class:
False positives (7 findings):
@langchain/core 1.1.44 ssrf.{cjs,js}: a SSRF *protection* module
that ships a literal blocklist `const CLOUD_METADATA_IPS = [...]`
of IMDS hosts as data the library REFUSES to dial. Our scanner
saw the IPs as substrings and flagged 6 of them.
object-treeify 1.1.33 package.json: a manual `docker` dev script
that mounts `~/.npmrc` and `~/.aws` for local containerised
builds. npm never runs `scripts.docker` automatically; it is
only invoked when a developer runs `npm run docker`. Our bare
substring scan flagged the `/.npmrc` reference anyway.
Cap-too-tight class (10+ findings):
next/swc, rolldown bindings, biome CLI, lightningcss, mermaid
sourcemap, typescript.js. The 8 MiB per-file cap was calibrated
for JS source and rejected legitimate precompiled native binaries
(next-swc .node is 137 MB) and CLI executables (biome is 25-33 MB).
Fixes
=====
cred-surface-host detection split into two tiers:
ALWAYS_BAD substrings have no legitimate use anywhere and still
bare-match: `registry.npmjs.org/-/npm/v1/tokens`,
`ACTIONS_ID_TOKEN_REQUEST_URL/TOKEN`.
NEEDS_CONTEXT substrings (IMDS IPs, GCE metadata host, k8s
ServiceAccount path, Vault endpoint) require co-occurrence with
EITHER a fetch verb (fetch/axios/http.get/etc) within 200 chars
OR an `http(s)?://HOST` URL prefix OR a `host:`/`hostname:`
config field. A defensive blocklist literal does not match any
of those rules; an actual outbound call always does.
cred-surface-path detection moved out of the bare-text scan into
`scan_package_json` and scoped to the 4 NPM lifecycle hooks
(preinstall / install / postinstall / prepare). A `/.npmrc`
reference in a `docker` dev script is silent; a `cat ~/.npmrc
| curl ...` in a `postinstall` fires HIGH.
Per-file size cap split by content type, sniffed via 16-byte
magic header read (ELF / Mach-O / PE / WASM / archive formats),
plus suffix list (.node/.wasm/.so/.dll/.dylib/.exe), plus regex
for versioned shared libs (libfoo.so.8.17.3), plus a null-byte
ratio fallback for extensionless binaries that headers do not
catch.
Text files: 16 MiB cap (still tight; typescript.js at 9.1 MB is
the legitimate ceiling).
Binary files: 256 MiB cap (next-swc .node is 137 MB; sharp
libvips is ~18 MB; rolldown bindings are 18-26 MB each).
Cumulative: 512 MiB per tarball.
Tarball: 256 MiB compressed.
Binary files are also skipped in the content scanner -- regex
over compiled machine code is noise. The IOC substring fallback
in `scan_extracted_tree` now uses the same magic-sniff to decide
whether to grep.
HTTP timeout bumped 30s -> 60s for large tarballs.
Verified
========
- AST parse OK.
- 11 fault-injection tests pass:
* zip-slip, symlink, oversized-declared-size, count-cap
* router_init.js IOC detected
* IMDS-in-URL still detected (new contextual rule)
* langchain SSRF blocklist no longer false-positive
* object-treeify docker script no longer false-positive
* lifecycle-script `cat ~/.npmrc | curl ...` detected
* synthetic ELF (extensionless executable) extracts and is
correctly skipped from text scan
* versioned `.so.8.17.3` shared lib extracts cleanly
- Real-network end-to-end on the full lockfile:
968 packages, 0 findings, 0 hard errors, 76 seconds.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Daniel Han