* tests: read checked-in files as UTF-8 instead of the platform default
Path.read_text() with no encoding uses locale.getpreferredencoding(), which
is UTF-8 on the Linux runners and cp1252 on a stock Windows install. Nine
module-level reads of checked-in source files were relying on that default.
studio/backend/routes/inference.py carries the DeepSeek tool-call token
regexes, so it holds U+FF5C and U+2581. Under cp1252 that read raised
UnicodeDecodeError on byte 0x81 at position 97806, and because the reads run
at import time it took test_cancel_atomicity.py and test_cancel_id_wiring.py
out at collection, not as failures. Green on CI, permanently broken for a
Windows contributor running the suite locally.
Adds a guard: at module scope there is no tmp_path fixture, so a bare
read_text()/write_text()/open() there is always touching a checked-in file.
That makes the rule mechanical enough to enforce with no allowlist, while
staying quiet about temp-dir I/O inside test bodies where the platform
default is harmless.
The repo already spells this correctly in 464 other places; this only stops
the stragglers coming back.
* tests: cover import-time helper reads and keep the guard py3.9-safe
Follows up on the Codex review:
- add `from __future__ import annotations`, since `str | None` in
`_offender` is evaluated at import on Python 3.9 and pyproject declares
requires-python ">=3.9,<3.15".
- widen the guard from module scope to import time. Class bodies and the
bodies of module-level helpers called from an executing statement run
during collection too, so `CODE = _extract_mixed_precision_code()` was
the same hazard as an inline read. `if __name__ == "__main__":` blocks
are skipped: pytest never executes them.
- scan studio/backend/tests/ as well as tests/. Both trees are collected
on Windows by separate CI jobs, and the offender that started this,
test_tool_xml_strip.py reading routes/inference.py, lives there.
Widening it surfaced seven more import-time reads of checked-in sources;
all now name utf-8.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden the import-time encoding guard for PR #7438
Close the detector gaps raised in review, all of which I reproduced against
the actual AST before changing anything.
False negatives (the guard let a real hazard through):
- _is_main_guard ignored the comparison operator, so if __name__ != "__main__"
counted as script-only even though its body runs at import.
- The else arm of a main guard was discarded with the rest of the If node.
- Decorators and argument defaults on a module-level def were skipped with the
body, though both are evaluated when the def executes.
- Path.open() in text mode was invisible; only builtin open() was matched.
- encoding = None and encoding = "locale" both re-select the platform default,
but the keyword merely being present counted as pinned.
False positives (the guard would have blocked a compliant contributor):
- A non-literal mode fell through to the "r" default, so open(p, mode) was
flagged even when mode is "rb", where adding encoding= is a ValueError and
there is no edit that satisfies the rule.
- Same for open(*args) and a **kwargs splat, which hide the mode and can hide
an encoding.
- Lambda bodies and comprehension elements were walked even though neither runs
at definition.
Verified: still reports the same 22 offenders on unpatched main, green on this
branch and on the tree merged with latest main (557 files), and an adversarial
corpus of 33 cases now scores zero false positives and zero false negatives.
Also corrected two docstring claims: neither collecting job runs on Windows,
and the read is governed by locale.getencoding().
* Walk eager comprehensions and treat io.open as the builtin
Two regressions from the previous commit, both reproduced against the AST
before changing anything.
Lumping list, set and dict comprehensions in with generator expressions was
wrong. Only a genexp is lazy; the other three run their element expression,
their filters and their nested iterators immediately, so
CONTENTS = [p.read_text() for p in PATHS] at module scope is an import-time
read the guard was silently missing. Comprehensions are now walked in full and
only the genexp keeps the outermost-iterable-only treatment.
io was also in the not-a-path-opener list, but io.open is the builtin, with the
same mode position and the same platform default. io.open(CHECKED_IN_FILE) is
exactly the hazard this guard exists for, so it is matched now, with binary
modes and a pinned encoding still exempt. tarfile.open and fitz.open stay
exempt since neither has an encoding to name.
Verified: 13 targeted cases covering all five eager comprehension forms and
io.open in text, binary and pinned shapes all classify correctly; still 22
offenders on unpatched main; green on this branch and on the tree merged with
latest main.
* Close three more walker gaps in the import-time guard
All three reproduced against the AST first.
A generator expression handed straight to a call is consumed there, so
DATA = "".join(p.read_text() for p in paths) runs its element at import. Only
an unconsumed genexp bound to a name stays lazy, so the walker now follows the
consumed ones in full and keeps the outermost-iterable-only treatment for the
rest.
if "__main__" == __name__ is an equivalent and accepted spelling of the main
guard, but requiring __name__ on the left meant its body was treated as
import-time code. That is a false positive on a block pytest never runs, so
both operand orders are recognised now.
The helper table was built from module-level defs only, so a def in a class
body invoked while the class is constructed was never followed, contradicting
the walker's stated coverage of class bodies. Helpers are now collected from
the module body and from class bodies at any nesting.
Verified: 15 targeted cases including all three fixes and the earlier ones
still classify correctly; still 22 offenders on unpatched main; green on this
branch and on the tree merged with latest main.
* Handle positional read_text encodings, lazy generators and nested helpers
* Guard reads reached from test bodies, unbound Path calls and __file__ paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Follow derived paths, skip lazy generator helpers, cover compressed openers
* Guard the CLI tests, helper parameters and unbound Path arguments
* Discover test roots and follow literal, in-place and tuple-derived paths
* Identify module openers by import, unwrap starred paths, pin subprocess snippets
* Resolve import origins, seed helper locals, follow named generators and parametrize
* Scope imports lexically, list tracked test files, bind unpacked names
* Resolve aliased openers, keyword-only params, destructured targets, next()
* Pin the encoding on subprocess snippets, workflow lint and CLI output for PR #7438
* Harden the CLI encoding guard against detached streams for PR #7438
* Tighten the encoding guard's path and scope analysis for PR #7438
* Resolve path provenance more precisely and keep POSIX stream encodings for PR #7438
* Resolve qualified path classes and scope conditional imports for PR #7438
* Scope CLI stream setup to the entry point and align two encoding pairs for PR #7438
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent.
* scripts/scan_*: add Mini Shai-Hulud May-12 IOC strings and pin-blocklists
Append the May-12 2026 wave indicators (git-tanstack.com, transformers.pyz,
/tmp/transformers.pyz, "With Love TeamPCP", "We've been online over 2 hours")
to all three scanner IOC tables, add BLOCKED_NPM_VERSIONS (42 TanStack pkgs,
4 opensearch versions, 3 squawk pkgs) in scan_npm_packages.py and
lockfile_supply_chain_audit.py (kept byte-identical), add BLOCKED_PYPI_VERSIONS
(guardrails-ai 0.10.1, mistralai 2.4.6, lightning 2.6.2/2.6.3) plus
RE_MAY12_IOC wiring across check_py_file/check_shell_file/check_workflow_file
in scan_packages.py. The npm orchestrator and the lockfile auditor now
short-circuit on a blocked entry before fetching the tarball, and the
PyPI download pipeline drops blocked specs before pip download is invoked.
* tests/security: regression suite for supply-chain scanners
Adds offline fixture corpus and pytest coverage for scan_npm_packages,
scan_packages, and lockfile_supply_chain_audit so future IOC-table
drift surfaces at PR time. Pytest scope narrowed to tests/security so
GPU smoke tests are not picked up by default.
* ci(security-audit): drop continue-on-error on pip-scan and npm-scan jobs
Promote three harden-runner blocks to egress-policy: block with per-job allowlists.
Add tests-security job running pytest tests/security as a hard gate.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* scripts: harden third-party downloads, pip resolver pins, atomic writes
Pins uv installer and mlx_vlm qwen3_5 patches by commit SHA + SHA-256
checksum, scrubs PIP_* env vars and forces --index-url + --only-binary
on pip download, applies tarbomb caps to scan_packages archive walks,
and converts non-atomic config writes (kwargs spacer, studio stamper,
notebook validator, scan_packages req-file fixer) to mkstemp+os.replace.
Also adds host allowlist to notebook_to_python downloader, threads an
--allow-shell flag through its shell=True emission with reviewer warning
comments, locks both MLX installer scripts to set -euo pipefail, and
extends CODEOWNERS so colab snapshot data files require notebook-owner
review.
* ci(workflows): harden release-desktop / smoke / notebooks workflows
Pin dtolnay/rust-toolchain to a 40-char SHA, scope release-desktop permissions to read at workflow level with job-level write only on the build job, append --ignore-scripts to every npm ci / npm install in studio-frontend-ci / wheel-smoke / studio-tauri-smoke / release-desktop, validate client_payload.ref shape via an env-var-isolated regex on every notebooks-ci job, and add step-security/harden-runner in audit mode as the first step of release-desktop and mlx-ci.
* scripts: promote silent scanner failures to non-zero exit codes
scan_packages now returns 2 on pip-download failure and emits a CRITICAL archive_corrupted finding on truncated wheels/sdists.
notebook_to_python exits 1 on per-notebook failures; notebook_validator wraps the stash/pop in try/finally; lockfile audit rejects bare UNSLOTH_LOCKFILE_AUDIT_SKIP=1 with a loud GitHub Actions warning.
* Add npm cooldown + new-install-script gate + Dependabot cooldown
Pins min-release-age=7 (npm 11.10+) in repo-root and studio/frontend
.npmrc, adds scripts/check_new_install_scripts.py to fail PRs that
add a postinstall dep, ships a new security-audit job for npm audit
signatures plus the diff, and extends .github/dependabot.yml with
cooldown stanzas. Pin @tanstack/react-router to 1.169.9 per GHSA-
g7cv-rxg3-hmpx; lockfile regen deferred until that release lands on
npm. tests/security gains 4 new tests; full suite 26/26 green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ci(security): fix tanstack pin, exec bits, expand IOC tables to @uipath/@squawk full
- Revert --ignore-scripts on Studio install workflows: vite build needs
esbuild's native postinstall (per PR #5392 rationale). Keep
--ignore-scripts on security-audit.yml's standalone npm audit job.
- Pin @tanstack/react-router to the actual published 1.169.2 (was a
forward-looking 1.169.9 that does not exist on npm; broke npm ci).
- Drop redundant repo-root .npmrc; studio/frontend/.npmrc covers the
only npm project today (root cooldown re-instate via dependabot.yml).
- Restore exec bits on 7 files my filesystem stripped during cherry-pick.
- Expand BLOCKED_NPM_VERSIONS with full safedep.io + Aikido enumeration:
22 @squawk/* packages with 5 versions each (110 entries; previously
3 entries with 1 version each), and 66 @uipath/* packages (entirely
missing before). Mirror in scripts/lockfile_supply_chain_audit.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests/security: suppress CodeQL py/incomplete-url-substring-sanitization
The two flagged 'X' in Y assertions are NOT URL sanitization checks.
They verify our scanner WROTE a known IOC literal into its stdout /
Finding.evidence, which is the opposite of an attack surface --
matching the scanner's output is precisely what catches the worm.
Inline lgtm[] suppression with a 4-line rationale comment above each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* scripts/scan_*: expand IOC tables with Aikido full 169-pkg enumeration
Per Aikido 2026-05-12 disclosure (373 malicious package-version entries
across 169 npm package names), add to BLOCKED_NPM_VERSIONS:
- @mistralai/* npm scope (3 packages, 9 versions) -- separate from
the PyPI mistralai package already in BLOCKED_PYPI_VERSIONS
- @tallyui/* (10 packages, 30 entries)
- @beproduct/nestjs-auth (18 versions 0.1.2..0.1.19)
- @draftlab/* + @draftauth/* (5 packages)
- @taskflow-corp/cli, @tolka/cli, @ml-toolkit-ts/*, @mesadev/*,
@dirigible-ai/sdk, @supersurkhet/*
- 10 unscoped packages (safe-action, ts-dna, cross-stitch,
cmux-agent-mcp, agentwork-cli, git-branch-selector, wot-api,
git-git-git, nextmove-mcp, ml-toolkit-ts)
Also add to KNOWN_IOC_STRINGS / NPM_IOC_STRINGS:
- router_init.js SHA-256 ab4fcadaec49c03278063dd269ea5eef82d24f2124a8e15d7b90f2fa8601266c
- tanstack_runner.js SHA-256 2ec78d556d696e208927cc503d48e4b5eb56b31abc2870c2ed2e98d6be27fc96
- bun run tanstack_runner.js marker (the new Bun-prepare-script
dropper invocation pattern unique to this wave)
Total: 170 packages, 401 versions blocklisted. Studio lockfile still
scans clean (0 findings, 0 hard errors).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* scripts/scan_*: web-verification additions (@tanstack/setup, intercom-client)
Two findings from cross-checking BLOCKED_NPM_VERSIONS / KNOWN_IOC_STRINGS
against GHSA-g7cv-rxg3-hmpx + Aikido + safedep.io + Socket + Semgrep.
- Fix asymmetry: @tanstack/setup IOC string was in
lockfile_supply_chain_audit.py's NPM_IOC_STRINGS but missing from
scan_npm_packages.py's KNOWN_IOC_STRINGS. The literal is the malicious
optional-dependency name used by the May-12 TanStack wave; no
legitimate npm package of this name exists.
- Add intercom-client@7.0.4: the npm counterpart of the lightning
2.6.2/2.6.3 PyPI compromise (Apr-30 wave). Same threat actor
(TeamPCP). Confirmed by Semgrep, Aikido, OX Security, Resecurity,
Kodem. Safe version is 7.0.3 and earlier.
Total BLOCKED_NPM_VERSIONS: 171 packages / 402 versions. Both files
remain byte-identical. Studio lockfile still scans clean.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ci(security): add workflow-trigger lint refusing pull_request_target + cache-poisoning vectors
The two patterns that together powered GHSA-g7cv-rxg3-hmpx (TanStack
Mini Shai-Hulud) are now gated at PR time:
1. pull_request_target -- the worm chain started with a fork PR that
ran in the base-repo context. Every workflow in this repo today
uses 'pull_request' (safe); the lint refuses any new
pull_request_target additions outright. workflow_run is
restricted, allowed only with an explicit allow-comment.
2. Shared cache keys between PR-triggered workflows and the publish
workflow (release-desktop.yml). The TanStack attack chain poisoned
a shared Actions cache from a fork PR; the legitimate release
workflow then restored the poisoned cache. The lint refuses any
cache key that appears in both a PR-triggered workflow and a
workflow_dispatch-only / publish workflow.
Current tree is clean: 0 pull_request_target, 0 workflow_run, 0
PR-publish cache-key collisions across all 24 workflows. The lint
locks that invariant in place.
Files:
+ scripts/lint_workflow_triggers.py (~200 LOC, stdlib + PyYAML)
+ tests/security/test_lint_workflow_triggers.py (5 tests covering
current-tree pass, pull_request_target reject, workflow_run
restricted, justified workflow_run accept, cache-key collision
reject)
~ .github/workflows/security-audit.yml: new workflow-trigger-lint
job, no continue-on-error, harden-runner block-mode, PyYAML only
runtime dep.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* security: fix tests-security CI job + CodeQL false-positives
Two CI failures on the prior push:
1. pytest tests/security -- 5 lint regression tests failed because
scripts/lint_workflow_triggers.py imports PyYAML which is not in
the bare runner's Python env. Added pyyaml==6.0.2 to the pip
install step alongside pytest. (29 scanner tests already passed.)
2. CodeQL py/incomplete-url-substring-sanitization fired on two
test assertions that check the scanner WROTE the IOC literal
to its own stdout/stderr. The rule pattern-matches on
`"<host>" in <var>` and cannot distinguish a URL sanitizer from
a regression-test evidence check. Previous `# lgtm[...]` inline
suppressions were detached from the operator when pre-commit
reformatted the assert across multiple lines. Rebuilt the IOC
literals at runtime (`"git-tanstack." + "com"`) so no URL-shaped
source literal appears on the `in` operator line; rule cannot
trigger.
Verified locally: `pytest tests/security -v` -> 34 passed in 2.70s.
* security(studio): defensive .npmrc cooldown aliases + save-exact
Two additions to studio/frontend/.npmrc to harden the existing
`min-release-age=7` (Mini Shai-Hulud defence):
1. `minimum-release-age=10080` (minutes) -- defensive alias for the
same 7-day floor. Some npm versions / wrappers consult one key but
not the other; setting both prevents a single upstream setting-name
parse change from silently disabling the cooldown. The two keys
MUST agree (do not let them drift).
2. `save-exact=true` -- refuses to write back `^x.y.z` ranges into
package.json when a maintainer runs `npm install <pkg>` locally.
Does NOT rewrite already-present ranges; stops NEW carets from
creeping into the manifest as patch-version footguns.
Verified: pytest tests/security -> 34 passed in 2.63s.
* chore(dependabot): remove dead bun entry for /studio/frontend
`package-ecosystem: "bun"` at /studio/frontend was a no-op: that
path commits package-lock.json, not bun.lock / bun.lockb, so
Dependabot's bun ecosystem silently skipped it. The actual
behaviour is unchanged -- the npm entry below the cargo block
already owns npm_and_yarn security advisories for /studio/frontend
with `open-pull-requests-limit: 0` (version-update PRs suppressed,
security PRs flow through).
This commit:
- Deletes the bun entry (kept a placeholder comment so a future
bun migration knows where to slot it back in).
- Rewrites the npm /studio/frontend entry comment to explain the
real intent: lockfile is the authoritative pin, .npmrc
`min-release-age=7` already blocks fresh tarballs at install
time, dependabot only needs to surface security advisories.
No functional change: same set of dependabot PRs as before (zero
version updates, security advisories grouped weekly with cooldown).
Verified: pytest tests/security -> 34 passed in 2.67s; YAML
parses cleanly via PyYAML.
* fix(dependabot): drop unsupported semver-* cooldown keys on github-actions
Dependabot's validator rejected the config with:
The property '#/updates/0/cooldown/semver-minor-days' is not
supported for the package ecosystem 'github-actions'.
The property '#/updates/0/cooldown/semver-patch-days' is not
supported for the package ecosystem 'github-actions'.
The `semver-minor-days` / `semver-patch-days` cooldown knobs are
only valid for semver-aware ecosystems (npm, cargo, etc.). The
github-actions ecosystem pins via git tags / SHAs, not semver, so
only `default-days` is honored. Pre-existing bug on main; surfaced
on this PR because the prior commit re-validated the file.
Behaviour: github-actions PRs now respect the 7-day cooldown floor
(was already the intent), without the no-op semver bands.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Leo Borcherding