CI(lint): turn the studio-backend ruff stub into a real Python gate
Rename the job to "Python lint (syntax + ruff + safety nets)" and
expand it from one non-blocking ruff invocation over studio/backend
into four real gates over the whole tree. Total CI time goes from
~8 s to ~12 s, but the previous job was informational; this one
blocks merges on actual breakage.
Steps (in order):
1. AST/syntax (HARD GATE)
`python -m compileall -q -j 0 unsloth unsloth_cli studio tests
cli.py unsloth-cli.py`. Same parser the interpreter uses;
anything broken here would also crash at `import X` on a user's
machine. ~3.5 s across 350+ files locally.
2. ruff check whole repo (HARD GATE)
The narrow rule set in pyproject.toml [tool.ruff.lint] (E9 /
F63 / F7 / F82) catches undefined names, broken comparisons,
and syntax. The whole repo passes today, so the previous
studio/backend-only `|| true` was masking real breakage on
the wider tree. <1 s.
3. Debugger-leftover scan (HARD GATE)
AST-walk over every committed .py looking for `breakpoint()`,
`pdb.set_trace()`, or `ipdb.set_trace()` call sites. AST-based
so commented-out debugger lines don't false-positive (which
is why a bare grep would not work -- there are three commented
`# breakpoint()` markers in unsloth/models/rl* today). 0 hits
locally across 350 files.
4. SPDX-License-Identifier on studio/backend (WARNING)
Surfaces drift in the one tree where we already have a strict
SPDX policy. Currently 3 files missing; warned, not blocked,
so the rollout can be a separate PR.
5. ruff format drift (INFO)
Counts files that would be reformatted by plain `ruff format`.
Non-blocking because the canonical formatter is
scripts/run_ruff_format.py = ruff format + the kwarg-spacing
pass, so plain `ruff format --check` always reports a large
diff. Once that custom pipeline is wired in, drop
continue-on-error and add it to the gate.
ruff is pinned to 0.15.12 to match .pre-commit-config.yaml so a
CI-only ruff bump cannot start disagreeing with what pre-commit
already accepted.
This commit is contained in:
parent
cee324725c
commit
42ec2590a2
1 changed files with 99 additions and 6 deletions
105
.github/workflows/studio-backend-ci.yml
vendored
105
.github/workflows/studio-backend-ci.yml
vendored
|
|
@ -12,7 +12,11 @@
|
|||
# - -k 'not llama_cpp_load_progress_live': spawns a real llama.cpp process,
|
||||
# not appropriate for CPU-only runners.
|
||||
#
|
||||
# ruff is non-blocking initially; remove `|| true` once the backend lints clean.
|
||||
# Three jobs:
|
||||
# - pytest matrix (3.10/3.11/3.12/3.13) over studio/backend/tests
|
||||
# - repo-cpu-tests: auto-discovered tests/ + state-isolated spoof files
|
||||
# - python-lint: whole-repo Python gate (compileall + ruff +
|
||||
# debugger-leftover scan)
|
||||
|
||||
name: Backend CI
|
||||
|
||||
|
|
@ -203,8 +207,12 @@ jobs:
|
|||
echo "::endgroup::"
|
||||
done
|
||||
|
||||
ruff:
|
||||
name: Backend ruff lint (non-blocking)
|
||||
python-lint:
|
||||
# Whole-repo Python gate. Fast (~10-12 s) so it fits the same
|
||||
# short-feedback budget as the old studio/backend-only ruff job,
|
||||
# but actually blocks merges on real breakage rather than just
|
||||
# printing the lint count.
|
||||
name: Python lint (syntax + ruff + safety nets)
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
|
|
@ -213,6 +221,91 @@ jobs:
|
|||
with:
|
||||
python-version: '3.12'
|
||||
cache: 'pip'
|
||||
- run: pip install ruff
|
||||
- name: ruff check (non-blocking until accumulated drift is cleared)
|
||||
run: ruff check studio/backend || true
|
||||
# Pin to match .pre-commit-config.yaml so a CI-only ruff bump
|
||||
# cannot disagree with what pre-commit accepted.
|
||||
- run: pip install 'ruff==0.15.12'
|
||||
|
||||
- name: AST/syntax check (every committed .py must compile)
|
||||
# python -m compileall uses the same parser the interpreter
|
||||
# uses, so anything broken here would also crash at
|
||||
# `import X` on a user's machine. Sub-second across 350+
|
||||
# files. Hard gate.
|
||||
run: |
|
||||
python -m compileall -q -j 0 \
|
||||
unsloth unsloth_cli studio tests cli.py unsloth-cli.py
|
||||
|
||||
- name: ruff check (whole repo)
|
||||
# The narrow rule set in pyproject.toml [tool.ruff.lint]
|
||||
# selects E9 / F63 / F7 / F82 -- syntax errors, broken
|
||||
# comparisons, undefined names. The whole repo passes today,
|
||||
# so this is now a hard gate; the prior studio/backend-only
|
||||
# `|| true` was masking real breakage on the wider tree.
|
||||
run: |
|
||||
ruff check unsloth unsloth_cli studio tests cli.py unsloth-cli.py
|
||||
|
||||
- name: No leftover debugger / pdb / breakpoint calls
|
||||
# Catches the "I'll just stick a breakpoint() here" mistake
|
||||
# before it ships. AST-based so commented-out debugger
|
||||
# markers don't false-positive (grep would). Sub-second.
|
||||
run: |
|
||||
python <<'PY'
|
||||
import ast, pathlib, sys
|
||||
|
||||
SKIP_PARTS = {".venv", "venv", "build", "dist", ".git",
|
||||
"unsloth_compiled_cache", "node_modules"}
|
||||
|
||||
bad = []
|
||||
scanned = 0
|
||||
for path in sorted(pathlib.Path(".").rglob("*.py")):
|
||||
if any(part in SKIP_PARTS for part in path.parts):
|
||||
continue
|
||||
scanned += 1
|
||||
try:
|
||||
tree = ast.parse(path.read_text(encoding="utf-8", errors="replace"))
|
||||
except SyntaxError:
|
||||
continue # compileall step above already fails this
|
||||
for node in ast.walk(tree):
|
||||
if not isinstance(node, ast.Call):
|
||||
continue
|
||||
fn = node.func
|
||||
if isinstance(fn, ast.Name) and fn.id == "breakpoint":
|
||||
bad.append((path, node.lineno, "breakpoint()"))
|
||||
elif (isinstance(fn, ast.Attribute) and fn.attr == "set_trace"
|
||||
and isinstance(fn.value, ast.Name)
|
||||
and fn.value.id in {"pdb", "ipdb"}):
|
||||
bad.append((path, node.lineno, f"{fn.value.id}.set_trace()"))
|
||||
|
||||
if bad:
|
||||
for path, lineno, what in bad:
|
||||
print(f"::error file={path},line={lineno}::leftover {what} -- remove before merging")
|
||||
sys.exit(1)
|
||||
print(f"no leftover debugger calls (scanned {scanned} files)")
|
||||
PY
|
||||
|
||||
- name: SPDX-License-Identifier on every studio/backend .py (warning)
|
||||
# studio/backend is the only tree where we have a strict
|
||||
# SPDX policy right now (every committed .py opens with the
|
||||
# AGPL-3.0-only line). Surface drift without blocking; the
|
||||
# whole-repo rollout is a separate cleanup.
|
||||
continue-on-error: true
|
||||
run: |
|
||||
missing=$(git ls-files 'studio/backend/*.py' \
|
||||
| xargs grep -L "SPDX-License-Identifier" 2>/dev/null || true)
|
||||
if [ -n "$missing" ]; then
|
||||
count=$(echo "$missing" | wc -l)
|
||||
echo "::warning::$count studio/backend Python files are missing SPDX-License-Identifier"
|
||||
echo "$missing" | head -20
|
||||
else
|
||||
echo "all studio/backend .py files have SPDX-License-Identifier"
|
||||
fi
|
||||
|
||||
- name: ruff format drift (informational; whole-repo count)
|
||||
# The repo's canonical formatter is scripts/run_ruff_format.py
|
||||
# = `ruff format` + scripts/enforce_kwargs_spacing.py. Plain
|
||||
# `ruff format --check` reports the kwarg-spacing diff as
|
||||
# drift, which is expected. Surface the count so we can
|
||||
# track it; keep non-blocking until the custom pipeline is
|
||||
# wired in here.
|
||||
continue-on-error: true
|
||||
run: |
|
||||
ruff format --check unsloth unsloth_cli studio tests cli.py unsloth-cli.py
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue