ci(notebooks): diff Colab oracle against committed snapshots

Extend notebook_validator.py with a colab-diff subcommand that
fetches three files from googlecolab/backend-info:

  pip-freeze.gpu.txt   -> snapshot at scripts/data/colab_pip_freeze.gpu.txt
  apt-list-gpu.txt     -> snapshot at scripts/data/colab_apt_list.gpu.txt
  os-info-gpu.txt      -> snapshot at scripts/data/colab_os_info.gpu.txt

Each file is parsed with a format-specific parser (pip ==, apt
listing, free-form os-info) and compared against the committed
snapshot. The diff reports NEW / REMOVED / CHANGED keys per file.

Wired into Notebooks CI two ways:
- PR-time static job: advisory step (continue-on-error: true) so
  upstream Colab rotations surface in the PR check UI without
  blocking authors.
- Daily static-with-pypi cron: --strict step so backend-info drift
  fails the cron within ~24h and the maintainer can refresh the
  snapshots intentionally.

Catches the same bug classes the existing R-INST-002/003/004/005
rules catch, but earlier: when Colab bumps libcudnn / Python /
torch wheels, we hear about it before a notebook breaks.

Add baseline snapshots from current backend-info HEAD: 1136 apt
packages, 4 os-info entries, 720 pip-freeze entries.
This commit is contained in:
Daniel Han 2026-05-08 11:39:38 +00:00
commit 420f588205
4 changed files with 1318 additions and 0 deletions

View file

@ -101,6 +101,18 @@ jobs:
--out unsloth/scripts/data/colab_pip_freeze.gpu.txt \
|| echo "::warning::refresh-colab failed; using committed snapshot"
- name: Diff Colab oracle vs committed snapshots (advisory)
# Pulls pip-freeze.gpu.txt + apt-list-gpu.txt + os-info-gpu.txt
# from googlecolab/backend-info and prints NEW / REMOVED /
# CHANGED entries against scripts/data/colab_*.txt. Non-blocking
# on PRs; the daily cron job below runs the same step with
# --strict so upstream rotations surface within ~24h.
continue-on-error: true
working-directory: ${{ github.workspace }}
run: |
python unsloth/scripts/notebook_validator.py colab-diff \
--snapshot-dir unsloth/scripts/data
- name: Drift check (re-run update_all_notebooks.py + git diff)
working-directory: ${{ github.workspace }}
# Reported as non-blocking until the upstream `unslothai/notebooks`
@ -169,6 +181,14 @@ jobs:
run: |
python unsloth/scripts/notebook_validator.py refresh-colab \
--out unsloth/scripts/data/colab_pip_freeze.gpu.txt
- name: Diff Colab oracle vs committed snapshots (--strict on cron)
# Cron-only escalation of the advisory PR-time check. Fails if
# any of pip-freeze.gpu.txt / apt-list-gpu.txt / os-info-gpu.txt
# has drifted from scripts/data/colab_*.txt; refresh the
# snapshots in this repo to acknowledge.
run: |
python unsloth/scripts/notebook_validator.py colab-diff \
--snapshot-dir unsloth/scripts/data --strict
- name: Lint with live PyPI metadata
run: |
python unsloth/scripts/notebook_validator.py lint \