Tests + CI guard: batched left-padded generation can never silently regress again (#1066, #3699) (#6145)
* Add regression guard for batched left-padded generation (#1066, #3699) Three layers of tests plus a path-filtered CI workflow so the left-padding position_ids / attention-mask bug class cannot silently return: - tests/utils/test_prepare_inputs_ast_guard.py: import-free AST checks on _fast_prepare_inputs_for_generation (cumsum-from-mask branch present, cache_position only as fallback, no mask truncation, model families wired) - tests/utils/test_prepare_inputs_leftpad.py: CPU behavioral unit test with synthetic left-padded masks and fake caches; exact expected position_ids for prefill and cached decode - tests/utils/test_batched_leftpad_generation_gpu.py: optional GPU e2e, solo vs batched prefix match, skipped without CUDA - .github/workflows/batch-inference-guard.yml: ubuntu-latest CPU job running the two deterministic layers on PRs touching unsloth/models/** Validated: all pass on main; both CPU layers fail at 6d0f8643~1 (pre #4100) and at 332eabf3~1 (pre #2216), reproducing the historical bug signatures. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cite staging proof in batch-inference-guard header (staging-2 PRs 170/171) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fold left-padding guard into consolidated Core CI; merge AST + behavioral tests No new workflow and no new CI job: the guard now runs as one HARD GATE step inside consolidated-tests-ci.yml, right after the callback signature drift detector, where the CPU torch stack is already installed. The AST structural checks and the behavioral unit tests live in a single file (tests/utils/test_prepare_inputs_leftpad.py); the AST layer stays stdlib-only with unsloth imported lazily inside the behavioral tests, so import breakage cannot mask the structural checks. Revalidated after the merge: 11 assertions pass on main, 8 fail at 6d0f8643~1 (pre #4100). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update staging proof reference for consolidated gate (PRs 170/172) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
parent
bc8b9dc917
commit
184141db99
3 changed files with 530 additions and 0 deletions
14
.github/workflows/consolidated-tests-ci.yml
vendored
14
.github/workflows/consolidated-tests-ci.yml
vendored
|
|
@ -333,6 +333,20 @@ jobs:
|
|||
run: |
|
||||
python -m pytest -v --tb=short tests/test_callback_signature_drift.py
|
||||
|
||||
- name: batched left-padding generation guard (HARD GATE)
|
||||
# Guards _fast_prepare_inputs_for_generation against the bug class of
|
||||
# issues #1066 / #3699: position_ids taken from cache_position (which
|
||||
# counts left-pad tokens) or the 2D attention mask truncated to its
|
||||
# last column. Both shipped in cc4c5d77 and were fixed by #2216 and
|
||||
# #4100; nothing tested this path, so each regression reached users.
|
||||
# Layer 1 in the file is stdlib-ast-only (survives unsloth import
|
||||
# breakage), layer 2 calls the real function on CPU via the
|
||||
# tests/conftest.py CUDA spoof. Validated to fail on the pre-#2216
|
||||
# and pre-#4100 code states; staging proof on GPU-less runners:
|
||||
# danielhanchen/unsloth-staging-2 PR 170 (green, gate passed in all combos) / PR 172 (red, gate failed in all combos).
|
||||
run: |
|
||||
python -m pytest -v --tb=short tests/utils/test_prepare_inputs_leftpad.py
|
||||
|
||||
- name: unsloth Bucket-A — CPU tests not in Repo tests (CPU)
|
||||
# CPU tests across 6 files under tests/saving/, tests/utils/, tests/python/
|
||||
# that Repo tests (CPU) --ignores. AST/protobuf/regex plus tiny CPU model
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue