unsloth/tests/utils
alkinun 9e334d552c
Fix text-only VLM CPT packing truncation (#7211)
* Fix text-only VLM CPT packing truncation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Handle streaming vision datasets in packing

* Harden multimodal packing detection

* Preserve safe packing boundaries

* Scope stream packing checks to VLMs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Narrow VLM packing detection

* Align packing mode and eval safety

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add qwen3_5/qwen3_next to PADDING_FREE_BLOCKLIST to avoid packed-sequence contamination

* Detect hybrid linear-attention models structurally instead of by name for packing guard

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Install wrapped-packing setup at the signature, not the Zoo license comment

The _unsloth_wrapped_packing / _inspect setup block was injected by matching the
exact 'All Unsloth Zoo code licensed under LGPLv3' comment line in the sourced
sft_prepare_dataset. The unsloth_zoo dependency is only lower-bounded, so a newer
Zoo that moves or drops that header made the setup a silent no-op while the
truncation and pack_dataset rewrites still emitted references to those names,
raising NameError on every SFT dataset preparation.

Anchor the setup on the function signature instead (a structural location that
always exists) and fail loudly if it cannot be found, so the helper variables are
always defined before they are referenced across Zoo versions.

Adds a regression test that patches in a Zoo source without the license header.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherl <61019402+Etherll@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-20 00:23:37 -07:00
..
__init__.py Revert "[FIX] Vllm guided decoding params (#3662)" 2025-12-01 05:43:45 -08:00
aime_eval.md reroute merge logic language models + comprehensive tests + eval kits (#2673) 2025-06-02 20:32:57 -07:00
aime_eval.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
cleanup_utils.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
data_utils.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
generate_dataset_with_none.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
hf_utils.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
ocr_eval.md Fix Typos in Documentation and Comments (#2721) 2025-06-17 04:34:51 -07:00
ocr_eval.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
os_utils.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
perplexity_eval.md reroute merge logic language models + comprehensive tests + eval kits (#2673) 2025-06-02 20:32:57 -07:00
perplexity_eval.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
run_none_detect_tests.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
test_attention_dispatch_dora_dtype.py Probe xformers support on sm_120 instead of disabling it by version (#6828) 2026-07-14 00:01:25 -03:00
test_attention_masks.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
test_batched_leftpad_generation_gpu.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
test_packing.py Fix text-only VLM CPT packing truncation (#7211) 2026-07-20 00:23:37 -07:00
test_prepare_inputs_leftpad.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
test_q_galore.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
test_qat.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
test_rope_scaling_drift.py Keep native RoPE scaling when extending context; carry rope_theta for linear (#7028) 2026-07-09 04:20:41 -07:00
test_trunc_normal_patch.py Reduce and tighten comments and docstrings across the test suite (#6429) 2026-06-18 01:07:09 -07:00
test_xformers_capability_gate.py Probe xformers support on sm_120 instead of disabling it by version (#6828) 2026-07-14 00:01:25 -03:00