unsloth/unsloth
Daniel Han 4929c5f769
Keep pad-named pad_tokens (e.g. <|vision_pad|>); fix Qwen3-Base load crash (#6652)
* Keep pad-named pad_tokens; defer pad repair to shared unsloth_zoo.pad_token

A pad-named token (e.g. <|vision_pad|>) is a valid pad. The narrow fallback that
stripped vision pad tokens on text-only models is now a no-op; the active path
delegates to the shared fix_pad_token in unsloth_zoo, which keeps pad-named tokens
and only heals missing / eos-collision / out-of-range pads.

This fixes the Qwen3-4B-Base load crash (its config ships pad_token=<|vision_pad|>):
the old swap could not find a safe text pad (eos is <|endoftext|>, no unk_token) and
left the tokenizer broken. Removes the unused _VISION_PAD_TOKENS / _SAFE_TEXT_PAD_TOKENS
sets. Tests updated.

Pairs with unslothai/unsloth-zoo#831.

* Remove _fix_vision_pad_token; inline the no-op fallback

A pad-named token (e.g. <|vision_pad|>) is a valid pad, so the old vision-pad swap
helper has no purpose. _fix_pad_token now returns the tokenizer unchanged when the
shared unsloth_zoo.pad_token module is unavailable, instead of routing through a
no-op helper. Test WANTED set updated.
2026-06-25 04:41:09 -07:00
..
dataprep Fix SyntheticDataKit.chunk_data dropping single-chunk documents (#6595) 2026-06-23 16:39:38 -03:00
kernels Windows installer: fix DiskPart UAC mid-install, drive-root cache, and spurious unsloth.exe rename warning (#6296) 2026-06-22 03:09:08 -07:00
models Fix CPOTrainer crash with multimodal processors (Gemma 4) (#6522) 2026-06-23 06:29:22 -07:00
optimizers Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
registry Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
utils Fix FlashAttention fp32 crash with DoRA (use_dora=True) (#6526) 2026-06-23 01:29:19 -07:00
__init__.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
_auto_install.py Upgrade setuptools and wheel in the auto-install command (#6282) 2026-06-13 03:59:15 -07:00
_gpu_init.py Shim removed vllm.transformers_utils.tokenizer so fast_inference works on vLLM >= 0.22 (#6390) 2026-06-17 17:25:41 -07:00
chat_templates.py Fix construct_chat_template leaking {INPUT}/{OUTPUT} sentinel into the chat template (#6531) 2026-06-24 00:40:53 -07:00
device_type.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
import_fixes.py Suppress only torchao's cosmetic cpp-extensions warning (#6506) 2026-06-22 02:28:30 -07:00
ollama_template_mappers.py Re-apply #4939: updated models template mappers (#4950) 2026-04-15 07:52:12 -07:00
save.py Fix save crash for legacy list-form _tied_weights_keys (NemotronH) (#6540) 2026-06-22 02:11:46 -07:00
tokenizer_utils.py Keep pad-named pad_tokens (e.g. <|vision_pad|>); fix Qwen3-Base load crash (#6652) 2026-06-25 04:41:09 -07:00
trainer.py Reset torch.compile cache poisoned by a stray forward before trainer.train() (#6511) 2026-06-22 05:39:48 -07:00