6974e4d37b
Qwen3.5 has a 248,320-token vocabulary. At 8K context the full logits tensor is 8192 x 248320 x 4 = 7.68 GB, which causes OOM on T4/P100. The unsloth compiler already applies fused CE via apply_fused_lm_head, but this adds an explicit FastQwen3_5Model dispatch for cleaner routing and better error messages when Qwen3.5 is not supported. Changes: - Add unsloth/models/qwen3_5.py with FastQwen3_5Model that patches Qwen3_5ForConditionalGeneration and Qwen3_5ForCausalLM forwards to use unsloth_fused_ce_loss directly from hidden_states - Add loader dispatch for model_type == "qwen3_5" before "qwen3" - Version gate uses >= 5.0.0 (qwen3_5 only exists in transformers 5.x) - Guarded import in loader.py with try/except fallback - GDN layers intentionally left unpatched (flash-linear-attention) - 23 unit tests covering all 4 code paths Fixes from original PR #4331 by @vitalis: - Add explicit _get_dtype import (wildcard import skips _-prefixed names) - Single-token fast path now checks labels is None before returning early - Default model name corrected to Qwen/Qwen3.5-9B (8B does not exist) - Test assertion on nn.Linear removed (not a mock) - Unused imports removed Tested: Qwen3.5-0.8B 4bit training, 1.38 GB peak memory, 23/23 tests pass. Backwards compatible: import unsloth works on transformers 4.57.6. |
||
|---|---|---|
| .. | ||
| qlora | ||
| saving | ||
| utils | ||
| __init__.py | ||
| test_get_model_name.py | ||
| test_model_registry.py | ||
| test_raw_text.py | ||