fix: add Qwen3.5 version gate in loader dispatch (#4335)
* fix: add Qwen3.5 version gate in loader dispatch (#4188) Qwen3.5 (model_type qwen3_5) only exists in transformers >= 5.0.0. Without this gate, loading a Qwen3.5 model on transformers 4.x gives an unhelpful generic error. This adds a clear version check before the qwen3 dispatch to prevent substring misrouting and give a useful error message pointing users to upgrade. No dedicated FastQwen3_5Model is needed -- the compiler already applies fused CE automatically via apply_fused_lm_head for both Qwen3_5ForCausalLM and Qwen3_5ForConditionalGeneration. The generic FastModel fallback path handles everything. FORCE_FLOAT32 already has qwen3_5 on main. Tested on transformers 5.3.0: Qwen3.5-0.8B 4bit, 1.38 GB peak memory. Backwards compatible: import unsloth works on transformers 4.57.6. * fix: update FORCE_FLOAT32 comment for qwen3_5 The (1+w) RMSNorm pattern does not overflow float16 since Qwen3_5RMSNorm computes in float32 internally. The actual reason FORCE_FLOAT32 is needed is that Qwen3.5 GDN layers produce NaN grad norms during float16 training. Updated the comment to reflect the real reason. * fix: move qwen3_5 version check before dispatch chain The elif block intercepted qwen3_5 on transformers >= 5.0.0 without setting dispatch_model, causing UnboundLocalError at line 715. Move the version check before the if/elif dispatch chain so on transformers >= 5.0.0 the model_type falls through to the generic FastModel path as intended. * fix: qwen3_5 requires transformers >= 5.2.0, not 5.0.0 Checked all 5.x releases: - 5.0.0: no qwen3_5 module - 5.1.0: no qwen3_5 module - 5.2.0: qwen3_5 available * fix: move qwen3_5 version check into AutoConfig error handler The previous version check at the dispatch chain was unreachable -- AutoConfig.from_pretrained fails first with a generic "does not recognize this architecture" error on transformers < 5.2.0, so execution never reached the check. Move the qwen3_5-specific error message into the AutoConfig exception handler where "architecture" errors are caught. This intercepts the error before the generic message and gives users a clear upgrade path. Also remove the now-redundant check before the dispatch chain. Both FastLanguageModel and FastModel paths are covered. Tested: transformers 4.57.6 shows the Qwen3.5-specific error, transformers 5.3.0 loads and trains normally. --------- Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
This commit is contained in:
parent
262271a20d
commit
6912a15a42
1 changed files with 15 additions and 1 deletions
|
|
@ -106,7 +106,7 @@ FORCE_FLOAT32 = [
|
|||
"gemma3text", # Gemma3TextModel (EmbeddingGemma, standalone text-only Gemma3)
|
||||
"gemma3n",
|
||||
"gpt_oss",
|
||||
"qwen3_5", # Qwen3.5 RMSNorm uses (1+w) pattern like Gemma3, overflows float16
|
||||
"qwen3_5", # Qwen3.5 GDN layers produce NaN grad norms in float16 training
|
||||
]
|
||||
|
||||
global DISABLE_COMPILE_MODEL_NAMES
|
||||
|
|
@ -442,6 +442,13 @@ class FastLanguageModel(FastLlamaModel):
|
|||
except Exception as error:
|
||||
autoconfig_error = str(error)
|
||||
if "architecture" in autoconfig_error:
|
||||
if "qwen3_5" in autoconfig_error:
|
||||
raise ImportError(
|
||||
f"Unsloth: Your transformers version of {transformers_version} does not support Qwen3.5.\n"
|
||||
f"The minimum required version is 5.2.0.\n"
|
||||
f'Try `pip install --upgrade "transformers>=5.2.0"`\n'
|
||||
f"to obtain the latest transformers build, then restart this session."
|
||||
)
|
||||
raise ValueError(
|
||||
f"`{model_name}` is not supported yet in `transformers=={transformers_version}`.\n"
|
||||
f"Please update transformers via `pip install --upgrade transformers` and try again."
|
||||
|
|
@ -1051,6 +1058,13 @@ class FastModel(FastBaseModel):
|
|||
except Exception as error:
|
||||
autoconfig_error = str(error)
|
||||
if "architecture" in autoconfig_error:
|
||||
if "qwen3_5" in autoconfig_error:
|
||||
raise ImportError(
|
||||
f"Unsloth: Your transformers version of {transformers_version} does not support Qwen3.5.\n"
|
||||
f"The minimum required version is 5.2.0.\n"
|
||||
f'Try `pip install --upgrade "transformers>=5.2.0"`\n'
|
||||
f"to obtain the latest transformers build, then restart this session."
|
||||
)
|
||||
raise ValueError(
|
||||
f"`{model_name}` is not supported yet in `transformers=={transformers_version}`.\n"
|
||||
f"Please update transformers via `pip install --upgrade transformers` and try again."
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue