fix: add Qwen3.5 version gate in loader dispatch (#4335)

* fix: add Qwen3.5 version gate in loader dispatch (#4188)

Qwen3.5 (model_type qwen3_5) only exists in transformers >= 5.0.0.
Without this gate, loading a Qwen3.5 model on transformers 4.x gives
an unhelpful generic error. This adds a clear version check before the
qwen3 dispatch to prevent substring misrouting and give a useful error
message pointing users to upgrade.

No dedicated FastQwen3_5Model is needed -- the compiler already applies
fused CE automatically via apply_fused_lm_head for both
Qwen3_5ForCausalLM and Qwen3_5ForConditionalGeneration. The generic
FastModel fallback path handles everything.

FORCE_FLOAT32 already has qwen3_5 on main.

Tested on transformers 5.3.0: Qwen3.5-0.8B 4bit, 1.38 GB peak memory.
Backwards compatible: import unsloth works on transformers 4.57.6.

* fix: update FORCE_FLOAT32 comment for qwen3_5

The (1+w) RMSNorm pattern does not overflow float16 since Qwen3_5RMSNorm
computes in float32 internally. The actual reason FORCE_FLOAT32 is needed
is that Qwen3.5 GDN layers produce NaN grad norms during float16 training.
Updated the comment to reflect the real reason.

* fix: move qwen3_5 version check before dispatch chain

The elif block intercepted qwen3_5 on transformers >= 5.0.0 without
setting dispatch_model, causing UnboundLocalError at line 715.

Move the version check before the if/elif dispatch chain so on
transformers >= 5.0.0 the model_type falls through to the generic
FastModel path as intended.

* fix: qwen3_5 requires transformers >= 5.2.0, not 5.0.0

Checked all 5.x releases:
- 5.0.0: no qwen3_5 module
- 5.1.0: no qwen3_5 module
- 5.2.0: qwen3_5 available

* fix: move qwen3_5 version check into AutoConfig error handler

The previous version check at the dispatch chain was unreachable --
AutoConfig.from_pretrained fails first with a generic "does not
recognize this architecture" error on transformers < 5.2.0, so
execution never reached the check.

Move the qwen3_5-specific error message into the AutoConfig exception
handler where "architecture" errors are caught. This intercepts the
error before the generic message and gives users a clear upgrade path.

Also remove the now-redundant check before the dispatch chain.
Both FastLanguageModel and FastModel paths are covered.

Tested: transformers 4.57.6 shows the Qwen3.5-specific error,
transformers 5.3.0 loads and trains normally.

---------

Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
This commit is contained in:
Daniel Han 2026-03-16 20:37:42 -07:00 committed by GitHub
commit 6912a15a42
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -106,7 +106,7 @@ FORCE_FLOAT32 = [
"gemma3text", # Gemma3TextModel (EmbeddingGemma, standalone text-only Gemma3)
"gemma3n",
"gpt_oss",
"qwen3_5", # Qwen3.5 RMSNorm uses (1+w) pattern like Gemma3, overflows float16
"qwen3_5", # Qwen3.5 GDN layers produce NaN grad norms in float16 training
]
global DISABLE_COMPILE_MODEL_NAMES
@ -442,6 +442,13 @@ class FastLanguageModel(FastLlamaModel):
except Exception as error:
autoconfig_error = str(error)
if "architecture" in autoconfig_error:
if "qwen3_5" in autoconfig_error:
raise ImportError(
f"Unsloth: Your transformers version of {transformers_version} does not support Qwen3.5.\n"
f"The minimum required version is 5.2.0.\n"
f'Try `pip install --upgrade "transformers>=5.2.0"`\n'
f"to obtain the latest transformers build, then restart this session."
)
raise ValueError(
f"`{model_name}` is not supported yet in `transformers=={transformers_version}`.\n"
f"Please update transformers via `pip install --upgrade transformers` and try again."
@ -1051,6 +1058,13 @@ class FastModel(FastBaseModel):
except Exception as error:
autoconfig_error = str(error)
if "architecture" in autoconfig_error:
if "qwen3_5" in autoconfig_error:
raise ImportError(
f"Unsloth: Your transformers version of {transformers_version} does not support Qwen3.5.\n"
f"The minimum required version is 5.2.0.\n"
f'Try `pip install --upgrade "transformers>=5.2.0"`\n'
f"to obtain the latest transformers build, then restart this session."
)
raise ValueError(
f"`{model_name}` is not supported yet in `transformers=={transformers_version}`.\n"
f"Please update transformers via `pip install --upgrade transformers` and try again."