Compare commits

...
Sign in to create a new pull request.

1 commit

Author SHA1 Message Date
Daniel Han
027b7f8551 Route Gemma-4 float16 through UNSLOTH_FORCE_FLOAT32
Add "gemma4" to FORCE_FLOAT32 so FastVisionModel.from_pretrained with
dtype=torch.float16 flips UNSLOTH_FORCE_FLOAT32=1 and swaps the load
dtype to bf16, matching how Gemma-3 and Gemma-3n are handled. Without
this entry, fp16 Gemma-4 GRPO crashes at step 2 with a CUDA device-side
assert because the text-decoder MLP saturates in fp16 and produces
non-finite logits during generation.

The actual stability work lives in the paired unsloth-zoo patches on
Gemma4RMSNorm, Gemma4TextMLP, Gemma4TextAttention, and
Gemma4TextScaledWordEmbedding, all of which gate on UNSLOTH_FORCE_FLOAT32
and stay no-ops for bf16 runs.
2026-04-17 17:15:56 +00:00

View file

@ -105,6 +105,7 @@ FORCE_FLOAT32 = [
"gemma3,", # Add comma bc gemma3 will match gemma3n "gemma3,", # Add comma bc gemma3 will match gemma3n
"gemma3text", # Gemma3TextModel (EmbeddingGemma, standalone text-only Gemma3) "gemma3text", # Gemma3TextModel (EmbeddingGemma, standalone text-only Gemma3)
"gemma3n", "gemma3n",
"gemma4", # Gemma-4 E2B/E4B GRPO NaNs under float16 (MLP gate*up overflow)
"gpt_oss", "gpt_oss",
"qwen3_5", # Qwen3.5 GDN layers produce NaN grad norms in float16 training "qwen3_5", # Qwen3.5 GDN layers produce NaN grad norms in float16 training
] ]