ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

loader: import FORCE_FLOAT32 from unsloth_zoo (single source of truth) (#5610)

Browse source 
* loader: import FORCE_FLOAT32 from unsloth_zoo (single source of truth)

unsloth_zoo now owns the FORCE_FLOAT32 list in
unsloth_zoo/model_lists.py (re-exported as the top-level
`unsloth_zoo.FORCE_FLOAT32`). The CUDA loader here imports from there
so the MLX loader (unsloth_zoo.mlx.loader) and the CUDA loader stay in
sync from a single edit, and the bf16->fp16 downcast warning added in
unsloth-zoo PR #670 gates on the same list.

Companion to unsloth-zoo PR #670.

* loader: add inline FORCE_FLOAT32 fallback for old unsloth_zoo installs

If a user upgrades unsloth without upgrading unsloth_zoo, the previously
unconditional `from unsloth_zoo import FORCE_FLOAT32` would raise
ImportError at module import time, killing the whole package. Wrap the
import in try/except and fall back to an inline list that mirrors
unsloth_zoo.model_lists.FORCE_FLOAT32 byte-for-byte, so the module
loads cleanly on any zoo version while still preferring zoo as the
single source of truth when present.
This commit is contained in:
Daniel Han 2026-05-19 05:58:32 -07:00 • committed by GitHub
commit 06526f9d6a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 16 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

26
unsloth/models/loader.py
View file

@ -99,15 +99,22 @@ from ._utils import (
fast_inference_setup,
)
global FORCE_FLOAT32
# Forces float32 precision since float16 goes to infinity
FORCE_FLOAT32 = [
"gemma3,", # Add comma bc gemma3 will match gemma3n
"gemma3text", # Gemma3TextModel (EmbeddingGemma, standalone text-only Gemma3)
"gemma3n",
"gpt_oss",
"qwen3_5", # Qwen3.5 GDN layers produce NaN grad norms in float16 training
]
# Single source of truth is unsloth_zoo.model_lists. Re-exported so callers
# doing `from unsloth.models.loader import FORCE_FLOAT32` keep working.
# Fallback list mirrors zoo for users who upgrade unsloth without upgrading
# unsloth_zoo (so this module never fails at import).
try:
from unsloth_zoo import FORCE_FLOAT32 # noqa: F401
except ImportError:
global FORCE_FLOAT32
# Forces float32 precision since float16 goes to infinity
FORCE_FLOAT32 = [
"gemma3,", # Add comma bc gemma3 will match gemma3n
"gemma3text", # Gemma3TextModel (EmbeddingGemma, standalone text-only Gemma3)
"gemma3n",
"gpt_oss",
"qwen3_5", # Qwen3.5 GDN layers produce NaN grad norms in float16 training
]
global DISABLE_COMPILE_MODEL_NAMES
# Must be alphabetically sorted for each entry
@ -1381,7 +1388,6 @@ class FastModel(FastBaseModel):
for model_type_arch in model_types:
if model_type_arch != "siglip":
break
global FORCE_FLOAT32
for disable_name in FORCE_FLOAT32:
# add comma to model_types_all matching in case of exact match for end
if (