fix(studio): don't set trust_remote_code for Gemma 4 training (#4795)

Gemma 4 is a native transformers 5.5 model and does not need
trust_remote_code=True. The auto-enable logic (added for NemotronH)
was catching all transformers 5.x models, including Gemma 4.

When trust_remote_code=True, unsloth_compile_transformers() returns
early without running the compiler. This disables the fused cross
entropy patch, causing logged training loss to be inflated by the
gradient_accumulation_steps factor.

Exclude models matching "gemma-4" or "gemma4" from the auto-enable
so the compiler runs and applies fused cross entropy correctly.
This commit is contained in:
Daniel Han 2026-04-02 11:44:26 -07:00 committed by GitHub
commit 401621618b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -390,11 +390,16 @@ def run_training_process(
# Some newer architectures (e.g. NemotronH) have config parsing bugs in
# transformers that require trust_remote_code=True as a workaround.
# Only auto-enable for unsloth/* prefixed models (trusted source).
# Exclude Gemma 4 since it is a native transformers 5.5 model and
# trust_remote_code=True would bypass the compiler (disabling fused CE).
from utils.transformers_version import needs_transformers_5
_lowered = model_name.lower()
_is_native_t5 = any(x in _lowered for x in ("gemma-4", "gemma4"))
if (
needs_transformers_5(model_name)
and model_name.lower().startswith("unsloth/")
and _lowered.startswith("unsloth/")
and not _is_native_t5
and not config.get("trust_remote_code", False)
):
config["trust_remote_code"] = True