5c9a22b816* Fix Gemma3N audio training stride assertion with non-reentrant checkpointing
Gemma3N audio conformer processes variable-length audio tensors
that cause stride mismatches in AOT autograd compiled backward
when non-reentrant gradient checkpointing is used. The error
manifests as:
AssertionError: expected size 2==2, stride 1928==1936 at dim=0
This happens because the audio conformer's conv/norm layers produce
tensors whose strides vary with audio clip duration, but AOT autograd
traces the backward graph assuming fixed strides from the first batch.
The notebook sets gradient_checkpointing_kwargs={"use_reentrant": False}
and TRL 0.27.0+ also forces this. Both override Unsloth's own
use_reentrant=True set during prepare_model_for_training.
Fix: intercept gradient_checkpointing_enable on Gemma3N models to
always force use_reentrant=True, regardless of what the notebook
or TRL passes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| dataprep | ||
| kernels | ||
| models | ||
| optimizers | ||
| registry | ||
| utils | ||
| __init__.py | ||
| _auto_install.py | ||
| chat_templates.py | ||
| device_type.py | ||
| import_fixes.py | ||
| ollama_template_mappers.py | ||
| save.py | ||
| tokenizer_utils.py | ||
| trainer.py | ||