1080d0c4dc
Gemma3 models have a large vocabulary (262144 tokens) which causes training loss to explode when using int8 embedding quantization. This fix auto-detects Gemma3 models and switches from int8-int4 (phone-deployment) to int4 weight-only QAT for stable training. |
||
|---|---|---|
| .. | ||
| dataprep | ||
| kernels | ||
| models | ||
| registry | ||
| utils | ||
| __init__.py | ||
| _auto_install.py | ||
| chat_templates.py | ||
| device_type.py | ||
| import_fixes.py | ||
| ollama_template_mappers.py | ||
| save.py | ||
| tokenizer_utils.py | ||
| trainer.py | ||