The NVIDIA-CUDA gate imports torch, so the fake-gefen constructor-mapping tests
(loaded via file path to stay dependency-free) hit `import torch` before the
fake gefen module is used and failed with ModuleNotFoundError in a torch-less
environment. Stub `gefenx._require_nvidia_cuda` in the fake_gefen fixture, and
move gate coverage to test_gate_rejects_rocm_hip_build which exercises the real
gate directly (no fake_gefen). 35 tests pass; the 14 fake-gefen tests pass with
torch import blocked.
- make_gefenx_param_groups emits (name, param) pairs so gefen keeps the real
parameter names instead of synthesizing group_i_param_j; without this
GefenXConfig.period_one_substrings (matched against names) never fired.
- Only non-empty buckets become param groups: gefen rejects an empty parameter
group, so an embeddings-only run (all trainable params are modules_to_save)
with embedding_lr set crashed on the empty non_embeddings group.
- tests: (name, param) emission, embedding-only omits the empty group, and real
gefen regression tests for name preservation + embedding-only construction.
35 tests pass.
- optimizers/gefenx.py: expand _RESERVED_EXTRA_KWARGS with model / embedding_lr
/ embedding_learning_rate so a stray extra_kwargs entry can't collide with the
positional model arg or land as an unexpected keyword; skip None-valued config
fields in _collect_kwargs so gefen keeps its own runtime defaults instead of
being handed an explicit None (muon_lr / muon_weight_decay / backup_weight_decay).
- tests: cover the new reserved keys (model / embedding_*) and the None-skip
behaviour. 31 tests pass.
- optimizers/gefenx.py: drop reserved keys (params/lr/weight_decay/
backup_substrings/backup_lr_scale) from extra_kwargs with a warning, so a
stray override no longer collides with the builders' explicit args (TypeError).
- trainer.py: count scalar parameters with numel() in the gefenx enablement
message (was counting tensors); raise ValueError in UnslothTrainingArguments
when more than one of q_galore_config / gefenx_config / gefenx_muon_config is
set, instead of silently dispatching to the first.
- tests: skip the fused CUDA tests on ROCm/HIP (require torch.version.hip is
None, not just cuda.is_available); add tests for reserved-key filtering and
the mutually-exclusive-config ValueError. 29 tests pass.
Adds the Gefen-X optimizers via Unsloth's standard config-object pattern
(mirroring QGaloreConfig): GefenXConfig wraps gefen.Gefen (≈1 byte/param AdamW
replacement) and GefenXMuonConfig wraps gefen.GefenMuonHybrid (Muon on 2D hidden
weights, Gefen on embeddings/heads/norms/biases). Pass either via
UnslothTrainingArguments; UnslothTrainer.create_optimizer dispatches to the new
_create_gefenx_optimizer / _create_gefenx_muon_optimizer builders.
- unsloth/optimizers/gefenx.py: config->constructor mapping, param routing,
the axolotl recommended recipe defaults for the Muon hybrid
(backup_1d_period_one, adjust_lr_fn=match_rms_adamw, fused, backup_lr=0.5*lr),
and an NVIDIA-CUDA-only gate that rejects AMD/ROCm (HIP) and Intel XPU
(gefen ships CUDA-only kernels). gefen is imported lazily.
- unsloth/trainer.py: GefenXConfig / GefenXMuonConfig dataclasses, argument
plumbing on UnslothTrainingArguments, create_optimizer dispatch, __all__.
- tests: 26 tests — config mapping, param routing, the device gate, and real
end-to-end runs against gefen (CPU + fused CUDA, plain + muon) plus the full
UnslothTrainer.create_optimizer dispatch, all asserting parameters update.
MLX is unaffected (its separate trainer has no Gefen-X path).
