f58c3ddb07
lr_sched.step() runs once per outer optimizer step (after the gradient accumulation inner loop), for train_steps total. The scheduler was configured with num_warmup_steps and num_training_steps multiplied by gradient_accumulation_steps, so with accumulation > 1 a warmup or non-constant schedule stretched past the run and never reached the intended decay. Count both in optimizer steps. |
||
|---|---|---|
| .. | ||
| data_recipe | ||
| export | ||
| inference | ||
| rag | ||
| training | ||
| __init__.py | ||
| _torchao_stub.py | ||
| import_guards.py | ||
| tool_healing.py | ||