f58c3ddb07
lr_sched.step() runs once per outer optimizer step (after the gradient accumulation inner loop), for train_steps total. The scheduler was configured with num_warmup_steps and num_training_steps multiplied by gradient_accumulation_steps, so with accumulation > 1 a warmup or non-constant schedule stretched past the run and never reached the intended decay. Count both in optimizer steps. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| diffusion_lora_trainer.py | ||
| resume.py | ||
| s3_dataset.py | ||
| trainer.py | ||
| training.py | ||
| worker.py | ||