ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/scripts/benchmarks/results/stats/notebook_ref_10.json
Daniel Han d118195c8b Add Phase 0+1 GRPO backend comparison scaffolding 
Phase 0 (canonical reference):
- scripts/benchmarks/qwen3_grpo_notebook.py: notebook-to-script port of
  Qwen3_(4B)-GRPO.ipynb with StatisticsCallback from torch_debugging_utils
  and equivalence-friendly sampling (temp=0.1, top_p=0.97, min_p=0.5, top_k=5).
- scripts/benchmarks/results/notebook_ref_10.md: 10-step reference run table
  (median step post-warmup = 5.80s, peak 158.9 GB).
- scripts/benchmarks/results/stats/notebook_ref_10.json: full per-step logs
  for downstream compare_training_runs checks.

Phase 1 (rollout-only LoRA comparison scaffold):
- scripts/benchmarks/make_lora_adapter.py: one-shot that materializes a
  rank-32 LoRA at outputs/lora_rank32_fresh. Re-initializes lora_B with a
  tiny gaussian so the adapter isn't a no-op (otherwise LoRA kernels can
  short-circuit and we'd be measuring the base model).
- scripts/benchmarks/cb_vs_vllm_generation.py: extended with --lora_adapter
  for vLLM (LoRARequest) and tpaged (peft.PeftModel.from_pretrained,
  no merge_adapter), plus a new unsloth_fi_false backend that exercises the
  custom HF inference path (cached fp16 LoRA via fast_linear_forward).
  Sampling knobs are exposed and default to equivalence params.

Phase 2 scaffold:
- scripts/benchmarks/qwen3_grpo_unified.py: single entry point for all 5
  backends (vllm, unsloth_fi_false, cb_paged, cb_sdpa, naive_trl) sharing
  dataset, reward funcs, sampling, and StatisticsCallback. Skips the first
  3 steps when reporting median step wall.

No unsloth internals touched.
2026-04-20 13:54:06 +00:00

362 lines
No EOL
12 KiB
JSON
Raw Blame History

[
{
"step": 1,
"loss": 0.2423,
"grad_norm": 0.24541568756103516,
"learning_rate": 0.0,
"num_tokens": 5422.0,
"completions/mean_length": 1243.5,
"completions/min_length": 1019.0,
"completions/max_length": 1846.0,
"completions/clipped_ratio": 0.25,
"completions/mean_terminated_length": 1042.666748046875,
"completions/min_terminated_length": 1019.0,
"completions/max_terminated_length": 1089.0,
"rewards/match_format_exactly/mean": 2.25,
"rewards/match_format_exactly/std": 1.5,
"rewards/match_format_approximately/mean": 0.75,
"rewards/match_format_approximately/std": 1.5,
"rewards/check_answer/mean": -2.375,
"rewards/check_answer/std": 0.25,
"rewards/check_numbers/mean": -1.5,
"rewards/check_numbers/std": 0.0,
"reward": -0.875,
"reward_std": 2.75,
"frac_reward_zero_std": 0.0,
"completion_length": 1243.5,
"kl": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/high_max": 0.0,
"clip_ratio/region_mean": 0.0,
"epoch": 7.868439688409789e-05,
"time_ms": 60626.65366800502,
"memory_mb": 162696.66357421875,
"memory_gb": 158.883460521698
},
{
"step": 2,
"loss": 0.1559,
"grad_norm": 0.7674608826637268,
"learning_rate": 5e-06,
"num_tokens": 7690.0,
"completions/mean_length": 478.0,
"completions/min_length": 329.0,
"completions/max_length": 553.0,
"completions/clipped_ratio": 0.0,
"completions/mean_terminated_length": 478.0,
"completions/min_terminated_length": 329.0,
"completions/max_terminated_length": 553.0,
"rewards/match_format_exactly/mean": 0.75,
"rewards/match_format_exactly/std": 1.5,
"rewards/match_format_approximately/mean": -1.875,
"rewards/match_format_approximately/std": 2.25,
"rewards/check_answer/mean": -2.125,
"rewards/check_answer/std": 0.25,
"rewards/check_numbers/mean": -2.25,
"rewards/check_numbers/std": 0.5,
"reward": -5.5,
"reward_std": 4.0,
"frac_reward_zero_std": 0.0,
"completion_length": 478.0,
"kl": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/high_max": 0.0,
"clip_ratio/region_mean": 0.0,
"epoch": 0.00015736879376819577,
"time_ms": 3893.7715340289287,
"memory_mb": 160786.34130859375,
"memory_gb": 157.01791143417358
},
{
"step": 3,
"loss": -0.165,
"grad_norm": 0.47987309098243713,
"learning_rate": 4.444444444444444e-06,
"num_tokens": 12437.0,
"completions/mean_length": 1009.75,
"completions/min_length": 745.0,
"completions/max_length": 1319.0,
"completions/clipped_ratio": 0.0,
"completions/mean_terminated_length": 1009.75,
"completions/min_terminated_length": 745.0,
"completions/max_terminated_length": 1319.0,
"rewards/match_format_exactly/mean": 2.25,
"rewards/match_format_exactly/std": 1.5,
"rewards/match_format_approximately/mean": 0.375,
"rewards/match_format_approximately/std": 2.25,
"rewards/check_answer/mean": -1.375,
"rewards/check_answer/std": 1.9311050176620483,
"rewards/check_numbers/mean": -1.75,
"rewards/check_numbers/std": 0.5,
"reward": -0.5,
"reward_std": 5.0332231521606445,
"frac_reward_zero_std": 0.0,
"completion_length": 1009.75,
"kl": 0.0038291513919830322,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/high_max": 0.0,
"clip_ratio/region_mean": 0.0,
"epoch": 0.00023605319065229366,
"time_ms": 7929.127738985699,
"memory_mb": 161959.54833984375,
"memory_gb": 158.16362142562866
},
{
"step": 4,
"loss": 0.3177,
"grad_norm": 0.37925368547439575,
"learning_rate": 3.88888888888889e-06,
"num_tokens": 17403.0,
"completions/mean_length": 1076.5,
"completions/min_length": 546.0,
"completions/max_length": 1846.0,
"completions/clipped_ratio": 0.25,
"completions/mean_terminated_length": 820.0,
"completions/min_terminated_length": 546.0,
"completions/max_terminated_length": 1192.0,
"rewards/match_format_exactly/mean": 0.75,
"rewards/match_format_exactly/std": 1.5,
"rewards/match_format_approximately/mean": -0.75,
"rewards/match_format_approximately/std": 1.9364917278289795,
"rewards/check_answer/mean": -2.125,
"rewards/check_answer/std": 0.25,
"rewards/check_numbers/mean": -2.0,
"rewards/check_numbers/std": 0.5773502588272095,
"reward": -4.125,
"reward_std": 3.4970226287841797,
"frac_reward_zero_std": 0.0,
"completion_length": 1076.5,
"kl": 0.005913741886615753,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/high_max": 0.0,
"clip_ratio/region_mean": 0.0,
"epoch": 0.00031473758753639155,
"time_ms": 11084.477900003549,
"memory_mb": 162749.5458984375,
"memory_gb": 158.93510341644287
},
{
"step": 5,
"loss": -0.02,
"grad_norm": 0.6089861989021301,
"learning_rate": 3.3333333333333333e-06,
"num_tokens": 19235.0,
"completions/mean_length": 302.0,
"completions/min_length": 256.0,
"completions/max_length": 334.0,
"completions/clipped_ratio": 0.0,
"completions/mean_terminated_length": 302.0,
"completions/min_terminated_length": 256.0,
"completions/max_terminated_length": 334.0,
"rewards/match_format_exactly/mean": 3.0,
"rewards/match_format_exactly/std": 0.0,
"rewards/match_format_approximately/mean": 1.5,
"rewards/match_format_approximately/std": 0.0,
"rewards/check_answer/mean": -1.125,
"rewards/check_answer/std": 4.190763473510742,
"rewards/check_numbers/mean": -0.25,
"rewards/check_numbers/std": 2.5,
"reward": 3.125,
"reward_std": 6.650501251220703,
"frac_reward_zero_std": 0.0,
"completion_length": 302.0,
"kl": 0.015983864665031433,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/high_max": 0.0,
"clip_ratio/region_mean": 0.0,
"epoch": 0.00039342198442048943,
"time_ms": 2679.084858042188,
"memory_mb": 160465.05859375,
"memory_gb": 156.70415878295898
},
{
"step": 6,
"loss": 0.0,
"grad_norm": 0.002889552852138877,
"learning_rate": 2.7777777777777783e-06,
"num_tokens": 26958.0,
"completions/mean_length": 1833.75,
"completions/min_length": 1797.0,
"completions/max_length": 1846.0,
"completions/clipped_ratio": 0.75,
"completions/mean_terminated_length": 1797.0,
"completions/min_terminated_length": 1797.0,
"completions/max_terminated_length": 1797.0,
"rewards/match_format_exactly/mean": 0.0,
"rewards/match_format_exactly/std": 0.0,
"rewards/match_format_approximately/mean": -3.0,
"rewards/match_format_approximately/std": 0.0,
"rewards/check_answer/mean": -2.0,
"rewards/check_answer/std": 0.0,
"rewards/check_numbers/mean": -2.5,
"rewards/check_numbers/std": 0.0,
"reward": -7.5,
"reward_std": 0.0,
"frac_reward_zero_std": 1.0,
"completion_length": 1833.75,
"kl": 0.003961368463933468,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/high_max": 0.0,
"clip_ratio/region_mean": 0.0,
"epoch": 0.0004721063813045873,
"time_ms": 10652.5042289868,
"memory_mb": 162744.833984375,
"memory_gb": 158.9305019378662
},
{
"step": 7,
"loss": 0.0,
"grad_norm": 0.003148352960124612,
"learning_rate": 2.222222222222222e-06,
"num_tokens": 29616.0,
"completions/mean_length": 521.5,
"completions/min_length": 452.0,
"completions/max_length": 675.0,
"completions/clipped_ratio": 0.0,
"completions/mean_terminated_length": 521.5,
"completions/min_terminated_length": 452.0,
"completions/max_terminated_length": 675.0,
"rewards/match_format_exactly/mean": 0.0,
"rewards/match_format_exactly/std": 0.0,
"rewards/match_format_approximately/mean": -3.0,
"rewards/match_format_approximately/std": 0.0,
"rewards/check_answer/mean": -2.0,
"rewards/check_answer/std": 0.0,
"rewards/check_numbers/mean": -2.5,
"rewards/check_numbers/std": 0.0,
"reward": -7.5,
"reward_std": 0.0,
"frac_reward_zero_std": 1.0,
"completion_length": 521.5,
"kl": 0.009647021070122719,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/high_max": 0.0,
"clip_ratio/region_mean": 0.0,
"epoch": 0.0005507907781886852,
"time_ms": 4468.553012993652,
"memory_mb": 160983.70361328125,
"memory_gb": 157.21064805984497
},
{
"step": 8,
"loss": 0.0613,
"grad_norm": 0.4861072301864624,
"learning_rate": 1.6666666666666667e-06,
"num_tokens": 32984.0,
"completions/mean_length": 775.0,
"completions/min_length": 671.0,
"completions/max_length": 933.0,
"completions/clipped_ratio": 0.0,
"completions/mean_terminated_length": 775.0,
"completions/min_terminated_length": 671.0,
"completions/max_terminated_length": 933.0,
"rewards/match_format_exactly/mean": 0.0,
"rewards/match_format_exactly/std": 0.0,
"rewards/match_format_approximately/mean": -2.25,
"rewards/match_format_approximately/std": 1.5,
"rewards/check_answer/mean": -2.0,
"rewards/check_answer/std": 0.0,
"rewards/check_numbers/mean": -2.25,
"rewards/check_numbers/std": 0.5,
"reward": -6.5,
"reward_std": 2.0,
"frac_reward_zero_std": 0.0,
"completion_length": 775.0,
"kl": 0.003189136739820242,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/high_max": 0.0,
"clip_ratio/region_mean": 0.0,
"epoch": 0.0006294751750727831,
"time_ms": 5805.1959190052,
"memory_mb": 161361.3447265625,
"memory_gb": 157.5794382095337
},
{
"step": 9,
"loss": 0.006,
"grad_norm": 0.5726504921913147,
"learning_rate": 1.111111111111111e-06,
"num_tokens": 35116.0,
"completions/mean_length": 436.0,
"completions/min_length": 237.0,
"completions/max_length": 635.0,
"completions/clipped_ratio": 0.0,
"completions/mean_terminated_length": 436.0,
"completions/min_terminated_length": 237.0,
"completions/max_terminated_length": 635.0,
"rewards/match_format_exactly/mean": 1.5,
"rewards/match_format_exactly/std": 1.7320507764816284,
"rewards/match_format_approximately/mean": 0.75,
"rewards/match_format_approximately/std": 0.8660253882408142,
"rewards/check_answer/mean": -1.25,
"rewards/check_answer/std": 1.8484227657318115,
"rewards/check_numbers/mean": -1.5,
"rewards/check_numbers/std": 0.0,
"reward": -0.5,
"reward_std": 3.8297085762023926,
"frac_reward_zero_std": 0.0,
"completion_length": 436.0,
"kl": 0.0023976736702024937,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/high_max": 0.0,
"clip_ratio/region_mean": 0.0,
"epoch": 0.0007081595719568809,
"time_ms": 4304.089896031655,
"memory_mb": 160920.1044921875,
"memory_gb": 157.14853954315186
},
{
"step": 10,
"loss": 0.1582,
"grad_norm": 0.3651980459690094,
"learning_rate": 5.555555555555555e-07,
"num_tokens": 39155.0,
"completions/mean_length": 841.75,
"completions/min_length": 729.0,
"completions/max_length": 1108.0,
"completions/clipped_ratio": 0.0,
"completions/mean_terminated_length": 841.75,
"completions/min_terminated_length": 729.0,
"completions/max_terminated_length": 1108.0,
"rewards/match_format_exactly/mean": 2.25,
"rewards/match_format_exactly/std": 1.5,
"rewards/match_format_approximately/mean": 0.375,
"rewards/match_format_approximately/std": 2.25,
"rewards/check_answer/mean": -2.375,
"rewards/check_answer/std": 0.25,
"rewards/check_numbers/mean": -1.75,
"rewards/check_numbers/std": 0.5,
"reward": -1.5,
"reward_std": 4.0,
"frac_reward_zero_std": 0.0,
"completion_length": 841.75,
"kl": 0.00485160993412137,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/high_max": 0.0,
"clip_ratio/region_mean": 0.0,
"epoch": 0.0007868439688409789,
"time_ms": 6777.490795007907,
"memory_mb": 161641.44970703125,
"memory_gb": 157.8529782295227
}
]
Reference in a new issue View git blame Copy permalink