30-step results:
| Backend | Train wall | Median step | Peak mem | % of vLLM |
|----------------|-----------|-------------|----------|-----------|
| vLLM | 215.9 s | 5.14 s | 159 GB | 100 % |
| fi_false | 1165.4 s | 41.30 s | 10.7 GB | 12.4 % |
| cb_paged | 1564.5 s | 39.82 s | 61.9 GB | 12.9 % |
Pairwise diff vs vLLM (30 steps, compare_grpo_runs.py):
| Pair | max |loss| | max |kl| | max |reward| |
|---------------------------------|------------|------------|---------------|
| vLLM vs unsloth_fi_false | 0.39 | **0.015** | 9.25 (noisy) |
| vLLM vs cb_paged | 0.83 | (missing) | 6.25 (noisy) |
KL trajectory match between vLLM and unsloth_fi_false is the load-bearing
equivalence signal: both stay in [0, 0.015] across all 30 steps, so the
policy drift guardrail behaves the same. Reward diffs of ~3-9 are expected
because the rollout backends produce different completions even at
temperature=0.1 (kernel-level non-determinism).
Two caveats documented in results/grpo_equivalence.md:
- cb_paged's StatisticsCallback doesn't capture TRL's kl log entry because
TRL emits kl on a separate log call that doesn't include loss.
- cb_paged's grad_norm (~200-900) is unclipped pre-optimizer, while vLLM's
goes through Unsloth's internal max_grad_norm=1.0. Not a correctness bug,
just not apples-to-apples until cb_paged sets max_grad_norm in GRPOConfig.
Also updates the report with Phase 3 + Phase 4 status (CB sync driver eager
works; CUDA graph capture hangs on output_ids slice pending fix; torch.compile
on training step incompatible with both unsloth_fi_false and cb_paged).
2dd8339946