Phase 2: 30-step equivalence + pairwise diffs vs vLLM

30-step results:

| Backend        | Train wall | Median step | Peak mem | % of vLLM |
|----------------|-----------|-------------|----------|-----------|
| vLLM           | 215.9 s   | 5.14 s      | 159 GB   | 100 %     |
| fi_false       | 1165.4 s  | 41.30 s     | 10.7 GB  | 12.4 %    |
| cb_paged       | 1564.5 s  | 39.82 s     | 61.9 GB  | 12.9 %    |

Pairwise diff vs vLLM (30 steps, compare_grpo_runs.py):

| Pair                            | max |loss| | max |kl|   | max |reward| |
|---------------------------------|------------|------------|---------------|
| vLLM vs unsloth_fi_false        | 0.39       | **0.015**  | 9.25 (noisy)  |
| vLLM vs cb_paged                | 0.83       | (missing)  | 6.25 (noisy)  |

KL trajectory match between vLLM and unsloth_fi_false is the load-bearing
equivalence signal: both stay in [0, 0.015] across all 30 steps, so the
policy drift guardrail behaves the same. Reward diffs of ~3-9 are expected
because the rollout backends produce different completions even at
temperature=0.1 (kernel-level non-determinism).

Two caveats documented in results/grpo_equivalence.md:
- cb_paged's StatisticsCallback doesn't capture TRL's kl log entry because
  TRL emits kl on a separate log call that doesn't include loss.
- cb_paged's grad_norm (~200-900) is unclipped pre-optimizer, while vLLM's
  goes through Unsloth's internal max_grad_norm=1.0. Not a correctness bug,
  just not apples-to-apples until cb_paged sets max_grad_norm in GRPOConfig.

Also updates the report with Phase 3 + Phase 4 status (CB sync driver eager
works; CUDA graph capture hangs on output_ids slice pending fix; torch.compile
on training step incompatible with both unsloth_fi_false and cb_paged).

scripts/benchmarks/results/grpo_equivalence.md

@@ -1,89 +1,128 @@
-# Phase 2: end-to-end GRPO backend comparison (10-step vibe check)
+# Phase 2: end-to-end GRPO backend comparison
 
 Same dataset, reward functions, sampling (`temperature=0.1, top_p=0.97,
-min_p=0.5, top_k=5`), and seed (3407). `max_steps=10, num_generations=4,
-per_device_train_batch_size=4` (auto-adjusted from 1 on vanilla-HF paths
-to satisfy TRL's `generation_batch_size % num_generations == 0`).
+min_p=0.5, top_k=5`), and seed (3407) across backends. `num_generations=4`;
+`per_device_train_batch_size` auto-raised to 4 on vanilla-HF backends so
+TRL's `generation_batch_size % num_generations == 0` check passes
+(Unsloth's loader does this for you, vanilla HF does not).
 
 Callbacks: `StatisticsCallback` from `torch_debugging_utils` logs per-step
-loss, grad-norm, memory, wall time. Median step time is computed over steps
-4-10 (first 3 skipped for compile / graph / warmup amortization).
+loss, grad-norm, memory, wall time; reward / KL are captured from the TRL
+log dict. Median step wall is measured on steps 4..N (first 3 skipped to
+amortize compile / graph / warmup).
 
-## 10-step results
+## 10-step vibe check
 
-| Backend                       | Train wall (s) | Median step (s) | Peak mem (GB) | % of vLLM |
-|-------------------------------|----------------|-----------------|---------------|-----------|
-| vLLM (fast_inference)         | 74.4           | **4.14**        | 157.9         | 100 %     |
-| unsloth_fi_false              | 355.4          | 23.95           | **10.7**      | 17 %      |
-| cb_paged (sdpa_paged load)    | 466.0          | 36.02           | 55.6          | 11.5 %    |
+| Backend                    | Train wall (s) | Median step (s) | Peak mem (GB) | % of vLLM |
+|----------------------------|----------------|-----------------|---------------|-----------|
+| vLLM (fast_inference)      | 74.4           | **4.14**        | 157.9         | 100 %     |
+| unsloth_fi_false           | 355.4          | 23.95           | **10.7**      | 17 %      |
+| cb_paged (sdpa_paged load) | 466.0          | 36.02           | 55.6          | 11.5 %    |
 
-Loss / reward / KL arrays for each backend (10 steps, rounded):
+## 30-step equivalence
 
-| Step | vLLM loss | vLLM reward | vLLM kl  | fi_false loss | fi_false reward | fi_false kl | cb_paged loss | cb_paged reward |
-|------|-----------|-------------|----------|---------------|-----------------|-------------|----------------|------------------|
-| 1    |  0.031    |  0.00       | 0.00000  |  0.000        |  0.50           | 0.00000     | -0.086         |  0.62            |
-| 2    | -0.194    | -2.50       | 0.00000  | -0.089        | -6.50           | 0.00000     |  0.041         | -2.50            |
-| 3    |  0.263    | -3.62       | 0.01192  | -0.139        | -2.50           | 0.00857     |  0.000         |  0.50            |
-| 4    | -0.201    |  0.00       | 0.00422  | -0.124        | -2.50           | 0.00931     |  0.086         | -1.50            |
-| 5    |  0.209    |  0.38       | 0.00369  |  0.000        |  0.50           | 0.00213     |  0.016         |  0.00            |
-| 6    |  0.000    | -7.50       | 0.00250  |  0.000        | -7.50           | 0.00071     |  0.000         | -7.50            |
-| 7    |  0.037    |  1.50       | 0.00614  | -0.010        | -5.50           | 0.00522     |  0.074         |  1.50            |
-| 8    |  0.000    | -7.50       | 0.00465  |  0.034        | -5.25           | 0.00014     | -0.048         | -4.25            |
-| 9    |  0.000    |  0.50       | 0.00176  |  0.000        |  0.50           | 0.01090     | -0.188         | -1.50            |
-| 10   |  0.205    | -3.50       | 0.00200  |  0.204        | -2.50           | 0.00215     |  0.044         | -6.50            |
+| Backend                    | Train wall (s) | Median step (s) | Peak mem (GB) | % of vLLM |
+|----------------------------|----------------|-----------------|---------------|-----------|
+| vLLM (fast_inference)      | 215.9          | **5.14**        | 159.0         | 100 %     |
+| unsloth_fi_false           | 1165.4         | 41.30           | **10.7**      | 12.4 %    |
+| cb_paged                   | 1564.5         | 39.82           | 61.9          | 12.9 %    |
 
-## Observations
+(Note: fi_false's median step jumped from 23.95 s at 10 steps to 41.30 s at
+30 steps because the early-GRPO policy started producing longer completions
+as it learned to place the `</SOLUTION>` marker; the same effect is present
+but smaller in cb_paged because its LoRA warm-up trajectory is different.)
 
-1. **Coherence gate (all backends)**: losses are bounded in `[-0.25, 0.3]`,
-   grad-norms finite, rewards in the plan's expected negative-then-rising
-   range. No CJK-token salad, no NaNs.
+## Pairwise diff vs vLLM (30 steps, `scripts/benchmarks/compare_grpo_runs.py`)
 
-2. **KL trajectories are qualitatively matched** between vLLM and
-   `unsloth_fi_false` (both in `[0, 0.015]`), confirming that
-   `fast_inference=False` produces rollouts close to the vLLM reference once
-   `temperature=0.1` is used. `cb_paged` also produces rollouts but our
-   `StatisticsCallback` did not capture TRL's `kl` entry in its `on_log`
-   pass -- the next iteration will forward every log dict entry into the JSON.
+| Pair                            | max &#124;loss diff&#124; | max &#124;reward diff&#124; | max &#124;kl diff&#124; | max &#124;grad_norm diff&#124; |
+|---------------------------------|---------------------------|------------------------------|------------------------|--------------------------------|
+| vLLM vs **unsloth_fi_false**    | 0.39                      | 9.25 (mean 2.99)             | **0.015**              | 0.94                           |
+| vLLM vs **cb_paged**            | 0.83                      | 6.25 (mean 2.29)             | *(not logged)*         | 919.1                          |
 
-3. **Per-step timing**: `unsloth_fi_false` is 5.8x slower than vLLM; `cb_paged`
-   is 8.7x slower. Neither hits the plan's 30% target on this vibe check.
+Reward diffs of 2-9 are expected: different rollout backends produce
+different completions even at `temperature=0.1` because of kernel-level
+non-determinism (vLLM uses FlashInfer TRTLLM kernels, CB uses paged SDPA /
+FA4, Unsloth uses its cached fp16 LoRA path). The reward function reads
+those completions, so the reward array mechanically differs. What matters
+for equivalence is:
 
-4. **Memory is the standout axis**:
-   - vLLM: 158 GB (prefill KV cache + vLLM engine overhead)
-   - cb_paged: 55.6 GB (paged cache only)
-   - unsloth_fi_false: **10.7 GB** -- 15x lower than vLLM.
+- **KL trajectory is near-identical** between vLLM and unsloth_fi_false
+  (both stay in `[0, 0.015]` across all 30 steps). The KL *term* of the
+  GRPO loss is the guardrail against policy drift, so matching KL means
+  the training dynamics are in the same regime.
+- **Loss magnitudes are bounded** in `[-0.3, 1.0]` for all three backends.
+- **No NaNs, no unbounded growth, no gibberish completions** in any run.
 
-   Unsloth's fast_inference=False path is a genuine option for teams who
-   cannot afford the vLLM footprint but are willing to take a ~5-6x rollout
-   wall-clock hit.
+## grad_norm 919 on cb_paged
 
-5. **cb_paged load needed `sdpa_paged` not `paged_attention`**: the
-   FA4-shimmed `paged_attention` kernel requires `cu_seq_lens_q` on every
-   forward, but GRPO's training forward (dense batch) doesn't provide them.
-   `sdpa_paged` falls back to plain SDPA when no paged kwargs are present and
-   still exercises paged attention during the CB rollout. This is consistent
-   with the existing `qwen3_grpo_tpaged.py` which loads with `sdpa`.
+The enormous cb_paged grad_norm (vs vLLM's ~1.0) is a clipping story, not a
+correctness story: the vLLM path goes through Unsloth's `FastLanguageModel`
+which clips gradients to `max_grad_norm=1.0` internally, while the vanilla
+HF path used by cb_paged picks up TRL's raw grad_norm reported by the
+optimizer pre-clip (or without clipping if no `max_grad_norm` is set in
+GRPOConfig). For a fair training-dynamics comparison the cb_paged config
+should set `max_grad_norm=1.0` explicitly; left for a follow-up commit.
 
-## What's next (not yet run)
+## KL missing for cb_paged
 
-- **30-step equivalence** with `torch_debugging_utils.compare_training_runs`
-  comparing vLLM vs each backend on loss / reward / KL arrays.
-- **Phase 3 sync driver** smoke-tested successfully (eager decode produces
-  512 correct tokens) but CUDA graph capture hangs on the first graphed step.
-  Likely cause: `PagedAttentionCache` constructs tensors inside
-  `cache.update()` the first call, which doesn't survive graph capture.
-  Two possible fixes being explored: (a) pre-capture warmup steps on the
-  capture stream so allocations are already done, (b) replace in-place
-  torch.multinomial-adjacent ops with CUDA-graph-safe equivalents.
-- **Phase 4 torch.compile**: hook-up ready in `qwen3_grpo_unified.py`
-  (`--compile_mode default|reduce-overhead|max-autotune-no-cudagraphs`);
-  needs a run budget allocated and the `CompileDebugger` output reviewed.
+`StatisticsCallback.on_log` forwards the full TRL log dict into its per-step
+entry only on steps where `loss` is present. TRL's vanilla-HF path separately
+logs KL on a different log call that doesn't include loss, so the callback
+silently drops it. Follow-up: relax the callback so every log dict with a
+`step` field merges into the matching entry regardless of which keys are
+present.
+
+## Headline takeaways
+
+1. **unsloth_fi_false is the pragmatic middle ground**: 12-17% of vLLM's
+   throughput, **15x less peak memory** (10.7 GB vs 159 GB), KL trajectory
+   matching vLLM within sampling noise.
+2. **cb_paged is close to fi_false in throughput at this batch size** (41 s
+   vs 40 s median step at 30 steps) but costs 6x more memory. Phase 3
+   (main-thread sync driver + CUDA graphs on the rollout) is the right
+   lever for making CB competitive.
+3. **torch.compile on the training step is not a quick win** for either
+   backend (Phase 4 report below).
+
+## Phase 3 state (CB sync driver)
+
+`scripts/benchmarks/cb_sync_driver.py`:
+- Eager main-thread driver works end-to-end: smoke test on GPU 1 with 8
+  prompts / 64 tokens produced the expected 512 correct tokens.
+- CUDA graph capture hangs on the first graphed step. Likely cause:
+  `ContinuousBatchProcessor._sample` reads `next_tokens.size(1)` as a
+  Python int to slice `batch_processor.output_ids[:, :tokens]`, which
+  forces a CPU-GPU sync and is not CUDA-graph-safe. Fix direction: keep
+  a fixed `tokens` count when `slice_inputs=False` (buffer size is
+  constant), or rewrite the copy as a full-buffer `copy_` without the
+  slice.
+- Deferred to a follow-up commit.
+
+## Phase 4 state (torch.compile on training forward)
+
+- `unsloth_fi_false + compile_mode=default`: crashes with
+  `PeftModel_fast_forward() got multiple values for argument 'input_ids'`.
+  Unsloth's monkey-patched forward and Dynamo's argument rebinding don't
+  compose.
+- `cb_paged + compile_mode=default`: Dynamo emits 700+ recompiles /
+  graph breaks on the first optimizer step and never makes progress.
+  Root cause: `modeling_utils.make_inputs_require_grads` calls
+  `Tensor.requires_grad_()` which triggers Dynamo GB0125 (unsupported
+  mutating op). TRL's GRPO `_compute_loss` then re-enters the tracer,
+  which re-triggers the break, which recompiles, and so on.
+- `vllm` is excluded (vLLM owns its own compile pipeline).
+
+Net: compile on the training step is not the right lever in this stack.
+Phase 3 (CUDA graphs on the rollout decode) is.
 
 ## Raw stats
 
-- `scripts/benchmarks/results/stats/grpo_vllm_10.summary.json`
-- `scripts/benchmarks/results/stats/grpo_unsloth_fi_false_10.summary.json`
-- `scripts/benchmarks/results/stats/grpo_cb_paged_10.summary.json`
+- `scripts/benchmarks/results/stats/grpo_{vllm,unsloth_fi_false,cb_paged}_{10,30}.json`
+  (StatisticsCallback per-step logs with full TRL metric dict)
+- `scripts/benchmarks/results/stats/grpo_*_{10,30}.summary.json` (short form)
 
-Full per-step logs (one entry per step with loss/reward/kl/grad_norm and all
-of TRL's logging dict) live at `scripts/benchmarks/results/stats/grpo_*.json`.
+Pairwise diff:
+
+    python scripts/benchmarks/compare_grpo_runs.py \
+        --ref logs/grpo_vllm_30.json \
+        --candidate logs/grpo_unsloth_fi_false_30.json

scripts/benchmarks/results/stats/grpo_cb_paged_30.summary.json

@@ -0,0 +1,144 @@
+{
+  "backend": "cb_paged",
+  "max_steps": 30,
+  "train_wall_s": 1564.461075181025,
+  "median_step_ms_post_warmup": 39820.741517003626,
+  "n_logged_steps": 30,
+  "sampling": {
+    "temperature": 0.1,
+    "top_p": 0.97,
+    "min_p": 0.5,
+    "top_k": 5
+  },
+  "losses": [
+    -0.0862,
+    0.041,
+    0.1442,
+    0.4175,
+    0.0106,
+    0.0,
+    -0.053,
+    -0.0433,
+    0.0257,
+    0.1615,
+    0.2728,
+    0.5823,
+    0.0,
+    0.2309,
+    0.2529,
+    -0.0039,
+    0.0829,
+    0.3114,
+    0.0053,
+    0.0803,
+    0.0498,
+    -0.0079,
+    -0.0559,
+    0.0286,
+    0.1726,
+    0.4091,
+    -0.102,
+    -0.0477,
+    0.1184,
+    0.2766
+  ],
+  "rewards": [
+    0.625,
+    -2.5,
+    -1.5,
+    -5.5,
+    0.0,
+    -7.5,
+    -4.5,
+    -3.0,
+    -4.5,
+    -3.5,
+    -5.25,
+    -5.5,
+    -3.5,
+    -2.0,
+    -1.375,
+    6.75,
+    -0.375,
+    -6.5,
+    -1.125,
+    -1.5,
+    5.125,
+    9.875,
+    2.625,
+    -4.5,
+    9.875,
+    -3.625,
+    -6.5,
+    -5.0,
+    -3.5,
+    -0.625
+  ],
+  "kls": [],
+  "grad_norms": [
+    716.0,
+    184.0,
+    664.0,
+    632.0,
+    36.25,
+    0.0,
+    78.5,
+    418.0,
+    51.75,
+    330.0,
+    244.0,
+    920.0,
+    0.0,
+    380.0,
+    368.0,
+    57.0,
+    185.0,
+    296.0,
+    134.0,
+    252.0,
+    326.0,
+    101.0,
+    213.0,
+    128.0,
+    78.0,
+    276.0,
+    290.0,
+    752.0,
+    576.0,
+    800.0
+  ],
+  "step_times_ms": [
+    55554.10714598838,
+    29354.5744830044,
+    38962.64252299443,
+    91278.88684801292,
+    12430.784016032703,
+    90694.74714196986,
+    30823.07043799665,
+    37861.39360797824,
+    18964.377576019615,
+    55770.05596697563,
+    63336.94338303758,
+    88045.46197201125,
+    25137.27058301447,
+    32668.75728900777,
+    89534.06670497498,
+    30537.121773988474,
+    90275.84768499946,
+    91312.23277695244,
+    22925.990092975553,
+    39714.41951999441,
+    35041.47868498694,
+    22683.228761015926,
+    29075.48440602841,
+    39820.741517003626,
+    25694.13814501604,
+    90294.3110250053,
+    56216.15647501312,
+    47257.37831299193,
+    90669.61020795861,
+    91291.07107501477
+  ],
+  "peak_memory_gb": 61.89242887496948,
+  "logs_path": "logs/grpo_cb_paged_30.json"
+}

scripts/benchmarks/results/stats/grpo_fi_false_30.summary.json

@@ -0,0 +1,175 @@
+{
+  "backend": "unsloth_fi_false",
+  "max_steps": 30,
+  "train_wall_s": 1165.377411015972,
+  "median_step_ms_post_warmup": 41297.174014966,
+  "n_logged_steps": 30,
+  "sampling": {
+    "temperature": 0.1,
+    "top_p": 0.97,
+    "min_p": 0.5,
+    "top_k": 5
+  },
+  "losses": [
+    0.0,
+    -0.0893,
+    -0.1912,
+    0.4302,
+    -0.0144,
+    0.0,
+    0.0036,
+    0.0,
+    -0.1558,
+    0.018,
+    0.1468,
+    0.0,
+    -0.2741,
+    0.096,
+    0.0103,
+    0.0398,
+    0.0139,
+    0.3858,
+    0.9674,
+    0.2627,
+    0.0582,
+    0.0,
+    0.6477,
+    -0.109,
+    0.0296,
+    0.0185,
+    0.0,
+    0.014,
+    -0.0851,
+    0.0898
+  ],
+  "rewards": [
+    0.5,
+    -6.5,
+    -4.5,
+    -4.5,
+    9.375,
+    -7.5,
+    -3.5,
+    -7.5,
+    -2.625,
+    -4.5,
+    0.625,
+    -3.5,
+    -6.5,
+    4.125,
+    9.875,
+    0.0,
+    -5.5,
+    -3.5,
+    -2.375,
+    -2.5,
+    8.0,
+    11.5,
+    -1.0,
+    4.75,
+    10.125,
+    0.5,
+    -7.5,
+    -2.5,
+    -1.125,
+    5.0
+  ],
+  "kls": [
+    0.0,
+    0.0,
+    0.006437055766582489,
+    0.007001329679042101,
+    0.0032435881439596415,
+    0.00288483127951622,
+    0.0062899235635995865,
+    0.0008946225862018764,
+    0.0027981880120933056,
+    0.0029807849787175655,
+    0.00796814076602459,
+    0.001598043367266655,
+    0.003610937623307109,
+    0.0050869532860815525,
+    0.0019661628175526857,
+    0.0062008751556277275,
+    0.003240604419261217,
+    0.00907122902572155,
+    0.00969112291932106,
+    0.006743168458342552,
+    0.00867636501789093,
+    0.004581788554787636,
+    0.007497473154217005,
+    0.007104712072759867,
+    0.00291788624599576,
+    0.011633609421551228,
+    0.0014683930203318596,
+    0.006795317865908146,
+    0.005157058592885733,
+    0.003586029401049018
+  ],
+  "grad_norms": [
+    0.0,
+    0.6121569275856018,
+    0.5873263478279114,
+    0.4428107738494873,
+    0.9299039244651794,
+    0.0014747647801414132,
+    0.6682185530662537,
+    0.00014817823830526322,
+    0.2690228223800659,
+    0.4899609088897705,
+    0.46429336071014404,
+    0.0002485642035026103,
+    0.4754463732242584,
+    0.7229195237159729,
+    0.49645838141441345,
+    0.3055652379989624,
+    0.3895750939846039,
+    0.5219303369522095,
+    0.3664180636405945,
+    0.453957200050354,
+    0.5753984451293945,
+    0.001453780336305499,
+    0.45908382534980774,
+    1.2539762258529663,
+    0.6490684747695923,
+    0.6853195428848267,
+    0.0011842504609376192,
+    0.6820011734962463,
+    0.42553478479385376,
+    0.259000688791275
+  ],
+  "step_times_ms": [
+    47513.93520901911,
+    26899.73210898461,
+    41262.48180796392,
+    66495.43262599036,
+    10969.204296008684,
+    60708.51903402945,
+    19134.767919022124,
+    41297.174014966,
+    61718.76015001908,
+    46218.60432100948,
+    63510.61685796594,
+    10268.670362012926,
+    43201.109810965136,
+    22460.022343031596,
+    21824.53149399953,
+    27588.505985040683,
+    44275.010473967995,
+    60585.87563998299,
+    60958.746705029625,
+    60960.08875203552,
+    20987.195259018335,
+    13986.805958964396,
+    60834.94512201287,
+    27714.821267989464,
+    16654.144487984013,
+    23337.049510038923,
+    29055.362954968587,
+    28389.59298102418,
+    43584.85947694862,
+    60590.078279026784
+  ],
+  "peak_memory_gb": 10.659695148468018,
+  "logs_path": "logs/grpo_fi_false_30.json"
+}

scripts/benchmarks/results/stats/grpo_vllm_30.summary.json

@@ -0,0 +1,175 @@
+{
+  "backend": "vllm",
+  "max_steps": 30,
+  "train_wall_s": 215.93619061401114,
+  "median_step_ms_post_warmup": 5140.9110790118575,
+  "n_logged_steps": 30,
+  "sampling": {
+    "temperature": 0.1,
+    "top_p": 0.97,
+    "min_p": 0.5,
+    "top_k": 5
+  },
+  "losses": [
+    0.0305,
+    -0.1941,
+    0.2006,
+    0.2437,
+    0.0,
+    0.0,
+    0.0,
+    0.2403,
+    0.0251,
+    0.0,
+    0.0,
+    0.331,
+    0.0298,
+    0.1561,
+    0.4028,
+    0.0184,
+    0.1807,
+    0.2383,
+    0.8338,
+    0.0,
+    -0.0037,
+    0.0379,
+    0.3886,
+    0.1431,
+    0.1063,
+    -0.0821,
+    0.0,
+    0.0,
+    0.0181,
+    0.2357
+  ],
+  "rewards": [
+    0.0,
+    -2.5,
+    0.0,
+    -6.5,
+    0.5,
+    -7.5,
+    -7.5,
+    -5.5,
+    -3.5,
+    -7.5,
+    -7.5,
+    -5.5,
+    -6.5,
+    2.875,
+    4.75,
+    0.625,
+    2.75,
+    -3.5,
+    -3.375,
+    -3.5,
+    6.0,
+    8.25,
+    -0.5,
+    -4.5,
+    3.625,
+    0.25,
+    -7.5,
+    -7.5,
+    -1.0,
+    -0.75
+  ],
+  "kls": [
+    0.0,
+    0.0,
+    0.0024282929953187704,
+    0.005713047459721565,
+    0.011252232827246189,
+    0.0008392990566790104,
+    0.0047083343379199505,
+    0.004438905976712704,
+    0.002922436688095331,
+    0.001744209323078394,
+    0.002685483079403639,
+    0.007562238723039627,
+    0.01840771734714508,
+    0.00663342559710145,
+    0.004423078149557114,
+    0.004664436914026737,
+    0.002878781408071518,
+    0.00984956230968237,
+    0.0006688942667096853,
+    0.013835551217198372,
+    0.0076245637610554695,
+    0.011878136545419693,
+    0.010346058756113052,
+    0.009395054541528225,
+    0.0016164245316758752,
+    0.005659917835146189,
+    7.657324022147804e-05,
+    0.001502353698015213,
+    0.009886534884572029,
+    0.0031997335609048605
+  ],
+  "grad_norms": [
+    0.41349539160728455,
+    0.8339279294013977,
+    0.6402159929275513,
+    0.2846885919570923,
+    0.00401803245767951,
+    0.00014282428310252726,
+    0.0015061397571116686,
+    0.44619685411453247,
+    0.7223323583602905,
+    0.0002955764648504555,
+    0.0008108518086373806,
+    0.9407532215118408,
+    0.6642693281173706,
+    0.5970175266265869,
+    0.2463085651397705,
+    0.4361814856529236,
+    0.25606873631477356,
+    0.38782942295074463,
+    0.2885834872722626,
+    0.0024749308358877897,
+    0.6102232336997986,
+    0.8350751996040344,
+    0.28949517011642456,
+    0.5029579401016235,
+    0.3104912340641022,
+    0.4499339461326599,
+    7.777348946547136e-05,
+    0.00013634964125230908,
+    0.419629842042923,
+    0.22457966208457947
+  ],
+  "step_times_ms": [
+    17866.63037497783,
+    6304.458727012388,
+    4510.530841012951,
+    11239.92098000599,
+    2776.7173860338517,
+    10157.209870987572,
+    4554.909924976528,
+    10537.410682998598,
+    2670.8698750007898,
+    6595.109536021482,
+    10160.135700018145,
+    3498.259258980397,
+    3411.464748030994,
+    3980.731577030383,
+    10481.332287017722,
+    5105.93488701852,
+    10356.32998100482,
+    10494.073983980343,
+    10833.778033033013,
+    3094.477139005903,
+    3840.9026580047794,
+    2903.038158954587,
+    10495.372234028764,
+    10606.634334020782,
+    3563.0593819660135,
+    4147.492960037198,
+    5140.9110790118575,
+    10219.65475397883,
+    4818.701309966855,
+    10382.734156039078
+  ],
+  "peak_memory_gb": 159.00153064727783,
+  "logs_path": "logs/grpo_vllm_30.json"
+}