Daniel Han 06a1007c6c flex: double-copy LoRA rollout to avoid bf16 merge/unmerge drift ... PEFT's merge/unmerge pair is asymmetric at bf16 and leaks ~1 ULP per cycle onto base_layer.weight. Across hundreds of GRPO refreshes the base drifts, so the adapter trains against a moving target. Keep a pristine base_model on GPU and a deep-copied inference_model wrapped by PEFT. Before each rollout, restore the inference copy's LoRA-target base_layer weights in-place from pristine and call merge_adapter fresh. Never call unmerge_adapter. Adds --verify_no_drift which hashes base params before/after N perturb+refresh cycles and asserts bit-identical, and checks that the merged inference state is deterministic after restoring the LoRA. Update flex_vs_vllm.md with the double-copy row and memory cost.		2026-04-21 04:52:49 +00:00
..
stats	flex: switch to merge_adapter (reversible) + reframe writeup	2026-04-21 03:05:28 +00:00
flex_vs_vllm.md	flex: double-copy LoRA rollout to avoid bf16 merge/unmerge drift	2026-04-21 04:52:49 +00:00
grpo_equivalence.md	Phase 2: 30-step equivalence + pairwise diffs vs vLLM	2026-04-20 15:03:32 +00:00
lora_rollout_baselines.md	Phase 1+3: LoRA rollout benchmarks + CB sync driver + Phase 4 scaffold	2026-04-20 14:01:16 +00:00
notebook_ref_10.md	Add Phase 0+1 GRPO backend comparison scaffolding	2026-04-20 13:54:06 +00:00