Research map / Reinforcement learning / RL from feedback

Reinforcement Learning for Code Optimization

NeurIPS 2026 · Reinforcement learning › RL from feedback · approach: Scaling RL with verifiable rewards · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds DMC-Optim, a timing-aware competitive-programming benchmark and RL pipeline, and trains Qwen 2.5 and CWM 32B with GRPO using optimization-aware rewards to raise percentile-constrained pass@1 while keeping correctness.

Problem

RL for code correctness is established, but adding execution time to the reward for code optimization is brittle: timing noise, sparse rewards, weak tests, and GRPO instability overwhelm the signal, and naive rewards trade correctness for speed or fail to improve speed.

Approach

Makes execution time learnable by (i) cleaning DMC and adding correctness and large optimization tests with duration-filterability gating, (ii) running timing on an isolated remote sandbox with drift calibration, (iii) organizing optimization environments into pre-execution filtering, intra-execution timeouts, and post-execution percentile ranking, (iv) screening them with an offline simulator, an

Key results

• On DMC-Optim at p30 pass@1, CWM 32B rises from 13.7% (standard RLVR) to 30.9% (top-30% per-test percentile reward), a 125% relative gain with pure-correctness stable.
• On Qwen 2.5 7B at p50, pass@1 rises from 18.0% to 31.3%, and at p30 from 7.7% to 19.1%, with p100 about 46 vs 43.5 for RLVR.
• On LiveCodeBench, CWM 32B with p30 per-test-percentile training reaches an 82.9% median-sample speed win rate against standard RLVR while retaining 54.4% pass@1 in the pre-exec/related rows (reported 54.4% for a pre-execution config).

Surprising findings

• Naive raw-duration reward added on top of RLVR with optimization tests yields only ~5% strict-percentile gain while costing pure-correctness, and on base tests barely moves strict percentiles.
• Binary collapsed rewards consistently outperform bucketed and additive-blend rewards at stricter percentiles, and optimization-only reward causes complete policy collapse.

Limitations

• Small training prompt budget (about 1,000 DMC-Optim train problems) compared with larger code-RL runs.
• CWM 32B may have seen DMC problem statements during SFT/midtraining, so decontamination is not guaranteed.
• LCB tests are too short to measure optimization differences, so LCB speed results are less reliable and timeouts are not used.

Experimental setup

Datasets: DeepMind Code Contests (DMC) LiveCodeBench BigO(Bench) OpenCodeReasoning-2 OpenMathReasoning

Benchmarks: DMC-Optim test LiveCodeBench

Metrics: pass@1 pass@10 pairwise speed win rate (WR_b, WR_m)

Baselines: Standard RLVR naive raw-duration reward Afterburner Afterburner-style additive blend Two-gate bucketed reward

Builds on: GRPO RLVR Dr. GRPO DeepSeek-R1 binary reward PIE Afterburner

Closest papers by meaning

• CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models (NeurIPS 2026 · similarity 0.89)
• Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization (NeurIPS 2025 · similarity 0.89)
• $\mathcal{B}$-Coder: Value-Based Deep Reinforcement Learning for Program Synthesis (ICLR 2024 · similarity 0.87)
• RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning (ICML 2025 · similarity 0.87)
• Extrapolative Weight Averaging Reveals Correctness–Efficiency Frontiers in Code RL (NeurIPS 2026 · similarity 0.87)
• Spurious Rewards: Rethinking Training Signals in RLVR (ICML 2026 · similarity 0.86)