Research map / Large language models / Reasoning and chain of thought

Extrapolative Weight Averaging Reveals Correctness–Efficiency Frontiers in Code RL

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that linear extrapolation between two RL checkpoints trained under nested input-size unit-test thresholds continues a correctness-efficiency frontier on LiveCodeBench hard, and that ensembling extrapolated checkpoints raises pass@250 by 3.3%.

Problem

Whether extrapolative weight averaging between RL checkpoints can extend the trade-off frontier between optimization failures (timeouts) and correctness failures in competitive-programming RL, producing useful new checkpoints without extra training.

Approach

Train RL runs under nested verifier-strictness thresholds (input-length cutoffs 10^k) from a shared SFT checkpoint; these form an ordered family along a correctness-efficiency axis. Linear interpolation recovers the frontier and linear extrapolation continues it beyond the RL endpoints, and extrapolated checkpoints are complementary, so pooling them improves matched-budget pass@k.

Key results

• Extrapolated five-checkpoint pool improves LCB/hard pass@250 by 3.3% over the best single checkpoint (bootstrap 95% CI [0.8%, 6.5%]).
• For alpha in [-0.5, 1.5], solve rate stays nearly flat on LCB/hard while optimization failures fall and correctness failures rise along the frontier.
• Online-fixing on 32B pure reasoning raises fix@1 on LCB/hard from 14.2% to 17.9% versus a compute-matched resample-only control.

Surprising findings

• Reward shaping over the same nested threshold outcomes lands on the same frontier rather than dominating it.
• Stricter verification does not increase solve rate on hard problems; it rotates failures between timeouts and wrong answers.

Limitations

• Ensemble comparisons match number of generations rather than token or wall-clock cost.
• Input-length thresholds are a coarse proxy for efficiency pressure.
• The 7B extrapolation range is narrower and degrades earlier from format errors.

Experimental setup

Datasets: CodeContestsPlus CodeContests OpenCodeReasoning-2 OpenMathReasoning LiveCodeBench

Benchmarks: LiveCodeBench v5 LiveCodeBench v6

Metrics: solve rate (pass@1) pass@k pass n@k optimization failure rate correctness failure rate format error rate fix@1 Jaccard similarity of solved sets

Baselines: best single checkpoint interpolation-only checkpoint pool resample-only control reward-shaped RL

Builds on: Rewarded Soups Model soups ExPO Dr.GRPO DAPO

Closest papers by meaning

• Reinforcement Learning for Code Optimization (NeurIPS 2026 · similarity 0.87)
• Scaling Test-Time Compute Without Verification or RL is Suboptimal (ICML 2025 · similarity 0.86)
• SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning (NeurIPS 2026 · similarity 0.86)
• Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (NeurIPS 2025 · similarity 0.86)
• Primal Generation, Dual Judgment: Self-Training from Test-Time Scaling (NeurIPS 2026 · similarity 0.85)
• CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models (NeurIPS 2026 · similarity 0.85)