Research map / Large language models / Reasoning and chain of thought

CURE: Co-Evolving Coders and Unit Testers via Reinforcement Learning

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

CURE co-evolves a single LLM as code generator and unit-test generator via RL using an execution-matrix reward derived from a reward-precision analysis, requiring no ground-truth code, yielding ReasonFlux-Coder 4B/7B/14B.

Problem

Training unit test generators typically requires ground-truth code solutions, which are costly; the paper asks whether code and unit-test generators can co-evolve without ground-truth code to improve coding ability and test-time selection (Best-of-N).

Approach

A self-play RL framework where a single policy generates both code and unit tests, with rewards for tests derived from an execution matrix against generated solutions (estimating a precision-like quantity mu = p_u(1-p_01) - (1-p_u)p_00), plus a response-length-guided reward transformation for long-CoT testers.

Key results

• ReasonFlux-Coder-7B and 14B improve one-shot code accuracy by 5.3% and BoN accuracy by 9.0% on average over their Qwen2.5-Instruct bases, and average 37.8% gain in unit test accuracy.
• ReasonFlux-Coder-4B (long-CoT, from Qwen3-4B) consistently outperforms Qwen3-4B while reducing unit test generation response length to 64.8% of original.
• Using GPT-4o-mini as coder with ReasonFlux-Coder-4B as unit tester yields a 7.0% improvement over GPT-4o one-shot on average, at lower cost.

Measured effects reported in the paper

Surprising findings

• Using ReasonFlux-Coder-4B generated unit tests as RL rewards for Qwen2.5-14B-Instruct gives improvements comparable to ground-truth labeled unit tests.
• Accuracy gains are larger for standard base models than for long-CoT models, which gain less from BoN.

Limitations

• Experiments limited to coding tasks; mathematical reasoning not addressed.
• Long-CoT variant and reward transformation add complexity; long-CoT gains smaller.
• Training uses only 4.5K problems and a limited number of steps (350 for 7B/14B, 50 for 4B).

Experimental setup

Datasets: CodeContests CodeForces LiveBench MBPP LiveCodeBench

Benchmarks: LiveBench MBPP LiveCodeBench CodeContests CodeForces

Metrics: unit test accuracy one-shot code accuracy Best-of-N accuracy response length API cost

Baselines: Qwen2.5-Coder-Instruct Qwen2.5-7B-Instruct Qwen2.5-14B-Instruct DeepSeek-Coder-V2-16B Seed-Coder-8B-Instruct Qwen3-4B GPT-4o-mini GPT-4.1-mini GPT-4o MPSC AlphaCodium S*

Builds on: GRPO PPO UTGEN O1-Coder S* MPSC AlphaCodium

Closest papers by meaning

• CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models (NeurIPS 2026 · similarity 0.92)
• ReVeal: Self-Evolving Code Agents via Reliable Self-Verification (ICLR 2026 · similarity 0.90)
• Learning to Generate Unit Test via Adversarial Reinforcement Learning (ICLR 2026 · similarity 0.90)
• TTRL: Test-Time Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• $\textbf{Re}^{2}$: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving (ICLR 2026 · similarity 0.89)
• RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning (ICML 2025 · similarity 0.88)