Research map / Large language models / Reasoning and chain of thought

$\textbf{Re}^{2}$: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Re^2, an RL method (DAPO-based) where LLMs learn to abandon unproductive partial reasoning and restart from scratch, with the restart reward set to the out-of-group expected success rate of re-solving.

Problem

Even after RLVR training, LLMs generate unnecessary or low-quality CoT steps and overthink; once early reasoning is misguided they rarely recover, so test-time scaling via longer single chains is inefficient.

Approach

Add a third action, "resolve" (abandon the current prefix and restart), to RLVR. Prefixes are sampled by truncating responses, continuations are grouped per prefix, and a resolve continuation receives the out-of-group expected accuracy of re-solving (a geometric-series formula over at most R redo rounds), while final answers get 0/1 rewards. Training is pure RL with no SFT warm-up.

Key results

• Re^2 raises average accuracy over DAPO by +5.8 on Qwen2.5-7B Base (41.7 to 47.5) and +5.5 on Qwen2.5-14B Base (47.4 to 52.9).
• Re^2 amplifies redo behavior in vanilla models from 0.5% to over 30%.
• On DeepSeek-R1-Distill-Llama-8B, Re^2 improves AIME24 from 38.4 to 47.2 and averages +4.4 over DAPO.

Surprising findings

• Re^2 has about 11% higher rollout time than DAPO but still outperforms it under equal training time.
• Under few samples, Re^2 can underperform RLVR because it triggers more redo actions on hard problems, reducing valid final answers.

Limitations

• No explicit control over the probability of invoking re-solve at inference, which may affect efficiency.
• Only evaluated on LLMs and math/science reasoning; not vision or multimodal, tool use, or search tasks.
• Redo rounds may still be needed multiple times to reach a correct answer.

Experimental setup

Datasets: DAPO-Math-17K

Benchmarks: AIME 2024 AIME 2025 AMC 2023 GSM8K GPQA-Diamond MATH500

Metrics: accuracy average accuracy majority-vote accuracy under test-time scaling tokens consumed

Baselines: DAPO GRPO DLER DeepConf majority voting

Builds on: DAPO GRPO

Closest papers by meaning

• Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.93)
• Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.92)
• The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning (NeurIPS 2025 · similarity 0.92)
• Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.92)
• Advancing LLM Reasoning with Natural Language and Numerical Feedback (ICML 2026 · similarity 0.92)
• Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.92)