Research map / Large language models / Reasoning and chain of thought

Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Revisits pass@K for RLVR and introduces CoT-Pass@K, using an LLM verifier to show RLVR (DAPO-Qwen-32B, AceReason-Nemotron-7B) extends the reasoning boundary on math and code, with a theory that GRPO implicitly incentivizes correct CoTs.

Problem

Debate over whether RLVR truly improves LLM reasoning or only sampling efficiency, since Pass@K can be inflated by lucky guesses with flawed chain-of-thought in math.

Approach

Introduce CoT-Pass@K, which counts a sample as correct only if both its final answer and its CoT pass an LLM-as-a-CoT-Judge verifier (aggregated via any/all/majority voting), and give a theorem showing GRPO with answer-only rewards increases the probability of correct CoTs under a logic-prior assumption.

Key results

• On AIME 2024/2025, CoT-Pass@K shows a consistent gap between DAPO-Qwen-32B and Qwen2.5-32B across all K up to 1024, while standard Pass@K shows the base model catching up.
• On LiveCodeBench, AceReason-Nemotron-7B shows clear Pass@K gains over DeepSeek-R1-Distill-Qwen-7B on most versions.
• SFT on CoTs from later DAPO checkpoints steadily raises Pass@1 on AIME, ultimately matching DAPO-Qwen-32B's Pass@1.

Surprising findings

• Pass@K on math can be unreliable for base LLMs because correct final answers are often reached through incorrect reasoning, especially on hard problems with easily guessed answers.
• Even when P(CA) approaches 1 on training questions, a median of about 0.7 of correct-answer CoTs remain flawed, suggesting answer-only rewards do not fully remove unreasoned behaviors.

Limitations

• Relies on an LLM verifier (DeepSeek-R1-0528-Qwen3-8B) for CoT correctness, which can make false positives and negatives; manual checking is only partial.
• The theorem explains optimization dynamics but gives no generalization guarantee.
• Math results are for a single 32B model pair; the code results mainly use distilled 7B models.

Experimental setup

Datasets: DAPO-Math-17k AIME 2024 AIME 2025 MATH-500 AMC23 Minerva LiveCodeBench

Benchmarks: AIME 2024 AIME 2025 MATH-500 AMC23 Minerva LiveCodeBench

Metrics: Pass@K CoT-Pass@K Pass@1

Baselines: Qwen2.5-32B DeepSeek-R1-Distill-Qwen-7B Skywork-OR1-7B AceReason-Nemotron-7B

Builds on: GRPO DAPO pass@k estimator (Chen et al. 2021) DeepSeek-R1

Closest papers by meaning

• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (NeurIPS 2025 · similarity 0.96)
• On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation (ICLR 2026 · similarity 0.95)
• The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models (NeurIPS 2026 · similarity 0.94)
• Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning (ICML 2026 · similarity 0.94)
• The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning (NeurIPS 2025 · similarity 0.94)
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example (NeurIPS 2025 · similarity 0.94)