Research map / Large language models / Reasoning and chain of thought

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Systematically probes whether RLVR expands LLM reasoning beyond base models using pass@k at large k, finding RLVR mostly improves sampling efficiency and narrows coverage, while distillation can expand it.

Problem

Whether reinforcement learning with verifiable rewards (RLVR) truly elicits novel reasoning capacity in LLMs beyond what the base model already possesses, or merely reweights existing reasoning paths.

Approach

Uses pass@k at large k as a measure of reasoning boundary, comparing base and RLVR models across families, RL algorithms, and tasks; adds perplexity and coverage analyses showing RLVR outputs already lie within the base model's distribution, and contrasts with distillation.

Key results

• On Minerva with a 32B model, the base model outperforms the RL-trained model by about 9% at k=128, despite RL winning at k=1.
• On Omni-MATH-Train, Qwen2.5-7B pass@256 is 67.2 while GRPO reaches 66.3 and DAPO 66.1; pass@1 rises from 9.9 to 26.1 (GRPO) and 31.4 (DAPO).
• The distilled DeepSeek-R1-Distill-Qwen-7B shows a pass@k curve consistently above the base model, indicating expanded reasoning boundary.

Measured effects reported in the paper

Surprising findings

• Base models surpass RLVR models in pass@k at large k across all benchmarks and families, and the RL-trained models' solvable sets are nearly subsets of the base model's.
• Across six RL algorithms, the sampling efficiency gap stays above 40 points with only minor variation, suggesting current RLVR methods are far from the base model's upper bound.

Limitations

• Many strongest models and training pipelines are proprietary, limiting analysis of frontier RLVR.
• Pass@k at large k in math may be inflated by lucky guesses; manual checks cover only small subsets.
• Conclusions may not hold for longer RL training or larger compute budgets, which were not tested.

Experimental setup

Datasets: GSM8K MATH Minerva Olympiad AIME24 AMC23 Omni-MATH MATH500 LiveCodeBench HumanEval+ MBPP+ Geometry3K

Benchmarks: GSM8K MATH500 Minerva Olympiad AIME24 AMC23 LiveCodeBench v5 HumanEval+ MBPP+ MathVista-TestMini MathVision-TestMini AIME25

Metrics: pass@k pass@1 sampling efficiency gap perplexity solvable-problem coverage

Baselines: Qwen2.5-7B base Qwen2.5-Math-7B-Instruct DeepSeek-R1-Distill-Qwen-7B Qwen2.5-7B-Instruct GRPO PPO Reinforce++ RLOO ReMax DAPO

Builds on: GRPO PPO DAPO RLOO ReMax Reinforce++ pass@k estimator (Chen et al. 2021) SimpleRLZoo Oat-Zero Code-R1

Closest papers by meaning

• Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.96)
• On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation (ICLR 2026 · similarity 0.94)
• The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models (NeurIPS 2026 · similarity 0.94)
• Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.94)
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example (NeurIPS 2025 · similarity 0.93)
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.93)