Research map / Large language models / Reasoning and chain of thought

MATH-Beyond: A Benchmark for RL to Expand Beyond the Base Model

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MATH-Beyond, a math benchmark of problems that ≤8B open-weight base models fail at pass@1024, and uses it to measure whether RL post-training expands reasoning beyond the base model.

Problem

RL fine-tuning of LLMs for math mostly sharpens solutions that base models already reach under repeated sampling; existing benchmarks such as MATH-500 and AIME 2024 are saturated at pass@1024, so they cannot measure genuine reasoning-boundary expansion.

Approach

Build a benchmark by filtering DAPO-Math-17K and DeepScaleR problems to those unsolved at pass@1024 by a suite of ≤8B open-source base and supplementary models, verified against frontier models, and define an Expansion Rate metric (new problems solved by the post-trained model relative to its base model's unsolved set).

Key results

• Under pass@1024, RL models on the r1-1.5b base solve only 5.2-9.6% of base-unsolved MATH-B problems (Nemotron v1 7.83%, v2 9.57%, DeepScaleR 5.22%).
• Skywork-OR1-7B reaches a 21.21% expansion rate on the 99 problems unsolved by its R1-Distill-Qwen2.5-7B base.
• Long CoT distilled Qwen3-4B and Qwen3-8B reach expansion rates of 58.93% and 66.38%, far above the RL models.

Surprising findings

• Qwen2.5-based base models that appear strong on MATH-500 and AIME24 nearly all fail on MATH-B at pass@1024.
• Problems judged medium human difficulty (median 4/10, max 6.5 for intersection) are still robustly unsolved by models, showing a mismatch between human and model difficulty.

Limitations

• Only models up to 8B are evaluated; some items may not be zero-baseline for larger open-weight models.
• Benchmark is small (41 intersection / 181 union problems), so expansion-rate estimates are noisy.
• Rule-based verification pitfalls may remain despite filtering; answers restricted to integers.

Experimental setup

Datasets: DAPO-Math-17K DeepScaleR NuminaMath-1.5

Benchmarks: MATH-Beyond-Union (181 problems) MATH-Beyond-Intersection (41 problems) MATH-500 AIME 2024

Metrics: pass@1024 expansion rate pass@k shrinkage rate preservation rate consolidation

Baselines: Nemotron-Research-Reasoning-Qwen-1.5B DeepScaleR-1.5B-Preview Skywork-OR1-7B Qwen3-4B Qwen3-8B

Builds on: GRPO DAPO-Math-17K DeepScaleR Omni-MATH

Closest papers by meaning

• AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning (NeurIPS 2025 · similarity 0.93)
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example (NeurIPS 2025 · similarity 0.92)
• Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (NeurIPS 2025 · similarity 0.91)
• DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning (ICLR 2026 · similarity 0.91)
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.90)