Research map / Large language models / Agents, code and math
Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Guru, a 92K multi-domain RL reasoning dataset across six domains with domain-specific verifiers, studies cross-domain RL transfer, and trains Guru-7B/32B from Qwen2.5 base models that outperform open baselines on 17 benchmarks.
Problem
RL for LLM reasoning is concentrated on math and code, and there is no curated multi-domain RL dataset with reliable verifiers, so RL's cross-domain behavior is poorly understood.
Approach
Build a 92K-example RL corpus spanning Math, Code, Science, Logic, Simulation and Tabular with per-domain verifiers and a curation pipeline (dedup, heuristic filtering, difficulty filtering using weak/strong model pass rates), then use it to analyze domain-dependent RL transfer and train general reasoning models.
Key results
- Guru-7B averages 43.29 and Guru-32B averages 54.24 across 17 benchmarks, outperforming the best open baselines by about 9.0% and 6.7% respectively (text gives both 7.9% and 9.0% for 7B).
- Math, Code and Science show gains from cross-domain RL, while Logic, Simulation and Tabular need in-domain data.
- Difficulty-filtered math data raised AIME24 by +5.9 and AMC by +6.3 but reduced HumanEval by -9.2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with mixed-domain Guru data rather than the strongest reported baseline | Qwen2.5-7B-Base · 17-benchmark average | average score: 35.42 -> 43.29 | Helped |
| Train with mixed-domain Guru data rather than the strongest reported baseline | Qwen2.5-32B-Base · 17-benchmark average | average score: 47.53 -> 54.24 | Helped |
| Add difficulty filtering to Math RL training data (ablation) | Qwen2.5-7B-Base · MATH500 | accuracy: 75.8 -> 78.6 (+2.8) | Helped |
| Add difficulty filtering to Math RL training data (ablation) | Qwen2.5-7B-Base · AMC | accuracy: 52.1 -> 58.4 (+6.3) | Helped |
| Add difficulty filtering to Math RL training data (ablation) | Qwen2.5-7B-Base · AIME24 | accuracy: 15.8 -> 21.7 (+5.9) | Helped |
| Add difficulty filtering to Math RL training data (ablation) | Qwen2.5-7B-Base · HumanEval | accuracy: 82.3 -> 73.1 (-9.2) | Hurt |
| Add difficulty filtering to Math RL training data (ablation) | Qwen2.5-7B-Base · LiveCodeBench | accuracy: 11.1 -> 10.7 (-0.4) | Hurt |
| Add difficulty filtering to Math RL training data (ablation) | Qwen2.5-7B-Base · HiTab | accuracy: 56.5 -> 53.5 (-3.0) | Hurt |
| Add difficulty filtering to Math RL training data (ablation) | Qwen2.5-7B-Base · MultiHiertt | accuracy: 32.0 -> 35.5 (+3.5) | Helped |
Surprising findings
- Difficulty filtering of training data improves in-domain math but causes negative transfer to easier cross-domain tasks such as HumanEval.
- On Code, Logic and Tabular single-domain RL shortens responses, contrary to the common belief that RL lengthens outputs.
Limitations
- Training covers only 3 epochs for 7B and 2 epochs for 32B due to compute limits.
- Training starts directly from base models without SFT, so results may not transfer to SFT-then-RL pipelines.
- Only Python is used for code; no other languages are covered.
Experimental setup
Datasets: OR1 DAPO-Math-17k DeepScaleR LeetCode TACO-Verified PrimeIntellect LiveCodeBench WebInstruct-Verified ARC-AGI ARC-AGI-2 BARC Code I/O (PyEdu)
Benchmarks: AIME24 MATH500 AMC LiveCodeBench HumanEval MBPP GPQA-diamond SuperGPQA ARC-AGI Zebra Puzzle CodeI/O CruxEval-I
Metrics: accuracy pass@k average score
Baselines: General-Reasoner 7B Open-Reasoner-Zero 7B SimpleRL 7B Open-Reasoner-Zero 32B SimpleRL 32B
Builds on: GRPO verl DeepScaler Open-Reasoner-Zero SimpleRL-Zoo
Closest papers by meaning
- Tricks or Traps? A Deep Dive into RL for LLM Reasoning (ICLR 2026 · similarity 0.94)
- RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.94)
- Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning (NeurIPS 2025 · similarity 0.93)
- Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs (ICLR 2026 · similarity 0.93)
- On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.93)
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.93)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.