Research map / Large language models / Reasoning and chain of thought
General-Reasoner: Advancing LLM Reasoning Across All Domains
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes General-Reasoner, a Zero-RL training paradigm using a 230K-question multi-domain web-crawled verifiable dataset (WebInstruct-verified) and a 1.5B generative CoT answer verifier to replace rule-based rewards, reporting gains across 12 general and math benchmarks.
Problem
RL for LLM reasoning focuses on math and code because of data abundance and easy rule-based answer verification; rule-based verifiers fail on diverse non-math answer formats, limiting generalization to other domains.
Approach
Combine a large, filtered multi-domain set of web-crawled questions with short answers and a compact generative verifier (fine-tuned Qwen2.5-Math-1.5B) that judges answer equivalence with chain-of-thought, used as the GRPO reward in zero-RL training from base models.
Key results
- General-Reasoner-Qw3-14B reaches 70.3 on MMLU-Pro, 56.1 on GPQA-Diamond and 54.4 on TheoremQA, above GPT-4o's reported 74.6/50.0/43.6 on the latter two benchmarks (GPT-4o is higher on MMLU-Pro).
- The model-based verifier agrees with Gemini-2.0-Flash on 78.7% of Gemini-correct pairs versus 22.2% for the rule-based verifier.
- Training on the full diverse data beats math-only training on MMLU-Pro (58.9 vs 56.9 for 7B) and SuperGPQA (34.2 vs 29.8 for 7B).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with full diverse-domain data instead of math-only data (ablation) | Qwen2.5-7B Zero-RL · MMLU-Pro | accuracy: 56.9 -> 58.9 | Helped |
| Train with full diverse-domain data instead of math-only data (ablation) | Qwen2.5-7B Zero-RL · GPQA | accuracy: 32.8 -> 34.3 | Helped |
| Train with full diverse-domain data instead of math-only data (ablation) | Qwen2.5-7B Zero-RL · SuperGPQA | accuracy: 29.8 -> 34.2 | Helped |
| Train with full diverse-domain data instead of math-only data (ablation) | Qwen2.5-7B Zero-RL · Math-Related | accuracy: 49.1 -> 48.5 | Hurt |
| Train with full diverse-domain data instead of math-only data (ablation) | Qwen2.5-14B Zero-RL · MMLU-Pro | accuracy: 64.8 -> 66.6 | Helped |
| Train with full diverse-domain data instead of math-only data (ablation) | Qwen2.5-14B Zero-RL · GPQA | accuracy: 38.9 -> 43.4 | Helped |
| Train with full diverse-domain data instead of math-only data (ablation) | Qwen2.5-14B Zero-RL · SuperGPQA | accuracy: 35.6 -> 39.5 | Helped |
| Train with full diverse-domain data instead of math-only data (ablation) | Qwen2.5-14B Zero-RL · Math-Related | accuracy: 48.6 -> 53.9 | Helped |
| Use the generative General-Verifier instead of a rule-based verifier (ablation) | Qwen3-4B Zero-RL trained for 120 steps · MMLU-Pro | accuracy: approximately 58% -> about 60% | Helped |
| Use the generative General-Verifier instead of a rule-based verifier | answer verification · 50k Gemini-confirmed-correct answer-verification pairs | agreement: 22.2% -> 78.7% | Helped |
| Apply Zero RL with domain-diverse data and a generative verifier | Qwen2.5-7B-Base · MMLU-Pro | accuracy: 47.7 -> 58.9 | Helped |
| Apply Zero RL with domain-diverse data and a generative verifier | Qwen2.5-14B-Base · MMLU-Pro | accuracy: 53.3 -> 66.6 | Helped |
Surprising findings
- Math-only training with the 7B backbone gives a slightly higher math-related average than full-data training (49.1 vs 48.5).
Limitations
- Focuses on Zero RL; combination with SFT or distillation not explored.
- Does not specifically target code reasoning or olympiad-level math.
- Remaining gaps versus closed-source or closed-data models on some benchmarks.
Experimental setup
Datasets: WebInstruct-verified
Benchmarks: MMLU-Pro GPQA-Diamond SuperGPQA TheoremQA BBEH MATH-500 Olympiad Minerva GSM8K AMC AIME24 AIME25
Metrics: accuracy micro-average macro-average agreement with Gemini-2.0-Flash
Baselines: SimpleRL-Zoo Open-Reasoner-Zero Nemotron-CrossThink Qwen2.5-Instruct Qwen3-Instruct (non-think) rule-based verifier (Math-Verify)
Builds on: GRPO DeepSeek-R1-Zero WebInstruct SimpleRL Open-Reasoner-Zero
Closest papers by meaning
- Reinforcing General Reasoning Without Verifiers (ICLR 2026 · similarity 0.93)
- Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning (ICLR 2026 · similarity 0.93)
- Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.92)
- Position: General Intelligence Requires Reward-based Pretraining (ICML 2025 · similarity 0.91)
- Advancing LLM Reasoning with Natural Language and Numerical Feedback (ICML 2026 · similarity 0.91)
- QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation (ICLR 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.