Research map / Large language models / Reasoning and chain of thought
Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solver
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes rStar, a self-play mutual reasoning method that boosts small language models at inference via MCTS with five human-like reasoning actions plus a second SLM discriminator verifying trajectories by mutual consistency, without fine-tuning.
Problem
Small language models struggle with complex reasoning; self-exploration and self-rewarding during inference are unreliable for SLMs, so they fail to find and select correct reasoning trajectories without fine-tuning or stronger teachers.
Approach
Augments MCTS with a rich set of five human-like reasoning actions to generate diverse candidate trajectories, then uses a second SLM as discriminator that completes partial trajectories from sampled prefixes; trajectories whose completed answers agree with the original are mutually consistent and selected.
Key results
- On GSM8K, rStar raises LLaMA2-7B accuracy from 12.51% (few-shot CoT) to 63.91%, and Mistral-7B from 36.46% to 81.88%.
- rStar reaches 91.13% on GSM8K with LLaMA3-8B-Instruct, versus 74.53% few-shot CoT and 80.59% for RAP.
- Using GPT-4 as discriminator on LLaMA3-8B-Instruct gives 92.57% versus 91.13% with Phi3-mini discriminator.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use rStar with MCTS generation and mutual-consistency verification instead of SC@maj128 | LLaMA3-8B-Instruct · GSM8K | accuracy: 84.69 -> 91.13 | Helped |
| Use rStar instead of SC@maj128 | LLaMA3-8B-Instruct · GSM-Hard | accuracy: 31.16 -> 37.53 | Helped |
| Use rStar instead of SC@maj128 | LLaMA3-8B-Instruct · SVAMP | accuracy: 90.60 -> 94.29 | Helped |
| Use rStar instead of RAP | LLaMA3-8B-Instruct · StrategyQA | accuracy: 68.71 -> 71.57 | Helped |
| Use rStar instead of SC@maj128 | Phi3-mini-4k · MATH-500 | accuracy: 45.60 -> 48.60 | Helped |
| Add A1 to A2+A3+A4+A5 (ablation) | LLaMA3-8B rStar action space · GSM8K (200 sampled questions) | accuracy: 74.0 -> 75.0 | Helped |
| Add A2 to A3+A4+A5 (ablation) | LLaMA3-8B rStar action space · GSM8K (200 sampled questions) | accuracy: 73.5 -> 74.0 | Helped |
| Add A4 to A3+A5 (ablation) | LLaMA3-8B rStar action space · GSM8K (200 sampled questions) | accuracy: 72.5 -> 73.5 | Helped |
| Add A5 to A3 (ablation) | LLaMA3-8B rStar action space · GSM8K (200 sampled questions) | accuracy: 70.5 -> 72.5 | Helped |
| Replace the proposed reward with intermediate-action self-evaluation (ablation) | LLaMA3-8B generator · GSM8K | accuracy: 85.52 -> 82.18 | Hurt |
| Replace the proposed reward with intermediate-action self-evaluation (ablation) | LLaMA3-8B-Instruct generator · StrategyQA | accuracy: 71.57 -> 69.43 | No clear effect |
| Replace mutual-consistency discrimination with self-verification (ablation) | LLaMA3-8B generator · GSM8K | accuracy: 85.52 -> 75.52 | Hurt |
Surprising findings
- Self-evaluation rewards from SLMs are near-random; replacing RAP's self-evaluation score with random values barely changes accuracy.
- Self-verification by the SLM itself is ineffective, while the cross-model mutual discriminator substantially improves accuracy.
Limitations
- High inference cost from MCTS with many model calls per question (about 150-170 calls per GSM8K question).
- Relies on a second SLM of similar capability; discriminator choice matters and the method is evaluated mostly on math tasks.
- Self-rewarding for intermediate nodes is avoided; reward depends on majority voting confidence which requires many traces to be correct.
Experimental setup
Datasets: GSM8K GSM-Hard MATH SVAMP StrategyQA MultiArith
Benchmarks: GSM8K GSM-Hard MATH-500 SVAMP StrategyQA MultiArith
Metrics: accuracy
Baselines: Zero-shot CoT Few-shot CoT Self-consistency (SC@maj8/64/128) ToT RAP Self-verification MetaMath
Builds on: RAP Monte Carlo Tree Search self-consistency least-to-most prompting chain-of-thought
Closest papers by meaning
- rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking (ICML 2025 · similarity 0.90)
- rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset (NeurIPS 2025 · similarity 0.87)
- GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (ICLR 2025 · similarity 0.87)
- ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvement (ICLR 2025 · similarity 0.87)
- Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus (NeurIPS 2024 · similarity 0.87)
- Distilling LLM Agent into Small Models with Retrieval and Code Tools (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.