Research map / Large language models / Reasoning and chain of thought
Reasoning Planning for Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives accuracy bounds for majority voting and PRM-score aggregation, then trains EPIC, a contrastive two-tower router that picks a reasoning method per math question using a utility balancing accuracy and token cost.
Problem
Choosing the best reasoning method (sampling count, search strategy, aggregation) for each query under an accuracy-cost trade-off, rather than applying one method uniformly; prior work assumes more candidates always yield higher accuracy.
Approach
Theoretical bounds on how aggregation accuracy (majority vote, PRM sum, PRM max) changes with sample size N are used as a regularizer in a contrastive learning router that maps questions and reasoning-method embeddings into a shared space, selecting the top-1 method by similarity at inference.
Key results
- EPIC-0.25 reaches 86.4% accuracy on MATH500 with 1859.2 tokens, matching Best-of-16 (86.8%, 10036.2 tokens) at about 5x fewer tokens.
- EPIC-1.0 achieves 89.4% on MATH500 at 6921.7 tokens, versus 86.8% for Best-of-16 and an upper bound of 91.2% under the method universe.
- On GSM8K (trained on MATH), EPIC reaches 95.0% accuracy with 2085.5 tokens, above Best-of-16 (94.2%, 5019.2 tokens).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use EPIC-0.25 instead of Best-of-16 | MATH500 reasoning selection with Qwen2.5-Math-7B-Instruct · MATH500 | average token count: 10036.2 -> 1859.2 | Helped |
| use EPIC-0.25 instead of Beam-search | MATH500 reasoning selection with Qwen2.5-Math-7B-Instruct · MATH500 | accuracy: 85.2 -> 86.4 | Helped |
| use EPIC-0.25 instead of Beam-search | MATH500 reasoning selection with Qwen2.5-Math-7B-Instruct · MATH500 | average token count: 2638.1 -> 1859.2 | Helped |
| increase the accuracy weight from λ=0.25 to λ=1.0 | EPIC on MATH500 · MATH500 | accuracy: 86.4 -> 89.4 | Helped |
| increase the accuracy weight from λ=0.25 to λ=1.0 | EPIC on MATH500 · MATH500 | average token count: 1859.2 -> 6921.7 | Hurt |
| use EPIC instead of Best-of-8 | MATH-trained selector transferred to GSM8K · GSM8K test | accuracy: 94.3 -> 95.0 | Helped |
| use EPIC instead of Best-of-8 | MATH-trained selector transferred to GSM8K · GSM8K test | tokens: 2412.3 -> 2085.5 | Helped |
| use EPIC (λ=0.25) instead of 7B-Best-of-4 in a multi-model method pool | Cost-aware multi-model reasoning · Multi-model reasoning | cost: 17495.8 -> 8047.8 | Helped |
| use EPIC (λ=1.0) instead of 7B-Best-of-16 in a multi-model method pool | Cost-aware multi-model reasoning · Multi-model reasoning | accuracy: 86.8 -> 89.0 | Helped |
| use EPIC (λ=1.0) instead of 7B-Best-of-16 in a multi-model method pool | Cost-aware multi-model reasoning · Multi-model reasoning | cost: 70253.4 -> 35705.4 | Helped |
| increase representation dimension d (ablation) | EPIC with λ=0.5 · MATH500 | accuracy | Helped |
| replace the proposed utility function with an alternative functional form (ablation) | EPIC · MATH500 | performance | Hurt |
Surprising findings
- Theory and experiments indicate that more candidate samples do not always yield higher accuracy under fixed generation distributions, which motivates selecting methods per query.
Limitations
- Performance is capped by the capacity of the fixed universe of 81 reasoning methods (upper bound 91.2%).
- Bounds are valid only for large N; for small N the empirical accuracy is used as the regularization target.
- Cost is measured by token count, and the method embedding for new methods requires a textual description network left for future work.
Experimental setup
Datasets: MATH LiveCodeBench GSM8K
Benchmarks: MATH500 GSM8K LiveCodeBench
Metrics: accuracy average token count
Baselines: CoT-G Best-of-2 Best-of-4 Best-of-8 Best-of-16 MCTS Beam-search Random Allocation Offline Ada-BoK DRA-λ CL-λ DeepSeek-V3
Builds on: Self-Consistency best-of-N Monte Carlo Tree Search InfoNCE OpenR Ada-BoK
Closest papers by meaning
- Improving Factuality and Reasoning in Language Models through Multiagent Debate (ICML 2024 · similarity 0.88)
- TACT: Advancing Complex Aggregative Reasoning with Information Extraction Tools (NeurIPS 2024 · similarity 0.87)
- Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.87)
- Training Language Models to Reason Efficiently (NeurIPS 2025 · similarity 0.87)
- CodePlan: Unlocking Reasoning Potential in Large Language Models by Scaling Code-form Planning (ICLR 2025 · similarity 0.87)
- Scalable Best-of-N Selection for Large Language Models via Self-Certainty (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.