Research map / Large language models / Reasoning and chain of thought
Bag of Tricks for Inference-time Computation of LLM Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Benchmarks inference-time computation methods for LLM reasoning and studies "tricks" (prompt type, temperature, top-p, reward model type, self-evaluation) across eight reasoning tasks using Llama, Qwen, and Mistral models.
Problem
Inference-time computation methods (Best-of-N, beam search, MCTS, self-consistency, self-refine) are sensitive to implementation details and have inconsistent effectiveness across reasoning tasks, lacking standardized evaluation.
Approach
Systematically ablates low-attention configuration choices (instruction prompt, temperature, top-p, reward type, self-evaluation) and benchmarks six inference-time computation methods under a fixed token budget on a common set of tasks, reporting that simple tricks such as temperature tuning can shift accuracy by several points.
Key results
- Tuning temperature to 0.8 improves reasoning accuracy by roughly 2.32% to 4.83% across four datasets relative to other settings, per the authors.
- Under a fixed token budget, Self-Consistency and Best-of-N achieve the highest accuracy on several tasks; e.g., Llama-3.1-8B Best-of-N reaches 94.0% on PrOntoQA and Qwen-2.5-7B Self-Consistency reaches 70.2% on MATH.
- Self-Refine underperforms Self-Consistency broadly; e.g., Self-Refine on FEVER drops to 19.1% (Llama-3.1-8B) and 16.8% (Qwen-2.5-7B).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Tune inference temperature to 0.8 (ablation) | Best-of-N reasoning · four reasoning datasets | accuracy: 2.32% to 4.83% | Helped |
| Tune top-p to 0.9 (ablation) | Best-of-N reasoning · reasoning tasks | accuracy: 2.32%–5.88% | Helped |
| Use CoT instead of IO prompting (ablation) | LLM inference-time reasoning · Bamboogle and MATH | accuracy (%): 42.7 -> 52.4; 11.8 -> 58.4 | Helped |
| Add reflection-based CoT instead of standard CoT (ablation) | LLM inference-time reasoning · Bamboogle and MATH | accuracy (%): 52.4 -> 56.4; 58.4 -> 58.6 | Mixed |
| Use self-evaluation to select candidates instead of random selection or majority voting (ablation) | LLM inference-time reasoning · four benchmark tasks | accuracy | Mixed |
| Use process-based LLM-as-Judge rewards (ablation) | inference-time reasoning · complex reasoning, mathematics, and code generation | accuracy / pass@1 | Helped |
| Use Proof-Critical Reward while scaling test-time computation (ablation) | LLM reasoning · MATH | accuracy | Hurt |
| Combine Reflect CoT with temperature 0.8 and top-p 0.9 (ablation) | LLM inference-time reasoning · Bamboogle and MATH | accuracy | No clear effect |
| Use Self-Refine rather than Self-Consistency | LLM reasoning under fixed token budget · reasoning tasks | accuracy | Hurt |
| Increase completion tokens with Beam Search | LLM reasoning · reasoning tasks | accuracy | No clear effect |
| Use Best-of-N instead of other inference-time methods | LLM reasoning · knowledge-based reasoning tasks | accuracy | Helped |
Surprising findings
- Self-evaluation can perform worse than random selection in some settings.
- Improvements from combining tricks are not additive; Reflect CoT with temperature 0.8 did not help on Bamboogle and MATH.
Limitations
- Findings may not generalize to all datasets and models, as the authors note.
- Reward models show generalization issues, producing inconsistent gains across tasks and performance inflation.
- Combined tricks are not reliably additive.
Experimental setup
Datasets: GSM8K GSM-Hard MATH500 PrOntoQA HumanEval Bamboogle FEVER HotpotQA
Benchmarks: GSM8K GSM-Hard MATH500 PrOntoQA HumanEval Bamboogle FEVER HotpotQA
Metrics: accuracy pass@1 token consumption
Baselines: Best-of-N Step-Level Best-of-N Beam Search MCTS Self-Consistency Self-Refine
Builds on: Best-of-N Self-Consistency Monte Carlo Tree Search Self-Refine Chain-of-Thought
Closest papers by meaning
- Fixing the Broken Compass: Diagnosing and Improving Inference-Time Reward Modeling (ICLR 2026 · similarity 0.93)
- Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving (ICLR 2025 · similarity 0.92)
- A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning (NeurIPS 2025 · similarity 0.91)
- Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning (NeurIPS 2025 · similarity 0.91)
- A State-Transition Framework for Efficient LLM Reasoning (ICLR 2026 · similarity 0.91)
- RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning (ICLR 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.