Research map / Large language models / Reasoning and chain of thought
Toward Adaptive Reasoning in Large Language Models with Thought Rollback
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Thought Rollback (TR), a zero-shot prompting framework where LLMs analyze their reasoning chain, roll back to an earlier mistaken thought, and accumulate error analyses as experience in the prompt to build adaptive, cyclic thought structures.
Problem
Forward-only, unidirectional thought structures (chains, trees, graphs) propagate errors and hallucinations in LLM step-by-step reasoning, so a single wrong thought can derail the whole solution.
Approach
LLMs perform step-level error analysis on their thought chain, roll back to the step before the first identified bad thought, and add the accumulated error analyses as experiences in the prompt when regenerating from that point; solutions from multiple rollback paths are combined by experience-weighted majority voting.
Key results
- GPT-4 with TR + W-Voting reaches 72.11% on MATH (CoT5 variant) versus 66.33% for BoT+CoT, and 71.89% without CoT examples.
- On MATH level 5 (324 samples), GPT-4 + TR + W-Voting reaches 52.16% solve rate at about 62 interactions, versus 24.38% for GPT-4 ToT Reasoning at about 150 interactions.
- On Game of 24, GPT-4 with TR + W-Voting achieves 87% solve rate with about 32 interactions, compared with 74% for ToT (30 interactions) and 83.7% for BoT (724 interactions).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add weighted majority voting to TR solutions (ablation) | GPT-4 + TR · MATH | solve rate (%): 55 -> 71.89 | Helped |
| Add weighted majority voting to TR solutions (ablation) | GPT-4 + TR · Game of 24 | solving rate: 70 -> 87 | Helped |
| Add CoT5 examples to TR with weighted voting (ablation) | GPT-4 + TR + W-Voting · MATH | solve rate (%): 71.89 -> 72.11 | Helped |
| Add CoT5 examples to TR with weighted voting (ablation) | GPT-3.5-turbo + TR + W-Voting · MATH | solve rate (%): 39.78 -> 35.89 | Hurt |
| Add CoT5 examples to TR with weighted voting (ablation) | GPT-3.5-turbo + TR + W-Voting · AQuA-RAT | solve rate (%): 63.39 -> 62.6 | Hurt |
| Add CR demonstrations to TR with weighted voting (ablation) | GPT-4 + TR + W-Voting · Game of 24 | solving rate: 87 -> 93 | Helped |
| Use thought rollback rather than BoT | GPT-4 · AQuA-RAT | solve rate (%): +6.3 | Helped |
| Use thought rollback rather than BoT | GPT-4 · MATH | solve rate (%): +9.45 | Helped |
| Use TR with zero-shot prompting rather than PHP+C-CoT | GPT-4 · MATH | solve rate (%): 17.99 | Helped |
| Use TR with zero-shot prompting rather than PHP+C-CoT | GPT-3.5-turbo · MATH | solve rate (%): 3.28 | Helped |
| Use TR + W-Voting rather than ZeroShot-CoT | GPT-4 · MATH | solve rate (%): 27.19% | Helped |
| Use TR + W-Voting rather than ZeroShot-CoT | GPT-4 · TheoremQA | solve rate (%): 13% | Helped |
Surprising findings
- For GPT-3.5-turbo, adding CoT examples to TR significantly decreased performance on AQuA-RAT and MATH, which the authors attribute to weak LLMs struggling with multiple instructions in the prompt.
- TR's token cost is very high: on MATH, generated tokens are about 35x and prompt tokens about 425x those of zero-shot CoT.
Limitations
- Token cost is very high (tens to hundreds of times zero-shot CoT), limiting use for users with limited resources.
- Performance depends on the LLM's ability to perform correct error analysis; gains are small for weaker models such as GPT-3.5-turbo and Llama2.
- Hard problems lead to over-complex thought structures with more than 100 thoughts.
Experimental setup
Datasets: GSM8K SVAMP AQuA-RAT MATH TheoremQA Game of 24 MMLU
Benchmarks: GSM8K SVAMP AQuA-RAT MATH TheoremQA Game of 24 MMLU
Metrics: solve rate interaction number token cost
Baselines: Zero-shot CoT CoT Self-consistency Complex CoT Boosting of Thoughts Tree of Thoughts Progressive-Hint Prompting CSV Cumulative Reasoning Faithful-CoT
Builds on: chain-of-thought tree of thoughts boosting of thoughts progressive-hint prompting cumulative reasoning
Closest papers by meaning
- THOUGHT PROPAGATION: AN ANALOGICAL APPROACH TO COMPLEX REASONING WITH LARGE LANGUAGE MODELS (ICLR 2024 · similarity 0.90)
- Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models (ICLR 2024 · similarity 0.90)
- Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.90)
- Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models (ICML 2024 · similarity 0.89)
- Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.89)
- Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.