Research map / Large language models / Reasoning and chain of thought
Chain-of-thought training and analysis
Train and analyze chain-of-thought reasoning, including its faithfulness and information content.
127 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2023–2026) · an approach in Reasoning and chain of thought, keyword cluster “chain thought · thought cot · cot reasoning”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 4 of 6 tracked benchmarks.
23 papers in 2023–24 and 104 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 17.7% to 7.3%.
Open questions the data points to
- GSM8K (accuracy): best went from 89.84 (up to 2024) to 99.9 in 2025 — Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning (NeurIPS 2025 · 1 citation)
- MultiArith (accuracy): best went from 96.3 (up to 2024) to 98.2 in 2025 — Rethinking Chain-of-Thought from the Perspective of Self-Training (ICML 2025 · 0 citations)
- CSQA (accuracy): best went from 74.4 (up to 2024) to 82.72 in 2025 — Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models (ICML 2025 · 0 citations)
- AQuA (accuracy): best went from 36.5 (up to 2024) to 70.1 in 2025 — Rethinking Chain-of-Thought from the Perspective of Self-Training (ICML 2025 · 0 citations)
- Papers report Zero-Shot-CoT + SC on SVAMP (accuracy) anywhere from 47.6 to 82.2: comparisons against it depend on whose number you take.
- Papers report Zero-Shot-CoT (+ SC) on SingleEq (accuracy (%)) anywhere from 68.11 to 88.8: comparisons against it depend on whose number you take.
What looks settled
- SVAMP (accuracy): best up to 2024 87.33, best in 2025–26 87.5 from 3 papers, at most 0.2% better, if at all.
- AddSub (accuracy): best up to 2024 94.7, best in 2025–26 88.4 from 2 papers, at most 0.2% better, if at all.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2023 · Automatic Chain of Thought Prompting in Large Language Models (ICLR 2023 · 233 citations)
- 2024 · KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning (AAAI 2024 · 38 citations)
- 2025 · Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns (AAAI 2025 · 15 citations)
- 2026 · When More is Less: Understanding Chain-of-Thought Length in LLMs (ICLR 2026 · 5 citations)
Earliest papers here
- Automatic Chain of Thought Prompting in Large Language Models (ICLR 2023 · 233 citations)
- Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting (NeurIPS 2023 · 107 citations)
- Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective (NeurIPS 2023 · 27 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
GSM8K · accuracy (higher is better)
16 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 62.8 | Automatic Chain of Thought Prompting in Large Language Models (ICLR 2023 · 233 citations) |
| 2024 | 89.84 | Instance-adaptive Zero-shot Chain-of-Thought Prompting (NeurIPS 2024 · 13 citations) |
| 2025 | 99.9 | Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning (NeurIPS 2025 · 1 citation) |
MultiArith · accuracy (higher is better)
7 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 93.2 | Automatic Chain of Thought Prompting in Large Language Models (ICLR 2023 · 233 citations) |
| 2024 | 96.3 | Get an A in Math: Progressive Rectification Prompting (AAAI 2024 · 3 citations) |
| 2025 | 98.2 | Rethinking Chain-of-Thought from the Perspective of Self-Training (ICML 2025 · 0 citations) |
SVAMP · accuracy (higher is better)
7 papers, 2023–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 69.5 | Automatic Chain of Thought Prompting in Large Language Models (ICLR 2023 · 233 citations) |
| 2024 | 87.33 | Instance-adaptive Zero-shot Chain-of-Thought Prompting (NeurIPS 2024 · 13 citations) |
| 2025 | 87.5 | Rethinking Chain-of-Thought from the Perspective of Self-Training (ICML 2025 · 0 citations) |
CSQA · accuracy (higher is better)
5 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 74.4 | Automatic Chain of Thought Prompting in Large Language Models (ICLR 2023 · 233 citations) |
| 2025 | 82.72 | Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models (ICML 2025 · 0 citations) |
AQuA · accuracy (higher is better)
4 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 36.5 | Automatic Chain of Thought Prompting in Large Language Models (ICLR 2023 · 233 citations) |
| 2025 | 70.1 | Rethinking Chain-of-Thought from the Perspective of Self-Training (ICML 2025 · 0 citations) |
AddSub · accuracy (higher is better)
4 papers, 2023–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 91.9 | Automatic Chain of Thought Prompting in Large Language Models (ICLR 2023 · 233 citations) |
| 2024 | 94.7 | Get an A in Math: Progressive Rectification Prompting (AAAI 2024 · 3 citations) |
What has been tried
64 papers here report 713 measured changes (a component added, removed or swapped, compared with a baseline): helped 65%, no clear effect 8%, hurt 22%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add knowledge graph grounding (ConceptNet subgraph encoder) to image+caption-context model | ScienceQA test · Avg accuracy | 87.74 -> 92.62 | Helped | KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning (AAAI 2024 · 38 citations) |
| remove QA-PCoT signals (train on QA-CoT only) | ScienceQA test · accuracy (avg %) | -5.76 (91.75 to 85.99) | Hurt | T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Large Language Model Signals for Science Question Answering (AAAI 2024 · 37 citations) |
| Use top-k, top-p, beam, or temperature decoding instead of greedy decoding | GSM8K · accuracy | 9.9% -> 4.9%; 6.4%; 6.7%; 7.5% | Hurt | Chain-of-Thought Reasoning Without Prompting (NeurIPS 2024 · 36 citations) |
| associable instruction tuning (LoRA on Oogiri-GO LoT instructions) | Oogiri-GO IT2T/I2T/T2T multilingual · avg accuracy/NDCG | +6.8, +6.7, +4.3 | Helped | Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation (CVPR 2024 · 22 citations) |
| Train CPO on uniform QA data rather than single-task data | Bamboogle · accuracy | 32.0 -> 35.2 | Helped | Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs (NeurIPS 2024 · 19 citations) |
| Use adaptive k | GSM8K · accuracy (%) | 27.45 -> 28.05 | Helped | Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns (AAAI 2025 · 15 citations) |
| Use few-shot CoT instead of zero-shot CoT | Reasoning datasets · performance | No clear effect | To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning (ICLR 2025 · 13 citations) | |
| Encourage diversity among self-generated exemplars | GSM8K · accuracy | 75.9% -> 77.8% | Helped | Large Language Models as Analogical Reasoners (ICLR 2024 · 13 citations) |
| Change IAP-ss candidate order to #9, 8, 5, 4, 3 rather than the default #1-9 | SVAMP · accuracy | 77.33 -> 63.66 | Hurt | Instance-adaptive Zero-shot Chain-of-Thought Prompting (NeurIPS 2024 · 13 citations) |
| Add distinct conditions for longer and shorter CoT during training | GSM8K · accuracy | 34.50 -> 36.92 | Helped | C3oT: Generating Shorter Chain-of-Thought Without Compromising Effectiveness (AAAI 2025 · 10 citations) |
| Add the user preference analyzer (PA) | ML100K · N@10 | 0.4283 -> 0.4411 | Helped | CoT4Rec: Revealing User Preferences Through Chain of Thought for Recommender Systems (AAAI 2025 · 10 citations) |
| Use incorrect rather than manual CoT examples | CoinFlip · accuracy | 98.89% -> 96.76% | Hurt | Chain of Thoughtlessness? An Analysis of CoT in Planning (NeurIPS 2024 · 9 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| Zero-Shot-CoT + SC | SVAMP | accuracy | 47.6 – 82.2 | 3 papers, e.g. AAAI 2025 and ICML 2025 |
| Zero-Shot-CoT (+ SC) | SingleEq | accuracy (%) | 68.11 – 88.8 | 2 papers, e.g. AAAI 2025 and ICML 2025 |
| Zero-Shot-CoT (+ SC) | MultiArith | accuracy (%) | 79.83 – 95.7 | 2 papers, e.g. AAAI 2025 and ICML 2025 |
| Zero-Shot-CoT + SC | GSM8K | accuracy | 70.7 – 79.2 | 2 papers, e.g. AAAI 2024 and ICML 2025 |
What reviewers will expect
Datasets and baselines named in the 104 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- GSM8K · 30 of 104 papers
- MATH · 17 of 104 papers
- CommonsenseQA · 9 of 104 papers
- GPQA · 8 of 104 papers
- StrategyQA · 7 of 104 papers
- MMLU · 7 of 104 papers
- MATH-500 · 7 of 104 papers
- MultiArith · 6 of 104 papers
Baselines
- Coconut · 7 of 104 papers
- Zero-Shot-CoT · 6 of 104 papers
- CoT · 6 of 104 papers
- GPT-4o · 5 of 104 papers
- SFT · 5 of 104 papers
- CODI · 5 of 104 papers
- Self-Refine · 4 of 104 papers
- GRPO · 4 of 104 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 127 papers; from OpenAlex and the papers' affiliations).
- Google · 6 papers
- Meta · 6 papers
- Chinese Academy of Sciences · 5 papers
- Harvard University · 5 papers
- Amazon · 5 papers
- University of Illinois, Urbana Champaign · 5 papers
- New York University · 4 papers
- The Chinese University of Hong Kong · 4 papers
- University of Electronic Science and Technology of China · 3 papers
- Singapore Management University · 3 papers
Use it
Download BibTeX of 25 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Long CoT dynamics and efficiency (136 papers)
- Search and self-correction in reasoning (123 papers)
- Theory of autoregressive reasoning (28 papers)
Elsewhere in the topic
- RL with verifiable rewards for reasoning (270 papers)
- Efficient and adaptive thinking (231 papers)
- Test-time compute scaling (205 papers)
- Routing, pruning and distillation for reasoning (135 papers)
- Efficient LLM serving (126 papers)
- Speculative decoding (92 papers)
- KV cache compression (45 papers)
- Process reward models (39 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).