Research map / Large language models / Language, safety and interpretability
Reasoning mechanisms and failure modes
Analyze symbolic mechanisms, attention glitches, and hallucination in LLM reasoning.
114 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Language, safety and interpretability, keyword cluster “languages · pretraining · multilingual”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
25 papers in 2023–24 and 76 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 3.6% to 4.1%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Deep Learning for Predicting Human Strategic Behavior (NeurIPS 2016 · 63 citations)
- 2018 · Measuring abstract reasoning in neural networks (ICML 2018 · 141 citations)
- 2019 · Abstract Reasoning with Distracting Features (NeurIPS 2019 · 30 citations)
- 2020 · Permutation Equivariant Models for Compositional Generalization in Language (ICLR 2020 · 65 citations)
- 2021 · Contextual Conditional Reasoning (AAAI 2021 · 3 citations)
- 2022 · Linking Emergent and Natural Languages via Corpus Transfer (ICLR 2022 · 7 citations)
- 2023 · Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task (ICLR 2023 · 62 citations)
- 2024 · A Tale of Tails: Model Collapse as a Change of Scaling Laws (ICML 2024 · 39 citations)
- 2025 · Improving Instruction-Following in Language Models through Activation Steering (ICLR 2025 · 8 citations)
- 2026 · When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models (AAAI 2026 · 9 citations)
Earliest papers here
- Deep Learning for Predicting Human Strategic Behavior (NeurIPS 2016 · 63 citations)
- Measuring abstract reasoning in neural networks (ICML 2018 · 141 citations)
- Abstract Reasoning with Distracting Features (NeurIPS 2019 · 30 citations)
What has been tried
63 papers here report 601 measured changes (a component added, removed or swapped, compared with a baseline): helped 55%, no clear effect 11%, hurt 27%, mixed 7%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add random effects for each participant and embedding dimension | horizon task, participant-level · negative log-likelihood | 25968.6 -> 23929.5 | Helped | Turning large language models into cognitive models (ICLR 2024 · 21 citations) |
| Zero out late attention heads | 14-task average, permuted labels · calibrated accuracy | 37.42 (1.47) -> 47.58 (1.47) | Helped | Overthinking the Truth: Understanding how Language Models Process False Demonstrations (ICLR 2024 · 6 citations) |
| suppress SAS until 3000 steps, then release the regularizer | 100K training steps · MLM loss | 1.77 ± 0.01 -> 1.75 ± 0.01 | Helped | Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs (ICLR 2024 · 4 citations) |
| Use the confirmatory 2α model instead of the four-rate full-feedback model | 2AFC full feedback · posterior probability | 0.92 ± 0.03 | Helped | In-Context Learning Agents Are Asymmetric Belief Updaters (ICML 2024 · 4 citations) |
| Add GMamba and LMamba to the baseline inference module | Pong · score | 8 -> 18 | Helped | GLAM: Global-Local Variation Awareness in Mamba-based World Model (AAAI 2025 · 3 citations) |
| Train with Train-II queries only, without atomic triples | Test-II · accuracy | Helped | How do Transformers Learn Implicit Reasoning? (NeurIPS 2025 · 2 citations) | |
| Increase context length from a single token to t=16 | Concept detection · accuracy | Mixed | A Language Model’s Guide Through Latent Space (ICML 2024 · 2 citations) | |
| Ablate the top 10% of CMR-like heads instead of the same number of randomly selected heads | C4 sampled texts · ICL score | Hurt | Linking In-context Learning in Transformers to Human Episodic Memory (NeurIPS 2024 · 2 citations) | |
| Add symbol abstraction heads | Identity rules, ABA and ABB · probability assigned to correct answer | Helped | Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models (ICML 2025 · 1 citation) | |
| Add queried-rule locating heads to the circuit | Normal samples · Δ^C_normal/Δ_normal | -0.97 -> 0.94 | Helped | A Implies B: Circuit Analysis in LLMs for Propositional Logical Reasoning (NeurIPS 2025 · 1 citation) |
| Increase the number of descriptive properties | Descriptive type checks · performance | Hurt | A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language (ICLR 2025 · 1 citation) | |
| Mask question and option contents in demonstrations while retaining formatting words and final answers | MMLU · probability of choosing a label in ABCD | No clear effect | Understanding In-Context Learning from Repetitions (ICLR 2024 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 41 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MMLU · 3 of 41 papers
- ARC Challenge · 3 of 41 papers
- TinyStories · 2 of 41 papers
- GSM8K · 2 of 41 papers
- MATH · 2 of 41 papers
- CommonsenseQA · 2 of 41 papers
- MedMCQA · 2 of 41 papers
- ARC Easy · 2 of 41 papers
Baselines
- Random · 3 of 41 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 101 papers; from OpenAlex and the papers' affiliations).
- University of California, Berkeley · 6 papers
- Google · 5 papers
- Microsoft · 5 papers
- Google DeepMind · 5 papers
- Peking University · 5 papers
- Stanford University · 5 papers
- Tsinghua University · 4 papers
- EPFL · 3 papers
- Massachusetts Institute of Technology · 3 papers
- Harvard University · 3 papers
Use it
Download BibTeX of 33 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Pruning and circuits in LLMs (135 papers)
- Linguistic encoding in LLM representations (123 papers)
- Task vectors and steering (84 papers)
Elsewhere in the topic
- LLM unlearning and benchmarks (298 papers)
- LLM pruning and scaling data (285 papers)
- Controlled text generation (220 papers)
- Faithfulness of LLM explanations (169 papers)
- Data and evaluation methodology (136 papers)
- Uncertainty quantification for LLMs (136 papers)
- Statistical frameworks for LLM evaluation (124 papers)
- Safety alignment of LLMs (123 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).