Research map / Architectures and efficiency / Attention and transformers
Grokking of Implicit Reasoning in Transformers: A Mechanistic Journey to the Edge of Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains GPT-2-style transformers from scratch on synthetic knowledge graphs to study implicit composition and comparison, showing grokking enables ID generalization and mechanistic circuits explain OOD (systematicity) differences.
Problem
Whether transformers can learn implicit reasoning over parametric knowledge (composition and comparison), which LLMs struggle with, and what mechanisms govern its generalization and systematicity.
Approach
Controlled synthetic data with atomic and inferred facts lets the authors show implicit reasoning emerges only via grokking, that the inferred/atomic ratio (not data size) drives grokking speed, and that the generalizing circuit configuration (cross-layer storage for composition vs parallel retrieval for comparison) determines OOD systematicity.
Key results
- Composition: ID generalization only via grokking (highest ID accuracy 9.2% before saturation, near-perfect after ~50x steps), OOD accuracy stays zero even after 2M steps.
- Comparison: transformer achieves both ID and OOD generalization through grokking, with a parallel circuit.
- Complex reasoning task: grokked transformer reaches 99.3% accuracy vs. 33.3% random chance best for GPT-4-Turbo (Direct+R 33.3%) and Gemini-Pro-1.5 (best 37.3%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the vanilla Transformer far beyond training-set saturation | Vanilla Transformer on composition · Composition test_inferredID | accuracy: 9.2% before extended training; approaches almost perfect accuracy after extended optimization | Helped |
| Train the vanilla Transformer for longer | Vanilla Transformer on composition · Composition test_inferredOOD | accuracy: zero | No clear effect |
| Increase the inferred-to-atomic fact ratio (ablation) | Vanilla Transformer on composition · Composition test_inferredID | generalization speed | Helped |
| Increase training data size while holding the inferred-to-atomic ratio fixed (ablation) | Vanilla Transformer on composition · Composition test_inferredID | generalization behavior | No clear effect |
| Increase weight decay (ablation) | Vanilla Transformer on composition · Composition test_inferredID | generalization speed | Helped |
| Share parameters between the first 4 and last 4 layers (ablation) | Transformer on composition · Composition test_inferredOOD | accuracy | Helped |
| Scale up the Transformer (ablation) | Transformer on composition · Composition test_inferredID | generalization behavior | No clear effect |
| Increase entity token multiplicity (ablation) | Transformer on composition · Composition test_inferredID | generalization speed | Hurt |
| Increase the inferred-to-atomic fact ratio (ablation) | Vanilla Transformer on comparison · Comparison test (ID) | generalization speed | Helped |
| Use chain-of-thought prompting instead of direct answering | Gemini-Pro-1.5 on the complex reasoning task · Complex reasoning task (OOD test) | accuracy (%): 37.3 -> 12.0 | Hurt |
| Add retrieval augmentation to Gemini-Pro-1.5 direct answering | Gemini-Pro-1.5 on the complex reasoning task · Complex reasoning task (OOD test) | accuracy (%): 28.7 -> 37.3 | Helped |
| Add retrieval augmentation to Gemini-Pro-1.5 chain-of-thought answering | Gemini-Pro-1.5 on the complex reasoning task · Complex reasoning task (OOD test) | accuracy (%): 11.3 -> 12.0 | Helped |
Surprising findings
- Training data size does not affect grokking speed when the inferred/atomic ratio is fixed, contradicting the critical data size hypothesis.
- Gemini-Pro-1.5 performs worse with chain-of-thought prompting (12.0%) than direct answering with retrieval (37.3%) on the complex reasoning task.
Limitations
- Abstract synthetic setup with controlled data; distance from practical pretraining settings.
- Task formulation covers only induction and application of inference rules over facts, not the full range of reasoning.
- No error bars or statistical significance reported; results from specified random seeds.
Experimental setup
Datasets: synthetic knowledge graph composition facts synthetic attribute comparison facts
Benchmarks: synthetic composition ID/OOD test sets synthetic comparison ID/OOD test sets complex reasoning task (OOD entity comparison)
Metrics: accuracy mean reciprocal rank causal strength Recall@3
Baselines: GPT-4-Turbo Gemini-1.5-Pro
Builds on: grokking (Power et al.) circuit efficiency logit lens causal tracing Universal Transformer
Closest papers by meaning
- How do Transformers Learn Implicit Reasoning? (NeurIPS 2025 · similarity 0.92)
- Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data? (ICLR 2025 · similarity 0.91)
- Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers (ICML 2025 · similarity 0.91)
- When can transformers reason with abstract symbols? (ICLR 2024 · similarity 0.90)
- Transformers Provably Learn Chain-of-Thought Reasoning with Length Generalization (NeurIPS 2025 · similarity 0.90)
- Transformers Provably Solve Parity Efficiently with Chain of Thought (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.