Research map / Large language models / Agents, code and math
Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the synthetic FTCT task, where Transformers train on fragmented causal chains and must chain them at test time, showing few-shot CoT enables compositional reasoning and attributing it to an underlying in-context-learning and parent-retrieval program.
Problem
Whether Transformers can compose knowledge fragments seen separately in training into complete causal reasoning chains at test time, and what mechanisms enable this.
Approach
Train small GPT-2-style Transformers on synthetic causal-graph child chains (fragmented, with noise vertices) and test on full-length chains using few-shot CoT prompts; explain success via a learned program combining induction heads (in-context learning of vertex order) and attention-based parent retrieval, with a constructive expressivity proof.
Key results
- 3-layer 3-head Transformer reaches 0.996 whole-chain testing accuracy at depth N=5, child length M=3, while 1L1H reaches only 0.022.
- Compositional reasoning shows a phase transition at relative knowledge ratio lambda = M/N around 0.3.
- Probing accuracy for predicting parent vertex values is about 77-82% versus about 2-12% for non-parent positions across shot numbers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add few-shot CoT examples at test time | Transformers trained on FTCT · FTCT chained testing | whole chain accuracy | Helped |
| increase relative knowledge ratio to at least 0.3 | Transformers trained on FTCT · FTCT compositional reasoning | optimal few-shot testing performance | Helped |
| add a second Transformer layer and use at least 2 heads (ablation) | FTCT Transformer · FTCT testing (N=5, M≤3) | whole chain accuracy: 0.022 -> 1.000 | Helped |
| reduce the architecture to 2 layers and 1 head (ablation) | 3L3H FTCT Transformer · FTCT testing (N=5, M≤3) | whole chain accuracy: 0.996 -> 0.766 | Hurt |
| reduce the architecture to 1 layer and 2 heads (ablation) | 3L3H FTCT Transformer · FTCT testing (N=5, M≤3) | whole chain accuracy: 0.996 -> 0.776 | Hurt |
| reduce the architecture to 1 layer and 1 head (ablation) | 3L3H FTCT Transformer · FTCT testing (N=5, M≤3) | whole chain accuracy: 0.996 -> 0.022 | Hurt |
| replace Transformer attention with an MLP (ablation) | FTCT model · FTCT testing (N=5, M≤3) | values accuracy: 1.000 -> 0.346 | Hurt |
| use more than one CoT example | FTCT Transformers at test time · FTCT chained testing | whole chain accuracy | Mixed |
Surprising findings
- Zero-shot testing performance is poor, but few-shot CoT with complete out-of-distribution reasoning paths lets the model reason correctly.
- MLPs generalize vertex order at test time better than at training time, which the authors attribute to noise vertices in training data.
Limitations
- Results are on synthetic FTCT data and may not transfer to natural language LLMs, as the authors acknowledge.
- Relies on simplified Transformer architecture for theory and arithmetic operations limited to addition and subtraction.
Experimental setup
Datasets: FTCT synthetic dataset (fragmented training chains, chained testing chains)
Benchmarks: FTCT
Metrics: whole chain accuracy vertices accuracy values accuracy validation loss probing accuracy
Baselines: MLP single-layer Transformer (1L1H, 1L2H, 2L1H)
Builds on: chain-of-thought prompting induction heads in-context learning
Closest papers by meaning
- Grokking of Implicit Reasoning in Transformers: A Mechanistic Journey to the Edge of Generalization (NeurIPS 2024 · similarity 0.91)
- When can transformers reason with abstract symbols? (ICLR 2024 · similarity 0.90)
- Transformers Provably Learn Chain-of-Thought Reasoning with Length Generalization (NeurIPS 2025 · similarity 0.90)
- How do Transformers Learn Implicit Reasoning? (NeurIPS 2025 · similarity 0.89)
- Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning (ICLR 2026 · similarity 0.88)
- Transformers Provably Solve Parity Efficiently with Chain of Thought (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.