Research map / Architectures and efficiency / Attention and transformers
What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains 2-layer attention-only transformers on a synthetic Omniglot few-shot task and uses a new JAX "artificial optogenetics" clamping framework to causally decompose induction head formation into three interacting subcircuits.
Problem
Induction heads emerge abruptly with a phase change in loss, but their diversity and the subcircuits driving their formation dynamics are poorly understood; prior progress measures are correlational.
Approach
Clamp (fix) subsets of activations throughout training, with gradients routed correctly, to causally isolate subcircuits; this reveals that induction head formation is driven by three smoothly evolving subcircuits (Layer 1 previous-token, QK match, copy) whose interaction yields the phase change.
Key results
- Single Layer 2 head ablations reveal additive induction heads: ablating Head 3 alone drops accuracy by about 1%, and keeping only Head 3 gives about 98% accuracy, similar to ablating the other heads together.
- Clamping Layer 1 and copy subcircuits shows the copy subcircuit (routing V and IH-copy) forms smoothly without internal phase change, and data-dependent delays are attributed to QK match (more classes) or copy (more labels).
- Clamped training converges with fewer sequences, and the JAX jit framework gives roughly 50x speedup (about 7 minutes vs about 330 minutes per run on a laptop).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multiple Layer 2 induction heads instead of restricting training to one head (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | training dynamics | Helped |
| Keep only Layer 2 Head 3 active throughout training (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | training dynamics | Hurt |
| Keep only Layer 2 Head 3 active in the trained network (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | accuracy: 98% | Hurt |
| Ablate Layer 1 head outputs while preserving Layer 2 activations (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | accuracy: 99.99% -> 99.98% | No clear effect |
| Ablate Layer 1 heads 1, 2, and 5 while preserving Layer 2 patterns (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | accuracy: 99.85% | No clear effect |
| Ablate Layer 1 heads 1, 2, and 5 while preserving Layer 2 values (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | accuracy: 69.87% | Hurt |
| Ablate Layer 1 heads 0, 3, 4, 6, and 7 while preserving Layer 2 patterns (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | accuracy: 34.27% | Hurt |
| Ablate Layer 1 heads 0, 3, 4, 6, and 7 while preserving Layer 2 values (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | accuracy: 99.32% | No clear effect |
| Ablate all Layer 1 heads while preserving Layer 2 attention patterns (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | accuracy: down to 55% | Hurt |
| Ablate all Layer 1 heads while preserving Layer 2 value vectors (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | accuracy: down to 11% | Hurt |
| Clamp one Layer 1 head to attend perfectly to the previous token (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | loss dynamics | No clear effect |
| Clamp Layer 1 attention and output (PT-attend and PT-copy) (ablation) | 2-layer attention-only transformer · Few-shot Omniglot task | phase-change duration: ≈7.5e4 iterations; compared to 2e5 | Helped |
Surprising findings
- Ablating any single induction head except the strongest has almost no effect on accuracy, so knockout ablations can be misleading due to redundancy.
- Networks use additional induction heads to learn faster during training even though a single head suffices at the end of training.
Limitations
- Small-scale setting: two-layer attention-only transformers on a synthetic few-shot task, not frontier LLMs.
- Clamping is restricted to activation-level clamps; weight-level clamping is confounded by routing/rotation identifiability.
- Clamped subcircuit analyses are costlier per iteration (about 3x) and require hand-designed clamps.
Experimental setup
Datasets: Omniglot
Benchmarks: held-out Omniglot classes (test exemplars) held-out label pairs (test relabel)
Metrics: accuracy cross-entropy loss induction head strength
Baselines: progress measures
Builds on: induction heads progress measures pattern-preserving ablation RoPE Omniglot
Closest papers by meaning
- Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence (ICML 2025 · similarity 0.92)
- Selective Induction Heads: How Transformers Select Causal Structures in Context (ICLR 2025 · similarity 0.91)
- Linking In-context Learning in Transformers to Human Episodic Memory (NeurIPS 2024 · similarity 0.89)
- Strategy Coopetition Explains the Emergence and Transience of In-Context Learning (ICML 2025 · similarity 0.89)
- Phase Transitions in Attention: A Bayesian Theory of Copy Head Emergence (NeurIPS 2026 · similarity 0.89)
- The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.