Research map / Large language models / Language, safety and interpretability
Which Attention Heads Matter for In-Context Learning?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Compares induction heads and function-vector (FV) heads across 12 decoder-only LMs (70M–7B), finding few-shot ICL depends mainly on FV heads (especially at scale), and that many FV heads evolve from induction heads during training.
Problem
Two proposed mechanisms explain in-context learning: induction heads that copy tokens and function-vector heads that encode the task. It is unclear which drives few-shot ICL, and prior conclusions conflicted.
Approach
Measures induction and FV scores per head, performs ablations with "exclusion" to control for overlap between the two head sets, separates few-shot ICL accuracy from token-loss difference, and tracks individual heads over training checkpoints to find that FV heads often arise from induction heads.
Key results
- Ablating top 2% FV heads substantially degrades few-shot ICL accuracy, while ablating induction heads with low FV scores has effects comparable to random ablation in models above 1B parameters.
- Seven of twelve models show zero overlap between top 2% induction and FV heads; the remaining show 5-15% overlap.
- Induction heads emerge around step 1,000 of 143,000 in Pythia training, while FV heads appear around step 16,000; many FV heads show high induction scores earlier in training, but the reverse is not observed.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Preserve FV heads rather than ablate them, excluding top induction heads (ablation) | Decoder-only transformer models · Few-shot ICL accuracy on 19 natural language tasks | accuracy | Helped |
| Preserve induction heads with low FV scores rather than ablate them (ablation) | Models exceeding 1B parameters · Few-shot ICL accuracy on 19 natural language tasks | accuracy | No clear effect |
| Preserve induction heads rather than ablate them (ablation) | Models below 160M parameters · Token-loss difference | token-loss difference | No clear effect |
| Preserve FV heads rather than ablate them (ablation) | Models below 160M parameters · Token-loss difference | token-loss difference | No clear effect |
| Preserve induction heads rather than ablate them (ablation) | Models over 345M parameters · Token-loss difference | token-loss difference | Helped |
| Preserve FV heads rather than ablate them (ablation) | Models over 345M parameters · Token-loss difference | token-loss difference | Helped |
| Increase model scale while comparing FV-head and induction-head ablations (ablation) | Decoder-only transformer models · Few-shot ICL accuracy | accuracy | Mixed |
| Ablate induction heads without excluding heads that also score highly for FV (ablation) | Decoder-only transformer models · Few-shot ICL accuracy | accuracy | Hurt |
| Ablate induction heads while excluding the top 2% FV heads (ablation) | Decoder-only transformer models · Few-shot ICL accuracy | accuracy | No clear effect |
Surprising findings
- Ablating induction heads matters more than FV heads for token-loss difference in models above 345M, reversing the pattern seen for few-shot ICL accuracy.
- Previously reported induction-head importance for ICL is largely explained by overlap with FV-like heads rather than induction behavior alone.
Limitations
- Conjecture that induction heads are precursors to FV heads is speculative and does not explain FV heads with low induction scores throughout training.
- Pythia 70M and Llama 2 show unusually low FV scores, with causes unconfirmed.
- Relies on top-2% head thresholds and a fixed set of ICL tasks, with results on Pythia checkpoints only for training dynamics.
Experimental setup
Datasets: 37 natural language ICL tasks (Todd et al.) 8 new ICL tasks (4 variants, 4 binding tasks) The Pile (token-loss difference)
Benchmarks: 19 natural language ICL tasks (few-shot accuracy) The Pile token-loss difference
Metrics: few-shot ICL accuracy token-loss difference induction score FV score
Baselines: random head ablation token-loss difference (Olsson et al. metric)
Builds on: induction heads function vectors causal mediation analysis TransformerLens
Closest papers by meaning
- Capturing In-Context Learning Dynamics with Task Operators (NeurIPS 2026 · similarity 0.90)
- Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning (NeurIPS 2025 · similarity 0.89)
- Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers (NeurIPS 2024 · similarity 0.89)
- Dual Operating Modes of In-Context Learning (ICML 2024 · similarity 0.89)
- Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability (ICLR 2024 · similarity 0.89)
- Linking In-context Learning in Transformers to Human Episodic Memory (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.