Research map / Large language models / Language, safety and interpretability
Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces In-Context Meta-Learning (ICML), a copy-task extension where the model must infer a task from context, and shows a 2-layer attention-only transformer passes through three circuit phases (bigram, semi-context, full-context) during training.
Problem
Induction heads explain ICL only when the answer appears in context; the mechanism by which transformers acquire meta-learning (task inference from examples) during training is unclear.
Approach
Extend the copy task to an in-context meta-learning setting where tasks define different item-label mappings, so the query answer must be inferred from context; track training dynamics via novel attention-based circuit metrics (bigram, label attention, chunk example) that reveal three phases with distinct circuits.
Key results
- Training of a 2-layer attention-only transformer on ICML shows three accuracy phases plateauing near 30-40%, ~75%, and 100%, with circuit metrics shifting at phase boundaries.
- With T=1 and p_B=1 the setup reduces to standard induction-head emergence as a single phase.
- In a multi-head (2-head) model, accuracy improves smoothly while circuit metrics reveal hidden abrupt changes: one head becomes NCC-like and the other FCC-like.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the full-context circuit (FCC), binding example pairs and attending to labels | ICML two-layer attention-only transformer · ICML setting (T=3) | accuracy: approximately 75% -> 100% | Helped |
| retain only the non-context circuit (NCC) (ablation) | ICML two-layer attention-only transformer · ICML setting (T=3) | accuracy: 100% -> 30--40% | Hurt |
| retain only the semi-context circuit (SCC) (ablation) | ICML two-layer attention-only transformer · ICML setting (T=3) | accuracy: 100% -> approximately 75% | Hurt |
| provide the full-context circuit (FCC) from the beginning (ablation) | ICML two-layer attention-only transformer · ICML setting (T=3) | training steps to 100% accuracy: single step | Helped |
| increase the number of tasks | ICML model · ICML setting | Phase 1 accuracy: approximately proportional to 1/T | Hurt |
| set the number of tasks to 1 instead of 3 | ICML model · ICML setting | number of learning phases | Helped |
| increase the number of classes to large values (e.g., K=128,256) | ICML model · ICML setting | Phase 2 visibility: K=128,256 | Hurt |
| increase within-class variation epsilon | ICML model · ICML setting | Phase 2 visibility: epsilon=1 | Hurt |
| increase class-sampling bias alpha | ICML model · ICML setting | emergence of NCC or SCC | Hurt |
| increase task-sampling bias beta | ICML model · ICML setting | average accuracy across tasks | No clear effect |
| increase the number of attention heads | ICML attention-only transformer · ICML setting | smoothness of accuracy improvement | Helped |
| randomly permute labels within context pairs | ICML model · random-label OOD evaluation | accuracy | Mixed |
Surprising findings
- Multi-head models show smooth accuracy curves even though individual heads still undergo abrupt circuit transitions (hidden circuit emergence).
- Models trained with random label permutations still achieve accuracy rising during the semi-context phase, offering an explanation for random-label robustness in LLMs.
Limitations
- Model is a small 2-layer attention-only transformer, far from large-scale LLMs.
- Real-LLM evidence is limited to attention analysis of GPT2-XL on SST2 2-shot prompts, with no training dynamics.
Experimental setup
Datasets: SST2 (GLUE)
Benchmarks: SST2 (GLUE)
Metrics: accuracy loss bigram attention label attention chunk example attention random-label accuracy
Baselines: induction head (single-task copy setting)
Builds on: induction heads Data distributional properties drive emergent in-context learning Transformers learn in-context by gradient descent
Closest papers by meaning
- What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation (ICML 2024 · similarity 0.92)
- In-Context Learning through the Bayesian Prism (ICLR 2024 · similarity 0.91)
- Competition Dynamics Shape Algorithmic Phases of In-Context Learning (ICLR 2025 · similarity 0.90)
- Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers (NeurIPS 2024 · similarity 0.90)
- Position: Do pretrained Transformers Learn In-Context by Gradient Descent? (ICML 2024 · similarity 0.89)
- How Do Nonlinear Transformers Learn and Generalize in In-Context Learning? (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.