Research map / Large language models / Language, safety and interpretability
Revisiting In-context Learning Inference Circuit in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a three-step inference circuit (input text encode, semantics merge, feature retrieval and copy) that explains ICL inference phenomena in large LMs such as Llama 3 70B.
Problem
Existing explanations of in-context learning inner mechanisms (often based on synthetic or toy settings) fail to capture the full set of inference phenomena observed in large language models.
Approach
Decomposes ICL inference into three atomic operations on Transformer layers (input text encoding on forerunner tokens, semantics merge of demonstration text into label tokens, and induction-style feature retrieval and copy to the query), validated with kernel alignment, centroid probes, head counting, and ablations on real LLMs.
Key results
- Ablating the proposed Step 3 (label to query forerunner attention) on Llama 3 8B with 100% of layers drops accuracy from 68.55 to about 58.3 (-10.22), versus -0.06 for random controls.
- Early-exit with a centroid classifier at layer 34 of Llama 3 70B reaches 84.27% accuracy at 2.38x speed, versus 66.19% with the full model plus LM head.
- Forerunner tokens produce the best input text encoding (kernel alignment) among the token types, emerging in early layers and staying high to the end of inference.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a centroid classifier instead of the LM head | Full ICL inference on Llama 3 70B · ICL classification (average across six datasets) | accuracy: 66.19% -> 83.24% | Helped |
| Prune layers after Layer34 and use a centroid classifier | ICL inference on Llama 3 70B · ICL classification (average across six datasets) | accuracy: 83.24% -> 84.27% | Helped |
| Disconnect demo text x_i → forerunner s_i attention (ablation) | ICL on Llama 3 8B · ICL classification | accuracy change: -4.98, -15.82, -23.43, -30.60 | Hurt |
| Disconnect query text x_q → forerunner s_q attention (ablation) | ICL on Llama 3 8B · ICL classification | accuracy change: -13.87, -21.10, -24.74, -28.38 | Hurt |
| Disconnect demo forerunner s_i → label y_i attention (ablation) | ICL on Llama 3 8B · ICL classification | accuracy change: -2.24, -3.45, -3.39, -3.42 | Hurt |
| Disconnect label y_i → query forerunner s_q attention (ablation) | ICL on Llama 3 8B · ICL classification | accuracy change: -5.14, -10.03, -11.36, -10.22 | Hurt |
| Disconnect forerunner-to-forerunner shortcut attention (ablation) | ICL on Llama 3 70B · ICL classification | accuracy change: -1.30, -0.75, -0.78, -1.56 | Hurt |
Surprising findings
- Ablating induction (Step 3) leaves accuracy above the zero-shot level, contrary to the expectation that it should match zero-shot performance, suggesting bypass mechanisms.
- Forerunner token heads copy text representations to label tokens without selectivity toward label correctness, yet label denoising emerges from subspace overlap.
Limitations
- Only classification tasks are studied; extension to non-classification tasks is untested.
- Results may not hold for in-weight learning scenarios without ground-truth labels in context.
- Operations are not functionally indivisible; the circuit is not reduced to a minimal set of attention heads.
Experimental setup
Datasets: SST-2 MR Financial Phrasebank SST-5 TREC AGNews
Benchmarks: SST-2 MR Financial Phrasebank SST-5 TREC AGNews
Metrics: accuracy kernel alignment centroid classification accuracy accuracy drop under ablation
Baselines: zero-shot random prediction PPL-ICL ICCL
Builds on: induction circuit mutual nearest-neighbor kernel alignment centroid classifier probing
Closest papers by meaning
- Capturing In-Context Learning Dynamics with Task Operators (NeurIPS 2026 · similarity 0.92)
- In-Context Learning Learns Label Relationships but Is Not Conventional Learning (ICLR 2024 · similarity 0.91)
- Implicit In-context Learning (ICLR 2025 · similarity 0.91)
- In-Context Language Learning: Architectures and Algorithms (ICML 2024 · similarity 0.91)
- Many-Shot In-Context Learning (NeurIPS 2024 · similarity 0.90)
- The Representation Landscape of Few-Shot Learning and Fine-Tuning in Large Language Models (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.