Research map / Architectures and efficiency / Attention and transformers
Circuit Component Reuse Across Tasks in Transformer Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that the GPT2-Medium circuit for Indirect Object Identification (IOI) is largely reused for a Colored Objects copying task (about 78% head overlap), and that intervening on inhibition and negative mover heads raises Colored Objects accuracy from 49.6% to 93.7%.
Problem
Whether circuits found by mechanistic interpretability are task-specific or whether components and algorithmic building blocks are reused across different tasks in transformer language models.
Approach
Reproduces the GPT2-Small IOI circuit on GPT2-Medium, then uses path patching to show that a seemingly different Colored Objects task recruits largely the same heads (mover, induction/duplicate, with content gatherers replacing inhibition heads), and repairs its errors by forcing inhibition and negative mover heads to attend to incorrect colors.
Key results
- Path patching finds about 78% (25/32) overlap in top in-circuit attention heads between IOI and Colored Objects on GPT2-Medium.
- Intervening on three inhibition heads and the negative mover head raises Colored Objects accuracy from 49.6% to 93.7%, with zero new mistakes introduced.
- Forcing content gatherer heads to ignore obj2 drops Colored Objects accuracy to about 35%, near random among the three test colors.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Force three inhibition heads and negative mover head 19.1 to attend to incorrect color options | GPT2-Medium · Colored Objects | accuracy: 49.6% to 93.7% | Helped |
| Add the negative-mover intervention alongside the inhibition-head intervention (ablation) | GPT2-Medium with inhibition-head intervention · Colored Objects | accuracy: 81.5% -> 93.7% | Helped |
| Add the inhibition-head intervention alongside the negative-mover intervention (ablation) | GPT2-Medium with negative-mover intervention · Colored Objects | accuracy: 78.1% -> 93.7% | Helped |
| Block content-gatherer attention to the queried object token (ablation) | GPT2-Medium · Colored Objects | accuracy: null | Hurt |
| Block content-gatherer attention to the token “color” (ablation) | GPT2-Medium · Colored Objects | accuracy | No clear effect |
| Block content-gatherer attention to hypothesized question tokens at the end position (ablation) | GPT2-Medium · Colored Objects | accuracy: to 35% | Hurt |
| Block attention for three randomly sampled non-content-gatherer heads instead (ablation) | GPT2-Medium · Colored Objects | accuracy: 49.9%±1.0 | No clear effect |
| Force inhibition and negative mover heads to attend to incorrect color options | GPT2-Medium · Colored Objects | attention to incorrect colors: -8.7% | Helped |
| Force inhibition and negative mover heads to attend to incorrect color options | GPT2-Medium · Colored Objects | attention to correct color: 2.7% | Helped |
Surprising findings
- Inhibition heads are active on Colored Objects but receive noisy signals and do not help the mover heads, and the negative mover head contributes little to Colored Objects logits.
- Overlap of heads is not trivial: a Greater Than circuit is reported as looking fundamentally different from IOI.
Limitations
- Overlap quantification is approximate, depends on a 2% top-head threshold, with no exact solution offered.
- Study is limited to GPT2-Medium, a relatively small model, and two tasks.
- Intervention is a hand-designed proof-of-concept that forces attention patterns rather than learned behavior.
Experimental setup
Datasets: IOI dataset (1000 examples) Colored Objects dataset derived from BIG-Bench (1000 examples) Greater Than dataset (200 examples)
Benchmarks: IOI Colored Objects
Metrics: accuracy logit difference head overlap (%) attention to wrong colors
Baselines: GPT2-Small IOI circuit random head ablation
Builds on: path patching IOI circuit (Wang et al.) BIG-Bench reasoning about colored objects TransformerLens
Closest papers by meaning
- Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits (NeurIPS 2025 · similarity 0.91)
- Finding Interpretable Prompt-Specific Circuits in Language Models (NeurIPS 2026 · similarity 0.90)
- Talking Heads: Understanding Inter-Layer Communication in Transformer Language Models (NeurIPS 2024 · similarity 0.90)
- Causal Head Gating: A Framework for Interpreting Roles of Attention Heads in Transformers (NeurIPS 2025 · similarity 0.89)
- Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition (ICLR 2025 · similarity 0.88)
- Extrapolation by Association: Length Generalization Transfer In Transformers (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.