Research map / Architectures and efficiency / Attention and transformers
Finding Transformer Circuits With Edge Pruning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Frames automated circuit discovery as gradient-based edge pruning on a disentangled residual stream, yielding sparser, more faithful circuits in GPT-2 and scaling to CodeLlama-13B.
Problem
Automated circuit discovery in Transformers is either too slow (greedy ACDC search) or relies on inaccurate first-order approximations (EAP), limiting faithfulness, scale, and dataset size.
Approach
Learn continuous hard-concrete masks over edges between components (rather than over heads, neurons or layers), replacing pruned edges with corrupted activations via interchange ablation, using a disentangled residual stream so each node can read from arbitrary upstream nodes, and optimizing with a Lagrangian sparsity target.
Key results
- On IOI, Edge Pruning finds a 98.8%-sparse circuit as faithful as ACDC's 96.8%-sparse circuit, using over 2.65x fewer edges.
- At 100K IOI examples, Edge Pruning takes about 3,042 s versus ACDC's prohibitive runtime and EAP's 12,260 s, with the lowest KL (0.20).
- On CodeLlama-13B Boolean Expressions, circuits with 1,041 (IP) and 1,464 (FS) edges (>99.96% sparsity) match full-model accuracy within about 2-3 points; the IP and FS circuits overlap by 62.7% of the sparser circuit's edges.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Edge Pruning instead of ACDC or EAP | GPT-2 Small circuit discovery · IOI, 200 examples | KL divergence: 0.92 (ACDC); 3.47 (EAP); 0.25 (Edge Pruning) | Helped |
| Use Edge Pruning instead of ACDC or EAP | GPT-2 Small circuit discovery · IOI, 400 examples | KL divergence: 0.88 (ACDC); 3.66 (EAP); 0.22 (Edge Pruning) | Helped |
| Use Edge Pruning instead of EAP | GPT-2 Small circuit discovery · IOI, 100K examples | KL divergence: 3.78 (EAP); 0.20 (Edge Pruning) | Helped |
| Use Edge Pruning instead of EAP | GPT-2 Small circuit discovery · IOI, 100K examples | time (s): 12,260 (EAP); 3,042 (Edge Pruning) | Helped |
| Increase the training data from 200 to 400 examples | ACDC · IOI | KL divergence: 0.92 -> 0.88 | Helped |
| Add node masks alongside edge masks (ablation) | Edge Pruning · Circuit discovery at high sparsity | sparsity | Helped |
| Use the instruction-prompted circuit rather than the full model | CodeLlama-13B-Instruct · Boolean Expressions, instruction prompted | accuracy (%): 82.00 -> 79.25 | Hurt |
| Use the few-shot circuit rather than the full model | CodeLlama-13B-Instruct · Boolean Expressions, few-shot | accuracy (%): 89.25 -> 87.25 | Hurt |
| Use the IP circuit under few-shot prompting | CodeLlama-13B-Instruct IP circuit · Boolean Expressions, few-shot | accuracy (%): 79.25 (instruction prompted) -> 74.50 (few-shot) | Hurt |
| Use the FS circuit under instruction prompting | CodeLlama-13B-Instruct FS circuit · Boolean Expressions, instruction prompted | accuracy (%): 87.25 (few-shot) -> 75.75 (instruction prompted) | Hurt |
| Restrict the circuit to edges shared by IP and FS circuits | CodeLlama-13B-Instruct · Boolean Expressions, instruction prompted and few-shot | accuracy (%): 79.25 -> 72.50 (instruction prompted); 87.25 -> 68.25 (few-shot) | Hurt |
Surprising findings
- Circuits found with few-shot prompting perform well under instruction prompting and vice versa, suggesting shared mechanisms.
- Node masks were needed in addition to edge masks; edge masks alone had difficulty reaching high sparsities.
Limitations
- Slower than EAP on small datasets and requires more GPU memory due to the disentangled residual stream, especially at scale.
- Circuits for large models still contain hundreds of edges and remain hard to interpret; only a small region of the CodeLlama circuit was analyzed manually.
- Faithfulness metrics do not guarantee that a circuit represents the true computation (interpretability illusion).
Experimental setup
Datasets: IOI (single template, IOI-t1) IOI (30 templates, HuggingFace fahamu/ioi) Greater Than Gendered Pronoun Tracr xproportion and reverse Boolean Expressions (BBH-derived, in-house generated)
Benchmarks: IOI-t1 IOI Greater Than Gendered Pronoun Tracr xproportion Tracr reverse Boolean Expressions
Metrics: KL divergence logit difference probability difference exact match Kendall's Tau accuracy runtime sparsity ROC AUC vs manual circuits
Baselines: ACDC EAP
Builds on: ACDC EAP (edge attribution patching) CoFi pruning hard concrete distribution (L0 regularization) interchange ablation
Closest papers by meaning
- Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition (ICLR 2025 · similarity 0.91)
- EAP-GP: Mitigating Saturation Effect in Gradient-based Automated Circuit Identification (NeurIPS 2025 · similarity 0.88)
- Transcoders find interpretable LLM feature circuits (NeurIPS 2024 · similarity 0.87)
- Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models (NeurIPS 2025 · similarity 0.86)
- The Computational Complexity of Circuit Discovery for Inner Interpretability (ICLR 2025 · similarity 0.86)
- Circuit Component Reuse Across Tasks in Transformer Language Models (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.