Research map / Architectures and efficiency / RNNs and normalization
Towards Automated Circuit Discovery for Mechanistic Interpretability
Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, Adrià Garriga-Alonso
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Minimize task-specific metric instead of KL divergence (ablation) | ACDC · Docstring edge ROC | AUC: 0.982 -> 0.972 | No clear effect |
| Minimize task-specific metric instead of KL divergence (ablation) | ACDC · Greater-Than edge ROC | AUC: 0.853 -> 0.461 | Hurt |
| Minimize task-specific metric instead of KL divergence (ablation) | ACDC · IOI edge ROC | AUC: 0.869 -> 0.589 | Hurt |
| Patch with zero activations instead of corrupted activations (ablation) | ACDC · Tracr-Reverse edge ROC | AUC: 0.200 -> 1.000 | Improved |
| Patch with zero activations instead of corrupted activations (ablation) | ACDC · Tracr-Proportion edge ROC | AUC: 0.679 -> 1.000 | Improved |
| Patch with zero activations instead of corrupted activations (ablation) | ACDC · IOI edge ROC, KL | AUC: 0.869 -> 0.539 | Hurt |
| Patch with zero activations instead of corrupted activations (ablation) | Subnetwork Probing · Docstring edge ROC, KL | AUC: 0.937 -> 0.428 | Hurt |
| Use ACDC instead of SP and HISP | ACDC · IOI edge ROC, KL, corrupted | AUC: 0.823 -> 0.869 | Improved |
| Use ACDC instead of HISP when optimizing task loss | ACDC · Tracr-Reverse edge ROC, corrupted | AUC: 0.577 -> 0.200 | Hurt |
| Use ACDC instead of SP/HISP, number of edges above 20 | ACDC · Induction, KL on held-out set | KL divergence per edges | Improved |
| Use zero activations instead of corrupted activations (ablation) | ACDC, SP, HISP · Induction | KL divergence | Improved |
| Evaluate on reset (permuted weight) network versus trained network (ablation) | HISP and SP · Induction reset network | KL divergence | Hurt |
Closest papers by meaning
- Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition (ICLR 2025 · similarity 0.92)
- The Computational Complexity of Circuit Discovery for Inner Interpretability (ICLR 2025 · similarity 0.88)
- Finding Transformer Circuits With Edge Pruning (NeurIPS 2024 · similarity 0.88)
- Transcoders find interpretable LLM feature circuits (NeurIPS 2024 · similarity 0.87)
- InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniques (NeurIPS 2024 · similarity 0.87)
- Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
