Research map / Large language models / Agents, code and math
Hypothesis Testing the Circuit Hypothesis in LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formalizes the circuit hypothesis into testable criteria (mechanism preservation, localization, minimality) and applies hypothesis tests to six benchmark circuits, finding synthetic circuits align while discovered circuits satisfy the criteria only partially.
Problem
Evaluating whether LLM capabilities are implemented by small subnetworks (circuits) that satisfy idealized properties of preservation, localization, and minimality, which has lacked principled quantitative tests.
Approach
Translate the idealized circuit criteria into statistical hypothesis tests (equivalence, independence, minimality as stringent tests; sufficiency and partial necessity as flexible tests) that compare candidate circuits against random reference circuits, avoiding exhaustive circuit search.
Key results
- Synthetic Tracr circuits pass the equivalence, independence, and minimality tests (rejecting the non-ideal nulls), aligning with the idealized properties.
- Induction circuit passes independence and minimality tests; Docstring circuit passes minimality; IOI, G-T, and Docstring fail equivalence and independence (for IOI/DS, independence rejected).
- For G-T roughly 50% of edges and for IOI about 20% of edges can be removed while retaining faithfulness; for two circuits removing 20% of edges had little impact.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the Induction circuit instead of the full model | Induction circuit · Induction | proportion candidate outperforms model: 0.03 | Hurt |
| Use the IOI circuit instead of the full model | IOI circuit · IOI | proportion candidate outperforms model: 0.24 | Hurt |
| Use the Greater-Than circuit instead of the full model | Greater-Than circuit · G-T | proportion candidate outperforms model: 0.07 | Hurt |
| Use the Docstring circuit instead of the full model | Docstring circuit · Docstring | proportion candidate outperforms model: 0.1 | Hurt |
| Use the candidate circuit instead of a random circuit at 90% of model size | IOI and Greater-Than circuits · IOI; G-T | faithfulness | Helped |
| Use the candidate circuit instead of a random circuit at 30% of model size | Docstring and Induction circuits · Docstring; Induction | faithfulness | Helped |
| Remove around 50% of the canonical circuit edges (ablation) | Greater-Than circuit · G-T | faithfulness: 50% | No clear effect |
| Remove about 20% of the canonical circuit edges (ablation) | IOI circuit · IOI | faithfulness: 20% | No clear effect |
| Remove a few minimal edges from the canonical circuit (ablation) | Canonical circuits · Circuit tasks | faithfulness | No clear effect |
| Knock out the candidate circuit rather than a same-size random circuit from the model | Docstring circuit · Docstring | faithfulness after knockdown | No clear effect |
| Use STR instead of zero-ablation for knockdown (ablation) | IOI and Greater-Than circuits · IOI; G-T | relative significance of candidate-circuit knockdown | Mixed |
Surprising findings
- Knocking out the complete model has less impact than knocking out the candidate circuit for G-T and IOI under STR ablation, suggesting knockdown metrics alone are insufficient.
- None of the natural (discovered) circuits passed the equivalence test, and all except IOI perform much worse than the original model.
Limitations
- Idealized tests are stringent and flexible tests are sensitive to circuit size measurements and null hypothesis design.
- Type II error of non-equivalence and non-independence tests was not empirically studied.
- Results are for a single experimental setup; circuits are not robust to changes in setup.
Experimental setup
Datasets: IOI dataset OpenWebText validation split (induction) Docstring dataset greater-than dataset Tracr task inputs
Benchmarks: IOI Induction Docstring Greater-Than Tracr-R Tracr-P
Metrics: p-value faithfulness (l2 norm) HSIC proportion of inputs where circuit outperforms model
Baselines: ACDC
Builds on: activation patching automatic circuit discovery (ACDC) TransformerLens
Closest papers by meaning
- Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates (NeurIPS 2025 · similarity 0.88)
- Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis (ICML 2025 · similarity 0.87)
- LLM Circuit Analyses Are Consistent Across Training and Scale (NeurIPS 2024 · similarity 0.87)
- Transcoders find interpretable LLM feature circuits (NeurIPS 2024 · similarity 0.87)
- Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities (NeurIPS 2025 · similarity 0.86)
- Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.