Research map / Probabilistic and scientific modeling / Causal discovery
Mechanistic Interpretability as Statistical Estimation: A Variance Analysis
Maxime Méloux, François Portet, Maxime Peyrard
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use bootstrap resampling instead of meta-dataset resampling | EAP-IG circuit discovery · averaged over models and tasks | pairwise Jaccard mean: 0.790 -> 0.561 | Hurt |
| use bootstrap resampling instead of prompt paraphrasing | EAP-IG circuit discovery · averaged over models and tasks | circuit error: 0.150 -> 0.440 | Hurt |
| replace exact CMA scores with EAP estimates | edge importance scoring · gpt2-small Greater-Than | coefficient of variation of edge scores: CV≈0.5 -> CV>1 | Hurt |
| scale from gpt2-small to Llama-3.2 (1B and Instruct) | circuit discovery · IOI, SVA, Greater-Than | stability of circuits | Hurt |
| instruction-tune Llama-3.2-1B | circuit discovery · IOI, SVA, Greater-Than | circuit stability | No clear effect |
| change aggregation sum to median and mean to patching for EAP-IG-inputs (ablation) | EAP-IG-inputs · Greater-Than (Llama-3.2-1B-Instruct) | Jaccard to median circuit: 1.000 -> 0.086 | Hurt |
| swap EAP-IG-inputs for Clean-corrupted (ablation) | EAP-IG-inputs, sum, patching · IOI (Llama-3.2-1B-Instruct) | Jaccard to median circuit: 0.182 -> 0.071 | Hurt |
| use mean-positional instead of patching counterfactual (ablation) | EAP-IG-inputs, sum · Greater-Than (Llama-3.2-1B-Instruct) | circuit error: 0.20 -> 0.41 | Hurt |
| use EAP-IG-activations instead of EAP-IG-inputs (ablation) | EAP-IG, sum, patching · Greater-Than (Llama-3.2-1B-Instruct) | Jaccard to median circuit: 0.571 -> 0.098 | Hurt |
| increase Gaussian noise amplitude of the counterfactual (ablation) | EAP-IG circuit discovery · gpt2-small | Jaccard index CV | Mixed |
| use Clean-corrupted instead of EAP (ablation) | circuit size · IOI (Llama-3.2-1B) | circuit size (edges): 7 -> 448 | Hurt |
Closest papers by meaning
- Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable? (ICLR 2025 · similarity 0.89)
- Mechanistic Circuit Identification for Controllable Data Generation (NeurIPS 2026 · similarity 0.88)
- Since Faithfulness Fails: The Performance Limits of Neural Causal Discovery (ICML 2025 · similarity 0.88)
- Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery (NeurIPS 2026 · similarity 0.87)
- Compact Proofs of Model Performance via Mechanistic Interpretability (NeurIPS 2024 · similarity 0.86)
- MIB: A Mechanistic Interpretability Benchmark (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
