Research map / Large language models / Language, safety and interpretability
All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMs
Xi Chen, Mingyu Jin, Jingcheng Niu, Yutong Yin, Jinman Zhao, Bangwei Guo, Dimitris N. Metaxas, Zhaoran Wang, Yutao Yue, Gerald Penn
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add overlap-aware sheaf repulsion penalty across discovery runs (ablation) | DiscoGP · IOI, 20 sheaves | mutual IoU: 0.30% -> 0.15% | Improved |
| add overlap-aware sheaf repulsion penalty across discovery runs (ablation) | DiscoGP · IOI, 20 sheaves | average accuracy: 99.95 -> 99.59 | No clear effect |
| add overlap-aware sheaf repulsion penalty across discovery runs (ablation) | DiscoGP · BLiMP, 20 sheaves | mutual IoU: 1.03% -> 0.70% | Improved |
| add overlap-aware sheaf repulsion penalty across discovery runs (ablation) | DiscoGP · ANA, 20 sheaves | mutual IoU: 0.57% -> 0.20% | Improved |
| add overlap-aware sheaf repulsion penalty across discovery runs (ablation) | DiscoGP · DNA, 20 sheaves | mutual IoU: 0.54% -> 0.49% | No clear effect |
| increase number of discovered sheaves from two to 20 (ablation) | DiscoGP with OASR · IOI | overlap (IoU): pairwise 4.1% -> mutual 0.15% | Hurt |
| vary attention head traversal order in ACDC (ablation) | ACDC · IOI | circuit stability: interchange |E∩|=174 vs |E∪|=608 | Hurt |
| use zero ablation instead of interchange ablation (ablation) | ACDC · IOI | edge density: 1.09% -> 6.76% | Hurt |
| vary only names in IOI prompts, increasing top-k edges (ablation) | EAP · IOI | pairwise IoU between circuits | Hurt |
| replace KL-divergence loss with task-specific cross entropy loss (ablation) | Edge Pruning · IOI | min pairwise IoU: 0.340 -> 0.064 | Hurt |
| replace KL-divergence loss with task-specific cross entropy loss (ablation) | Edge Pruning · Docstring | accuracy: 88.25 -> 99.75 | Improved |
| exclude the three-edge core from sheaf discovery on whole IOI (ablation) | DiscoGP on IOI · IOI | accuracy: to 52.3% | Hurt |
Closest papers by meaning
- Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates (NeurIPS 2025 · similarity 0.87)
- Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis (ICML 2025 · similarity 0.87)
- Hypothesis Testing the Circuit Hypothesis in LLMs (NeurIPS 2024 · similarity 0.86)
- Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition (ICLR 2025 · similarity 0.86)
- Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery (NeurIPS 2026 · similarity 0.86)
- Towards Global-level Mechanistic Interpretability: A Perspective of Modular Circuits of Large Language Models (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
