Research map / Large language models / Language, safety and interpretability
Towards Global-level Mechanistic Interpretability: A Perspective of Modular Circuits of Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ModCirc, a framework that discovers a task-agnostic vocabulary of modular circuits (MCs) in LLMs via reusable subgraph extraction, RL-based GNN graph partitioning, and LLM-generated functional interpretations.
Problem
Existing mechanistic interpretability finds circuits task by task, which generalizes poorly across tasks and is costly because each circuit node needs a human or GPT-4 functional interpretation.
Approach
Define five criteria (consistency, locality, reusability, composability, globality) for a modular circuit vocabulary, then discover reusable subgraphs across task circuits, partition them with a policy-gradient-trained GNN, and assign cached functional interpretations so new tasks can reuse them.
Key results
- ModCirc reusability 0.32-0.37 across four medical evaluation datasets vs. baselines below 0.21.
- On transfer to IOI with GPT-2 Small, ModCirc recovers 92% (23/25) of ground-truth circuit components.
- On acronym detection, ModCirc recovers 87.5% (7/8) of ground-truth circuit nodes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ModCirc instead of the strongest listed baseline | ModCirc · Coreference | reusability: 0.21 ± 0.01 -> 0.37 ± 0.01 | Helped |
| use ModCirc instead of the strongest listed baseline | ModCirc · MedAttr | reusability: 0.18 ± 0.01 -> 0.34 ± 0.01 | Helped |
| use ModCirc instead of the strongest listed baseline | ModCirc · MedStatus | reusability: 0.17 ± 0.01 -> 0.33 ± 0.00 | Helped |
| use ModCirc instead of the strongest listed baseline | ModCirc · PubMed Summ. | reusability: 0.18 ± 0.01 -> 0.32 ± 0.01 | Helped |
| use ModCirc rather than the strongest listed baseline | ModCirc · Coreference | consistency: 0.39 ± 0.02 -> 0.40 ± 0.02 | Helped |
| use ModCirc rather than the strongest listed baseline | ModCirc · MedAttr | consistency: 0.34 ± 0.01 -> 0.33 ± 0.01 | Hurt |
| use ModCirc rather than the strongest listed baseline | ModCirc · MedStatus | consistency: 0.26 ± 0.00 -> 0.26 ± 0.01 | No clear effect |
| use ModCirc rather than the strongest listed baseline | ModCirc · PubMed Summ. | consistency: 0.34 ± 0.01 -> 0.34 ± 0.03 | No clear effect |
| use ModCirc rather than the strongest listed baseline | ModCirc · Coreference | composability: 0.80 ± 0.10 -> 0.70 ± 0.10 | Hurt |
| use ModCirc rather than the strongest listed baseline | ModCirc · MedAttr | composability: 1.00 ± 0.00 -> 0.95 ± 0.10 | Hurt |
| use ModCirc rather than the strongest listed baseline | ModCirc · MedStatus | composability: 1.00 ± 0.00 | No clear effect |
| use ModCirc rather than the strongest listed baseline | ModCirc · PubMed Summ. | composability: 0.75 ± 0.00 -> 0.70 ± 0.10 | Hurt |
Surprising findings
- ModCirc does not consistently outperform baselines on composability; some baselines reach perfect composability scores.
- Composability decreases when enforcing connected MCs, reflecting a trade-off with consistency and reusability.
Limitations
- Evaluated on only one LLM (MedLLaMA 8B) for the main experiments, plus GPT-2 Small for transfer.
- Consistency and reusability are not included in the RL objective; ModCirc is weaker on composability than some baselines.
- Relies on GPT-4 for FI generation and on the reusable-subgraph heuristic.
Experimental setup
Datasets: MEDAL Medical Abstracts MedMCQA Symptom to Diagnosis AGNews MPQA Universal Dependency TREC
Benchmarks: MedStatus MedAttr Coreference PubMed Summ. IOI acronym detection
Metrics: composability consistency reusability recovery rate of ground-truth circuit components
Baselines: Random Frequent Random KMeans Activation-based grouping
Builds on: activation patching edge attribute patching integrated gradients REINFORCE Nanda modularity hypothesis
Closest papers by meaning
- Extracting Interpretable Task-Specific Circuits from Large Language Models for Faster Inference (AAAI 2025 · similarity 0.92)
- Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis (ICML 2025 · similarity 0.87)
- Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models (ICLR 2025 · similarity 0.87)
- The Same but Different: Structural Similarities and Differences in Multilingual Language Modeling (ICLR 2025 · similarity 0.87)
- Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures (ICLR 2025 · similarity 0.87)
- Transcoders find interpretable LLM feature circuits (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.