Research map / Architectures and efficiency / RNNs and normalization
Weight-sparse transformers have interpretable circuits
Leo Gao, Achyuta Rajaram, Jacob Coxon, Soham V. Govande, Bowen Baker, Daniel P Mossing
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train weight-sparse model instead of dense model at matched pretraining loss | GPT-2 style transformer on Python code · 20 binary tasks | minimal circuit size: roughly 16-fold smaller circuits | Improved |
| decrease L0 (make weights sparser) at fixed total parameters (ablation) | weight-sparse transformer · pretraining loss vs circuit size | capability and interpretability | Mixed |
| increase model width d_model at fixed L0 (ablation) | weight-sparse transformer · pretraining loss vs circuit size | capability and interpretability | Improved |
| scale total parameter count of sparse models (ablation) | weight-sparse transformer · capability-interpretability frontier | Pareto frontier | Improved |
| ablate the circuit nodes only, leaving rest of network intact (ablation) | sparse circuits · task loss | task loss | Hurt |
| lengthen list context to dilute the averaged bracket-depth feature | bracket counting circuit · flat vs nested list closing | p(correct) | Hurt |
| steer sparse-model quote-type channel and map via bridges to dense model | bridged dense model · single_double_quote 4-layer | P(single quote) | Improved |
| steer sparse-model colon channel and map via bridges to dense model | bridged dense model · while_return_true 4-layer | P(colon) | Improved |
| use weight-sparse training instead of dense training | training compute · comparable capability | training and inference compute: 100-1000x more compute | Hurt |
Closest papers by meaning
- Transcoders find interpretable LLM feature circuits (NeurIPS 2024 · similarity 0.92)
- Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models (ICLR 2025 · similarity 0.92)
- Extracting Interpretable Task-Specific Circuits from Large Language Models for Faster Inference (AAAI 2025 · similarity 0.89)
- Neurons to Words: A Novel Method for Automated Neural Network Interpretability and Alignment (AAAI 2025 · similarity 0.89)
- Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures (ICLR 2025 · similarity 0.89)
- Towards Global-level Mechanistic Interpretability: A Perspective of Modular Circuits of Large Language Models (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
