Research map / Architectures and efficiency / RNNs and normalization
Causal Interpretation of Neural Network Computations with Contribution Decomposition (CODEC)
Joshua Brendan Melander, Zaki Alaoui, Shenghua Liu, Surya Ganguli, Stephen Baccus
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| decompose contributions instead of activations with SAE (ablation) | CODEC on ResNet-50 · ImageNet val, class correlation of modes | mode-class correlation | Improved |
| use contribution modes instead of individual channels (ablation) | CODEC on ResNet-50 · ImageNet val, class correlation | correlation with class | Improved |
| select channels by contribution modes rather than activation modes for ablation/preservation (ablation) | ResNet-50 network control · ImageNet, target vs off-target class | ablation specificity | Improved |
| analyze contributions instead of activations | ResNet-50 layers · ImageNet val | Hoyer sparsity | Improved |
| analyze contributions instead of activations | ResNet-50 class-averaged channel weightings · ImageNet val | dimensionality (components for 95% variance) | Improved |
| set dictionary size to equal or smaller than channel count (ablation) | CODEC SAE · ResNet-50 layers 3,7,13,15 | SAE R^2 / stability | Hurt |
| raise SAE threshold from 0.5 to 0.9 (ablation) | CODEC SAE · ResNet-50 | SAE R^2 | Hurt |
| vary L1 regularization, random seed, non-negativity constraint (ablation) | CODEC SAE · ResNet-50 | SAE reconstruction R^2 | No clear effect |
| shrink SAE encoder MLP hidden dimension (ablation) | CODEC SAE · ResNet-50 | reconstruction | Hurt |
| apply CODEC contribution-SAE instead of activation-SAE on ViT-B (ablation) | ViT-B token, MLP and attention layers · ImageNet ablation | ablation specificity | Improved |
| use contrastive target instead of top-1 logit (ablation) | CODEC · ResNet-50 four layers | sparsity: R^2 above 0.85 | No clear effect |
| group ViT attention dimensions into learned heads (ablation) | ViT-B attention · ImageNet | Hoyer sparsity vs random grouping | No clear effect |
Closest papers by meaning
- Learning Reveals Invisible Structure in Low-Rank RNNs (NeurIPS 2026 · similarity 0.86)
- Decomposing stimulus-specific sensory neural information via diffusion models (NeurIPS 2025 · similarity 0.86)
- Explaining the Implicit Neural Canvas: Connecting Pixels to Neurons by Tracing their Contributions (CVPR 2024 · similarity 0.86)
- Towards Learning and Explaining Indirect Causal Effects in Neural Networks (AAAI 2024 · similarity 0.86)
- Since Faithfulness Fails: The Performance Limits of Neural Causal Discovery (ICML 2025 · similarity 0.85)
- Linear Explanations for Individual Neurons (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
