Research map / Architectures and efficiency / RNNs and normalization
Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Reuse the Pair 1 mask for Pair 2 rather than train a separate mask | FNN · Double addition | accuracy (%): 100 ± 0.0 -> 1 ± 0.1 | Hurt |
| Reuse the Pair 2 mask for Pair 1 rather than train a separate mask | FNN · Double addition | accuracy (%): 100 ± 0.0 -> 1 ± 0.1 | Hurt |
| Reuse the Pair 1 mask for Pair 2 rather than train a separate mask | LSTM · Double addition | accuracy (%): 100 ± 0.0 -> 1 ± 0.1 | Hurt |
| Reuse the Pair 2 mask for Pair 1 rather than train a separate mask | LSTM · Double addition | accuracy (%): 100 ± 0.0 -> 1 ± 0.1 | Hurt |
| Use an inverted Pair 1 mask, removing weights needed for Pair 1 | FNN · Double addition, Pair 2 | accuracy (%): 100 ± 0.0 -> 94 ± 6.7 | No clear effect |
| Use an inverted Pair 1 mask, removing weights needed for Pair 1 | LSTM · Double addition, Pair 2 | accuracy (%): 100 ± 0.0 -> 100 ± 0.2 | No clear effect |
| Use an inverted Pair 1 mask, removing weights needed for Pair 1 | LSTM (forced) · Double addition, Pair 2 | accuracy (%): 100 ± 0.0 -> 96 ± 4.1 | No clear effect |
| Initialize masks to favor already occupied weights | FNN on permuted MNIST · Permuted MNIST | weight sharing | Helped |
| Mask trained on the easy split rather than use the full network | SCAN LSTM encoder-decoder and Transformer · SCAN difficulty/generalization splits | test performance | Hurt |
| Apply a mask trained on the easy split to the hard split | Neural networks on Mathematics Dataset · Mathematics Dataset difficulty splits | performance | Hurt |
| Remove weights found unnecessary for one CIFAR10 class | CNNs on CIFAR10 · CIFAR10 validation classes | class accuracy | Hurt |
| Remove weights found unnecessary for one CIFAR10 class | CNNs on CIFAR10 · CIFAR10 non-target classes | class accuracy | No clear effect |
Closest papers by meaning
- Breaking Neural Network Scaling Laws with Modularity (ICLR 2025 · similarity 0.89)
- Neural networks are more modular than single neurons suggest (NeurIPS 2026 · similarity 0.88)
- Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks (NeurIPS 2025 · similarity 0.86)
- Learning Useful Representations of Recurrent Neural Network Weight Matrices (ICML 2024 · similarity 0.85)
- Expand-and-Cluster: Parameter Recovery of Neural Networks (ICML 2024 · similarity 0.84)
- Equivariant Polynomial Functional Networks (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.