Research map / Architectures and efficiency / RNNs and normalization
Modular Networks: Learning to Decompose Neural Computation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use learned EM-based modular routing instead of REINFORCE routing with M=15 and K=1 | Modular Networks · Penn Treebank test | perplexity: 240.760 -> 229.651 | Helped |
| Use learned EM-based modular routing instead of REINFORCE routing with M=5 and K=1 | Modular Networks · Penn Treebank test | perplexity: 240.450 -> 236.809 | Helped |
| Use learned EM-based modular routing instead of REINFORCE routing with M=15 and K=3 | Modular Networks · Penn Treebank test | perplexity: 274.060 -> 246.493 | Helped |
| Use learned EM-based modular routing instead of REINFORCE routing with M=5 and K=3 | Modular Networks · Penn Treebank test | perplexity: 267.585 -> 236.314 | Helped |
| Use learned EM-based modular routing instead of noisy top-k routing with M=15 and K=1 | Modular Networks · Penn Treebank test | perplexity: 422.636 -> 229.651 | Helped |
| Use learned EM-based modular routing instead of noisy top-k routing with M=5 and K=1 | Modular Networks · Penn Treebank test | perplexity: 338.275 -> 236.809 | Helped |
| Select one module dynamically rather than use one fixed module | Modular Networks (M=15) · Penn Treebank test | perplexity: 247.408 -> 229.651 | Helped |
| Select one module dynamically rather than use three fixed modules | Modular Networks (M=15) · Penn Treebank test | perplexity: 241.294 -> 229.651 | Helped |
| Select three modules rather than one with M=15 (ablation) | EM Modular Networks · Penn Treebank test | perplexity: 229.651 -> 246.493 | Hurt |
| Select three modules rather than one with M=5 (ablation) | EM Modular Networks · Penn Treebank test | perplexity: 236.809 -> 236.314 | No clear effect |
Closest papers by meaning
- Breaking Neural Network Scaling Laws with Modularity (ICLR 2025 · similarity 0.88)
- Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks (ICLR 2021 · similarity 0.88)
- Neural networks are more modular than single neurons suggest (NeurIPS 2026 · similarity 0.86)
- Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · similarity 0.85)
- Scalable Optimization in the Modular Norm (NeurIPS 2024 · similarity 0.85)
- ModHiFi: Identifying High Fidelity predictive components for Model Modification (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.