Research map / Architectures and efficiency / RNNs and normalization
Neural Sculpting: Uncovering hierarchically modular task structure in neural networks through pruning and network analysis
Shreyas Malakarjun Patil, Loizos Michael, Constantine Dovrolis
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with plain dense training instead of pruning | Standard NN training · Input separable function graph | edge-weight product test (null hypothesis rejected) | Hurt |
| Standard training on reused sub-function | Standard NN training · Reused sub-function graph | units covering 90% of path weight | Hurt |
| Add iterative unit and edge pruning to training | NN training · Boolean function graphs | module detection success rate | Improved |
| Increase NN depth beyond the function graph depth | Neural Sculpting · Boolean function graphs | module detection success rate | Improved |
| Increase number of output nodes reusing the gate sub-function | Neural Sculpting · Single reused sub-function graph | success rate | Improved |
| Increase input overlap between two separable sub-functions | Neural Sculpting · Two input separable sub-functions graph | success rate of input module detection | Hurt |
| Decrease number of times each sub-function is used | Neural Sculpting · Two input separable sub-functions graph | success rate | Hurt |
| Use NN depth below the number of hierarchical levels (5) | Neural Sculpting · Hierarchical function graph | sub-function recovery | Hurt |
| Use NN depth at least the number of hierarchical levels (5) | Neural Sculpting · Hierarchical function graph | sub-function recovery | Improved |
| Use two-digit MNIST classification task | Neural Sculpting · Two-MNIST-digit classification | completely separable modules: 33 out of 36 | Improved |
| Increase NN depth | Neural Sculpting · Hierarchical MNIST task | recovery of two input separable modules | Improved |
| Increase NN depth | Neural Sculpting · Hierarchical MNIST task | middle layer unit separability | No clear effect |
Closest papers by meaning
- Neural networks are more modular than single neurons suggest (NeurIPS 2026 · similarity 0.92)
- Breaking Neural Network Scaling Laws with Modularity (ICLR 2025 · similarity 0.89)
- On the Modularity of Hypernetworks (NeurIPS 2020 · similarity 0.88)
- Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks (ICLR 2021 · similarity 0.87)
- How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Model (ICML 2024 · similarity 0.85)
- Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
