Research map / Vision-language and multimodal / Human and embodied modeling
Break It Down: Evidence for Structural Compositionality in Neural Networks
Michael A. Lepori, Thomas Serre, Ellie Pavlick
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Discover sparse subnetworks for individual task subroutines and ablate each discovered subnetwork (ablation) | Trained ResNet50 and BERT-Small models · Compositional vision and language tasks | difference in target-versus-other subroutine accuracy | Improved |
| Initialize BERT-Small with pretrained language-model weights rather than training from scratch | BERT-Small · Language tasks | modularity of discovered subnetworks | Improved |
| Initialize ResNet50 with SimCLR-pretrained weights rather than random initialization | ResNet50 · Vision tasks | structural compositionality | No clear effect |
| Ablate the Contact subnetwork (ablation) | ResNet50 trained on Number-Contact · Number-Contact, Contact subroutine | target-versus-other subroutine accuracy difference | Improved |
| Ablate the Number subnetwork (ablation) | ResNet50 trained on Number-Contact · Number-Contact, Number subroutine | target-versus-other subroutine accuracy difference | No clear effect |
| Use a ViT backbone for the vision compositional tasks | ViT · Inside-Contact, Number-Contact, and Inside-Number | accuracy | Hurt |
| Use sparse subnetworks to test whether each known subroutine is independently represented | Neural networks trained on compositional vision and language tasks · Compositional vision and language tasks | evidence of structural compositionality | Improved |
Closest papers by meaning
- Scaling can lead to compositional generalization (NeurIPS 2025 · similarity 0.90)
- When does compositional structure yield compositional generalization? A kernel theory. (ICLR 2025 · similarity 0.89)
- Neural networks are more modular than single neurons suggest (NeurIPS 2026 · similarity 0.88)
- Separating the 'what' and 'how' of compositional computation to enable reuse and continual learning (NeurIPS 2025 · similarity 0.88)
- A Second-Order Perspective on Model Compositionality and Incremental Learning (ICLR 2025 · similarity 0.88)
- Discovering modular solutions that generalize compositionally (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
