Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Improving Compositionality of Neural Networks by Decoding Representations to Inputs
Mike Wu, Noah Goodman, Stefano Ermon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a jointly trained decoder that reconstructs inputs from every classifier activation | Standard neural network · MNIST, FashionMNIST, CelebA | accuracy: 97.1 -> 97.8; 87.7 -> 88.8; 90.8 -> 90.8 | Improved |
| Recursively compose the decodable classifier with itself | DecNN · MNIST, FashionMNIST, CelebA | accuracy: 97.1 -> 97.5; 87.7 -> 88.1; 90.8 -> 90.8 | No clear effect |
| Use recursive-path disagreement as an uncertainty score | ReDecNN · CelebA correctly vs. incorrectly classified | ROC-AUC: 0.657 -> 0.692 | Improved |
| Use recursive-path disagreement as an uncertainty score | ReDecNN · MNIST and FashionMNIST correctly vs. incorrectly classified | ROC-AUC: 0.878 -> 0.869; 0.818 -> 0.795 | Hurt |
| Use recursive-path disagreement for OOD detection | ReDecNN · MNIST trained; FashionMNIST OOD | ROC-AUC: 0.888 -> 0.812 | Hurt |
| Use recursive-path disagreement for OOD detection | ReDecNN · CelebA trained; held-out wearing-a-hat attribute | ROC-AUC: 0.519 -> 0.604 | Improved |
| Use recursive composition | Standard neural network · MNIST, FashionMNIST, CelebA test | ECE: 1.18 -> 0.33; 0.56 -> 0.21; 2.75 -> 1.92 | Improved |
| Increase recursive depth from 2 through 8 (ablation) | ReDecNN · MNIST trained; FashionMNIST OOD | ROC-AUC: 0.665 -> 0.788 -> 0.805 -> 0.812 | Improved |
| Increase recursive depth from 2 through 8 (ablation) | ReDecNN · MNIST test | accuracy: 97.7; 97.6; 97.7; 97.5 | No clear effect |
| Compose with a pretrained linear classifier | ReDecNN · MNIST trained; FashionMNIST OOD | ROC-AUC: 0.812 -> 0.845 | Improved |
| Compose with a pretrained ResNet-18 | ReDecNN · MNIST trained; FashionMNIST OOD | ROC-AUC: 0.812 -> 0.650 | Hurt |
| Regularize predictions on reconstructions rather than on the original input (ablation) | ReDecNN with pretrained linear model · MNIST; FashionMNIST OOD | ROC-AUC: 0.699 -> 0.845 | Improved |
Closest papers by meaning
- Position: A Theory of Deep Learning Must Include Compositional Sparsity (ICML 2025 · similarity 0.87)
- How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learning (ICLR 2025 · similarity 0.86)
- Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · similarity 0.85)
- Scaling can lead to compositional generalization (NeurIPS 2025 · similarity 0.85)
- A Second-Order Perspective on Model Compositionality and Incremental Learning (ICLR 2025 · similarity 0.85)
- When does compositional structure yield compositional generalization? A kernel theory. (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
