Research map / Architectures and efficiency / RNNs and normalization
A Rainbow in Deep Network Black Boxes
Florentin Guth, Brice Ménard, Gaspar Rochette, Stéphane Mallat
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase network width scaling factor s | Learned scattering networks, CIFAR-10 · CIFAR-10 | mean-square distance of aligned activations to limit | Improved |
| increase width of ResNet-18 | ResNet-18, ImageNet · ImageNet | mean-square distance of aligned activations | Mixed |
| project neuron weights onto the first principal components of the weight covariance (about 30 of 128 dimensions) (ablation) | Learned scattering network, CIFAR-10 (3 layers) · CIFAR-10 | test accuracy: approximately 30 of 128 dimensions | No clear effect |
| average weight covariance estimates over more trained networks N (ablation) | Weight covariance estimation, CIFAR-10 · CIFAR-10 | approximation accuracy | No clear effect |
Closest papers by meaning
- Asymptotics of Learning with Deep Structured (Random) Features (ICML 2024 · similarity 0.88)
- Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization? (ICML 2020 · similarity 0.86)
- Neural Redshift: Random Networks are not Random Functions (CVPR 2024 · similarity 0.85)
- Initial Guessing Bias: How Untrained Networks Favor Some Classes (ICML 2024 · similarity 0.84)
- Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning (NeurIPS 2026 · similarity 0.84)
- Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
