Research map / Architectures and efficiency / RNNs and normalization
Redundant representations help generalization in wide neural networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase network width beyond the interpolation threshold | Regularized DenseNet40 on CIFAR10 · CIFAR10 | test error: W^{-1/2} | Helped |
| Use random chunks larger than the clone threshold from the widest representation | Wide neural network final representations · pMNIST, CIFAR10, CIFAR100 | test error: w_c^{-1/2} | Helped |
| Train without weight decay (ablation) | Wide-ResNet28-8 · CIFAR10 | test error scaling with chunk size | Hurt |
| Stop training before reaching zero training error (ablation) | Wide neural networks · CIFAR10 and CIFAR100 | chunk residual correlation | Hurt |
| Use a network that does not interpolate the training set (ablation) | ResNet50-4 · ImageNet | chunk test error scaling | Hurt |
| Increase training duration after the full network reaches zero training error so clones also fit the training data (ablation) | Wide-ResNet28-8 · CIFAR10 | test error | Helped |
| Increase training duration from epoch 150 to epoch 185 (ablation) | Wide-ResNet28-8 · CIFAR100 | test error: from 0.194 to 0.1765 | Helped |
Closest papers by meaning
- Position: Deep Learning is Not So Mysterious or Different (ICML 2025 · similarity 0.87)
- What Variables Affect Out-of-Distribution Generalization in Pretrained Models? (NeurIPS 2024 · similarity 0.87)
- To Grok or not to Grok: Disentangling Generalization and Memorization on Corrupted Algorithmic Datasets (ICLR 2024 · similarity 0.86)
- Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers (NeurIPS 2025 · similarity 0.86)
- Feature Contamination: Neural Networks Learn Uncorrelated Features and Fail to Generalize (ICML 2024 · similarity 0.86)
- On the Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamics (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.