Research map / Optimization / Implicit bias and generalization
Understanding deep learning requires rethinking generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on random labels instead of true labels (no other changes) (ablation) | Inception, Alexnet, MLPs · CIFAR10 | training accuracy | No clear effect |
| turn off random crop and weight decay (ablation) | Inception · CIFAR10 | test accuracy: 89.05 -> 85.75 | Hurt |
| add data augmentation (random crop) and weight decay (ablation) | Alexnet · CIFAR10 | test accuracy: 76.07 -> 81.22 | Helped |
| random labels with no regularization (ablation) | Inception · CIFAR10 | test accuracy: 85.75 -> 9.78 | Hurt |
| remove batch normalization (ablation) | Inception · CIFAR10 | test accuracy: 85.75 -> 82.00 (no crop/wd) | Hurt |
| add weight decay to random-label training (ablation) | Alexnet · CIFAR10 | fit of random labels | Hurt |
| add weight decay with default coefficient to random-label training (ablation) | Inception, MLP 3x512, MLP 1x512 · CIFAR10 | training accuracy on random labels | No clear effect |
| turn off all regularizers (augmentation, dropout, weight decay) (ablation) | Inception v3 · ImageNet | top-1 test accuracy: 77.84 -> 59.80 | Hurt |
| add data augmentation (keeping dropout and weight decay off vs all on) (ablation) | Inception v3 · ImageNet | top-1 test accuracy: 59.80 -> 72.95 | Helped |
| random labels on ImageNet with no regularization (ablation) | Inception v3 · ImageNet | top-1 test accuracy: 59.80 -> 0.11 (random labels) | Hurt |
| early stopping (best test accuracy along training) | Inception v3 · ImageNet | top-1 test accuracy: 59.80 -> 63.16 (best during training) | Helped |
| add batch normalization (ablation) | Inception · CIFAR10 | test accuracy: 82.00 -> 85.75 (no crop/wd) | Helped |
Closest papers by meaning
- Train longer, generalize better: closing the generalization gap in large batch training of neural networks (NeurIPS 2017 · similarity 0.90)
- Do Neural Networks Need Gradient Descent to Generalize? A Theoretical Study (NeurIPS 2025 · similarity 0.89)
- Position: Deep Learning is Not So Mysterious or Different (ICML 2025 · similarity 0.89)
- To Grok or not to Grok: Disentangling Generalization and Memorization on Corrupted Algorithmic Datasets (ICLR 2024 · similarity 0.89)
- The Master Key Filters Hypothesis: Deep Filters Are General (AAAI 2025 · similarity 0.88)
- Understanding the Generalization of Stochastic Gradient Adam in Learning Neural Networks (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.