Research map / Optimization / Implicit bias and generalization
Deep Double Descent: Where Bigger Models and More Data Hurt
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase model width through the interpolation threshold and into the over-parameterized regime | CNNs, ResNets, and Transformers · CIFAR-10, CIFAR-100, IWSLT'14, and WMT'14 | test error or test loss | Mixed |
| Train a sufficiently large model for more epochs, passing the interpolation threshold | ResNet18 and 5-layer CNN · CIFAR-10 and CIFAR-100 | test error | Mixed |
| Increase training epochs for medium-sized models that barely reach near-zero training error | ResNet18 · CIFAR-10 with label noise | test error | Hurt |
| Increase the number of training samples near the critical model-size regime | 5-layer CNN · CIFAR-10 | test error | No clear effect |
| Increase the number of training samples in some critical regimes | Transformer · IWSLT'14 English-to-German | test performance | Hurt |
| Add data augmentation | 5-layer CNN · CIFAR-10 | test error peak location | Mixed |
| Add label noise | CNNs and ResNets · CIFAR-10 and CIFAR-100 | test error | Mixed |
| Use an ensemble of 5 independently trained models | ResNet18 · CIFAR-10 with 15% label noise | test error | Helped |
| Vary weight decay as a complexity control | 5-layer CNN · CIFAR-10 with label noise | test error | Mixed |
Closest papers by meaning
- Understanding Double Descent through Universal Compression (NeurIPS 2026 · similarity 0.90)
- Unraveling the Enigma of Double Descent: An In-depth Analysis through the Lens of Learned Feature Space (ICLR 2024 · similarity 0.90)
- On the Role of Optimization in Double Descent: A Least Squares Study (NeurIPS 2021 · similarity 0.90)
- DSD²: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free? (AAAI 2024 · similarity 0.87)
- On Double Descent in Reinforcement Learning with LSTD and Random Features (ICLR 2024 · similarity 0.86)
- A two-scale Complexity Measure for Deep Learning Models (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.