Research map / Optimization / Implicit bias and generalization
On the Role of Optimization in Double Descent: A Least Squares Study
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase hidden-layer width across the interpolation threshold | One-hidden-layer MLPs · MNIST and FashionMNIST | test error | Mixed |
| Increase the depth from one to three hidden layers | MLPs with increasing hidden-layer widths · MNIST | test error | Hurt |
| Use a narrow bottleneck in earlier layers while increasing the last hidden layer (ablation) | Three-hidden-layer MLP · MNIST | minimum positive eigenvalue of hidden features | Hurt |
| Use a narrow bottleneck in earlier layers while increasing the last hidden layer (ablation) | Three-hidden-layer MLP · MNIST | test error | Hurt |
| Add label noise | Gradient descent for least-squares regression · Random-design least squares | excess risk | Hurt |
Closest papers by meaning
- Unraveling the Enigma of Double Descent: An In-depth Analysis through the Lens of Learned Feature Space (ICLR 2024 · similarity 0.93)
- Phenomenology of Double Descent in Finite-Width Neural Networks (ICLR 2022 · similarity 0.93)
- Optimal Regularization can Mitigate Double Descent (ICLR 2021 · similarity 0.92)
- DSD²: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free? (AAAI 2024 · similarity 0.89)
- Double-Descent Curves in Neural Networks: A New Perspective Using Gaussian Processes (AAAI 2024 · similarity 0.89)
- Least Squares Regression Can Exhibit Under-Parameterized Double Descent (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.