Research map / Learning theory and algorithms / Kernels and regression theory
Phenomenology of Double Descent in Finite-Width Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase model capacity through the interpolation threshold | Finite-width neural networks trained with MSE · Downscaled MNIST | population loss | Mixed |
| Increase model capacity through the interpolation threshold | Finite-width neural networks trained with cross-entropy · Downscaled MNIST | population loss | Mixed |
| Use cross-entropy rather than MSE | Finite-width neural networks · Near the interpolation threshold | interpolation-threshold location: p≈Kn -> p≈n | Mixed |
| Train sufficiently long near the interpolation threshold | Finite-width neural networks trained with MSE · Downscaled MNIST | minimum non-zero Hessian eigenvalue | No clear effect |
| Train sufficiently long near the interpolation threshold | Finite-width neural networks trained with cross-entropy · Near p≈n | Hessian spectrum | Hurt |
| Add regularization of suitable magnitude | Finite-width neural networks · Double descent | population loss peak | Helped |
| Increase label noise | Finite-width neural networks · Double descent | population loss peak | Hurt |
| Include the second-order influence term in the leave-one-out estimate | Influence-function LOO for ordinary least squares · Ordinary least squares | LOO estimate fidelity | Helped |
| Use only the first-order influence term for leave-one-out estimation | Influence-function LOO for ordinary least squares · Ordinary least squares | LOO estimate fidelity | Hurt |
Closest papers by meaning
- On the Role of Optimization in Double Descent: A Least Squares Study (NeurIPS 2021 · similarity 0.93)
- Double-Descent Curves in Neural Networks: A New Perspective Using Gaussian Processes (AAAI 2024 · similarity 0.90)
- Unraveling the Enigma of Double Descent: An In-depth Analysis through the Lens of Learned Feature Space (ICLR 2024 · similarity 0.89)
- Investigating the Overlooked Hessian Structure: From CNNs to LLMs (ICML 2025 · similarity 0.86)
- Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss (ICML 2024 · similarity 0.86)
- DSD²: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free? (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.