Research map / Optimization / Implicit bias and generalization
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use an ℓ2-regularized neural net instead of the NTK kernel prediction function | Neural nets and NTK on synthetic distribution D · Synthetic distribution D | sample complexity: O(d) vs. Ω(d^2) | Helped |
| Add an explicit regularizer to the neural-net training objective | Two-layer neural net on synthetic data · Synthetic data | test accuracy | Helped |
| Add an explicit regularizer to the neural-net training objective | Two-layer neural net on synthetic data · Synthetic data | normalized margin | Helped |
| Increase hidden-layer width | Two-layer neural net · Synthetic data | test error | Helped |
| Increase hidden-layer width | Two-layer neural net · Synthetic data | margin | Helped |
| Add small uniform noise to Wasserstein gradient flow | Infinite-width regularized neural nets · Infinite-width two-layer networks | convergence rate: polynomial | Helped |
Closest papers by meaning
- Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks (ICML 2019 · similarity 0.90)
- Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee (ICLR 2020 · similarity 0.89)
- Gradient Descent Maximizes the Margin of Homogeneous Neural Networks (ICLR 2020 · similarity 0.89)
- Loss Landscapes are All You Need: Neural Network Generalization Can Be Explained Without the Implicit Bias of Gradient Descent (ICLR 2023 · similarity 0.89)
- Generalization in Kernel Regression Under Realistic Assumptions (ICML 2024 · similarity 0.89)
- Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.