Research map / Learning theory and algorithms / Kernels and regression theory
More is Better: when Infinite Overparameterization is Optimal and Overfitting is Obligatory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives theoretical backing to the "more is better" practice by showing that infinitely overparameterized, near-interpolating two-layer networks can be optimal and that overfitting is obligatory in certain regimes.
Problem
Empirical deep learning finds that larger models, more data, and more computation (lower training loss, near-interpolation) improve performance, but theory has not explained why this holds.
Approach
Provides theoretical analysis showing that model size, data, and compute each improve performance under near-interpolation, with infinite overparameterization being optimal and overfitting being necessary in the studied setting.
Key results
- Theoretically argues that model size, data, and compute each improve performance when optimizing to near-interpolation, per the abstract's framing.
Surprising findings
- Overfitting (near-interpolation) is argued to be obligatory rather than harmful in the studied regime.
Closest papers by meaning
- Theoretical Limitations of Ensembles in the Age of Overparameterization (ICML 2025 · similarity 0.89)
- Simplicity Bias in Overparameterized Machine Learning (AAAI 2024 · similarity 0.87)
- No Free Lunch from Random Feature Ensembles: Scaling Laws and Near-Optimality Conditions (ICML 2025 · similarity 0.87)
- Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks (ICML 2024 · similarity 0.87)
- Unveil Benign Overfitting for Transformer in Vision: Training Dynamics, Convergence, and Generalization (NeurIPS 2024 · similarity 0.86)
- Rethinking Benign Overfitting in Two-Layer Neural Networks (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.