Research map / Optimization / Implicit bias and generalization
Subquadratic Overparameterization for Shallow Neural Networks
Chaehwan Song, Ali Ramezani-Kebrya, Thomas Pethick, Armin Eftekhari, Volkan Cevher
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the initialization ratio \omega_2/\omega_1 while fixing the product \omega_1\omega_2 | Student shallow networks · MNIST teacher-student setup | generalization error | Improved |
| Make the initialization ratio \omega_2/\omega_1 tend to zero | Shallow neural networks · Lazy-training analysis | distance from the linearized training trajectory | Hurt |
| Make the initialization ratio \omega_2/\omega_1 tend to infinity | Shallow neural networks · Lazy-training analysis | distance from the linearized training trajectory | Improved |
Closest papers by meaning
- On the Explicit Role of Initialization on the Convergence and Implicit Bias of Overparametrized Linear Networks (ICML 2021 · similarity 0.90)
- High-Dimensional Analysis of Gradient Flow for Extensive-Width Quadratic Neural Networks (JMLR 2026 · similarity 0.89)
- An Improved Analysis of Training Over-parameterized Deep Neural Networks (NeurIPS 2019 · similarity 0.89)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.87)
- Simplicity Bias and Optimization Threshold in Two-Layer ReLU Networks (ICML 2025 · similarity 0.87)
- Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
