Research map / Optimization / Implicit bias and generalization
Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use refined iterate and Hessian eigenvalue bounds for GD stability analysis | GD for shallow neural networks · General case | overparameterization requirement: m\gtrsim (\eta T)^5 -> m\gtrsim (\eta T)^3 | Helped |
| Extend the stability analysis from GD to SGD | SGD for shallow neural networks · General case | excess risk bound: O(n^{-\frac{\alpha}{1+\alpha}}) | Helped |
| Use SGD instead of full-batch GD | Shallow neural networks · General case | computation: O(n^{\frac{\alpha+2}{\alpha+1}}) -> O(n) | Helped |
| Train in the low-noise setting where L(W*)=0 | GD for shallow neural networks · Low noise | excess risk bound: O(n^{-\frac{\alpha}{1+\alpha}}) -> O(n^{-\alpha}) | Helped |
| Train in the low-noise setting where L(W*)=0 | SGD for shallow neural networks · Low noise | excess risk bound: O(n^{-\frac{\alpha}{1+\alpha}}) -> O(n^{-\alpha}) | Helped |
| Use the refined stability and iterate analysis | GD for shallow neural networks · Low noise | overparameterization requirement: m\gtrsim (\eta T)^5\asymp n^5 -> m\gtrsim (\eta T)^3\asymp n^3 | Helped |
| Use SGD rather than GD | Shallow neural networks · Low noise | computation: O(n^{2}) -> O(n) | Helped |
Closest papers by meaning
- Stability & Generalisation of Gradient Descent for Shallow Neural Networks without the Neural Tangent Kernel (NeurIPS 2021 · similarity 0.94)
- Stability and Generalization of Learning Algorithms that Converge to Global Optima (ICML 2018 · similarity 0.93)
- A Precise Characterization of SGD Stability Using Loss Surface Geometry (ICLR 2024 · similarity 0.90)
- A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Bias (NeurIPS 2025 · similarity 0.90)
- Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks (ICML 2024 · similarity 0.90)
- On the Unstable Convergence Regime of Gradient Descent (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.