Research map / Optimization / Implicit bias and generalization
Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Require the Jacobian to be well-conditioned and have bounded or smooth variation near initialization | gradient descent on overparameterized nonlinear least-squares problems · overparameterized nonlinear learning | training residual | Helped |
| Use the proposed local-Jacobian conditions for stochastic gradient descent | SGD on overparameterized nonlinear least-squares problems · overparameterized nonlinear learning | training residual | Helped |
| Use the martingale-based neighborhood argument to permit larger learning rates | SGD on overparameterized nonlinear least-squares problems · overparameterized nonlinear learning | learning rate | Helped |
| Increase the MNIST sample size from n=500 to n=5000 | LeNet trained with Adam · MNIST, normalized misfit 0.2 | FC2 normalized distance: 0.25 -> 0.5 | Mixed |
Closest papers by meaning
- An Improved Analysis of Training Over-parameterized Deep Neural Networks (NeurIPS 2019 · similarity 0.91)
- Convergence Rates of Non-Convex Stochastic Gradient Descent Under a Generic Lojasiewicz Condition and Local Smoothness (ICML 2022 · similarity 0.90)
- Gradient Descent Provably Optimizes Over-parameterized Neural Networks (ICLR 2019 · similarity 0.90)
- Fast Convergence of Natural Gradient Descent for Over-Parameterized Neural Networks (NeurIPS 2019 · similarity 0.90)
- Convergence Rates for Gradient Descent on the Edge of Stability for Overparametrised Least Squares (NeurIPS 2025 · similarity 0.88)
- The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.