Research map / Optimization / Implicit bias and generalization
Benefit of deep learning with non-convex noisy gradient descent: Provable excess risk bound and superiority to kernel methods
Taiji Suzuki, Shunta Akiyama
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NGD with the additional smoothness condition indexed by q | NGD two-layer neural network · Teacher-student regression | expected excess risk: n^{-\frac{\gamma}{\alpha_1 + q \alpha_2 + 1}}+ \Xi_k | Improved |
| Use NGD's non-convex neural-network estimator rather than a linear estimator | NGD neural network and linear estimators · Teacher-student regression, fixed input dimension | excess risk dependence on input dimension: n^{-\frac{\gamma}{\alpha_1 + 1}} | Improved |
| Run NGD for more iterations and use a smaller step size | NGD two-layer neural network · Teacher-student regression | difference in expected risk from the stationary-distribution risk: C_{0} \exp(- \Lambda_\eta^* \eta k) + C_1 \frac{\sqrt{\beta}}{\Lambda^*_0}\eta^{1/2-a} | Improved |
Closest papers by meaning
- Diving into the shallows: a computational perspective on large-scale shallow learning (NeurIPS 2017 · similarity 0.89)
- Benign overfitting in ridge regression (JMLR 2023 · similarity 0.88)
- The Benefits of Implicit Regularization from SGD in Least Squares Problems (NeurIPS 2021 · similarity 0.88)
- On the Noisy Gradient Descent that Generalizes as SGD (ICML 2020 · similarity 0.88)
- Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods (ICLR 2025 · similarity 0.87)
- Risk Bounds of Accelerated SGD for Overparameterized Linear Regression (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
