Research map / Optimization / Implicit bias and generalization
Implicit Regularization of Accelerated Methods in Hilbert Spaces
Nicolò Pagliana, Lorenzo Rosasco
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use acceleration instead of gradient descent | Linear least-squares Hilbert-space learning, attainable case · Attainable case | excess risk upper bound: 1/t + t/n -> 1/t^2 + t^2/n | Mixed |
| Use accelerated methods with the stopping rule t^{acc}=O(n^{1/4}) instead of gradient descent with t^{GD}=O(n^{1/2}) | Linear least-squares Hilbert-space learning, attainable case · Attainable case | learning rate: both algorithms have learning rate of order 1/√n | Improved |
| Use acceleration instead of gradient descent | Linear least-squares Hilbert-space learning, general regularity · General regularity with source exponent r and effective-dimension exponent γ | excess risk upper bound: 1/t^{2r} + t^{1/γ}/n -> 1/t^{4r} + t^{2/γ}/n | Mixed |
| Use gradient descent rather than acceleration | Easy problems with large source regularity · Easy problems (large r) | learning accuracy | No clear effect |
Closest papers by meaning
- Provable Acceleration of Nesterov's Accelerated Gradient for Asymmetric Matrix Factorization and Linear Neural Networks (NeurIPS 2024 · similarity 0.88)
- Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent (ICLR 2023 · similarity 0.88)
- Convergence analysis of ODE models for accelerated first-order methods via positive semidefinite kernels (NeurIPS 2023 · similarity 0.88)
- Implicit Regularization in Matrix Factorization (NeurIPS 2017 · similarity 0.88)
- Nesterov acceleration despite very noisy gradients (NeurIPS 2024 · similarity 0.87)
- On the Implicit Bias of Adam (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
