Research map / Optimization / Nonconvex and smooth optimization
The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Schedule momentum as β₁ₜ = t/(t+2) instead of using constant momentum | Adaptive HB · Constrained convex problems, individual convergence | function suboptimality rate: O(\frac{\log t}{\sqrt {t}}) -> O(\frac{1}{\sqrt{t}}) | Helped |
| Use constant momentum β₁ₜ ≡ β < 1 rather than a time-varying schedule | Adaptive HB · Constrained convex problems, averaging convergence | function suboptimality rate: O(\frac{1}{\sqrt{t}}) | Helped |
| Use the schedule βₜ = t/(t+2) for momentum | HB · Constrained convex problems, individual convergence | function suboptimality rate: O(\frac{\log t}{\sqrt {t}}) -> O( \frac{1}{\sqrt{t}}) | Helped |
| Use βₜ = t/(t+2) rather than constant momentum | HB · Constrained convex problems, individual convergence | function suboptimality rate: O( \frac{1}{\sqrt{t}}) | Helped |
| Use the adaptive HB momentum schedule β₁ₜ = t/(t+2) | Adaptive HB · 4-layer CNN and ResNet-18 on CIFAR10, CIFAR100, and MNIST | training loss and test accuracy | Helped |
| Use Adaptive HB instead of PSG | Adaptive HB · Synthetic constrained convex function, T=1000 and T=5000 | individual function value | Helped |
| Use stochastic Adaptive HB on hinge-loss problems | Adaptive HB · A9a, W8a, Covtype, Ijcnn1, Rcv1, and Realsim | individual function suboptimality | Helped |
Closest papers by meaning
- Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise (ICLR 2024 · similarity 0.92)
- Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance (ICLR 2025 · similarity 0.92)
- Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration (NeurIPS 2026 · similarity 0.88)
- Accelerated Over-Relaxation Heavy-Ball Method: Achieving Global Accelerated Convergence with Broad Generalization (ICLR 2025 · similarity 0.88)
- Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent (ICML 2025 · similarity 0.88)
- Non-convex Stochastic Composite Optimization with Polyak Momentum (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.