Research map / Optimization / Nonconvex and smooth optimization
High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the adaptive AdaGrad stepsize without prior knowledge of smoothness or noise variance | AdaGrad · Smooth nonconvex stochastic optimization | average squared gradient norm: O (1 / \sqrt{T}) | Helped |
| Apply the analysis in the deterministic setting | AdaGrad · Smooth nonconvex deterministic optimization | average squared gradient norm: O \br{ \frac{(\Delta_1 + L)^2}{T} } | Helped |
| Use the particular Freedman martingale concentration bound | AdaGrad high-probability analysis · Smooth nonconvex stochastic optimization | dependence on probability margin: \log(1 / \delta) | Helped |
| Assume sub-Gaussian noise | AdaGrad · Smooth nonconvex optimization with sub-Gaussian noise | average squared gradient norm: 1 / \sqrt{T} \to 1/T | Helped |
| Add averaging with \alpha_t = 2/(t+1) | AdaGrad · Smooth nonconvex stochastic optimization | average squared gradient norm: O (1 / \sqrt{T}) | Helped |
| Use adaptive RSAG step sizes | AGD template · Smooth nonconvex stochastic optimization | average squared gradient norm: O (1 / \sqrt{T}) | Helped |
| Use uniform averaging with \alpha_t = 1/t instead of weighted averaging (ablation) | AGD template · Smooth nonconvex optimization | bound on the AGD-template coefficient: 2 -> \log(T+1) | Hurt |
| Use AcceleGrad's time-scale-separated step sizes (ablation) | AGD template analysis · Smooth nonconvex optimization | controllable growth of the second-term bound: \mathcal O (t^4) | Hurt |
Closest papers by meaning
- Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness (ICLR 2025 · similarity 0.91)
- Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction (NeurIPS 2024 · similarity 0.90)
- High-probability complexity bounds for stochastic non-convex minimax optimization (NeurIPS 2024 · similarity 0.89)
- Stability and Sharper Risk Bounds with Convergence Rate $\tilde{O}(1/n^2)$ (NeurIPS 2025 · similarity 0.88)
- High-Probability Bound for Non-Smooth Non-Convex Stochastic Optimization with Heavy Tails (ICML 2024 · similarity 0.88)
- AdaGrad under Anisotropic Smoothness (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.