Research map / Optimization / Nonconvex and smooth optimization
Adaptive Accelerated (Extra-)Gradient Methods with Variance Reduction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use accelerated extra-gradient variance reduction with adaptive steps when β is unknown | AdaVRAE · General convex finite-sum optimization | gradient evaluations to attain an O(ε)-suboptimal solution: \mathcal{O}\left(n\log\frac{\beta}{\epsilon}+\frac{\beta}{\epsilon}\right) -> \mathcal{O}\left(n\min\left\{ \log\log\fra | Helped |
| Use adaptive mirror descent with iterate-movement step sizes when β is unknown | AdaVRAG · General convex finite-sum optimization | gradient evaluations to attain an O(ε)-suboptimal solution: \mathcal{O}\left(n\log\frac{\beta}{\epsilon}+\frac{\beta}{\epsilon}\right) -> \mathcal{O}\left(n\min\left\{ \log\log\fra | Helped |
| Use additive rather than multiplicative adaptive step-size updates (ablation) | AdaVRAG · General convex finite-sum optimization | experimental training-objective convergence | Helped |
| Use additive rather than multiplicative adaptive step-size updates (ablation) | AdaVRAG · General convex finite-sum optimization | smoothness dependence of the convergence guarantee: \sqrt{\beta\log\beta} -> \beta | Hurt |
| Gradually increase the checkpoint-combination coefficient instead of setting it to 1/2 initially | AdaVRAG · General convex finite-sum optimization | first-term convergence complexity: \mathcal{O}(n\log n) -> \mathcal{O}(n\log\log n) | Helped |
| Use cumulative rather than epoch-reset step sizes | AdaVRAG and AdaVRAE · Real-world binary classification experiments | training objective | Helped |
| Use a fixed checkpoint coefficient of 1/2 in the first phase | VARAG · Real-world binary classification experiments | early training-objective convergence | Hurt |
Closest papers by meaning
- A unified variance-reduced accelerated gradient method for convex optimization (NeurIPS 2019 · similarity 0.92)
- Variance Reduction via Accelerated Dual Averaging for Finite-Sum Optimization (NeurIPS 2020 · similarity 0.91)
- Variance Reduction for Faster Non-Convex Optimization (ICML 2016 · similarity 0.89)
- Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction (NeurIPS 2024 · similarity 0.88)
- Accelerated Distance-adaptive Methods for Hölder Smooth and Convex Optimization (NeurIPS 2025 · similarity 0.87)
- Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.