Research map / Optimization / Nonconvex and smooth optimization
A unified variance-reduced accelerated gradient method for convex optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use direct acceleration with an epoch schedule that grows and then levels off | Varag · Smooth convex finite-sum optimization | gradient evaluations: ${\cal O} \left\{m \log m + \sqrt{\tfrac{m D_0}{\epsilon}} \right\}$ | Helped |
| Use a unified step-size policy that adapts to the strong-convexity regime | Varag · Smooth convex finite-sum optimization | gradient evaluations: ${\cal O}\left\{ m \log \tfrac{D_0}{\epsilon} \right\}; ${\cal O} \left\{m \log m +\sqrt{\tfrac{m D_0}{\epsilon}} \right | Helped |
| Allow strong convexity to come from the smooth data-fidelity term rather than requiring it in the regularizer | Varag · Lasso regression | training loss | Helped |
| Apply Varag to unconstrained logistic regression without a strongly convex perturbation | Varag · Unconstrained logistic regression | training loss | Helped |
| Use Varag with restarts under an error-bound condition | Varag · Finite-sum optimization under an error bound | gradient evaluations: ${\cal O}\left\{\big(m+\sqrt{\tfrac{m L}{\bar{\mu}}}\big)\log \tfrac{\psi({x}^0) - \psi(x^*)}{\epsilon}\right\} | Helped |
| Use mini-batches of size $b$ in Varag | Varag · Mini-batch finite-sum optimization | parallel speedup: factor $b$ | Helped |
| Use Varag rather than FGM | Finite-sum convex quadratic problems satisfying an error bound · Convex quadratic problems | training loss | Helped |
| Set $\alpha_s=1$ and $p_s=0$, removing the accelerated multi-sequence scheme (ablation) | Varag · Smooth strongly convex finite-sum optimization | gradient evaluations: ${\cal O}\{(m+L/\mu)\log(1/\epsilon)\} | Hurt |
Closest papers by meaning
- Adaptive Accelerated (Extra-)Gradient Methods with Variance Reduction (ICML 2022 · similarity 0.92)
- Variance Reduction via Accelerated Dual Averaging for Finite-Sum Optimization (NeurIPS 2020 · similarity 0.90)
- Dissipativity Theory for Accelerating Stochastic Variance Reduction: A Unified Analysis of SVRG and Katyusha Using Semidefinite Programs (ICML 2018 · similarity 0.90)
- Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite Sum Structure (NeurIPS 2017 · similarity 0.89)
- Decentralized Convex Finite-Sum Optimization with Better Dependence on Condition Numbers (ICML 2024 · similarity 0.89)
- Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.