Research map / Optimization / Nonconvex and smooth optimization
A Universally Optimal Multistage Accelerated Stochastic Gradient Method
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multistage accelerated gradient with geometrically growing stage lengths and shrinking stepsizes, without noise-level or initial-gap information | M-ASG · Strongly convex smooth optimization | expected suboptimality: \mathcal{O}(1) \left( \exp(-\tfrac{n}{C\sqrt{\kappa}})(f(x_0^0)-f^*) + \frac{\sigma^2 }{n \mu} \right) | Helped |
| Run M-ASG for one stage in the absence of noise | M-ASG · Strongly convex smooth optimization, noise-free | expected suboptimality: 2 \exp(-n/\sqrt{\kappa}) (f(x_0^0)-f^*) | Helped |
| Use an informed first-stage length based on the noise level and initial optimality gap | M-ASG* · Strongly convex smooth optimization | expected suboptimality: 36 (1+\log(8)) \frac{\sigma^2 }{(n-n_1) \mu} | Helped |
| Use the noise-agnostic one-stage stepsize | M-ASG · Strongly convex smooth optimization | expected suboptimality: \frac{2}{n^p} (f(x_0^0)-f^*) + \frac{p \sigma^2 \log n}{n \mu} | Helped |
| Use M-ASG or M-ASG* instead of the compared methods | M-ASG and M-ASG* · Strongly convex quadratic objective | function error | Mixed |
| Use M-ASG instead of the compared methods | M-ASG · M-NIST logistic regression, digit zero versus digit eight | function error | Mixed |
Closest papers by meaning
- A Generic Acceleration Framework for Stochastic Composite Optimization (NeurIPS 2019 · similarity 0.91)
- On the Convergence of Nesterov’s Accelerated Gradient Method in Stochastic Settings (ICML 2020 · similarity 0.91)
- Accelerated Zeroth-order Method for Non-Smooth Stochastic Convex Optimization Problem with Infinite Variance (NeurIPS 2023 · similarity 0.91)
- Universal Gradient Methods for Stochastic Convex Optimization (ICML 2024 · similarity 0.90)
- Nesterov acceleration despite very noisy gradients (NeurIPS 2024 · similarity 0.90)
- Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.