AI Paper Atlas

Research map / Optimization / Nonconvex and smooth optimization

How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD

Zeyuan Allen-Zhu

NeurIPS 2018 · Optimization › Nonconvex and smooth optimization · approach: Stochastic convex optimization complexity · 38 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
apply SGD after SGD instead of using naive SGD for small gradient normconvex stochastic optimization · online convex, gradient norm epsgradient complexity exponent: ε−4 -> ε−8/3Improved
add regularization with sigma proportional to eps before SGD1SGD1 · online convex, gradient norm epsgradient complexity exponent: ε−8/3 -> ε−5/2Improved
use recursive regularization with doubling sigma and warm starts instead of single regularizationSGD2 · online convex, gradient norm epsgradient complexity exponent: ε−5/2 -> ε−2 log^3(1/ε)Improved
apply the recursive-regularization idea to nonconvex objectives (SGD5)nonconvex SGD variants · online nonconvex, approximate local minimumgradient complexity exponent: ε−4 -> ε−3.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy