Research map / Optimization / Nonconvex and smooth optimization
Fast Rate Analysis of Some Stochastic Optimization Algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Relax strong convexity of the loss f to local strong convexity of the expected loss F around w* for Online Proximal Gradient (step size 2/(βt)) | Online Proximal Gradient (OPG) · stochastic convex learning with locally strongly convex F | convergence rate of G(avg w_T) - G(w*) | Helped |
| Modified RDA adding a strongly convex proximal term gamma * sum ||w_tau - w||^2 with step sizes beta_t = gamma(1+ln t) | Regularized Dual Averaging (RDA) · stochastic convex learning with locally strongly convex F | convergence rate of G(avg w_T) - G(w*) | Helped |
| Use locally strongly convex expected loss with step size 2γ/(βt) in OPG-ADMM | OPG-ADMM · stochastic convex learning with locally strongly convex F | convergence rate of G(avg x_T, avg y'_T) - G(x*,y*) | Helped |
| Analyze Diff_T - Reg_T directly instead of bounding regret and differences separately (martingale argument) | OPG, RDA, OPG-ADMM proof technique · theoretical analysis | convergence rate bound | Helped |
| Lasso experiments with d=300 and d=1000: empirical optimality gap decays with slope -1 on log-log scale after an initial phase T0 (about 300 for d=300, about 1000 for d=1000) | OPG, RDA, OPG-ADMM · synthetic LASSO, d=300 and d=1000, k=10 | optimality gap G(avg w_T)-G(w*) (log-log slope) | No clear effect |
Closest papers by meaning
- Simple Stochastic Gradient Methods for Non-Smooth Non-Convex Regularized Optimization (ICML 2019 · similarity 0.91)
- Stochastic Newton Proximal Extragradient Method (NeurIPS 2024 · similarity 0.90)
- Universal Gradient Methods for Stochastic Convex Optimization (ICML 2024 · similarity 0.89)
- Adaptive Algorithms with Sharp Convergence Rates for Stochastic Hierarchical Optimization (NeurIPS 2025 · similarity 0.88)
- Revisiting the Last-Iterate Convergence of Stochastic Gradient Methods (ICLR 2024 · similarity 0.88)
- Last-Iterate Convergence of Single-Loop Stochastic Methods for Constrained Convex-Concave Minimax Problems (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.