Research map / Optimization / Nonconvex and smooth optimization
SpiderBoost and Momentum: Faster Variance Reduction Algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase stepsize from eta = 0.01 (SPIDER) to eta = 0.05 (SpiderBoost) | SPIDER/SpiderBoost · logistic regression with nonconvex regularizer, a9a and w8a | function value gap vs passes: 0.01 -> 0.05 | Helped |
| increase stepsize from eta = 0.01 (SPIDER) to eta = 0.05 (SpiderBoost) | SPIDER/SpiderBoost · nonconvex robust linear regression, a9a and w8a | function value gap vs passes: 0.01 -> 0.05 | Helped |
| use a larger stepsize with SPIDER's normalized update | SPIDER · a9a, w8a | final function value gap | Hurt |
| add momentum to Prox-SpiderBoost (Prox-SpiderBoost-M) (ablation) | Prox-SpiderBoost · smooth nonconvex logistic and robust regression, a9a and w8a | function value gap vs epochs | Helped |
| Prox-SpiderBoost-M versus Katyusha-ns and ASVRG | momentum variance-reduced methods · smooth nonconvex problems | function value gap vs epochs | Helped |
| Prox-SpiderBoost-M versus other proximal variance-reduced algorithms with l1 regularizer weight 0.1 | Prox-SpiderBoost-M · nonsmooth nonconvex composite, a9a and w8a | function value gap vs epochs | Helped |
| SpiderBoost with a constant stepsize 1/(2L) instead of SPIDER's epsilon-dependent stepsize (theory) | SpiderBoost · nonconvex finite-sum, complexity | SFO complexity | No clear effect |
| Prox-SpiderBoost instead of ProxSVRG+ (theory) | composite nonconvex optimization · finite-sum complexity | SFO complexity: O(n + n^{2/3} eps^{-2}) -> O(n + n^{1/2} eps^{-2}) | Helped |
Closest papers by meaning
- Improved Zeroth-Order Variance Reduced Algorithms and Analysis for Nonconvex Optimization (ICML 2019 · similarity 0.89)
- Kill a Bird with Two Stones: Closing the Convergence Gaps in Non-Strongly Convex Optimization by Directly Accelerated SVRG with Double Compensation and Snapshots (ICML 2022 · similarity 0.89)
- Stochastic Nested Variance Reduction for Nonconvex Optimization (NeurIPS 2018 · similarity 0.88)
- Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient (ICML 2024 · similarity 0.87)
- Stochastic Optimization with Random Search (NeurIPS 2026 · similarity 0.87)
- Freya PAGE: First Optimal Time Complexity for Large-Scale Nonconvex Finite-Sum Optimization with Heterogeneous Asynchronous Computations (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.