Research map / Optimization / Nonconvex and smooth optimization
Kill a Bird with Two Stones: Closing the Convergence Gaps in Non-Strongly Convex Optimization by Directly Accelerated SVRG with Double Compensation and Snapshots
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Directly accelerated double-snapshot SVRG with compensated gradient estimator (DAVIS) versus Katyusha direct acceleration | non-SC unconstrained problem (1) · theoretical convergence | convergence rate: O(1/S2) to O(1/(nS2)) | Helped |
| Replace Katyusha-style reduction with DAVIS (double snapshot + compensation) to reach optimal complexity | non-SC unconstrained problem (1) · theoretical oracle complexity | oracle complexity: O(n/√ϵ + sqrt(nL/ϵ)) to O(n + sqrt(nL/ϵ)) | Helped |
| Double-snapshot DAVIS-ADMM with momentum weight θs/m (removing the θ1 ≤ constraint of ASVRG-ADMM) | equality-constrained problem (2) · theoretical convergence | convergence rate: O(1/S2) (with boundedness assumption) to O(1/(nS)) without it | Helped |
| Stochastic ADMM mini-batch/proximal variant ASVRG-ADMM versus DAVIS-ADMM on graph-guided fused Lasso | DAVIS-ADMM · graph-guided fused Lasso, Adult and Covtype | training loss vs effective passes | Helped |
| Replace SVRG estimator with DAVIS compensated estimator on ℓ1 logistic regression | DAVIS · ℓ1-norm regularized logistic regression, Adult and Covtype | training loss vs effective passes | Helped |
Closest papers by meaning
- Stochastic Variance Reduction for Nonconvex Optimization (ICML 2016 · similarity 0.91)
- Dissipativity Theory for Accelerating Stochastic Variance Reduction: A Unified Analysis of SVRG and Katyusha Using Semidefinite Programs (ICML 2018 · similarity 0.90)
- Variance Reduction via Accelerated Dual Averaging for Finite-Sum Optimization (NeurIPS 2020 · similarity 0.90)
- Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction (NeurIPS 2024 · similarity 0.89)
- Faster Accelerated First-order Methods for Convex Optimization with Strongly Convex Function Constraints (NeurIPS 2024 · similarity 0.88)
- Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.