Research map / Optimization / Nonconvex and smooth optimization
Doubly Accelerated Stochastic Variance Reduced Dual Averaging Method for Regularized Empirical Risk Minimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DASVRDA with double acceleration and variance reduction instead of Katyusha | Regularized ERM · Non-strongly convex objectives, mini-batch size b | total computational cost | Helped |
| Use DASVRDA with double acceleration and variance reduction instead of APCG, SPDC, or Katyusha | Regularized ERM · Strongly convex objectives, mini-batch size b | total computational cost | Helped |
| Use DASVRDA rather than Katyusha | Regularized ERM · Strongly convex objectives with L/μ ≥ n | necessary mini-batch size for optimal iteration complexity | Helped |
| Use DASVRDA rather than APCG, SPDC, or Katyusha | Regularized ERM · Non-strongly convex objectives with L/ε ≥ n log²(1/ε) | necessary mini-batch size for optimal iteration complexity | Helped |
| Add warm start to DASVRDA^ns (ablation) | DASVRDA^ns · Non-strongly convex objectives | convergence rate | Helped |
| Add heuristic adaptive restarting to DASVRDA (ablation) | DASVRDA · a9a and rcv1 with non-strongly convex objectives | objective gap | Helped |
| Add heuristic adaptive restarting to DASVRDA (ablation) | DASVRDA · Other experimental settings | objective gap | No clear effect |
| Combine Universal Catalyst with SVRG rather than using SVRG alone (ablation) | SVRG · Regularized logistic regression experiments | objective gap | No clear effect |
| Add Universal Catalyst to AccProxSVRG (ablation) | AccProxSVRG · Regularized logistic regression experiments | objective gap | Mixed |
| Use Katyusha rather than vanilla SVRG | Stochastic variance-reduced optimization · Regularized logistic regression experiments | objective gap | Mixed |
| Use APCG rather than vanilla SVRG | Stochastic variance-reduced optimization · Regularized logistic regression experiments | objective gap | Mixed |
| Use DASVRDA rather than other evaluated methods | Regularized logistic regression · a9a, rcv1, and sido0 experiments | objective gap | Helped |
Closest papers by meaning
- Stochastic Primal-Dual Method for Empirical Risk Minimization with O(1) Per-Iteration Complexity (NeurIPS 2018 · similarity 0.92)
- Variance Reduction via Accelerated Dual Averaging for Finite-Sum Optimization (NeurIPS 2020 · similarity 0.90)
- Variance-based Regularization with Convex Objectives (NeurIPS 2017 · similarity 0.90)
- Variance Reduction via Primal-Dual Accelerated Dual Averaging for Nonsmooth Convex Finite-Sums (ICML 2021 · similarity 0.90)
- A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates (ICML 2018 · similarity 0.89)
- Stochastic Variance Reduced Primal Dual Algorithms for Empirical Composition Optimization (NeurIPS 2019 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.