Research map / Optimization / Nonconvex and smooth optimization
Estimate Sequences for Variance-Reduced Stochastic Composite Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add acceleration to random-SVRG | random-SVRG · Strongly convex finite sums without perturbations | gradient evaluations to ε accuracy: O\left( \left(n + \frac{L_Q}{\mu}\right)\log\left(\frac{F(x_0)-F^*}{\varepsilon} \right) \right) -> O\left( \left(n + \s | Helped |
| Use decreasing step sizes instead of a constant step size (ablation) | random-SVRG · Finite sums with stochastic perturbations | asymptotic error: \frac{{\color{alizarin} \tilde{\sigma}^2}}{L} -> 0 | Helped |
| Use decreasing step sizes instead of a constant step size (ablation) | Accelerated random-SVRG · Finite sums with stochastic perturbations | asymptotic error: \frac{{\color{alizarin}\tilde{\sigma}^2}}{\sqrt{n \mu L}+n \mu} -> 0 | Helped |
| Add minibatching to accelerated SGD and decrease the step size (ablation) | SGD · No DropOut; alpha, gene, and ckn-cifar | objective function value | Helped |
| Add acceleration to SVRG (ablation) | random-SVRG · No DropOut; gene and ckn-cifar | objective function value | Helped |
| Add acceleration to SVRG (ablation) | random-SVRG · No DropOut; alpha | objective function value | No clear effect |
| Add acceleration to SVRG (ablation) | random-SVRG-d · DropOut; alpha | objective function value | Helped |
| Add acceleration to SVRG (ablation) | random-SVRG-d · DropOut; gene and ckn-cifar | objective function value | No clear effect |
| Use accelerated random-SVRG-d instead of SGD approaches | optimization methods · DropOut δ=0.01; alpha, gene, and ckn-cifar | objective function value | Helped |
| Use accelerated random-SVRG-d instead of SGD approaches | optimization methods · DropOut δ=0.1; alpha | objective function value | Helped |
| Use accelerated random-SVRG-d instead of SGD approaches | optimization methods · DropOut δ=0.1; gene and ckn-cifar | objective function value | No clear effect |
| Increase the perturbation using DropOut | accelerated SVRG · Perturbed finite-sum optimization | objective function value | Hurt |
Closest papers by meaning
- A Generic Acceleration Framework for Stochastic Composite Optimization (NeurIPS 2019 · similarity 0.91)
- Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction (NeurIPS 2024 · similarity 0.90)
- Universal Gradient Methods for Stochastic Convex Optimization (ICML 2024 · similarity 0.89)
- Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient (ICML 2024 · similarity 0.89)
- Nesterov acceleration despite very noisy gradients (NeurIPS 2024 · similarity 0.88)
- Last-Iterate Convergence of Single-Loop Stochastic Methods for Constrained Convex-Concave Minimax Problems (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.