Research map / Optimization / Nonconvex and smooth optimization
Nonconvex Variance Reduced Optimization with Arbitrary Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| importance sampling (optimal independent sampling S*) instead of uniform sampling (ablation) | SVRG, SAGA, SARAH non-convex minibatch methods · LIBSVM australian, covtype, ijcnn1, splice (logistic-type regression loss) | gradient norm / function value convergence: up to 4 orders of magnitude on australian (b=2) | Helped |
| minibatch size increase with optimal sampling S* (ablation) | SVRG, SAGA, SARAH · LIBSVM dataset (simulated multi-core, cores = minibatch size) | iterations/epochs to convergence: linear and sometimes superlinear speedup up to minibatch size 250 | Helped |
| approximate independent sampling Sa instead of independent sampling S* (ablation) | SVRG/SAGA/SARAH importance sampling · LIBSVM datasets | convergence speed | No clear effect |
Closest papers by meaning
- Stochastic Variance Reduction for Nonconvex Optimization (ICML 2016 · similarity 0.89)
- Variance Reduction for Faster Non-Convex Optimization (ICML 2016 · similarity 0.87)
- Stochastic Nested Variance Reduction for Nonconvex Optimization (NeurIPS 2018 · similarity 0.87)
- Towards closing the gap between the theory and practice of SVRG (NeurIPS 2019 · similarity 0.86)
- Online Variance Reduction with Mixtures (ICML 2019 · similarity 0.86)
- Hybrid Variance-Reduced SGD Algorithms For Minimax Problems with Nonconvex-Linear Function (NeurIPS 2020 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.