Research map / Optimization / Nonconvex and smooth optimization
Non-convex Finite-Sum Optimization Via SCSG Methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| SCSG with fixed batch and inner variance-reduced updates instead of a second fresh batch (SCSG vs mini-batch SGD at same IFO cost) | SCSG vs SGD on MNIST FCN and CNN · MNIST training and validation loss | training/validation log-loss vs IFO calls | Helped |
| Use time-varying batch sizes B_j = j^1.5 (capped at n) in SCSG | SCSG · MNIST training and validation loss | training/validation log-loss | Helped |
| Compare SGD with same time-varying batch sizes as SCSG (ablation) | SGD with increasing batch sizes · MNIST training loss | training log-loss | Helped |
| Increase the mini-batch ratio B_j/b_j in SCSG (ablation) | SCSG with time-varying batch sizes · MNIST training loss | training log-loss | Helped |
| SCSG vs SGD measured by wall clock time | SCSG with time-varying batch sizes · MNIST CNN, wall clock | training and validation log-loss vs seconds | Helped |
| Theoretical: SCSG with constant batch B=min(12H*/eps, n) vs SGD and SVRG on smooth nonconvex finite-sum | SCSG complexity · smooth nonconvex finite-sum | IFO complexity to eps-stationarity: O(eps^-5/3 ∧ n^2/3 eps^-1) vs SGD O(eps^-2) | Helped |
| Theoretical: stepsize scaling eta_j ∝ (B_j/b_j)^(-2/3) in SCSG | SCSG inner loop · smooth nonconvex | epoch gradient bound | No clear effect |
Closest papers by meaning
- Stochastic Variance Reduction for Nonconvex Optimization (ICML 2016 · similarity 0.91)
- Proximal Stochastic Methods for Nonsmooth Nonconvex Finite-Sum Optimization (NeurIPS 2016 · similarity 0.90)
- Universal Gradient Methods for Stochastic Convex Optimization (ICML 2024 · similarity 0.87)
- Stochastic Momentum Methods for Non-smooth Non-Convex Finite-Sum Coupled Compositional Optimization (NeurIPS 2025 · similarity 0.87)
- Stochastic Optimization with Random Search (NeurIPS 2026 · similarity 0.87)
- High-probability complexity bounds for stochastic non-convex minimax optimization (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.