Research map / Optimization / Nonconvex and smooth optimization
Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite Sum Structure
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use S-MISO variance reduction instead of SGD | Strongly convex stochastic finite-sum optimization · Strongly convex stochastic finite-sum objective | iteration complexity: SGD: O\left(\frac{L}{\mu} \log \frac{1}{\bar{\epsilon}} ~~+~~ \frac{\sigtot^2}{\mu \epsilon}\right); S-MISO: O\left(\lef | Helped |
| Replace SGD with S-MISO | STL-10 image classification with random crop and scaling · STL-10 training fold | training objective suboptimality | Helped |
| Replace SGD and other compared methods with S-MISO | Pretrained ResNet-50 last-layer retraining with random color adjustments · ImageNet-derived 100-image binary classification task | training objective suboptimality | Helped |
| Use S-MISO instead of SGD with Dropout | Breast cancer gene-expression classification · Gene expression data | training objective suboptimality | Helped |
| Use S-MISO-NU instead of uniform-sampling methods | IMDB sentiment classification with Dropout · IMDB labeled data | convergence rate | Helped |
| Add Dropout with δ = 0.1 | S-MISO-NU sentiment classifier · IMDB test set | test accuracy: 88.51% -> 88.68 ± 0.03% | Helped |
| Add Dropout with δ = 0.3 | S-MISO-NU sentiment classifier · IMDB test set | test accuracy: 88.51% -> 88.86 ± 0.11% | Helped |
| Use iterate averaging (ablation) | S-MISO · Strongly convex stochastic finite-sum objective | function-value complexity: O(L σp² / μ² ε) -> O(σp² / μ ε) | Helped |
| Use non-uniform sampling based on example smoothness (ablation) | S-MISO on heterogeneous examples · IMDB labeled data | initial convergence rate | Helped |
| Increase Dropout rate (ablation) | S-MISO · Gene-expression classification | training objective suboptimality | Hurt |
| Use N-SAGA instead of S-MISO | STL-10 image classification with augmentation · STL-10 training fold | training objective suboptimality | Hurt |
Closest papers by meaning
- Lightweight Stochastic Optimization for Minimizing Finite Sums with Infinite Data (ICML 2018 · similarity 0.91)
- Variance Reduction for Faster Non-Convex Optimization (ICML 2016 · similarity 0.91)
- Efficient Continual Finite-Sum Minimization (ICLR 2024 · similarity 0.90)
- Demystifying SGD with Doubly Stochastic Gradients (ICML 2024 · similarity 0.89)
- Adaptive Variance Reduction for Stochastic Optimization under Weaker Assumptions (NeurIPS 2024 · similarity 0.89)
- Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.