Research map / Optimization / Optimizers
Optimal Mini-Batch and Step Sizes for SAGA
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the practical expected-smoothness estimate to choose the SAGA step size and mini-batch size | b-nice SAGA · LIBSVM and UCI datasets | empirical total complexity | Helped |
| Use the practical step size and mini-batch size instead of the Hofmann settings with b=20 | b-nice SAGA · Ridge regression and regularized logistic regression datasets | empirical performance | Helped |
| Choose the practical mini-batch size instead of the grid-search mini-batch size | b-nice SAGA · Ridge regression, unscaled slice dataset | empirical total complexity | No clear effect |
| Increase the mini-batch size up to the predicted optimal size | b-nice SAGA · Strongly convex GLMs | total complexity | Helped |
| Increase the mini-batch size beyond the predicted optimal size | b-nice SAGA · Strongly convex GLMs | total complexity | Hurt |
| Use the practical expected-smoothness estimate instead of the exact expected-smoothness constant (ablation) | b-nice SAGA · Artificial ridge-regression datasets | expected smoothness | No clear effect |
| Use the simple bound instead of the Bernstein bound (ablation) | b-nice SAGA · Datasets with n much greater than d | expected-smoothness bound tightness | Helped |
| Use the Bernstein bound instead of the simple bound (ablation) | b-nice SAGA · Datasets with d close to or greater than n, or scaled data | expected-smoothness bound tightness | Helped |
| Use the practical step size instead of the Hofmann step size for mini-batches larger than 5 (ablation) | b-nice SAGA · Artificial and real datasets | step size | Helped |
Closest papers by meaning
- Fast Variance Reduction Method with Stochastic Batch Size (ICML 2018 · similarity 0.90)
- SAGA with Arbitrary Sampling (ICML 2019 · similarity 0.90)
- SGD: General Analysis and Improved Rates (ICML 2019 · similarity 0.88)
- A Short and Unified Convergence Analysis of the SAG, SAGA, and IAG Algorithms (ICML 2026 · similarity 0.88)
- A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates (ICML 2018 · similarity 0.87)
- Lightweight Stochastic Optimization for Minimizing Finite Sums with Infinite Data (ICML 2018 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.