Research map / Optimization / Optimizers
Starting Small - Learning with Adaptive Sample Sizes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| dynamically increase the sample size, adding one sample every other update | SAGA · SAGA bounds, T ≥ κ update steps | optimization error: O(log T · H(T)) -> O(H(T)) | Helped |
| use a dynamic sample-size schedule rather than a fixed full sample | SAGA · SAGA, sample size n | update steps to statistical accuracy: n log n -> 2n | Helped |
| force an update on each newly added sample, alternating with uniform resampling of existing data (ablation) | Starting Small · Real datasets | convergence | Helped |
| vary the regularizer to λ = 1/n^p, p < 1, instead of λ = 1/√n (ablation) | Starting Small · Four real datasets | convergence | No clear effect |
Closest papers by meaning
- On the Convergence of Loss and Uncertainty-based Active Learning Algorithms (NeurIPS 2024 · similarity 0.89)
- First-Order Adaptive Sample Size Methods to Reduce Complexity of Empirical Risk Minimization (NeurIPS 2017 · similarity 0.89)
- Using Statistics to Automate Stochastic Optimization (NeurIPS 2019 · similarity 0.89)
- Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws (ICLR 2025 · similarity 0.86)
- Gradient Descent Converges Arbitrarily Fast for Logistic Regression via Large and Adaptive Stepsizes (ICML 2025 · similarity 0.86)
- Understanding the Training Speedup from Sampling with Approximate Losses (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.