Research map / Optimization / Optimizers
First-Order Adaptive Sample Size Methods to Reduce Complexity of Empirical Risk Minimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive sample sizes with GD | GD on RCV1 · RCV1 | effective passes to reach 8% test error: 55 effective passes; GD can not improve the test error even after 100 passes | Helped |
| Use adaptive sample sizes with AGD | AGD on RCV1 · RCV1 | speed to reach 8% test error: about 5 times faster | Helped |
| Use adaptive sample sizes with SVRG | SVRG on RCV1 · RCV1 | computational complexity | Helped |
| Use adaptive sample sizes with GD | GD on MNIST with regularization O(1/sqrt(n)) · MNIST (digits 0 and 8), regularization O(1/sqrt(n)) | speed to reach suboptimality 10^-2: almost 6 times faster | Helped |
| Use adaptive sample sizes with AGD and SVRG | AGD and SVRG on MNIST with regularization O(1/sqrt(n)) · MNIST (digits 0 and 8), regularization O(1/sqrt(n)) | speed to reach suboptimality 10^-2 | Helped |
| Use adaptive sample sizes with GD, AGD, and SVRG | GD, AGD, and SVRG on MNIST · MNIST (digits 0 and 8), regularization O(1/sqrt(n)) | passes to reach minimum test error: 2.5% | Helped |
| Use adaptive sample sizes with GD, AGD, and SVRG | GD, AGD, and SVRG on MNIST · MNIST (digits 0 and 8), regularization O(1/n) | speed over fixed-sample-size methods: almost 10 times faster | Helped |
| Change regularization from order 1/sqrt(n) to order 1/n (ablation) | Adaptive and fixed-sample-size methods on MNIST · MNIST (digits 0 and 8) | minimum test error: 2.5% -> 1% | Helped |
| Change regularization from order 1/sqrt(n) to order 1/n (ablation) | Adaptive sample size methods on MNIST · MNIST (digits 0 and 8) | speed advantage over fixed-sample-size methods: almost 10 times faster | Helped |
Closest papers by meaning
- Exploiting Local Convergence of Quasi-Newton Methods Globally: Adaptive Sample Size Approach (NeurIPS 2021 · similarity 0.93)
- Adaptive Newton Method for Empirical Risk Minimization to Statistical Accuracy (NeurIPS 2016 · similarity 0.91)
- On the Asymptotic Distribution of the Minimum Empirical Risk (ICML 2024 · similarity 0.88)
- Boosting Test Performance with Importance Sampling--a Subpopulation Perspective (AAAI 2025 · similarity 0.87)
- On the Efficiency of ERM in Feature Learning (NeurIPS 2024 · similarity 0.87)
- Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.