Research map / Optimization / Nonconvex and smooth optimization
Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SGD with stochastic Armijo line-search instead of tuned constant step-size SGD and Adam | SGD · CIFAR-10/CIFAR-100 ResNet-34 and DenseNet-121, MNIST MLP | training loss and test accuracy | Helped |
| add Polyak momentum to SGD with Armijo line-search | SGD+Armijo · deep network classification | training loss and test accuracy | Helped |
| use Nesterov acceleration with Armijo line-search | SGD+Armijo · deep network classification | stability | Hurt |
| use stochastic extra-gradient with Lipschitz line-search instead of SGD+Armijo | deep networks · deep network classification | convergence and performance | Hurt |
| use Goldstein line-search instead of Armijo only (ablation) | SGD · deep network classification | performance | No clear effect |
| vary Armijo constant c in [0.1, 0.5] (ablation) | SGD+Armijo · CIFAR-10 ResNet-34 | performance | No clear effect |
| vary maximum step-size eta_max (ablation) | SGD+Armijo · CIFAR-10 ResNet-34 | performance | No clear effect |
| make model over-parametrized (rank 10 instead of rank 4 factors) with line-search SGD | SGD+Armijo/Goldstein · synthetic matrix factorization | training loss convergence | Helped |
Closest papers by meaning
- Armijo Line-search Can Make (Stochastic) Gradient Descent Provably Faster (ICML 2025 · similarity 0.92)
- SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochs (NeurIPS 2021 · similarity 0.91)
- On the Convergence of mSGD and AdaGrad for Stochastic Optimization (ICLR 2022 · similarity 0.91)
- Aiming towards the minimizers: fast convergence of SGD for overparametrized problems (NeurIPS 2023 · similarity 0.91)
- A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance (ICML 2021 · similarity 0.91)
- The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.