Research map / Optimization / Nonconvex and smooth optimization
On the Convergence of Stochastic Gradient Descent with Bandwidth-based Step Size
Xiaoyu Wang, Ya-xiang Yuan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add up-down policy (1/t Fix-Exp) to fixed-period exponentially decaying step size | SGD, ASGD, Momentum with Fix-Exp step size · CIFAR-10 (VGG-16) | test accuracy / loss | Improved |
| add up-down policy to step size for Adam | Adam with Fix-Exp step size · CIFAR-10 and CIFAR-100 | test accuracy / loss | No clear effect |
| use 1/t Grow-Exp instead of Grow-Exp step size | Epoch-SGD · CIFAR-100 (ResNet-18) | test performance | Improved |
| use 1/t Fix-Exp instead of Fix-Exp, triangular and cosine annealing step sizes | Momentum · CIFAR-100 (ResNet-18, 5 runs) | test loss and accuracy | Improved |
Closest papers by meaning
- A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance (ICML 2021 · similarity 0.92)
- On the Convergence of Step Decay Step-Size for Stochastic Optimization (NeurIPS 2021 · similarity 0.92)
- On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems (NeurIPS 2020 · similarity 0.91)
- The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations (ICML 2025 · similarity 0.89)
- Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent (AAAI 2025 · similarity 0.89)
- Gradient Descent Converges Arbitrarily Fast for Logistic Regression via Large and Adaptive Stepsizes (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
