Research map / Optimization / Optimizers
On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the distance-based diagnostic to schedule step-size reductions | SGD · Covertype and MNIST parity classification | testing loss | Helped |
| Use the distance-based diagnostic to schedule learning-rate reductions | ResNet18 · CIFAR-10 | accuracy | Helped |
| Use Pflug's consecutive-gradient inner-product statistic for restart decisions | SGD · Least-squares regression | loss | Hurt |
| Use the distance-based diagnostic instead of theoretically tuned averaged-SGD | SGD · Logistic regression on synthetic and real data | testing loss | Helped |
| Use the distance-based diagnostic instead of averaged-SGD with a tuned constant | SGD · Logistic regression | testing loss | No clear effect |
| Use the distance-based diagnostic instead of the tuned averaged-SGD baseline | SGD · SVM | loss | No clear effect |
| Use the distance-based diagnostic instead of the tuned averaged-SGD baseline | SGD · Lasso regression | loss | Helped |
| Use the distance-based diagnostic instead of the parameterized-rate SGD baseline | SGD · Uniformly convex functions | loss | No clear effect |
| Decrease the step size by different factors r (ablation) | Distance-based diagnostic · Least-squares regression | loss | No clear effect |
| Vary the distance-slope threshold (ablation) | Distance-based diagnostic · Logistic regression | loss | No clear effect |
| Vary the distance-slope threshold (ablation) | Distance-based diagnostic · SVM | loss | No clear effect |
| Vary the step-size decrease factor r (ablation) | Distance-based diagnostic · Uniformly convex functions | loss | No clear effect |
Closest papers by meaning
- Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent (AAAI 2025 · similarity 0.93)
- Understanding the unstable convergence of gradient descent (ICML 2022 · similarity 0.91)
- A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance (ICML 2021 · similarity 0.91)
- Derivatives of Stochastic Gradient Descent in parametric optimization (NeurIPS 2024 · similarity 0.88)
- Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime (NeurIPS 2025 · similarity 0.88)
- The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.