Research map / Optimization / Optimizers
Adaptive Gradient Descent without Descent
Yura Malitsky, Konstantin Mishchenko
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive stepsizes estimated from gradients instead of SGD's tuned schedule | CIFAR10 ResNet-18 training · CIFAR10 | test accuracy | Improved |
| Use adaptive stepsizes estimated from gradients instead of SGD's tuned schedule | CIFAR10 ResNet-18 training · CIFAR10 | early-stage training speed | Hurt |
| Use adaptive stepsizes estimated from gradients instead of SGD's tuned schedule | CIFAR10 ResNet-18 training · CIFAR10 | training stability | Hurt |
| Change the logistic-regression dataset | Barzilai-Borwein stepsize · Logistic regression, mushrooms/covtype | convergence | Mixed |
| Use Armijo line search rather than AdGD | Logistic regression · Logistic regression, mushrooms/covtype/w8a | iteration cost: approximately 2 times more expensive than AdGD | Hurt |
| Use Nesterov GD with line search rather than AdGD | Logistic regression · Logistic regression, mushrooms/covtype/w8a | iteration cost: 4 times more expensive | Hurt |
| Use the same stochastic sample to estimate the stepsize and compute the update | Adaptive SGD for overparameterized models · Neural network experiments | performance | Improved |
Closest papers by meaning
- Adaptive Proximal Gradient Method for Convex Optimization (NeurIPS 2024 · similarity 0.90)
- Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity (ICLR 2025 · similarity 0.89)
- Directional Smoothness and Gradient Methods: Convergence and Adaptivity (NeurIPS 2024 · similarity 0.89)
- Adaptive Proximal Gradient Methods Are Universal Without Approximation (ICML 2024 · similarity 0.89)
- Toward a Unified Theory of Gradient Descent under Generalized Smoothness (ICML 2025 · similarity 0.88)
- Adaptive backtracking line search (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
