Research map / Optimization / Optimizers
Do Stochastic, Feel Noiseless: Stable Stochastic Optimization via a Double Momentum Mechanism
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a stochastic optimizer (SGD variant with a double momentum mechanism) claimed to be more stable to learning-rate choice and to the noise level, as described in the abstract.
Problem
SGD is sensitive to the learning rate choice, requiring extensive hyperparameter tuning.
Approach
A new SGD variant using a double momentum mechanism that allows the use of a wide range of learning rates and improves stability with respect to stochastic noise, per the abstract's framing.
Experimental setup
Builds on: SGD
Closest papers by meaning
- An Improved Analysis of Stochastic Gradient Descent with Momentum (NeurIPS 2020 · similarity 0.92)
- Global Convergence and Stability of Stochastic Gradient Descent (NeurIPS 2022 · similarity 0.91)
- The Marginal Value of Momentum for Small Learning Rate SGD (ICLR 2024 · similarity 0.91)
- Dynamic Momentum Recalibration in Online Gradient Learning (CVPR 2026 · similarity 0.91)
- Random Scaling and Momentum for Non-smooth Non-convex Optimization (ICML 2024 · similarity 0.90)
- Gradient correlation is a key ingredient to accelerate SGD with momentum (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.