Research map / Optimization / Optimizers
Stochastic Modified Equations and Adaptive Stochastic Gradient Algorithms
Qianxiao Li, Cheng Tai, Weinan E
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the controlled learning-rate policy instead of Adagrad | SGD · M0 (MNIST) | test accuracy: 0.923 -> 0.925 | Improved |
| use the controlled learning-rate policy instead of Adam | SGD · M0 (MNIST) | test accuracy: 0.924 -> 0.925 | Improved |
| use the controlled learning-rate policy instead of Adagrad | SGD · C0 (CIFAR-10) | test accuracy: 0.457 -> 0.461 | Improved |
| use the controlled learning-rate policy instead of Adam | SGD · C0 (CIFAR-10) | test accuracy: 0.460 -> 0.461 | Improved |
| use the controlled learning-rate policy instead of Adam | SGD · C1 (CIFAR-10) | test accuracy: 0.881 -> 0.875 | Hurt |
| use the controlled momentum policy instead of fixed-momentum MSGD | MSGD · M0 (MNIST) | test accuracy: 0.914 -> 0.924 | Improved |
| use the controlled momentum policy instead of MSGD-A | MSGD · M0 (MNIST) | test accuracy: 0.908 -> 0.924 | Improved |
| use the controlled momentum policy instead of fixed-momentum MSGD | MSGD · C0 (CIFAR-10) | test accuracy: 0.453 -> 0.461 | Improved |
| use the controlled momentum policy instead of MSGD-A | MSGD · C0 (CIFAR-10) | test accuracy: 0.446 -> 0.461 | Improved |
| use the controlled momentum policy instead of fixed-momentum MSGD | MSGD · C1 (CIFAR-10) | test accuracy: 0.868 -> 0.876 | Improved |
| use the controlled momentum policy instead of MSGD-A | MSGD · C1 (CIFAR-10) | test accuracy: 0.869 -> 0.876 | Improved |
Closest papers by meaning
- Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations (JMLR 2019 · similarity 0.92)
- A Latent Variational Framework for Stochastic Optimization (NeurIPS 2019 · similarity 0.88)
- An Efficient High-dimensional Gradient Estimator for Stochastic Differential Equations (NeurIPS 2024 · similarity 0.88)
- Derivatives of Stochastic Gradient Descent in parametric optimization (NeurIPS 2024 · similarity 0.87)
- Adaptive Algorithms with Sharp Convergence Rates for Stochastic Hierarchical Optimization (NeurIPS 2025 · similarity 0.85)
- Convergence Rate in a Nonlinear Two-Time-Scale Stochastic Approximation with State (Time)-Dependence (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
