Research map / Optimization / Nonconvex and smooth optimization
Acceleration and Averaging in Stochastic Descent Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add primal-space averaging and tune the learning rate and inverse sensitivity to the noise decay | SAMD · Convex smooth minimization with σ_*(t) = O(t^{α_σ}), α_σ < 1/2 | expected function-value gap: O(t^{α_σ - 1/2}) vs O(t^{max(α_σ - 1/2, -1)}) | Helped |
| Add averaging in the primal space | SAMD · Persistent noise, including increasing volatility under the stated conditions | almost-sure convergence | Helped |
| Tune the dual learning rate and inverse sensitivity to the asymptotic noise rate | SAMD · Vanishing noise, σ_*(t) = O(t^{α_σ}), α_σ < 1/2 | expected function-value gap: O(t^{α_σ - 1/2}) | Helped |
| Use the theoretically predicted α_r = α_s - α_σ + .5 rather than the empirically better slower α_r | SAMD · Vanishing-noise simulations, α_σ = -.5 and α_σ = -1 | decay rate of expected function values: α_r = α_s - α_σ + .3; α_r = α_s - α_σ + .1 | Mixed |
| Use accelerated rather than non-accelerated stochastic mirror descent dynamics | SAMD versus SMD · Simplex-constrained sum-exponential and rank-1 quadratic examples | function-value convergence | Mixed |
| Use accelerated rather than non-accelerated stochastic mirror descent dynamics | SAMD versus SMD · Numerical examples with discretized stochastic dynamics | robustness to discretization step size | Helped |
| Increase the noise covariance magnitude | SAMD · Constant-noise numerical trajectories | oscillation amplitude | Hurt |
Closest papers by meaning
- Continuous and Discrete-time Accelerated Stochastic Mirror Descent for Strongly Convex Functions (ICML 2018 · similarity 0.92)
- Adaptive Averaging in Accelerated Descent Dynamics (NeurIPS 2016 · similarity 0.92)
- Bregman meets Lévy: Stochastic Mirror Descent with Heavy-Tailed Noise in Continuous and Discrete Time (ICML 2026 · similarity 0.91)
- The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations (ICML 2025 · similarity 0.90)
- Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization (NeurIPS 2025 · similarity 0.89)
- Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson–Romberg Extrapolation (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.