Research map / Optimization / Nonconvex and smooth optimization

Normalized SGD in the Convex Regime: First High-Probability Guarantees and Momentum Extension

NeurIPS 2026 · Optimization › Nonconvex and smooth optimization · approach: Stochastic gradient last-iterate analysis

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies normalized SGD for smooth convex optimization, giving first high-probability guarantees and a momentum extension.

Problem

Normalized SGD has been analyzed in non-convex stochastic settings, but its convex theory is much less developed.

Approach

Analyzes normalized gradient methods for smooth convex optimization, providing first high-probability guarantees for normalized SGD in the convex regime and extending the analysis to a momentum variant.

Experimental setup

Builds on: normalized SGD

Closest papers by meaning

• Robustness Analysis of Non-Convex Stochastic Gradient Descent using Biased Expectations (NeurIPS 2020 · similarity 0.92)
• An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias (NeurIPS 2021 · similarity 0.91)
• Error Analysis Affected by Heavy-Tailed Gradients for Non-Convex Pairwise Stochastic Gradient Descent (AAAI 2025 · similarity 0.91)
• Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions (NeurIPS 2025 · similarity 0.90)
• The Sample Complexity of Gradient Descent in Stochastic Convex Optimization (NeurIPS 2024 · similarity 0.90)
• Gradient correlation is a key ingredient to accelerate SGD with momentum (ICLR 2025 · similarity 0.90)