Research map / Optimization / Implicit bias and generalization

Gaussian Approximation and Concentration of Constant Learning-Rate Stochastic Gradient Descent

NeurIPS 2025 · Optimization › Implicit bias and generalization · approach: SGD noise and flat minima · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Develops finite-sample and asymptotic theory for constant learning-rate SGD, including a quenched CLT via a novel linear approximation technique and concentration results.

Problem

Understanding the distributional behavior (CLT, tail properties, concentration) of SGD iterates when the learning rate is held constant, rather than decaying.

Approach

A linear approximation technique yields a quenched central limit theorem for constant learning-rate SGD iterates with refined tail bounds, showing that fluctuations around the target converge to a multivariate normal regardless of initialization.

Key results

• Quenched CLT: regardless of initialization, SGD iterate fluctuations around the target converge to a multivariate normal distribution (as stated in the abstract).

Closest papers by meaning

• Statistical Guarantees for High-Dimensional Stochastic Gradient Descent (NeurIPS 2025 · similarity 0.92)
• Asymptotic theory of SGD with a general learning-rate (NeurIPS 2025 · similarity 0.92)
• Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance (NeurIPS 2021 · similarity 0.91)
• The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations (ICML 2025 · similarity 0.89)
• Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime (NeurIPS 2025 · similarity 0.89)
• Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson–Romberg Extrapolation (ICLR 2025 · similarity 0.89)