Research map / Optimization / Nonconvex and smooth optimization
Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descent
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use on-average model stability instead of requiring bounded gradients | SGD on convex smooth losses · Low-noise setting, F(w*) = 0 | excess generalization error: O(1/√n) -> O(1/n) | Helped |
| Relax strong smoothness to Hölder-continuous subgradients | SGD on convex losses · α ≥ 1/2 | excess generalization error: O(n^{-1/2}) | No clear effect |
| Relax smoothness to Hölder-continuous subgradients and increase iterations | SGD on convex losses · α < 1/2 | excess generalization error: O(n^{-1/2}); T ≍ n^{(2-α)/(1+α)} | Mixed |
| Use the low-noise condition F(w*) = 0 with Hölder-continuous subgradients | SGD on convex losses · Low-noise setting | excess generalization error: O(n^{-(1+α)/2}) | Helped |
| Apply the stability analysis to non-smooth hinge loss | SGD · Hinge loss (α = 0) | excess generalization error: O(n^{-1/2}); T ≍ n² | No clear effect |
| Require convexity of the empirical objective instead of convexity of every individual loss | SGD with smooth losses · Convex empirical objective, potentially non-convex individual losses | excess generalization error: O(n^{-1/2}) | No clear effect |
| Require strong convexity of the empirical objective instead of strong convexity of every individual loss | SGD with smooth losses · Strongly convex empirical objective, potentially non-convex individual losses | excess generalization error: O(E_S[1/(nσ_S)]) | No clear effect |
Closest papers by meaning
- Uniform Stability and Generalization Error of GD and SGD on Fixed-Point Parameters (NeurIPS 2026 · similarity 0.94)
- Error Analysis Affected by Heavy-Tailed Gradients for Non-Convex Pairwise Stochastic Gradient Descent (AAAI 2025 · similarity 0.94)
- Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm (ICML 2024 · similarity 0.93)
- Stability and Generalization Analysis of Decentralized SGD: Sharper Bounds Beyond Lipschitzness and Smoothness (ICML 2025 · similarity 0.92)
- A Precise Characterization of SGD Stability Using Loss Surface Geometry (ICLR 2024 · similarity 0.92)
- Stability and Generalization of Asynchronous SGD: Sharper Bounds Beyond Lipschitz and Smoothness (NeurIPS 2024 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.