Research map / Optimization / Nonconvex and smooth optimization
High Probability Guarantees for Nonconvex Stochastic Gradient Descent with Heavy Tails
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Assume gradient noise is heavy-tailed sub-Weibull with tail parameter theta in (1/2, 1] instead of sub-Gaussian light tails | nonconvex SGD optimization bound (gradient norm) · general nonconvex learning, Theorem 3.1 | high-probability bound on average weighted squared gradient norm | No clear effect |
| Increasing tail parameter theta beyond 1 (heavier tails) with bounded-step assumption | nonconvex SGD optimization bound · general nonconvex learning, Theorem 3.1(c) | high-probability bound on average weighted squared gradient norm | Hurt |
| Add PL (gradient dominance) curvature condition to SGD | optimization error of function values · Theorem 3.7 vs Theorem 3.1 | convergence rate order | Helped |
| Add Bernstein-type noise condition at optimum | excess risk bound · Theorem 3.15 vs Theorem 3.11 | excess risk rate in n | Helped |
| Gradient clipping (Algorithm 2) instead of bounded gradient assumption; stepsize independent of smoothness constant | nonconvex SGD with heavy-tailed sub-Weibull noise · Theorem 3.19 | average min(gradient norm, squared gradient norm) high-probability bound | No clear effect |
Closest papers by meaning
- Stability and Generalization of Nonconvex Optimization with Heavy-Tailed Noise (ICML 2026 · similarity 0.93)
- High-probability complexity bounds for stochastic non-convex minimax optimization (NeurIPS 2024 · similarity 0.93)
- Error Analysis Affected by Heavy-Tailed Gradients for Non-Convex Pairwise Stochastic Gradient Descent (AAAI 2025 · similarity 0.92)
- Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping (ICLR 2025 · similarity 0.90)
- Emergence of heavy tails in homogenized stochastic gradient descent (NeurIPS 2024 · similarity 0.90)
- Stochastic Optimization Schemes for Performative Prediction with Nonconvex Loss (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.