Research map / Optimization / Nonconvex and smooth optimization
Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use accelerated clipped-SSTM instead of non-accelerated clipped-SGD | Smooth convex stochastic optimization with heavy-tailed noise · Smooth convex, heavy-tailed noise | stochastic oracle calls: $O\left(\max\left\{\frac{LR_0^2}{\varepsilon}, \frac{\sigma^2R_0^2}{\varepsilon^2}\right\}\ln\left(\frac{LR_0^2}{\vareps | Helped |
| Add gradient clipping to SGD | SGD on heavy-tailed stochastic gradients · Logistic regression: heart and australian | convergence rate | Helped |
| Add gradient clipping to SSTM | SSTM on heavy-tailed stochastic gradients · Logistic regression: diabetes | convergence rate | Helped |
| Use clipped-SSTM instead of SSTM with small batchsizes | Accelerated stochastic optimization · Logistic regression datasets | oscillations | Helped |
| Use periodically decreasing clipping levels (ablation) | clipped-SGD · Logistic regression datasets | oscillations | Helped |
| Set the clipping level too high (ablation) | clipped-SGD · Logistic regression datasets | oscillations | No clear effect |
| Use constant batchsizes with a larger stepsize parameter (ablation) | clipped-SSTM · Smooth convex, heavy-tailed noise | stochastic oracle calls: $O\left(\max\left\{\sqrt{\frac{LR_0^2}{\varepsilon}}, \frac{\sigma^2R_0^2}{\varepsilon^2}\right\}\ln\frac{LR_0^2+\sigma | No clear effect |
| Use medium batchsizes and a larger stepsize parameter (ablation) | clipped-SSTM · Smooth convex, heavy-tailed noise | stochastic oracle calls: $O\left(\max\left\{\frac{LR_0^2}{\varepsilon}, \frac{\sigma^2R_0^2}{\varepsilon^2}\right\}\ln\frac{LR_0^2}{\varepsilon\b | No clear effect |
| Restart clipped-SSTM for strong convexity (ablation) | clipped-SSTM · Smooth strongly convex, heavy-tailed noise | stochastic oracle calls: $O\left(\max\left\{\sqrt{\frac{L}{\mu}}\ln\frac{\mu R^2}{\varepsilon}, \frac{\sigma^2}{\mu\varepsilon}\right\}\ln\left(\ | Helped |
Closest papers by meaning
- High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise (ICML 2024 · similarity 0.93)
- Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping (ICLR 2025 · similarity 0.92)
- Second-order Optimization under Heavy-Tailed Noise: Hessian Clipping and Sample Complexity Limits (NeurIPS 2025 · similarity 0.91)
- Stochastic Zeroth-Order Optimization Under Heavy-Tailed Noise (NeurIPS 2026 · similarity 0.91)
- Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions (NeurIPS 2025 · similarity 0.90)
- High-Probability Bound for Non-Smooth Non-Convex Stochastic Optimization with Heavy Tails (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.