Research map / Optimization / Nonconvex and smooth optimization
Improved Analysis of Clipping Algorithms for Non-convex Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the clipping framework instead of SGD | Stochastic non-convex optimization on (L0,L1)-smooth functions · Stochastic setting | gradient complexity: \mathcal{O}\left( \Delta (L_0+L_1M)\sigma^2\varepsilon^{-4} \right) -> \mathcal{O}\left(\Delta L_0\sigma^2\varepsilon^{- | Helped |
| Use the clipping framework instead of the earlier clipped SGD bound | Stochastic non-convex optimization on (L0,L1)-smooth functions · Stochastic setting | gradient complexity: \mathcal{O}\left((\Delta + (L_0+L_1\sigma)\sigma^2+\sigma L_0^2/L_1 )^2\varepsilon^{-4} \right) -> \mathcal{O}\left(\Del | Helped |
| Use clipping rather than SGD with momentum | ResNet-32 · CIFAR-10 | training speed | Helped |
| Use mixed clipping rather than the other clipping algorithms | ResNet-32 · CIFAR-10 | training speed | Helped |
| Use clipping rather than baseline SGD | ResNet-50 · ImageNet | training speed | Helped |
| Use mixed clipping rather than the other clipping algorithms | ResNet-50 · ImageNet | training speed | Helped |
| Use clipping rather than SGD without clipping | AWD-LSTM · Penn Treebank (PTB) | validation perplexity | Helped |
| Use mixed clipping rather than gradient clipping or momentum clipping | AWD-LSTM · Penn Treebank (PTB) | validation perplexity: more than 1 point | Helped |
Closest papers by meaning
- To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions (ICLR 2025 · similarity 0.92)
- Generalized Gradient Norm Clipping & Non-Euclidean $(L_0,L_1)$-Smoothness (NeurIPS 2025 · similarity 0.91)
- Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions (NeurIPS 2025 · similarity 0.89)
- Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity (ICLR 2025 · similarity 0.89)
- Clipped SGD Algorithms for Performative Prediction: Tight Bounds for Stochastic Bias and Remedies (ICML 2025 · similarity 0.89)
- Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.