Research map / Optimization / Nonconvex and smooth optimization
Why Gradient Clipping Accelerates Training: A Theoretical Justification for Adaptivity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply gradient clipping during LSTM language-model training | AWD-LSTM · PTB | training loss convergence | Helped |
| Apply gradient clipping during ResNet20 training | ResNet20 · CIFAR-10 | training loss convergence | Helped |
| Use clipped GD instead of SGD momentum | ResNet20 · CIFAR-10 test | accuracy: 95.0% -> 95.2% | No clear effect |
Closest papers by meaning
- Improved Analysis of Clipping Algorithms for Non-convex Optimization (NeurIPS 2020 · similarity 0.94)
- Generalized Gradient Norm Clipping & Non-Euclidean $(L_0,L_1)$-Smoothness (NeurIPS 2025 · similarity 0.90)
- To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions (ICLR 2025 · similarity 0.89)
- Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity (ICLR 2025 · similarity 0.89)
- Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed (ICML 2025 · similarity 0.88)
- Error Feedback under $(L_0,L_1)$-Smoothness: Normalization and Momentum (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.