Research map / Optimization / Optimizers
XGrad: Boosting Gradient-Based Optimizers With Weight Prediction
Lei Guan, Dongsheng Li, Yanqi Shi, Jian Meng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add weight prediction (XGrad) | SGDM · CIFAR-10, nine models | top-1 accuracy: 0.98% average gain, up to 1.81% | Improved |
| add weight prediction (XGrad) | SGDM · CIFAR-100, nine models | top-1 accuracy: 3.01% average gain, up to 5.54% | Improved |
| add weight prediction (XGrad) | Adam · CIFAR-10 etc., image models | top-1 accuracy: 0.76% average gain | Improved |
| add weight prediction (XGrad) | Adam · GNMT-8 WMT-16 EN-De | BLEU: 23.84 -> 24.58 | Improved |
| add weight prediction (XGrad) | AdamW · image classification | top-1 accuracy: 0.46% average gain, up to 0.62% | Improved |
| add weight prediction (XGrad) | AdaBelief · image classification | top-1 accuracy: 0.27% average gain | Improved |
| add weight prediction (XGrad) | AdaBelief · VGG-11 | top-1 accuracy: -0.21% | Hurt |
| add weight prediction (XGrad) | AdaM3 · BERT-base MRPC | dev accuracy: 68.38 -> 80.64 | Improved |
| add weight prediction (XGrad) | AdaM3 · image classification | top-1 accuracy: 0.28% average gain | Improved |
| use weight prediction step 4 instead of steps 1-3 (ablation) | XGrad on SGDM · ViT CIFAR-10 | top-1 accuracy: 61.96 vs 83.90 (S2) | Hurt |
| use weight prediction step 2/3/4 instead of 1 (ablation) | XGrad on Adam · LSTM-1 Penn Treebank | perplexity: 88.08 -> 88.83 (S2) | Hurt |
| add weight prediction (XGrad) | SGDM · training time per epoch, ResNet-101 | time (s): 131.73s -> 139.09s | Hurt |
Closest papers by meaning
- A Momentumized, Adaptive, Dual Averaged Gradient Method (JMLR 2022 · similarity 0.87)
- Learning Gradient Descent: Better Generalization and Longer Horizons (ICML 2017 · similarity 0.87)
- Learning Provably Improves the Convergence of Gradient Descent (NeurIPS 2025 · similarity 0.85)
- QLABGrad: A Hyperparameter-Free and Convergence-Guaranteed Scheme for Deep Learning (AAAI 2024 · similarity 0.85)
- Prodigy: An Expeditiously Adaptive Parameter-Free Learner (ICML 2024 · similarity 0.85)
- Understanding the Generalization of Stochastic Gradient Adam in Learning Neural Networks (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
