Research map / Optimization / Distributed and compressed
Momentum Ensures Convergence of SIGNSGD under Weaker Assumptions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add simple momentum (θ=0.9) to signSGD | signSGD-SIM on ResNet56 · CIFAR-100, batch 8 | test accuracy: +21.7 points (48.09% to 69.79%) | Helped |
| add simple momentum (θ=0.9) to signSGD | signSGD on ResNet56 · CIFAR-10, batch 8 | test accuracy: +4.19 points (88.32% to 92.51%) | Helped |
| add simple momentum (θ=0.9) to signSGD | signSGD on ResNet56 · CIFAR-100, batch 128 | test accuracy: +2.34 points (67.36% to 69.70%) | Helped |
| reduce batch size from 128 to 8 without momentum | signSGD on ResNet56 · CIFAR-100 | test accuracy: 67.36% to 48.09% | Hurt |
| add simple momentum to signSGD | signSGD on ResNet20/ResNet32 · CIFAR-10 and CIFAR-100, batch 8 | test accuracy: ResNet20 CIFAR-10 85.23% to 91.15%; ResNet32 CIFAR-100 47.11% to 67.94% | Helped |
| add simple momentum to signSGD with large batch (128) | signSGD on ResNet56 CIFAR-10 · CIFAR-10, batch 128 | test accuracy: 92.12% to 91.60% | Hurt |
| add simple momentum to federated/majority-vote signSGD (MV-STO-SIGNSGD-SIM) | MV-STO-SIGNSGD on MLP/MNIST and CNN/CIFAR-10 · MNIST and CIFAR-10, small batch sizes | test accuracy (qualitative, figures): qualitative; figures only, no numbers given | Helped |
| use momentum (SIGNSGD-SIM) instead of error feedback to remove bounded gradient assumption | SIGNSGD-SIM theory · Theorem 1 convergence | convergence rate: O(1/T^{1/4}) without bounded gradient or growing batch | Helped |
| add double momentum (extra momentum before gradient computation) to SIGNSGD-SIM | SIGNSGD-SIM theory · Second-order smooth objectives | convergence rate: O(1/T^{2/7}) vs O(1/T^{1/4}) under first-order smoothness | Helped |
Closest papers by meaning
- Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction (NeurIPS 2024 · similarity 0.91)
- When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds (NeurIPS 2026 · similarity 0.88)
- Gradient correlation is a key ingredient to accelerate SGD with momentum (ICLR 2025 · similarity 0.88)
- SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding (ICML 2024 · similarity 0.87)
- Exact risk curves of signSGD in High-Dimensions: quantifying preconditioning and noise-compression effects (ICML 2025 · similarity 0.87)
- Investigating the Role of Weight Decay in Enhancing Nonconvex SGD (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.