Research map / Optimization / Distributed and compressed
signSGD: Compressed Optimisation for Non-Convex Problems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Signum instead of Adam | ResNet-50 v2 · ImageNet | test accuracy | No clear effect |
| Use Signum instead of SGD with well-tuned weight decay | ResNet-50 v2 · ImageNet | test accuracy: about 2% | Hurt |
| Use Signum instead of SGD | ResNet-20 · CIFAR-10 | test accuracy | No clear effect |
| Use signSGD instead of SGD with separately tuned constant learning rates | Quadratic toy problem with sparse noise · 100-dimensional quadratic | convergence | Helped |
| Aggregate worker gradient signs with majority vote | Distributed signSGD · Non-convex optimization | gradient-noise variance bound: ||σ||_1 -> ||σ||_1/√M | Helped |
| Use sign-based updates rather than full-gradient SGD when gradients are dense and noise is sparse | signSGD · Non-convex optimization with sparse noise | convergence | Helped |
Closest papers by meaning
- Error Feedback Fixes SignSGD and other Gradient Compression Schemes (ICML 2019 · similarity 0.93)
- When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds (NeurIPS 2026 · similarity 0.90)
- Exact risk curves of signSGD in High-Dimensions: quantifying preconditioning and noise-compression effects (ICML 2025 · similarity 0.89)
- Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction (NeurIPS 2024 · similarity 0.89)
- SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding (ICML 2024 · similarity 0.89)
- On the Optimization and Generalization of Two-layer Transformers with Sign Gradient Descent (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.