Research map / Optimization / Distributed and compressed
signSGD with Majority Vote is Communication Efficient and Fault Tolerant
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use one-bit sign communication with majority-vote aggregation instead of full-precision distributed SGD | Distributed training · Distributed training | communication per iteration: 32× less communication | Helped |
| Use Signum with majority vote instead of NCCL-distributed SGD | ResNet50 · ImageNet | training time: 25% reduction in time | Helped |
| Use Signum with majority vote instead of NCCL-distributed SGD | ResNet50 · ImageNet | test accuracy | Hurt |
| Use Signum with majority vote instead of Adam aggregated with NCCL | QRNN · WikiText-103 | epoch time: roughly 3 times faster | Helped |
| Use Signum with majority vote instead of Adam aggregated with NCCL | QRNN · WikiText-103 | test perplexity after 2 hours: similar perplexity | No clear effect |
| Increase the per-worker batch size (ablation) | Signum with majority vote · WikiText-103, QRNN | performance | Helped |
| Increase the batch size (ablation) | Signum · WikiText-103, QRNN | performance | Helped |
| Use two-way compression for 1-bit L2 QSGD rather than one-way compression (ablation) | 1-bit L2 QSGD · Cifar-10, ResNet18 | convergence | Hurt |
| Increase the number of adversarial workers (ablation) | Signum with majority vote · ImageNet, ResNet50 | learning stability: 43% | Hurt |
| Retune the learning rate for the 43% adversarial setting (ablation) | Signum with majority vote · ImageNet, ResNet50 | learning stability: 43% | Helped |
| Increase adversarial workers beyond the security level f=2 (ablation) | Multi-Krum · Cifar-10, ResNet18 | training robustness: f=2 | Hurt |
Closest papers by meaning
- SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding (ICML 2024 · similarity 0.91)
- Exact risk curves of signSGD in High-Dimensions: quantifying preconditioning and noise-compression effects (ICML 2025 · similarity 0.87)
- On the Optimization and Generalization of Two-layer Transformers with Sign Gradient Descent (ICLR 2025 · similarity 0.87)
- When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds (NeurIPS 2026 · similarity 0.86)
- Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction (NeurIPS 2024 · similarity 0.85)
- Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.