Research map / Optimization / Distributed and compressed
CSER: Communication-efficient SGD with Error Reset
Cong Xie, Shuai Zheng, Sanmi Koyejo, Indranil Gupta, Mu Li, Haibin Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use error reset and partial synchronization with aggressive compression | CSER · CIFAR-100 | training speedup: nearly $10\times$ | Improved |
| Use error reset and partial synchronization with aggressive compression | CSER · ImageNet | training speedup: 4.5× | Improved |
| Use error reset rather than QSparse-local-SGD error feedback | CSER · CIFAR-100 test, R=256 | accuracy (%): diverge -> 86.68±0.07 | Improved |
| Use error reset rather than QSparse-local-SGD error feedback | CSER · CIFAR-100 test, R=1024 | accuracy (%): diverge -> 85.66±0.07 | Improved |
| Add gradient partial synchronization to model synchronization (ablation) | CSER · CIFAR-100 | test accuracy (%) | Improved |
| Remove gradient partial synchronization (ablation) | CSER-PL · CIFAR-100 | test accuracy (%) | Hurt |
| Use error assimilation without gradient partial synchronization (ablation) | CSEA · CIFAR-100 | test accuracy (%) | Hurt |
| Allocate some communication budget to gradient synchronization instead of all to model synchronization | CSER · theoretical compression-error example | compression error bound: 400 -> less than 236 | Improved |
| Use CSER error reset instead of QSparse-local-SGD error feedback | CSER · theoretical example, H=8 and δ₁=1/2 | compression error bound: 832 -> 576 | Improved |
Closest papers by meaning
- Local SGD Converges Fast and Communicates Little (ICLR 2019 · similarity 0.90)
- Communication-efficient SGD: From Local SGD to One-Shot Averaging (NeurIPS 2021 · similarity 0.90)
- Error Compensated Distributed SGD Can Be Accelerated (NeurIPS 2021 · similarity 0.89)
- The Error-Feedback framework: SGD with Delayed Gradients (JMLR 2020 · similarity 0.89)
- STL-SGD: Speeding Up Local SGD with Stagewise Communication Period (AAAI 2021 · similarity 0.89)
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
