Research map / Optimization / Distributed and compressed
DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-pass Error-Compensated Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use error compensation on both workers and the parameter server instead of worker-side compensation alone | Distributed SGD on CIFAR-10 with 1-bit compression · CIFAR-10 | iteration time under limited network bandwidth | Helped |
| Use error compensation on both workers and the parameter server instead of worker-side compensation alone | Distributed SGD on CIFAR-10 with 1-bit compression · CIFAR-10 | training loss | No clear effect |
| Add error compensation to worker-side 1-bit QSGD | Distributed SGD on CIFAR-10 · CIFAR-10 | training loss | Helped |
| Use Top-k compression with error compensation on workers and the parameter server | Distributed SGD on CIFAR-10 · CIFAR-10 | training loss | No clear effect |
| Remove error compensation from Top-k SGD | Distributed SGD on CIFAR-10 · CIFAR-10 | training loss | Hurt |
| Replace error-compensated Top-k compression with uncompensated Top-k compression | Distributed SGD on CIFAR-10 · CIFAR-10 | iteration time under limited network bandwidth | Hurt |
| Use worker-side error compensation without server-side compression and compensation (MEM-SGD) | Distributed SGD on CIFAR-10 with 1-bit compression · CIFAR-10 | training loss | No clear effect |
| Use worker-side error compensation without server-side compression and compensation (MEM-SGD) | Distributed SGD on CIFAR-10 with Top-k compression · CIFAR-10 | training loss | No clear effect |
Closest papers by meaning
- ErrorCompensatedX: error compensation for variance reduced algorithms (NeurIPS 2021 · similarity 0.90)
- Error Compensated Distributed SGD Can Be Accelerated (NeurIPS 2021 · similarity 0.90)
- Quantized Compressive Sampling of Stochastic Gradients for Efficient Communication in Distributed Deep Learning (AAAI 2020 · similarity 0.90)
- Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization (ICML 2018 · similarity 0.90)
- Don't Compress Gradients in Random Reshuffling: Compress Gradient Differences (NeurIPS 2024 · similarity 0.87)
- Leveraging partial stragglers within gradient coding (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.