Research map / Federated learning and privacy / Communication-efficient FL
Decentralized Deep Learning with Arbitrary Communication Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CHOCO-SGD instead of DeepSqueeze across the tested compression schemes | ResNet20 · CIFAR-10 test | top-1 accuracy: CHOCO-SGD achieves higher test accuracy in all listed compression settings | Helped |
| Use sign + norm compression instead of full-precision communication | CHOCO-SGD · ImageNet-1k validation | top-1 accuracy: 76.37\% -> 75.15\% | Hurt |
| Use decentralized CHOCO-SGD with sign + norm compression instead of centralized SGD | ResNet20 · CIFAR-10 test; social network | top-1 accuracy: 93.00 -> 91.80 | Hurt |
| Use decentralized CHOCO-SGD with sign + norm compression instead of centralized SGD | LSTM · WikiText-2 test; social network | perplexity: 89.39 -> 86.58 | Helped |
| Use CHOCO-SGD with sign compression instead of all-reduce | ResNet-50 · ImageNet-1k | training time: 20% | Helped |
| Increase the number of decentralized training nodes | CHOCO-SGD · CIFAR-10; fixed 1000 MB communication budget | test accuracy | Hurt |
| Use a torus instead of a ring topology at large network sizes | CHOCO-SGD · CIFAR-10; fixed 1000 MB communication budget | test accuracy | Helped |
| Use a torus instead of a ring topology in small networks | CHOCO-SGD · CIFAR-10; small networks | test accuracy | No clear effect |
| Replace random sparsification with top-k sparsification | DCD-PSGD · CIFAR-10 test | top-1 accuracy: 89.79 ± 0.40 (top-k 1%) vs diverges (random 1%) | Helped |
| Use 2-bit rather than 16-bit QSGD compression (ablation) | CHOCO-SGD · CIFAR-10 test | top-1 accuracy: 92.34 ± 0.19 -> 91.41 ± 0.11 | Hurt |
| Use 1% rather than 50% random sparsification (ablation) | CHOCO-SGD · CIFAR-10 test | top-1 accuracy: 92.54 ± 0.26 -> 91.32 ± 0.17 | Hurt |
| Use sign compression rather than full-precision communication (ablation) | CHOCO-SGD · CIFAR-10 test; ResNet20 | communication per weight: approximately $32\times$ less bits per weight | Helped |
Closest papers by meaning
- Practical Low-Rank Communication Compression in Decentralized Deep Learning (NeurIPS 2020 · similarity 0.92)
- Communication Compression for Decentralized Training (NeurIPS 2018 · similarity 0.89)
- On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning (AAAI 2020 · similarity 0.88)
- From Promise to Practice: Realizing High-performance Decentralized Training (ICLR 2025 · similarity 0.87)
- Subspace Networks: Scaling Decentralized Training with Communication-Efficient Model Parallelism (NeurIPS 2025 · similarity 0.87)
- Towards Faster Decentralized Stochastic Optimization with Communication Compression (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.