Research map / Federated learning and privacy / Communication-efficient FL
$\textbf{A}^2\textbf{CiD}^2$: Accelerating Asynchronous Communication in Decentralized Deep Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add continuous momentum (ablation) | Asynchronous decentralized training · ImageNet, ring graph, 32 workers, 1 communication/gradient | accuracy: 71.6 -> 73.4 | Helped |
| Add continuous momentum (ablation) | Asynchronous decentralized training · ImageNet, ring graph, 64 workers, 1 communication/gradient | accuracy: 64.1 -> 68.0 | Helped |
| Add continuous momentum (ablation) | Asynchronous decentralized training · ImageNet, ring graph, 16 workers, 1 communication/gradient | accuracy: 74.8 -> 74.7 | No clear effect |
| Add continuous momentum (ablation) | Asynchronous decentralized training · ImageNet, ring graph, 64 workers, 2 communications/gradient | accuracy: 68.2 -> 71.4 | Helped |
| Increase communication rate from 1 to 2 communications per gradient (ablation) | Asynchronous decentralized training with continuous momentum · ImageNet, ring graph, 64 workers | accuracy: 68.0 -> 71.4 | Helped |
| Add continuous momentum (ablation) | Asynchronous decentralized training · CIFAR-10, ring graph, 32 workers | accuracy: 93.95±0.11 -> 94.61±0.02 | Helped |
| Add continuous momentum (ablation) | Asynchronous decentralized training · CIFAR-10, ring graph, 64 workers | accuracy: 91.90±0.10 -> 93.08±0.20 | Helped |
| Add continuous momentum (ablation) | Asynchronous decentralized training · CIFAR-10, ring graph, 4 workers | accuracy: 95.02±0.06 -> 94.95±0.02 | No clear effect |
| Add continuous momentum (ablation) | Asynchronous decentralized training · CIFAR-10, ring graph, 8 workers | accuracy: 95.01±0.01 -> 95.01±0.10 | No clear effect |
Closest papers by meaning
- Asynchronous Decentralized SGD with Quantized and Local Updates (NeurIPS 2021 · similarity 0.90)
- A Quadratic Synchronization Rule for Distributed Deep Learning (ICLR 2024 · similarity 0.89)
- Asynchronous Decentralized Parallel Stochastic Gradient Descent (ICML 2018 · similarity 0.89)
- From Promise to Practice: Realizing High-performance Decentralized Training (ICLR 2025 · similarity 0.89)
- RelaySum for Decentralized Deep Learning on Heterogeneous Data (NeurIPS 2021 · similarity 0.88)
- AsyncMesh: Fully Asynchronous Optimization for Data and Pipeline Parallelism (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.