Research map / Federated learning and privacy / Communication-efficient FL
Decentralized peer-to-peer training
Train over decentralized data using asynchronous updates and compressed communication.
41 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2018–2026) · an approach in Communication-efficient FL, keyword cluster “devices · edge · dynamic”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
15 papers in 2023–24 and 5 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 10.1% to 4.3%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2018 · $D^2$: Decentralized Training over Decentralized Data (ICML 2018 · 174 citations)
- 2019 · Distributed Learning over Unreliable Networks (ICML 2019 · 20 citations)
- 2020 · The Non-IID Data Quagmire of Decentralized Machine Learning (ICML 2020 · 88 citations)
- 2021 · DRIVE: One-bit Distributed Mean Estimation (NeurIPS 2021 · 9 citations)
- 2022 · Learning To Collaborate in Decentralized Learning of Personalized Models (CVPR 2022 · 49 citations)
- 2023 · Global Update Tracking: A Decentralized Learning Algorithm for Heterogeneous Data (NeurIPS 2023 · 4 citations)
- 2024 · DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models (CVPR 2024 · 3 citations)
- 2025 · Distributed mediation analysis with communication efficiency (NeurIPS 2025 · 0 citations)
- 2026 · FusedRec: Fused Embedding Communication for Distributed Recommendation Training on GPUs (AAAI 2026 · 0 citations)
Earliest papers here
- $D^2$: Decentralized Training over Decentralized Data (ICML 2018 · 174 citations)
- COLA: Decentralized Linear Learning (NeurIPS 2018 · 42 citations)
- Distributed Learning over Unreliable Networks (ICML 2019 · 20 citations)
What has been tried
38 papers here report 313 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 12%, hurt 20%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Approximate interpolating measures instead of using exact interpolating measures | 2D Gaussian Wasserstein distance estimation · running time | at least 10 times faster | Helped | Federated Wasserstein Distance (ICLR 2024 · 2 citations) |
| remove hypernetwork | LR + MNIST · Total (s) | 41.72 -> 30.38(↓) | Helped | HyperPrism: An Adaptive Non-linear Aggregation Framework for Distributed Machine Learning over Non-IID Data and Time-varying Communication Links (NeurIPS 2024 · 1 citation) |
| Communicate 7 samples per client instead of 1 | CIFAR-10, 1 client per round · final test accuracy | 0.7516 -> 0.8028 | Helped | Universal Sample Coding (NeurIPS 2024 · 0 citations) |
| Set sampled hypotheses J=2 rather than J=K | Eight regression datasets · MSE | Helped | On the Necessity of Collaboration for Online Model Selection with Decentralized Data (NeurIPS 2024 · 0 citations) | |
| Add early stopping | MNIST · unlearning time (s) | 3.0 -> 2.3 | Helped | PDUDT: Provable Decentralized Unlearning under Dynamic Topologies (ICML 2025) |
What reviewers will expect
Datasets and baselines named in the 11 papers of this approach from 2024–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 7 of 11 papers
- MNIST · 3 of 11 papers
- CIFAR-100 · 2 of 11 papers
- Tiny ImageNet · 2 of 11 papers
- FashionMNIST · 2 of 11 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 20 papers; from OpenAlex and the papers' affiliations).
- Massachusetts Institute of Technology · 2 papers
- Alibaba Group · 2 papers
- Google · 2 papers
- Purdue University · 2 papers
Use it
Download BibTeX of 30 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Efficient local training in FL (77 papers)
- Communication-efficient heterogeneous FL (65 papers)
Elsewhere in the topic
- Federated optimization algorithms (70 papers)
- Client participation analysis (68 papers)
- Local drift correction (61 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).