Research map / Federated learning and privacy / Communication-efficient FL
The Non-IID Data Quagmire of Decentralized Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with skewed label partitions rather than IID partitions | Decentralized learning algorithms · CIFAR-10 | validation accuracy: 3% to 74% validation accuracy loss | Hurt |
| Train ResNet20 with non-IID rather than IID partitions, even using BSP | ResNet20 with BSP · CIFAR-10 | validation accuracy: 39% lower | Hurt |
| Replace BatchNorm with GroupNorm (ablation) | BN-LeNet with BSP · CIFAR-10, non-IID | validation accuracy: 79.2% | Helped |
| Replace BatchNorm with GroupNorm (ablation) | BN-LeNet with Gaia · CIFAR-10, non-IID | accuracy loss: 14.4%; better ... by an additive 10.7% | Helped |
| Replace BatchNorm with GroupNorm (ablation) | BN-LeNet with FedAvg · CIFAR-10, non-IID | accuracy loss: 8.9%; better ... by an additive 19.8% | Helped |
| Replace BatchNorm with GroupNorm (ablation) | BN-LeNet with DGC · CIFAR-10, non-IID | accuracy loss: 8.7%; better ... by an additive 60.2% | Helped |
| Adapt communication frequency using estimated cross-partition accuracy loss | Gaia · CIFAR-10, non-IID | communication savings over BSP: 9.6--34.1× | Helped |
| Increase label skew from 20% to 80% (ablation) | GN-LeNet with decentralized algorithms · CIFAR-10 | validation accuracy: accuracy gap ... as large as 7.4% (DGC) | Hurt |
| Increase the number of partitions from K=5 to K=10 (ablation) | Gaia and FedAvg · CIFAR-10, non-IID | accuracy loss vs. BSP: Gaia: 0% to 3%; FedAvg: 17% to 36% | Hurt |
| Use non-IID rather than IID real-world continent partitions | Gaia · FlickrMammal | accuracy: 3.7% | Hurt |
| Use non-IID rather than IID real-world continent partitions | FedAvg · FlickrMammal | accuracy: 3.2% | Hurt |
Closest papers by meaning
- Does Learning from Decentralized Non-IID Unlabeled Data Benefit from Self Supervision? (ICLR 2023 · similarity 0.87)
- Optimal Complexity in Decentralized Training (ICML 2021 · similarity 0.86)
- Consensus Control for Decentralized Deep Learning (ICML 2021 · similarity 0.86)
- Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training (NeurIPS 2025 · similarity 0.86)
- RelaySum for Decentralized Deep Learning on Heterogeneous Data (NeurIPS 2021 · similarity 0.86)
- Exploiting Label Skews in Federated Learning with Model Concatenation (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.