Research map / Optimization / Distributed and compressed
Quasi-global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace local-gradient momentum with quasi-global momentum based on consecutive synchronized-model differences | DSGDm-N · CIFAR-10, Ring, ResNet-EvoNorm-20 | test top-1 accuracy: 89.98 ± 0.10 -> 91.28 ± 0.38; 77.48 ± 2.67 -> 82.20 ± 1.27 | Helped |
| Use quasi-global momentum instead of local momentum | DSGDm-N · ImageNet, Ring, ResNet-18 EvoNorm | test top-1 accuracy: 68.77 ± 0.05 -> 69.20 ± 0.08; 53.15 ± 0.14 -> 56.50 ± 0.01 | Helped |
| Extend quasi-global momentum to Adam | DAdam · AG News, α=0.1 | test accuracy: 87.29 ± 0.60 -> 88.33 ± 0.67 | Helped |
| Add local HeavyBall momentum (ablation) | DSGD · CIFAR-10, α=1, Ring, ResNet-EvoNorm-20 | test top-1 accuracy: 88.88 ± 0.26 -> 89.67 ± 0.33 | Helped |
| Add local Nesterov momentum (ablation) | DSGD · CIFAR-10, α=0.1, Ring, ResNet-EvoNorm-20 | test top-1 accuracy: 74.55 ± 2.07 -> 77.48 ± 2.67 | Helped |
| Synchronize local momentum buffers over the ring (ablation) | DSGDm-N · CIFAR-10, α=0.1, Ring, ResNet-EvoNorm-20 | test top-1 accuracy: 77.48 ± 2.67 -> 79.83 ± 2.29 | Helped |
| Synchronize local gradients over the ring (ablation) | DSGDm-N · CIFAR-10, α=0.1, Ring, ResNet-EvoNorm-20 | test top-1 accuracy: 77.48 ± 2.67 -> 78.58 ± 4.12 | Helped |
| Replace local momentum with QG-DSGDm-N (ablation) | DSGDm-N · CIFAR-10, α=0.1, Ring, ResNet-EvoNorm-20 | test top-1 accuracy: 77.48 ± 2.67 -> 82.20 ± 1.27 | Helped |
| Replace GN with EvoNorm (ablation) | ResNet-20-x2 · ImageNet-32, α=0.1, Ring, QG-DSGDm-N | test top-1 accuracy: 21.42 ± 0.81 -> 19.53 ± 0.91 | Hurt |
| Replace GN with EvoNorm (ablation) | ResNet-18 · ImageNet, α=0.1, Ring, QG-DSGDm-N | test top-1 accuracy: 47.86 ± 1.05 -> 56.50 ± 0.01 | Helped |
| Use QG-DSGDm-N rather than gradient tracking with DSGDm-N | DSGDm-N · CIFAR-10, α=0.1, Ring (n=16), ResNet-EvoNorm-20 | test top-1 accuracy: 78.64 ± 1.84 -> 82.20 ± 1.27 | Helped |
| Use the multiple-step QG momentum variant (τ > 1) instead of the consecutive-model version (τ=1) (ablation) | QG-DSGDm · Not specified in the provided text | performance | Hurt |
Closest papers by meaning
- Momentum Benefits Non-iid Federated Learning Simply and Provably (ICLR 2024 · similarity 0.87)
- From Promise to Practice: Realizing High-performance Decentralized Training (ICLR 2025 · similarity 0.87)
- Overcoming Data and Model heterogeneities in Decentralized Federated Learning via Synthetic Anchors (ICML 2024 · similarity 0.86)
- Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training (NeurIPS 2025 · similarity 0.86)
- Widening the Network Mitigates the Impact of Data Heterogeneity on FedAvg (ICML 2025 · similarity 0.86)
- Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.