Research map / Optimization / Distributed and compressed
Ordered Local Momentum for Asynchronous Distributed Learning Under Arbitrary Delays
Chang-Wei Shi, Shi-Shang Wang, Wu-Jun Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use local momentum (MSGD) at workers instead of local SGD (ablation) | local OrMo-DA · CIFAR10 SqueezeNet, K=16, S=32 homogeneous | test accuracy: 77.96 -> 91.05 | Improved |
| use local momentum at workers instead of local SGD (ablation) | local OrMo-DA · CIFAR100 SqueezeNet, K=16, S=32 homogeneous | test accuracy: 49.09 -> 65.65 | Improved |
| use local momentum at workers (small S) (ablation) | local OrMo-DA · CIFAR10 SqueezeNet, K=8, S=8 homogeneous | test accuracy: 92.02 -> 92.11 | No clear effect |
| use ordered local momentum asynchronous aggregation instead of AL-SGD | AL-SGD · CIFAR10 SqueezeNet, K=8, S=8 homogeneous | test accuracy: 91.46 -> 92.11 | Improved |
| use OrLoMo (asynchronous) instead of synchronous PRSGDm | PRSGDm · CIFAR10 SqueezeNet, K=16, S=32 homogeneous | test accuracy: 90.10 -> 91.05 | Improved |
| use OrLoMo instead of PRSGDm | PRSGDm · CIFAR10 SqueezeNet, K=8, S=8 homogeneous | test accuracy: 92.18 -> 92.11 | No clear effect |
| use OrLoMo instead of PRSGDm | PRSGDm · wall-clock training, heterogeneous workers | training speed | Improved |
| increase slow/fast ratio from 1 to 50 (max delay 34 to 9449) (ablation) | OrLoMo · CIFAR10 ResNet20, K=16, S=8 | test accuracy: 91.38 -> 91.44 | No clear effect |
| use local momentum instead of local SGD at small S with heterogeneous workers (ablation) | local OrMo-DA · CIFAR100 ResNet20 heterogeneous K=8 S=8 | test accuracy: 67.83 -> 67.62 | No clear effect |
Closest papers by meaning
- Ordered Momentum for Asynchronous SGD (NeurIPS 2024 · similarity 0.95)
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity (NeurIPS 2026 · similarity 0.87)
- Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity (ICML 2025 · similarity 0.87)
- A Quadratic Synchronization Rule for Distributed Deep Learning (ICLR 2024 · similarity 0.87)
- Convergence of Distributed Adaptive Optimization with Local Updates (ICLR 2025 · similarity 0.86)
- Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
