Research map / Optimization / Distributed and compressed
Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models
Yunfei Teng, Wenbo Gao, François Chalus, Anna E Choromanska, Donald Goldfarb, Adrian Weller
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace EASGD's averaged elastic force with a pull toward the leader worker (LSGD), 16 workers | CNN7 distributed training · CIFAR-10 | convergence speed: roughly 1.5 times faster than EASGD | Improved |
| LSGD instead of EASGD with 16 workers | CNN7 distributed training · CIFAR-10 | test error | Improved |
| LSGD instead of EASGD with 4 workers | VGG16 distributed training · CIFAR-10 | test error | No clear effect |
| LSGD instead of DOWNPOUR | VGG16 distributed training · CIFAR-10 | convergence speed and test error | Improved |
| LSGD instead of EASGD with 4 workers | ResNet20 distributed training · CIFAR-10 | test error | Improved |
| LSGD instead of EASGD with 16 workers | ResNet20 distributed training · CIFAR-10 | convergence speed: roughly 2 times faster than EASGD | Improved |
| LSGD and EASGD instead of DOWNPOUR and SGD | CNN7 and ResNet20 on CIFAR-10 · CIFAR-10 | test error | Improved |
| LSGD instead of EASGD with 4 workers | ResNet50 distributed training · ImageNet | test error | No clear effect |
| LSGD instead of DOWNPOUR | ResNet50 distributed training · ImageNet | convergence speed | Improved |
Closest papers by meaning
- Local SGD Converges Fast and Communicates Little (ICLR 2019 · similarity 0.89)
- Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks (ICML 2020 · similarity 0.88)
- signSGD: Compressed Optimisation for Non-Convex Problems (ICML 2018 · similarity 0.88)
- Distributed Stochastic Gradient Descent with Event-Triggered Communication (AAAI 2020 · similarity 0.88)
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity (NeurIPS 2026 · similarity 0.88)
- AdaGK-SGD: Adaptive Global Knowledge Guided Distributed Stochastic Gradient Descent (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
