Research map / Optimization / Distributed and compressed
Local SGD Converges Fast and Communicates Little
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use local SGD with synchronization interval H=O(√(T/(Kb))) instead of synchronizing every iteration | Local SGD · Strongly convex finite-sum optimization | convergence rate: O(1/(KTb)) | No clear effect |
| Use local SGD with synchronization interval H=O(√(T/(Kb))) instead of synchronizing every iteration | Local SGD · Strongly convex finite-sum optimization | communication rounds: O(√(T/(Kb))) | Helped |
| Increase the number of workers K | Local SGD · Strongly convex finite-sum optimization | convergence rate: O(1/(KTb)) | Helped |
| Increase the local mini-batch size b | Local SGD · Strongly convex finite-sum optimization | convergence rate: O(1/(KTb)) | Helped |
| Set the synchronization interval to H=T (one-shot averaging) | Local SGD · Strongly convex finite-sum optimization | convergence rate | Hurt |
| Allow asynchronous synchronization with delay τ and H+τ=O(√(T/K)) | Asynchronous local SGD · Strongly convex finite-sum optimization | convergence rate: O(G^2/(KT)) | No clear effect |
Closest papers by meaning
- Don't Use Large Mini-batches, Use Local SGD (ICLR 2020 · similarity 0.95)
- STL-SGD: Speeding Up Local SGD with Stagewise Communication Period (AAAI 2021 · similarity 0.94)
- Communication-efficient SGD: From Local SGD to One-Shot Averaging (NeurIPS 2021 · similarity 0.94)
- Communication trade-offs for Local-SGD with large step size (NeurIPS 2019 · similarity 0.92)
- The Effect of Network Width on the Performance of Large-batch Training (NeurIPS 2018 · similarity 0.92)
- LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.