Research map / Optimization / Distributed and compressed
STL-SGD: Speeding Up Local SGD with Stagewise Communication Period
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a stagewise increasing communication period while decreasing the learning rate | STL-SGD^sc · a9a and MNIST, IID and Non-IID | communication rounds to reach 10^-4 objective gap: 184 -> 61; 10068 -> 4417; 289 -> 79; 2642 -> 1518 | Helped |
| Use geometric Option 1 rather than linear Option 2 scheduling (ablation) | STL-SGD^nc on CIFAR10 · ResNet18 and VGG16, IID and Non-IID | communication rounds to reach 99% training accuracy: 470 -> 434; 1158 -> 954; 696 -> 602; 2732 -> 2179 | Helped |
| Use stagewise increasing communication periods for strongly convex optimization | STL-SGD · Strongly convex, IID | communication complexity: $O(N^{\frac{3}{2}} T^{\frac{1}{2}}) -> $O(N \log{T}) | Helped |
| Use stagewise increasing communication periods for strongly convex optimization | STL-SGD · Strongly convex, Non-IID | communication complexity: $O(N^{\frac{3}{4}} T^{\frac{3}{4}}) -> $O(N^{\frac{1}{2}} T^{\frac{1}{2}}) | Helped |
| Use stagewise scheduling for a non-convex objective satisfying PL | STL-SGD · Non-convex + PL, IID | communication complexity: $O(N^{\frac{1}{3}} T^{\frac{1}{3}}) -> $O(N \log{T}) | Helped |
| Increase communication periods stagewise for general non-convex optimization | STL-SGD · General non-convex, IID and Non-IID | communication complexity | No clear effect |
| Increase communication periods geometrically rather than linearly (ablation) | STL-SGD^nc on CIFAR10 · ResNet18 and VGG16, IID and Non-IID | communication rounds: 470 -> 434; 1158 -> 954; 696 -> 602; 2732 -> 2179 | Helped |
Closest papers by meaning
- Local SGD Converges Fast and Communicates Little (ICLR 2019 · similarity 0.94)
- Communication-efficient SGD: From Local SGD to One-Shot Averaging (NeurIPS 2021 · similarity 0.94)
- On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization (ICML 2019 · similarity 0.91)
- On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimization (ICML 2019 · similarity 0.91)
- LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging (NeurIPS 2026 · similarity 0.89)
- Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.