Research map / Optimization / Distributed and compressed
Cooperative SGD: A Unified Framework for the Design and Analysis of Local-Update SGD Algorithms
Jianyu Wang, Gauri Joshi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| set EASGD elasticity alpha to 2/(m+2)=0.2 instead of 0.9/m=0.1125 (ablation) | EASGD · CIFAR-10 VGG-16 | training loss: alpha 0.1125 -> 0.2 | Improved |
| increase alpha beyond 2/(m+1) approx 0.2222 (ablation) | EASGD · CIFAR-10 VGG-16 | convergence: alpha greater than 0.2222 | Hurt |
| add overlap of auxiliary-variable update with worker computation | EASGD (tau=1) · CIFAR-10 VGG-16 | training time: reduces about 50% training time | Improved |
| increase communication period tau from 4 to 24 (ablation) | PSASGD · CIFAR-10 VGG-16 | training loss after same epochs: tau = 24 has the highest loss | Hurt |
| use a smaller learning rate with periodic averaging (ablation) | PSASGD · CIFAR-10 VGG-16 | gap between communication periods: learning rate 0.04 vs smaller | Improved |
| add local updates (tau=10) to decentralized SGD with zeta=0.75 (ablation) | D-PSGD · CIFAR-10 VGG-16 | wall-clock time to training loss 0.1: requires about 4x less wall-clock time | Improved |
| add local updates (higher tau) in epoch terms (ablation) | D-PSGD / fully sync SGD · CIFAR-10 VGG-16 | convergence per epoch: tau = 1 -> tau = 10 | Hurt |
| add an auxiliary variable node with alpha=(1+zeta)/(m+1+zeta) | D-PSGD · theory (Theorem 4) | second largest eigenvalue: zeta -> m*zeta/(m+1+zeta) | Improved |
| large communication period in PSASGD before learning-rate decay (ablation) | PSASGD · CIFAR-10 VGG-16 | test accuracy | Improved |
Closest papers by meaning
- Local SGD Converges Fast and Communicates Little (ICLR 2019 · similarity 0.93)
- STL-SGD: Speeding Up Local SGD with Stagewise Communication Period (AAAI 2021 · similarity 0.92)
- Communication-efficient SGD: From Local SGD to One-Shot Averaging (NeurIPS 2021 · similarity 0.92)
- LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging (NeurIPS 2026 · similarity 0.90)
- Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent (AAAI 2024 · similarity 0.89)
- AdaGK-SGD: Adaptive Global Knowledge Guided Distributed Stochastic Gradient Descent (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
