Research map / Optimization / Nonconvex and smooth optimization
Decentralized Learning: Theoretical Optimality and Practical Improvements
Yucheng Lu, Christopher De Sa
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DeTAG instead of D-PSGD on increasingly unshuffled (decentralized) data | LeNet 8 workers ring · CIFAR10 (X% shuffled) | training loss convergence | Improved |
| use DeTAG instead of D2 and DSGT on unshuffled data | LeNet 8 workers ring · CIFAR10 (X% shuffled) | training loss convergence | Improved |
| use DeTAG on fully shuffled data | LeNet 8 workers ring · CIFAR10 (100% shuffled) | training loss convergence | No clear effect |
| use DeTAG instead of baselines as spectral gap shrinks | Resnet20 16 workers ring · CIFAR100 (varying spectral gap) | training loss convergence | Improved |
| use DeTAG* instead of D-PSGD, D2, DSGT | Resnet18 32-GPU ring · ImageNet | training loss and validation accuracy per round | Improved |
| add multi-round gossip (R-1 extra rounds) to larger mini-batch (ablation) | DeTAG* vs naive R-times batch · ImageNet (R=2,4,8) | validation accuracy | Improved |
Closest papers by meaning
- Enhancing Parallelism in Decentralized Stochastic Convex Optimization (ICML 2025 · similarity 0.88)
- Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training (NeurIPS 2025 · similarity 0.88)
- From Promise to Practice: Realizing High-performance Decentralized Training (ICLR 2025 · similarity 0.88)
- Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent (AAAI 2024 · similarity 0.88)
- Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence (AAAI 2025 · similarity 0.87)
- Stability and Generalization Analysis of Decentralized SGD: Sharper Bounds Beyond Lipschitzness and Smoothness (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
