Research map / Optimization / Nonconvex and smooth optimization
Optimal Complexity in Decentralized Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use factorized consensus matrices to achieve exact average consensus in D to 2D communication steps | DeFacto · Stochastic non-convex training on arbitrary graphs | iteration complexity: O\left(\frac{\Delta L\sigma^2}{nB\epsilon^4} + \frac{\Delta LD}{\epsilon^2}\right) | Helped |
| Combine gradient tracking and accelerated gossip | DeTAG · Stochastic non-convex decentralized training | communication complexity: O\left(\frac{\Delta L\log\left(n + \frac{\varsigma_0n}{\epsilon\sqrt{\Delta L}}\right)}{\epsilon^2\sqrt{1-\lambda}} \rig | Helped |
| Use DeTAG rather than D-PSGD, D², or DSGT | LeNet on CIFAR10 · CIFAR10, less-shuffled local data | convergence speed | Helped |
| Use DeTAG rather than decentralized baselines | ResNet20 · CIFAR100, Ring graph with varied spectral gaps | convergence speed | Helped |
| Decrease κ to make the network sparser | DeTAG · CIFAR100, ResNet20 on a Ring graph | convergence robustness | Helped |
| Remove accelerated gossip (set η=0) (ablation) | DeTAG · Stochastic non-convex decentralized training | communication complexity: R will become \tilde{O}\left(\frac{1}{1-\lambda}\right) | Hurt |
| Set phase length R=1 and acceleration η=0 (ablation) | DeTAG · Decentralized training | performance | No clear effect |
| Replace factorized consensus matrices with repeated use of one gossip matrix (ablation) | DeFacto-style algorithm · Stochastic non-convex decentralized training | communication complexity: O(n\varsigma) | Hurt |
Closest papers by meaning
- Enhancing Parallelism in Decentralized Stochastic Convex Optimization (ICML 2025 · similarity 0.88)
- Faster Adaptive Decentralized Learning Algorithms (ICML 2024 · similarity 0.88)
- Stability and Generalization Analysis of Decentralized SGD: Sharper Bounds Beyond Lipschitzness and Smoothness (ICML 2025 · similarity 0.87)
- Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training (NeurIPS 2025 · similarity 0.87)
- Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent (AAAI 2024 · similarity 0.87)
- Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.