Research map / Optimization / Implicit bias and generalization
Stability and Generalization of Decentralized Stochastic Gradient Descent
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use decentralized updates instead of centralized SGD | SGD · General convex | uniform stability: The centralized bound is \frac{2B^2\sum_{t=1}^{T-1}\alpha_t}{mn}; D-SGD adds 4B^2\sum_{t=1}^{T-1}\Big[(1+\alpha_t B)\sum | Hurt |
| Use decentralized updates instead of centralized SGD | SGD · Strongly convex | uniform stability: The centralized bound is {2B^2}/{(mnν)}; D-SGD adds \frac{4( 1+\alpha B)B^2}{\nu} \frac{1_{\lambda\neq 0}}{1-\lambda} | Hurt |
| Use a smaller learning rate | D-SGD · Body Fat | absolute loss difference | Helped |
| Use a complete graph rather than other tested graph structures | D-SGD · Body Fat | absolute loss difference | Helped |
| Increase the learning rate | D-SGD · ijcnn1 | absolute loss difference | Hurt |
| Train a nonconvex model rather than convex or strongly convex models | D-SGD · CIFAR10 | stability | Hurt |
| Use a cycle graph rather than a more favorable tested graph | D-SGD · Body Fat | absolute loss difference | Hurt |
Closest papers by meaning
- Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent (AAAI 2024 · similarity 0.94)
- Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm (ICML 2024 · similarity 0.94)
- Stability and Generalization Analysis of Decentralized SGD: Sharper Bounds Beyond Lipschitzness and Smoothness (ICML 2025 · similarity 0.92)
- Error Analysis Affected by Heavy-Tailed Gradients for Non-Convex Pairwise Stochastic Gradient Descent (AAAI 2025 · similarity 0.91)
- Stability and Generalization of Asynchronous SGD: Sharper Bounds Beyond Lipschitz and Smoothness (NeurIPS 2024 · similarity 0.90)
- Stability and Generalization of Zeroth-Order Decentralized Stochastic Gradient Descent with Changing Topology (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.