Research map / Optimization / Distributed and compressed
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD
Kun Yuan, Sulaiman A. Alghunaim, Xinmeng Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-round gossip and gradient accumulation to D2/Exact-Diffusion (ablation) | D2/Exact-Diffusion · transient stage, generally convex | transient stage order: Omega(n^3/(1-beta)^2) -> Omega~(n/(1-beta)) | Improved |
| add multi-round gossip and gradient accumulation to D2/Exact-Diffusion (ablation) | D2/Exact-Diffusion · transient stage, strongly convex | transient stage order: Omega~(n/(1-beta)) -> Omega~(1/(1-beta)^(1/2)) | Improved |
| remove data heterogeneity influence by using D2/Exact-Diffusion instead of D-SGD | D-SGD · transient stage, generally convex | transient stage order: Omega(n^3/(1-beta)^4) -> Omega(n^3/(1-beta)^2) | Improved |
| remove data heterogeneity influence by using D2/Exact-Diffusion instead of D-SGD | D-SGD · transient stage, strongly convex | transient stage order: Omega~(n/(1-beta)^2) -> Omega~(n/(1-beta)) | Improved |
| grow cycle from 32 to 64 nodes (1/(1-beta) from 78.07 to 311.51) (ablation) | D-SGD · logistic regression, cycle graph | convergence performance: 78.07 -> 311.51 | Hurt |
| homogeneous data (sigma_h^2 = 0) instead of heterogeneous (ablation) | D-SGD vs D2/ED · least squares, cycle of 32 nodes | convergence | No clear effect |
Closest papers by meaning
- A Unified Theory of Decentralized SGD with Changing Topology and Local Updates (ICML 2020 · similarity 0.94)
- Improved Convergence Analysis of Topology Dependence in Decentralized SGD (ICML 2026 · similarity 0.91)
- Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm (ICML 2024 · similarity 0.89)
- Achieving Linear Speedup and Near-Optimal Complexity for Decentralized Optimization over Row-stochastic Networks (ICML 2025 · similarity 0.89)
- Lower Bounds and Optimal Algorithms for Non-Smooth Convex Decentralized Optimization over Time-Varying Networks (NeurIPS 2024 · similarity 0.88)
- Enhancing Parallelism in Decentralized Stochastic Convex Optimization (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
