Research map / Optimization / Distributed and compressed
DSA: Decentralized Double Stochastic Averaging Gradient Algorithm
Aryan Mokhtari, Alejandro Ribeiro
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DSA instead of EXTRA (fewer gradients per iteration) | EXTRA · logistic regression Q=500, N=20 | gradient evaluations to error 1e-8: 1725 -> 380 | Improved |
| use DSA instead of EXTRA | EXTRA · logistic regression Q=500, N=20 | iterations to error 1e-8: 69 -> 380 | Hurt |
| add double descent (EXTRA mixing) to stochastic averaging gradient (ablation) | Decentralized SAGA · logistic regression Q=500, N=20 | convergence | Improved |
| add stochastic averaging to EXTRA instead of plain stochastic gradient (ablation) | Stochastic EXTRA · logistic regression Q=500, N=20 | convergence | Improved |
| reduce step size from 1e-2 to 1e-3 (ablation) | DGD, sto-EXTRA, D-SAGA · logistic regression Q=500, N=20 | accuracy vs speed: alpha = 10^-2 -> 10^-3 | Mixed |
| use complete graph instead of line graph (ablation) | DSA · N=50, Q=500 | iterations to error 1e-8: 1819 -> 247 | Improved |
| use cycle graph instead of line graph (ablation) | DSA · N=50, Q=500 | iterations to error 1e-8: 1819 -> 1133 | Improved |
| use random graph pc=0.35 instead of pc=0.25 (ablation) | DSA · N=50, Q=500 | iterations to error 1e-8: 504 -> 310 | Improved |
| increase total samples Q from 100 to 5000 (q from 5 to 250) (ablation) | DSA · N=20 random graph | convergence time | Hurt |
| larger graph condition number (ablation) | DSA vs EXTRA · N=50 topologies | DSA advantage over EXTRA | Improved |
| choose N=125 (q=4) nodes instead of very small or large N (ablation) | DSA · Q=500 | convergence | Improved |
| use DSA instead of EXTRA on large problem | EXTRA · protein homology, Q=1.45x10^5, N=200 | gradient evaluations to target accuracy | Improved |
Closest papers by meaning
- Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance (AAAI 2026 · similarity 0.89)
- Stability and Generalization of the Decentralized Stochastic Gradient Descent Ascent Algorithm (NeurIPS 2023 · similarity 0.89)
- Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent (AAAI 2024 · similarity 0.89)
- Decentralized Convex Finite-Sum Optimization with Better Dependence on Condition Numbers (ICML 2024 · similarity 0.88)
- Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence (AAAI 2025 · similarity 0.88)
- Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
