Research map / Optimization / Distributed and compressed
GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning
Anis Elgabli, Jihong Park, Amrit S. Bedi, Mehdi Bennis, Vaneet Aggarwal
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GADMM instead of LAG-WK | linear regression, synthetic data · Linear regression (synthetic, N=24) | iterations to 1e-4: 44,933 -> 558 | Improved |
| use GADMM instead of GD | logistic regression · Logistic regression (N=26) | total communication cost: 31,104 -> 1,712 | Improved |
| use GADMM instead of LAG-WK with few workers | linear regression, N=14 · Linear regression (N=14) | total communication cost: 820 -> 1,092 | Hurt |
| use GADMM instead of LAG-WK in linear regression | synthetic data N=24, rho 3 and 5 · Linear regression (synthetic, N=24) | total communication cost: 6 and 9 times lower than that of LAG-WK | Improved |
| vary penalty rho between 3, 5 and 7 (ablation) | GADMM · Linear regression (synthetic, N=24) | iterations to 1e-4: less than 1,000 iterations for rho 3, 5, 7 | No clear effect |
| lower rho when local data are highly correlated across workers (ablation) | GADMM · Body Fat (real linear regression) | convergence speed | Improved |
| higher rho when local data are independent across workers (ablation) | GADMM · Synthetic linear regression | convergence speed | Improved |
| add dynamic chain reconstruction (D-GADMM) to GADMM (ablation) | GADMM, linear regression N=50 · Synthetic linear regression (N=50, topology change every 15 iterations) | iterations and TC to 1e-4 | Improved |
| use D-GADMM with graph changed every iteration instead of standard ADMM | linear regression N=24 · Synthetic linear regression (N=24, rho=1) | total communication cost: 40 times less | Improved |
| use standard ADMM instead of GADMM | linear regression N=24 · Synthetic linear regression (N=24) | iterations / TC to 1e-4: 4 times higher cost than GADMM | Mixed |
Closest papers by meaning
- Fast ADMM Algorithm for Distributed Optimization with Adaptive Penalty (AAAI 2016 · similarity 0.88)
- A Communication Efficient Stochastic Multi-Block Alternating Direction Method of Multipliers (NeurIPS 2019 · similarity 0.88)
- Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees (ICLR 2025 · similarity 0.87)
- HyperPrism: An Adaptive Non-linear Aggregation Framework for Distributed Machine Learning over Non-IID Data and Time-varying Communication Links (NeurIPS 2024 · similarity 0.86)
- LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging (NeurIPS 2026 · similarity 0.86)
- AdaGK-SGD: Adaptive Global Knowledge Guided Distributed Stochastic Gradient Descent (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
