Research map / Optimization / Distributed and compressed
Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add acceleration to compressed gradient descent | CGD · Single-machine strongly convex | iteration complexity: O\left((1+\omega)\frac{L}{\mu}\log \frac{1}{\epsilon}\right) -> O\left((1+\omega)\sqrt{\frac{L}{\mu}}\log \frac{1}{\epsi | Helped |
| Add acceleration to compressed gradient descent | CGD · Single-machine convex | iteration complexity: O\left((1+\omega)\frac{L}{\epsilon}\right) -> O\left((1+\omega)\sqrt{\frac{L}{\epsilon}}\right) | Helped |
| Add acceleration and variance reduction to distributed compressed gradient descent | DIANA · Distributed strongly convex, n ≤ ω | iteration complexity: O\left(\omega\big(1+\frac{L}{n\mu}\big)\log \frac{1}{\epsilon}\right) -> O\left(\omega\Big(1+\sqrt{\frac{L}{n\mu}}\Big)\ | Helped |
| Add acceleration and variance reduction to distributed compressed gradient descent | DIANA · Distributed strongly convex, n > ω | iteration complexity: O\left(\big(\omega+\frac{L}{\mu}\big)\log \frac{1}{\epsilon}\right) -> O\left(\Big(\omega+\sqrt{\frac{L}{\mu}}+\sqrt{\sq | Helped |
| Use ADIANA instead of DIANA | ADIANA and DIANA · a5a, mushrooms; random sparsification, random dithering, and natural compression | communication efficiency | Helped |
| Use ADIANA instead of its uncompressed version | ADIANA · a5a and mushrooms with random sparsification | communication efficiency | Hurt |
| Use random dithering compression with ADIANA rather than no compression | ADIANA · a5a and mushrooms | communication efficiency: about twice faster | Helped |
| Use natural compression with ADIANA rather than no compression | ADIANA · a5a and mushrooms | communication efficiency: about twice faster | Helped |
| Use variance-reduced compression rather than vanilla compressed gradient descent | DIANA and ADIANA versus DCGD · Regularized logistic regression on a5a and mushrooms | convergence to optimum | Helped |
Closest papers by meaning
- Accelerated Methods with Compressed Communications for Distributed Optimization Problems Under Data Similarity (AAAI 2025 · similarity 0.92)
- CANITA: Faster Rates for Distributed Convex Optimization with Communication Compression (NeurIPS 2021 · similarity 0.92)
- Scalable Distributed Stochastic Optimization via Bidirectional Compression: Beyond Pessimistic Limits (NeurIPS 2026 · similarity 0.91)
- Federated Optimization with Doubly Regularized Drift Correction (ICML 2024 · similarity 0.89)
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity (NeurIPS 2026 · similarity 0.88)
- Towards Faster Decentralized Stochastic Optimization with Communication Compression (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.