Research map / Optimization / Distributed and compressed
Federated Accelerated Stochastic Gradient Descent
Honglin Yuan, Tengyu Ma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedAc-I instead of FedAvg | Distributed stochastic optimization · Strongly convex, smooth objectives | communication overhead term: \frac{L \sigma^2}{\mu^2 T R} -> \frac{L \sigma^2}{\mu^2 T R^2} | Improved |
| Use FedAc-II instead of FedAvg | Distributed stochastic optimization · Strongly convex, smooth objectives | communication overhead term: \frac{L \sigma^2}{\mu^2 T R} -> \frac{L^2 \sigma^2}{\mu^3 T R^3} | Improved |
| Use FedAc instead of FedAvg | Distributed stochastic optimization · General convex objectives | communication-dependent convergence term: \frac{1}{T^{\frac{1}{3}} R^{\frac{1}{3}}} -> \frac{1}{T^{\frac{1}{3}} R^{\frac{2}{3}}} | Improved |
| Use FedAc instead of Minibatch-SGD | Distributed stochastic optimization · General convex objectives | communication-dependent convergence term: \frac{1}{R} -> \frac{1}{T^{\frac{1}{3}} R^{\frac{2}{3}}} | Improved |
| Apply FedAc-II under third-order smoothness instead of FedAvg | Distributed stochastic optimization · Strongly convex, third-order-smooth objectives | communication overhead term: \frac{Q^2 \sigma^4}{\mu^5 T^2 R^2} -> \frac{Q^2 \sigma^4}{\mu^5 T^2 R^6} | Improved |
| Use FedAc-I rather than FedAvg | Logistic regression · a9a, 8192 workers, suboptimality 10^{-3} | communication rounds: 4096 -> 32 | Improved |
| Use FedAc-I rather than Minibatch-SGD | Logistic regression · a9a, 8192 workers, suboptimality 10^{-3} | communication rounds: 1024 -> 32 | Improved |
| Use FedAc-I rather than Minibatch-Accelerated-SGD | Logistic regression · a9a, 8192 workers, suboptimality 10^{-3} | communication rounds: 128 -> 32 | Improved |
| Use FedAc-I rather than FedAvg | Logistic regression · epsilon, 2048 workers, suboptimality 10^{-4} | communication rounds: 4096 -> 64 | Improved |
| Use FedAc-I rather than Minibatch-SGD | Logistic regression · epsilon, 2048 workers, suboptimality 10^{-4} | communication rounds: 4096 -> 64 | Improved |
| Use FedAc-I rather than Minibatch-Accelerated-SGD | Logistic regression · epsilon, 2048 workers, suboptimality 10^{-4} | communication rounds: 256 -> 64 | Improved |
| Remove the acceleration-stability tradeoff and use full acceleration (ablation) | FedAc · Direct parallelization with infrequent synchronization | stability | Hurt |
Closest papers by meaning
- Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization (ICML 2020 · similarity 0.90)
- On the Convergence of FedAvg on Non-IID Data (ICLR 2020 · similarity 0.89)
- Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning (NeurIPS 2022 · similarity 0.88)
- Federated Optimization with Doubly Regularized Drift Correction (ICML 2024 · similarity 0.88)
- Faster Rates For Federated Variational Inequalities (NeurIPS 2026 · similarity 0.87)
- Stabilized Proximal-Point Methods for Federated Optimization (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
