Research map / Optimization / Distributed and compressed
Trading Redundancy for Communication: Speeding up Distributed SGD for Non-convex Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| infuse data redundancy (mu=0.1, 0.25, 0.5) into periodic model averaging (tau=50) versus no redundancy | RI-SGD vs PR-SGD on ResNet CIFAR10/CIFAR100 · CIFAR10 and CIFAR100 | convergence speed and test accuracy | Helped |
| full redundancy (gamma=1) RI-SGD vs cooperative SGD sampling from entire dataset | RI-SGD with full redundancy · CIFAR10 ResNet44 | training performance | Helped |
| infuse 25% redundancy (mu=0.25) vs PR-SGD (mu=0) at equal batch size 512 | RI-SGD · CIFAR10 ResNet44 | training performance / saturation | Helped |
| increase number of local updates tau | RI-SGD / model averaging · CIFAR CIFAR10/CIFAR100 | accuracy vs communication rounds | Mixed |
| redundancy at equal minibatch size controlled for computation (RI-SGD with minibatch reduced inversely to redundancy) | RI-SGD · CIFAR datasets | accuracy | Helped |
Closest papers by meaning
- Convergence of Distributed Adaptive Optimization with Local Updates (ICLR 2025 · similarity 0.88)
- Accelerated Methods with Compressed Communications for Distributed Optimization Problems Under Data Similarity (AAAI 2025 · similarity 0.88)
- LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging (NeurIPS 2026 · similarity 0.88)
- Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence (AAAI 2025 · similarity 0.88)
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity (NeurIPS 2026 · similarity 0.88)
- Towards Faster Decentralized Stochastic Optimization with Communication Compression (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.