Research map / Optimization / Distributed and compressed
Communication-efficient Distributed Learning for Large Batch Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| joint sparsification of gradient computation (skip computing dropped blocks) on top of communication sparsification | JointSpar-LARS · MNIST, SVHN, Fashion-MNIST, CIFAR10, CIFAR100 | wall-clock speedup to target accuracy vs baselines: close to 2x in most cases, up to 4x; as low as 1.1x for FullPrecision on SVHN | Helped |
| bandit-based distribution learning for block selection (vs. full-gradient compression baselines) | JointSpar-LARS · ImageNet AlexNet top-5 accuracy vs time | convergence speed in wall-clock time | Helped |
| per-machine computation share under large batch: full-gradient compression keeps compute dominant | TernGrad and GSPAR compression baselines · ImageNet AlexNet, 8 machines | tcomp/tcomm per epoch: compute time 1205 and 1079 vs SGD 1021; comm reduced to 446 and 519 from 1156 | No clear effect |
Closest papers by meaning
- Accelerated Methods with Compressed Communications for Distributed Optimization Problems Under Data Similarity (AAAI 2025 · similarity 0.90)
- Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients (NeurIPS 2019 · similarity 0.90)
- Gradient Sparsification for Communication-Efficient Distributed Optimization (NeurIPS 2018 · similarity 0.90)
- LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning (NeurIPS 2018 · similarity 0.90)
- Approximate Gradient Coding for Distributed Learning with Heterogeneous Stragglers (NeurIPS 2025 · similarity 0.90)
- Towards Faster Decentralized Stochastic Optimization with Communication Compression (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.