Distributed and compressed: research map
349 accepted papers on Distributed and compressed in Optimization, from ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026), grouped into 2 clusters and 7 approaches. The busiest year so far is 2026.
Within Optimization, its share held steady from 8.2% in 2023–24 to 7.7% in 2025–26 (71 → 75 papers at ICML, NeurIPS, CVPR and AAAI, the venues with data for all four years).
Explore Distributed and compressed in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
Approaches and key papers
decentralized · asynchronous · byzantine · 240 papers
Approaches in this cluster:
- Decentralized optimization algorithms (94 papers)
Design and analyze decentralized and distributed optimization methods with faster convergence. - Local and asynchronous SGD (84 papers)
Analyze asynchronous, local, and minibatch SGD for communication-efficient distributed training. - Byzantine-robust learning (32 papers)
Defend distributed training against adversarial workers with robust aggregation. - Compressed distributed optimization (30 papers)
Reduce communication in decentralized and distributed training via compression and error feedback.
Most cited and most cited since 2024:
- On the Convergence of FedAvg on Non-IID Data (ICLR 2020 · 979 citations)
- Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent (NeurIPS 2017 · 967 citations)
- Byzantine-Robust Decentralized Learning via Remove-then-Clip Aggregation (AAAI 2024 · 9 citations)
- Jointly Improving the Sample and Communication Complexities in Decentralized Stochastic Minimax Optimization (AAAI 2024 · 5 citations)
communication distributed · gradient compression · error feedback · 109 papers
Approaches in this cluster:
- Gradient quantization and sparsification (57 papers)
Compress gradients with quantization and sparsification to cut communication in distributed SGD. - Communication complexity theory (35 papers)
Prove lower bounds and communication-optimal distributed algorithms with compression. - Error feedback theory (17 papers)
Analyze error feedback mechanisms for compressed distributed optimization.
Most cited and most cited since 2024:
- QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding (NeurIPS 2017 · 911 citations)
- TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning (NeurIPS 2017 · 662 citations)
- Exponential Quantum Communication Advantage in Distributed Inference and Learning (NeurIPS 2024 · 2 citations)
- Sketched Adaptive Distributed Deep Learning: A Sharp Convergence Analysis (NeurIPS 2025 · 1 citations)
Related topics in Optimization
- Implicit bias and generalization (958)
- Nonconvex and smooth optimization (1,448)
- Optimizers (1,124)