AI Paper Atlas

Research map / Optimization

Distributed and compressed: research map

388 papers on Distributed and compressed in Optimization, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 2 clusters and 7 approaches. The busiest year so far is 2021.

What it is

Training on many machines is limited by communication. This topic designs and analyses methods that send less (compressed or quantised gradients), sync less often (local and asynchronous SGD), have no central server (decentralised) or tolerate faulty and malicious workers (Byzantine robustness).

Terms you will see on this page:

Gradient compression
sending fewer or lower-precision gradient values.
Local SGD
workers take several steps alone before averaging.
Byzantine robustness
still converging when some workers send arbitrary wrong updates.

Start reading with the most cited papers here:

Within Optimization, its share held steady from 8.2% in 2023–24 to 7.7% in 2025–26 (71 → 75 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).

2016: 9162017: 17172018: 39182019: 35192020: 41202021: 47212022: 38222023: 38232024: 38242025: 40252026: 4626

Explore Distributed and compressed in the interactive map

Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.

What changed recently

Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):

Papers from 2025–2026 that are already cited most:

Approaches and key papers

Decentralized and robust distributed training · 273 papers
Keywords: decentralized, asynchronous, byzantine

Approaches in this cluster:

Most cited and most cited since 2024:

Communication-efficient training · 115 papers
Keywords: communication distributed, gradient compression, error feedback

Approaches in this cluster:

Most cited and most cited since 2024:

Related topics in Optimization

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy