AI Paper Atlas

Research map / Optimization

Optimizers: research map

1,206 papers on Optimizers in Optimization, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 4 clusters and 16 approaches. The busiest year so far is 2025.

What it is

The algorithms that actually train networks, such as Adam and SGD with momentum, and how to make them faster, more memory-efficient or better understood. The topic also covers gradients through inner optimisation (bilevel and meta-learning), sampling-based optimisers and zeroth-order methods.

Terms you will see on this page:

Adam
an adaptive optimizer that scales each parameter's step by running gradient statistics.
Preconditioning
reshaping gradient steps using curvature information.
Bilevel optimization
an optimisation problem nested inside another, as in hyperparameter tuning.

Start reading with the most cited papers here:

Within Optimization, its share held steady from 27.9% in 2023–24 to 29.7% in 2025–26 (242 → 290 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).

2016: 60162017: 52172018: 68182019: 102192020: 111202021: 115212022: 132222023: 122232024: 142242025: 154252026: 14826

Explore Optimizers in the interactive map

Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.

What changed recently

Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):

Papers from 2025–2026 that are already cited most:

Approaches and key papers

Differentiable losses and solvers · 469 papers
Keywords: differentiable, functions, loss

Approaches in this cluster:

Most cited and most cited since 2024:

Adaptive optimizers · 321 papers
Keywords: adaptive, optimizers, momentum

Approaches in this cluster:

Most cited and most cited since 2024:

Sampling and gradient estimators · 313 papers
Keywords: variational, sampling, inference

Approaches in this cluster:

Most cited and most cited since 2024:

Bilevel and meta-learning optimization · 103 papers
Keywords: bilevel, inner, shot

Approaches in this cluster:

Most cited and most cited since 2024:

Related topics in Optimization

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy