AI Paper Atlas

Research map / Bandits and online learning

Multi-armed bandits: research map

537 papers on Multi-armed bandits in Bandits and online learning, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 2 clusters and 7 approaches. The busiest year so far is 2021.

What it is

A bandit problem is choosing repeatedly among options ("arms") with unknown payoffs, as in A/B testing or ad selection. The learner must trade off trying arms to learn about them against playing the one that looks best; pure-exploration variants just aim to identify the best arm quickly.

Terms you will see on this page:

Arm
one of the options a bandit algorithm can choose.
Exploration-exploitation trade-off
gathering information versus using what is known.
Best-arm identification
finding the best arm with as few pulls as possible.

Start reading with the most cited papers here:

Within Bandits and online learning, its share grew from 19.0% in 2023–24 to 21.0% in 2025–26 (106 → 125 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).

2016: 21162017: 24172018: 26182019: 45192020: 46202021: 72212022: 54222023: 70232024: 47242025: 64252026: 6826

Explore Multi-armed bandits in the interactive map

Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.

What changed recently

Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):

Papers from 2025–2026 that are already cited most:

Approaches and key papers

Stochastic multi-armed bandits · 380 papers
Keywords: multi armed, armed bandit, stochastic

Approaches in this cluster:

Most cited and most cited since 2024:

Best-arm identification · 157 papers
Keywords: identification, best arm, pure exploration

Approaches in this cluster:

Most cited and most cited since 2024:

Related topics in Bandits and online learning

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy