AI Paper Atlas

Research map / Bandits and online learning

RL theory and MDPs: research map

275 papers on RL theory and MDPs in Bandits and online learning, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 2 clusters and 6 approaches. The busiest year so far is 2021.

What it is

The mathematics of reinforcement learning: how many interactions an agent needs to learn a near-optimal policy in a Markov decision process, with or without function approximation, and how to handle changing or adversarial environments.

Terms you will see on this page:

MDP
Markov decision process, the standard model of sequential decisions with states, actions and rewards.
Function approximation
representing values or policies with a parametric model instead of a table.
Regret bound
a guarantee on how much reward is lost while learning.

Start reading with the most cited papers here:

Within Bandits and online learning, its share shrank from 13.1% in 2023–24 to 10.1% in 2025–26 (73 → 60 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).

2016: 1162017: 9172018: 4182019: 14192020: 23202021: 46212022: 41222023: 37232024: 38242025: 32252026: 3026

Explore RL theory and MDPs in the interactive map

Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.

What changed recently

Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):

Papers from 2025–2026 that are already cited most:

Approaches and key papers

RL with function approximation · 156 papers
Keywords: rl, function approximation, linear function

Approaches in this cluster:

Most cited and most cited since 2024:

Markov decision processes · 119 papers
Keywords: markov decision, decision processes, processes mdps

Approaches in this cluster:

Most cited and most cited since 2024:

Related topics in Bandits and online learning

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy