AI Paper Atlas

Research map / Reinforcement learning

Offline RL: research map

646 papers on Offline RL in Reinforcement learning, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 2 clusters and 8 approaches. The busiest year so far is 2026.

What it is

Offline RL learns a policy entirely from a fixed dataset of past interactions, without trying new actions, which matters when exploration is costly or unsafe (healthcare, robots). The core difficulty is that the policy may choose actions the data never covered, so methods stay conservative.

Terms you will see on this page:

Offline RL
learning a policy from logged data only.
Distribution shift (in RL)
the learned policy visiting states and actions the dataset lacks.
Conservatism
penalising values of actions that the data does not support.

Start reading with the most cited papers here:

Within Reinforcement learning, its share shrank from 15.3% in 2023–24 to 10.7% in 2025–26 (240 → 247 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).

2016: 1162017: 0172018: 4182019: 2192020: 14202021: 47212022: 76222023: 128232024: 123242025: 107252026: 14426

Explore Offline RL in the interactive map

Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.

What changed recently

Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):

Papers from 2025–2026 that are already cited most:

Approaches and key papers

Offline RL algorithms · 409 papers
Keywords: offline reinforcement, offline rl, conservative

Approaches in this cluster:

Most cited and most cited since 2024:

Offline-to-online and sequence models · 237 papers
Keywords: offline online, transformer, online reinforcement

Approaches in this cluster:

Most cited and most cited since 2024:

Related topics in Reinforcement learning

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy