Research map / Reinforcement learning
MDPs and planning: research map
726 papers on MDPs and planning in Reinforcement learning, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 2 clusters and 8 approaches. The busiest year so far is 2024.
What it is
The theory of reinforcement learning in Markov decision processes: how much data is needed to learn good policies, how policy optimisation converges, and how to act when the agent cannot observe the full state (POMDPs) or the model is uncertain.
Terms you will see on this page:
- Sample complexity
- how many interactions are needed to learn a near-optimal policy.
- POMDP
- a decision process where the agent sees only partial observations of the state.
- Robust MDP
- planning against the worst model within a set of plausible ones.
Start reading with the most cited papers here:
- Is Q-Learning Provably Efficient? (NeurIPS 2018 · 306 citations)
- Mitigating Bias in Face Recognition Using Skewness-Aware Reinforcement Learning (CVPR 2020 · 229 citations)
Within Reinforcement learning, its share shrank from 12.9% in 2023–24 to 7.6% in 2025–26 (203 → 175 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).
Explore MDPs and planning in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
What changed recently
Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):
- Average-reward and regularized MDPs rising: 14% → 22% of the topic (29 → 39 papers)
- Robust MDP policy gradient rising: 12% → 18% of the topic (24 → 31 papers)
- POMDP policy optimization rising: 8% → 10% of the topic (16 → 18 papers)
- Function approximation in RL theory falling: 18% → 11% of the topic (37 → 19 papers)
- POMDPs and confounded RL falling: 16% → 12% of the topic (33 → 21 papers)
Papers from 2025–2026 that are already cited most:
- Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach (AAAI 2025 · 5 citations)
- Probabilistic Shielding for Safe Reinforcement Learning (AAAI 2025 · 5 citations)
- Real-Time Recurrent Reinforcement Learning (AAAI 2025 · 4 citations)
- Overcoming the Curse of Dimensionality in Reinforcement Learning Through Approximate Factorization (ICML 2025 · 2 citations)
Approaches and key papers
MDPs and partial observability · 424 papers
Keywords: markov decision, processes, observable
Approaches in this cluster:
- Average-reward and regularized MDPs (146 papers)
Develop theory for regularized, average-reward, and non-stationary MDPs. - POMDPs and confounded RL (109 papers)
Study RL under partial observability and confounding. - Robust MDP policy gradient (98 papers)
Optimize policies in robust MDPs with rectangular uncertainty. - POMDP policy optimization (71 papers)
Solve POMDPs with Monte Carlo, transformers, and policy-based methods.
Most cited and most cited since 2024:
- Mitigating Bias in Face Recognition Using Skewness-Aware Reinforcement Learning (CVPR 2020 · 229 citations)
- A Theory of Regularized Markov Decision Processes (ICML 2019 · 116 citations)
- Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach (AAAI 2025 · 5 citations)
- Principal-Agent Reward Shaping in MDPs (AAAI 2024 · 5 citations)
Sample complexity of RL · 302 papers
Keywords: sample complexity, function approximation, epsilon
Approaches in this cluster:
- Function approximation in RL theory (97 papers)
Establish sample-efficient RL with general and low-rank function approximation. - Offline RL sample complexity (87 papers)
Prove near-optimal offline RL guarantees with pessimism and variance reduction. - Policy optimization theory (64 papers)
Analyze policy finetuning, reward-free exploration, and effective horizon. - Rich-observation exploration (54 papers)
Explore provably with state abstraction in rich-observation and model-based RL.
Most cited and most cited since 2024:
- Is Q-Learning Provably Efficient? (NeurIPS 2018 · 306 citations)
- Contextual Decision Processes with low Bellman rank are PAC-Learnable (ICML 2017 · 143 citations)
- Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL (NeurIPS 2024 · 11 citations)
- Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo (ICLR 2024 · 5 citations)
Related topics in Reinforcement learning
- Visual and representation RL (2,264)
- Policy gradient and actor-critic (1,570)
- Imitation learning (450)
- RL from feedback (758)
- Offline RL (646)
