Research map / Reinforcement learning / Imitation learning
Reward learning from demonstrations
Infer reward functions from suboptimal or limited demonstrations with scalable inverse RL algorithms.
70 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Imitation learning, keyword cluster “inverse reinforcement · irl · reward function”
State History Tried Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
18 papers in 2023–24 and 14 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 15.1% to 12.1%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Showing versus doing: Teaching by demonstration (NeurIPS 2016 · 72 citations)
- 2017 · Inverse Reward Design (NeurIPS 2017 · 61 citations)
- 2018 · OptionGAN: Learning Joint Reward-Policy Options Using Generative Adversarial Inverse Reinforcement Learning (AAAI 2018 · 59 citations)
- 2019 · Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications (AAAI 2019 · 56 citations)
- 2020 · Predicting Goal-Directed Human Attention Using Inverse Reinforcement Learning (CVPR 2020 · 100 citations)
- 2021 · Scalable Bayesian Inverse Reinforcement Learning (ICLR 2021 · 13 citations)
- 2022 · Programmatic Reward Design by Example (AAAI 2022 · 8 citations)
- 2023 · The Perils of Trial-and-Error Reward Design: Misdesign through Overfitting and Invalid Task Specifications (AAAI 2023 · 48 citations)
- 2024 · Massively Scalable Inverse Reinforcement Learning in Google Maps (ICLR 2024 · 2 citations)
- 2025 · Inverse Reinforcement Learning by Estimating Expertise of Demonstrators (AAAI 2025 · 3 citations)
- 2026 · Recovering Hidden Reward in Diffusion-Based Policies (ICML 2026 · 0 citations)
Earliest papers here
- Showing versus doing: Teaching by demonstration (NeurIPS 2016 · 72 citations)
- Distance Minimization for Reward Learning from Scored Trajectories (AAAI 2016 · 24 citations)
- Inverse Reward Design (NeurIPS 2017 · 61 citations)
What has been tried
53 papers here report 480 measured changes (a component added, removed or swapped, compared with a baseline): helped 60%, no clear effect 12%, hurt 20%, mixed 8%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add demonstrator-specific accuracy deviations and learnable precision to IQ | Cartpole and Lunar Lander, three suboptimal dataset settings · return relative to best demonstrator | Helped | Inverse Reinforcement Learning by Estimating Expertise of Demonstrators (AAAI 2025 · 3 citations) | |
| Set RHIP horizon to H=10 instead of using MaxEnt | Experimental region · training time | 70% faster | Helped | Massively Scalable Inverse Reinforcement Learning in Google Maps (ICLR 2024 · 2 citations) |
| reduce expert demonstrations from 64 to 5 trajectories | MuJoCo low-data regime · convergence speed | Helped | Hybrid Inverse Reinforcement Learning (ICML 2024 · 2 citations) | |
| Set entropy regularization τ to 0 | MVTec-AD mean · Detection AUC | 97.0 -> 67.9 | Hurt | Maximum Entropy Inverse Reinforcement Learning of Diffusion Models with Energy-Based Models (NeurIPS 2024 · 2 citations) |
| Add self-reflection | IsaacGym tasks · task success | 233.92 -> 114.66 (Cartpole); 464.40 -> 153.52 (Ball Balance) | Helped | ELEMENTAL: Interactive Learning from Demonstrations and Vision-Language Models for Reward Design in Robotics (ICML 2025 · 0 citations) |
| adaptively select demonstration environments using maximin regret instead of selecting environments uniformly at random | Continuous maze · normalized return | Demo: 52±07 -> 68±04; Test: 53±12 -> 71±02 | Helped | Environment Design for Inverse Reinforcement Learning (ICML 2024 · 0 citations) |
| Add L2 regularization instead of clipping with reward normalization | Five MuJoCo environments · feasible rewards | Helped | Simplifying Constraint Inference with Inverse Reinforcement Learning (NeurIPS 2024 · 0 citations) |
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 32 papers; from OpenAlex and the papers' affiliations).
- Carnegie Mellon University · 3 papers
- TU Darmstadt · 3 papers
- Google DeepMind · 3 papers
- Google · 2 papers
- Georgia Institute of Technology · 2 papers
- Technische Universität Darmstadt · 2 papers
- University of Oxford · 2 papers
- Cornell University · 2 papers
- The Chinese University of Hong Kong · 2 papers
- Mila · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Theory of inverse RL (44 papers)
Elsewhere in the topic
- Offline imitation from imperfect data (108 papers)
- Imitation combined with RL (108 papers)
- Behavior cloning theory and models (68 papers)
- Adversarial imitation learning (33 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).