Research map / Reinforcement learning / Imitation learning
Imitation combined with RL
Mix demonstrations with reinforcement learning, including hierarchical and self-correcting policies.
108 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Imitation learning, keyword cluster “il · adversarial imitation · cloning”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
26 papers in 2023–24 and 38 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 21.8% to 32.8%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Third Person Imitation Learning (ICLR 2017 · 146 citations)
- 2018 · Policy Optimization with Demonstrations (ICML 2018 · 78 citations)
- 2019 · Virtual-Taobao: Virtualizing Real-World Online Retail Environment for Reinforcement Learning (AAAI 2019 · 145 citations)
- 2020 · Task-Oriented Dialog Systems That Consider Multiple Appropriate Responses under the Same Context (AAAI 2020 · 134 citations)
- 2021 · Learning to Reach Goals via Iterated Supervised Learning (ICLR 2021 · 23 citations)
- 2022 · Self-Adaptive Imitation Learning: Learning Tasks with Delayed Rewards from Sub-optimal Demonstrations (AAAI 2022 · 11 citations)
- 2023 · Reinforcement Learning for Branch-and-Bound Optimisation Using Retrospective Trajectories (AAAI 2023 · 20 citations)
- 2024 · BAKU: An Efficient Transformer for Multi-Task Policy Learning (NeurIPS 2024 · 17 citations)
- 2025 · ReSim: Reliable World Simulation for Autonomous Driving (NeurIPS 2025 · 5 citations)
- 2026 · Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning (ICML 2026 · 0 citations)
Earliest papers here
- Third Person Imitation Learning (ICLR 2017 · 146 citations)
- Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction (ICML 2017 · 80 citations)
- Policy Optimization with Demonstrations (ICML 2018 · 78 citations)
What has been tried
74 papers here report 747 measured changes (a component added, removed or swapped, compared with a baseline): helped 68%, no clear effect 6%, hurt 22%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Replace the MLP observation trunk with a transformer | LIBERO-90 · success rate | 0.81 -> 0.90 | Helped | BAKU: An Efficient Transformer for Multi-Task Policy Learning (NeurIPS 2024 · 17 citations) |
| vary epsilon over {0.01, 0.05, 0.1, 0.5, 1, 5} | ablation on epsilon · return | Mixed | No Prior Mask: Eliminate Redundant Action for Deep Reinforcement Learning (AAAI 2024 · 13 citations) | |
| add online demonstrations and integrate IL with RL | NoCrash, regular task, testing conditions · success rate (%) | 52±4 -> 86±4 | Helped | RLfOLD: Reinforcement Learning from Online Demonstrations in Urban Autonomous Driving (AAAI 2024 · 12 citations) |
| Add marginal state distribution matching loss to action loss | R2R val · SR | 72 -> 74 | Helped | Vision-Language Navigation with Energy-Based Policy (NeurIPS 2024 · 11 citations) |
| Add simulated CARLA data to training | Waymo Open validation · Trajectory Difference | Helped | ReSim: Reliable World Simulation for Autonomous Driving (NeurIPS 2025 · 5 citations) | |
| add behavior cloning to TD3+HER | sequential subtasks · success rate | 0.00 -> 0.64; 0.00 -> 0.40; 0.00 -> 0.24; 0.00 -> 0.15 | Helped | SurgicAI: A Hierarchical Platform for Fine-Grained Surgical Policy Learning and Benchmarking (NeurIPS 2024 · 4 citations) |
| Add online RL finetuning to offline SORREL pretraining | SC, MIS, CA, CFL, MK standard test · number of nodes | 33 -> 32; 47 -> 44; 261 -> 253; 241 -> 239; 234 -> 142 | Helped | SORREL: Suboptimal-Demonstration-Guided Reinforcement Learning for Learning to Branch (AAAI 2025 · 3 citations) |
| Use iterative adversarial training through lower-3 + upper-2 rather than lower-2 + upper-2 | CMU, hard · E_vel (m/s) | 0.2892 -> 0.2202 | Helped | Adversarial Locomotion and Motion Imitation for Humanoid Policy Learning (NeurIPS 2025 · 2 citations) |
| Replace equivariant ψ U-Net with a traditional U-Net | RLBench stack-blocks, stack-wine, phone-on-base, and put-plate (5 demos) · success rate (%) | 76 -> 76; 100 -> 96; 96 -> 96; 32 -> 24 | Mixed | Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3D (ICLR 2024 · 2 citations) |
| add reward augmentation to DAIL's imitation-learning reward estimator | HalfCheetah, broken source · return | 6402±362 -> 7067±176 | Helped | Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitation (NeurIPS 2024 · 2 citations) |
| Use decoupled translation and rotation action representation | RLBench, 24 unseen tasks · success-rate change | -15 -> 0 | Helped | Instant Policy: In-Context Imitation Learning via Graph Diffusion (ICLR 2025 · 1 citation) |
| add forward contrast to backward coherence | Push-T, open-loop average · success rate | 35.1 ± 1.0 -> 37.8 ± 1.6 | Helped | Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling (ICLR 2025 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 22 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- Robomimic · 4 of 22 papers
- Franka-Kitchen · 3 of 22 papers
- Push-T · 2 of 22 papers
- LIBERO-10 · 2 of 22 papers
Baselines
- TD3 · 2 of 22 papers
- PPO · 2 of 22 papers
- RLPD · 2 of 22 papers
- BC · 2 of 22 papers
- CQL · 2 of 22 papers
- Filtered BC · 2 of 22 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 64 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 3 papers
- Stanford University · 3 papers
- Google · 3 papers
- Carnegie Mellon University · 2 papers
- Shanghai Jiao Tong University · 2 papers
- University of Massachusetts at Amherst · 2 papers
- University of Chicago · 2 papers
- University of California, Berkeley · 2 papers
- Technion · 2 papers
- Zhejiang University · 2 papers
Use it
Download BibTeX of 39 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Offline imitation from imperfect data (108 papers)
- Behavior cloning theory and models (68 papers)
- Adversarial imitation learning (33 papers)
Elsewhere in the topic
- Reward learning from demonstrations (70 papers)
- Theory of inverse RL (44 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).