Research map / Reinforcement learning / Imitation learning
Offline imitation from imperfect data
Imitate from suboptimal or limited demonstrations using distribution correction and offline data.
108 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Imitation learning, keyword cluster “il · adversarial imitation · cloning”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
26 papers in 2023–24 and 31 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 21.8% to 26.7%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Smooth Imitation Learning for Online Sequence Prediction (ICML 2016 · 14 citations)
- 2018 · Exponentially Weighted Imitation Learning for Batched Historical Data (NeurIPS 2018 · 45 citations)
- 2019 · Imitating Latent Policies from Observation (ICML 2019 · 55 citations)
- 2020 · Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences (ICML 2020 · 41 citations)
- 2021 · Primal Wasserstein Imitation Learning (ICLR 2021 · 36 citations)
- 2022 · Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations (ICML 2022 · 9 citations)
- 2023 · Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching (AAAI 2023 · 9 citations)
- 2024 · Harnessing Network Effect for Fake News Mitigation: Selecting Debunkers via Self-Imitation Learning (AAAI 2024 · 6 citations)
- 2025 · Learning from Demonstrations via Capability-Aware Goal Sampling (NeurIPS 2025 · 1 citation)
- 2026 · Distinguishing Imitation Error from Intrinsic Motion Learning Difficulty (ICML 2026 · 0 citations)
Earliest papers here
- Smooth Imitation Learning for Online Sequence Prediction (ICML 2016 · 14 citations)
- Exponentially Weighted Imitation Learning for Batched Historical Data (NeurIPS 2018 · 45 citations)
- Imitating Latent Policies from Observation (ICML 2019 · 55 citations)
What has been tried
81 papers here report 757 measured changes (a component added, removed or swapped, compared with a baseline): helped 59%, no clear effect 13%, hurt 21%, mixed 7%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add negative samples to GASIL | Synthetic Twitter networks, ablation study · episodic reward | Helped | Harnessing Network Effect for Fake News Mitigation: Selecting Debunkers via Self-Imitation Learning (AAAI 2024 · 6 citations) | |
| Add expert-provided sub-task labels | Fetch Pick-and-Place tasks · cumulative reward | Helped | GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimation (AAAI 2024 · 4 citations) | |
| Add the reward regularizer | Mujoco tasks, N=2 and N=3 · normalized return | Helped | SPRINQL: Sub-optimal Demonstrations driven Offline Imitation Learning (NeurIPS 2024 · 3 citations) | |
| give RPI access to three diversified oracles instead of only one | Cartpole · return | below 600 -> 645 | Helped | Blending Imitation and Reinforcement Learning for Robust Policy Improvement (ICLR 2024 · 2 citations) |
| replace soft-TV distance with soft-χ2 divergence | hopper, rebalanced by state, p(D1)=0.5 · Normalized score | 67.1 ± 8.2 -> 60.9 ± 7.6 | Hurt | Mitigating Covariate Shift in Behavioral Cloning via Robust Stationary Distribution Correction (NeurIPS 2024 · 2 citations) |
| Use capability-aware goal sampling instead of always selecting the final demonstration state | Disassemble, StickPush, and Pen · success rate | Helped | Learning from Demonstrations via Capability-Aware Goal Sampling (NeurIPS 2025 · 1 citation) | |
| Replace KD-tree with Ball-tree search | D4RL MuJoCo locomotion (9 tasks) · total normalized score | 713.3 -> 712.1 | No clear effect | SEABO: A Simple Search-Based Method for Offline Imitation Learning (ICLR 2024 · 1 citation) |
| Remove the selected-data cloning term (importance-sampling weighting only) | Offline imitation learning benchmarks · normalized score | Hurt | How to Leverage Diverse Demonstrations in Offline Imitation Learning (ICML 2024 · 1 citation) | |
| Use rollout augmentation (RA) during AED training | Pick & Place · AUROC and AUPRC | Helped | AED: Adaptable Error Detection for Few-shot Imitation Policy (NeurIPS 2024 · 1 citation) | |
| Combine policy loss and dynamics loss, with higher weight on policy loss | Walker2d · average reward | Helped | Markov Balance Satisfaction Improves Performance in Strictly Batch Offline Imitation Learning (AAAI 2025 · 0 citations) | |
| replace learned distance d with goal metric h | fetch_pick_and_place-all · W_min | 0.070±0.009 -> 0.207±0.026 | Hurt | Zero-Shot Offline Imitation Learning via Optimal Transport (ICML 2025 · 0 citations) |
| Use 1 critic network instead of multiple critics | Two-state hard-exploration MDP · return | Hurt | IL-SOAR : Imitation Learning with Soft Optimistic Actor cRitic (ICML 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 19 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Baselines
- BC · 3 of 19 papers
- GAIL · 3 of 19 papers
- ValueDICE · 2 of 19 papers
- SQIL · 2 of 19 papers
- JSRL · 2 of 19 papers
- Cal-QL · 2 of 19 papers
- RLPD · 2 of 19 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 57 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 4 papers
- University of Illinois, Urbana Champaign · 3 papers
- EPFL · 3 papers
- University of California, Berkeley · 3 papers
- Stanford University · 2 papers
- NVIDIA · 2 papers
- The University of Tokyo · 2 papers
- Nanjing University · 2 papers
- University of California, Davis · 2 papers
- University of Houston · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Imitation combined with RL (108 papers)
- Behavior cloning theory and models (68 papers)
- Adversarial imitation learning (33 papers)
Elsewhere in the topic
- Reward learning from demonstrations (70 papers)
- Theory of inverse RL (44 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).