Research map / Reinforcement learning / Imitation learning
How to Leverage Diverse Demonstrations in Offline Imitation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ILID, an offline imitation learning method that selects imperfect behaviors by whether their resultant states fall in the expert state manifold, then trains a weighted behavior cloning policy on expert and selected data.
Problem
Offline imitation learning with a few expert demonstrations and large imperfect datasets: existing weighting methods select data by state-action similarity to experts, discarding useful diverse behaviors and failing to recover from out-of-distribution states.
Approach
Select imperfect behaviors by their resultant states (using a state-only discriminator to identify expert states, then rolling back K causal state-actions), and apply a lightweight weighted BC objective with importance-sampling weights for expert data and a threshold indicator for selected data.
Key results
- ILID outperforms baselines on 20/21 tasks in the main comparison, typically by 2-5x.
- ILID reaches 72.3 on antmaze-umaze vs 22.0 for DWBC and 9.9 for ISWBC.
- ILID runtime (~40 min) is comparable to BC (~30 min) for 250k steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use resultant-state-based selection and weighted behavior cloning (ILID) instead of the strongest reported baselines | Offline imitation learning · 21-task benchmark suite | tasks outperforming baselines: 20/21 | Helped |
| Use resultant-state-based behavior selection and weighted cloning | ILID · AntMaze medium | normalized score: 6.4±0.3 -> 64.6±5.2 | Helped |
| Use resultant-state-based behavior selection and weighted cloning | ILID · Robomimic lift | normalized score: 56.9±2.4 -> 90.4±2.4 | Helped |
| Remove the selected-data cloning term (importance-sampling weighting only) (ablation) | ILID · Offline imitation learning benchmarks | normalized score | Hurt |
| Remove importance-sampling weighting; use the BC loss for the first term (ablation) | ILID · Continuous-control benchmarks | normalized score | Hurt |
| Replace selected complementary data with all imperfect data (ablation) | ILID · Offline imitation learning benchmarks | normalized score | Hurt |
| Set beta(s,a) identically to 1 (ablation) | ILID · Offline imitation learning benchmarks | training effectiveness and stability | Hurt |
| Increase rollback step K (ablation) | ILID · All benchmarks | normalized score | Mixed |
Surprising findings
- Naive BC on union data (BCU) performs near zero on many tasks, worse than expert-only BC on several domains.
Limitations
- Requires state overlap between expert and imperfect data; fails when only the expert can reach expert states.
- Hyperparameters such as threshold sigma and rollback K were chosen per setup; the method assumes near-deterministic dynamics for theory.
Experimental setup
Datasets: D4RL robomimic
Benchmarks: D4RL MuJoCo D4RL AntMaze D4RL Adroit D4RL FrankaKitchen robomimic vision vision-based MuJoCo
Metrics: normalized score
Baselines: BCE BCU DWBC ISWBC CSIL MLIRL
Builds on: behavior cloning DWBC ISWBC D4RL robomimic
Closest papers by meaning
- Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.91)
- SEABO: A Simple Search-Based Method for Offline Imitation Learning (ICLR 2024 · similarity 0.90)
- Mitigating Covariate Shift in Behavioral Cloning via Robust Stationary Distribution Correction (NeurIPS 2024 · similarity 0.90)
- Robust Visual Imitation Learning with Inverse Dynamics Representations (AAAI 2024 · similarity 0.90)
- Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloning (NeurIPS 2025 · similarity 0.89)
- Imitation from Observations with Trajectory-Level Generative Embeddings (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.