Research map / Image generation / Fast diffusion sampling
Imitation Learning from Purified Demonstrations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DP-IL, which purifies sub-optimal demonstrations with a forward-then-reverse DDPM diffusion process trained on a few optimal demos, then runs BC or GAIL on the purified data, with theoretical bounds on purification distance.
Problem
Imitation learning from imperfect demonstrations where only a small set of optimal demonstrations is available alongside many sub-optimal ones, which degrade standard IL methods.
Approach
Instead of integrating confidence weighting into IL training (bi-level optimization), purify the sub-optimal demonstrations first via a two-step diffusion process (forward noising to smooth perturbations, reverse denoising using a diffusion model trained on optimal demos), then apply any IL method (BC or GAIL) to the purified data.
Key results
- On MuJoCo offline BC, DP-BC achieves the best score in most of the 24 settings, e.g., Ant D1-L1 261±54 vs BC-all -22±33 and BCND 28±28.
- On Robosuite NutAssembly reaching with human demos, DP-BC reaches 0.86 success rate vs 0.82 for DWBC, 0.76 for BC, and 0.56 for BCND.
- With mixed demonstrations, DP-BC reaches e.g. 3146 on HalfCheetah vs 1255 for BC and 921 for BCND.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| purify demonstrations with diffusion before behavior cloning | BC on MuJoCo · Ant-v2 D1-L1 | average cumulative reward: 28±28 -> 261±54 | Helped |
| purify demonstrations with diffusion before behavior cloning | BC on MuJoCo · HalfCheetah-v2 D1-L3 | average cumulative reward: 834±249 -> 4042±80 | Helped |
| purify demonstrations with diffusion before behavior cloning | BC on MuJoCo · Walker2d-v2 D2-L3 | average cumulative reward: 2637±134 -> 3076±205 | Helped |
| purify demonstrations with diffusion before behavior cloning | BC on MuJoCo · Hopper-v2 D1-L3 | average cumulative reward: 3207±7 -> 1825±137 | Hurt |
| purify human demonstrations with diffusion before behavior cloning | BC on RoboSuite Nut Assembly reaching · RoboSuite Nut Assembly reaching | success rate: 0.82 -> 0.86 | Helped |
| replace diffusion purification with Gaussian filter (ablation) | DP-BC · Ant-v2 D1-L1 | average cumulative reward: 242 vs 261 | Hurt |
| replace diffusion purification with mean filter (ablation) | DP-BC · Ant-v2 D1-L1 | average cumulative reward: 182 vs 261 | Hurt |
| replace diffusion purification with median filter (ablation) | DP-BC · Ant-v2 D1-L2 | average cumulative reward: 437 vs 803 | Hurt |
| use uniform or salt-and-pepper perturbations instead of Gaussian perturbations (ablation) | DP-BC · MuJoCo Ant and HalfCheetah | average cumulative reward | Mixed |
| set reverse point i_r to 3 (ablation) | DP-BC on RoboSuite Nut Assembly reaching · RoboSuite Nut Assembly reaching | success rate: 0.78 -> 0.86 | Helped |
| increase reverse point i_r from 3 to 10 (ablation) | DP-BC on RoboSuite Nut Assembly reaching · RoboSuite Nut Assembly reaching | success rate: 0.86 -> 0.60 | Hurt |
| use mixed L1, L2, and L3 demonstrations (ablation) | DP-BC · MuJoCo Ant | average cumulative reward: 193 -> 601 | Helped |
Surprising findings
- Using only the small optimal set (BC-opt) sometimes performs worse than BC trained on all demonstrations including sub-optimal ones.
- Filter-based denoisers (mean, median, Gaussian) are outperformed by diffusion purification, with the mean filter the strongest among them.
Limitations
- Robosuite evaluation uses a single simple reaching task with only 10 human demonstrations, limiting generality.
- Optimal reverse step i_r must be tuned per demonstration quality, with performance sensitive to it (e.g., i_r=10 gives 0.60 success vs 0.86 at i_r=3).
Experimental setup
Datasets: MuJoCo Ant-v2 MuJoCo HalfCheetah-v2 MuJoCo Walker2d-v2 MuJoCo Hopper-v2 RoboTurk human demonstrations (Robosuite Sawyer NutAssembly reaching)
Benchmarks: MuJoCo Ant-v2 MuJoCo HalfCheetah-v2 MuJoCo Walker2d-v2 MuJoCo Hopper-v2 Robosuite Sawyer NutAssembly reaching
Metrics: average cumulative ground-truth reward success rate
Baselines: BC BCND DWBC DemoDICE 2IWIL IC-GAIL WGAIL mean filter median filter Gaussian filter
Builds on: DDPM DiffPure GAIL BCND DWBC
Closest papers by meaning
- Diffusion Model-Augmented Behavioral Cloning (ICML 2024 · similarity 0.90)
- Causal Imitation for Markov Decision Processes: a Partial Identification Approach (NeurIPS 2024 · similarity 0.88)
- Limited Preference Aided Imitation Learning from Imperfect Demonstrations (ICML 2024 · similarity 0.88)
- Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrations (NeurIPS 2025 · similarity 0.88)
- Diffusion Imitation from Observation (NeurIPS 2024 · similarity 0.88)
- Imitation from Observations with Trajectory-Level Generative Embeddings (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.