Research map / Reinforcement learning / Imitation learning
Diffusion Model-Augmented Behavioral Cloning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Diffusion Model-Augmented Behavioral Cloning (DBC), which adds a diffusion-model loss on expert state-action pairs to the BC loss so the policy learns both conditional p(a|s) and joint p(s,a) signals without environment interaction.
Problem
Behavioral cloning models the conditional p(a|s) and generalizes poorly to unseen states, while joint-probability generative models generalize better but are slow at action retrieval and suffer from manifold overfitting, in environment-free imitation learning.
Approach
Train a DDPM on concatenated expert state-action pairs and use its noise-prediction loss (normalized by the expert diffusion loss, hinged at zero) on policy-predicted actions as an extra learning signal added to the BC loss, so the policy stays efficient at inference while benefiting from joint-distribution guidance.
Key results
- On Maze, DBC achieves 95.4% ± 1.7% success vs. 95.5% for Diffusion Policy and 92.1% for BC; on FetchPick DBC reaches 97.5% vs. 91.6% for BC.
- On AntReach DBC achieves 70.1% success vs. 61.8% for Diffusion Policy and 56.2% for BC.
- Inference speed on Maze: DBC 30.79 episodes/s vs. Diffusion Policy 1.38 episodes/s and Implicit BC 9.92 episodes/s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine BC loss with diffusion-model loss (ablation) | DBC · FetchPick | success rate: 91.6% ± 5.8% -> 97.5% ± 1.9% | Helped |
| combine BC loss with diffusion-model loss (ablation) | DBC · Maze | success rate: 79.6% ± 9.6% -> 95.4% ± 1.7% | Helped |
| use diffusion model instead of the second-best GAN guide (ablation) | DBC with BC · Maze | success rate: +2.4% | Helped |
| remove expert diffusion-loss normalization (ablation) | DBC · Maze | success rate: 95.4% ± 1.7% -> 94.7% ± 1.9% | Hurt |
| remove expert diffusion-loss normalization (ablation) | DBC · FetchPick | success rate: 96.9% ± 1.7% -> 96.6% ± 1.7% | Hurt |
| set diffusion-loss coefficient λ to 0 (BC only) (ablation) | DBC · FetchPick | success rate: 97.5% -> 91.7% | Hurt |
| set diffusion-loss coefficient λ to 10 (ablation) | DBC · FetchPick | success rate: 51.46% | Hurt |
| train the diffusion model with noise level 0.01 instead of 0 (ablation) | DM-guided policy · spiral maze | success rate: 12% -> 32% | Helped |
| use DM guidance rather than BC alone (ablation) | policy learning on spiral maze · spiral maze | success rate: 85% vs 32% | Hurt |
| use diffusion-model data augmentation for BC (ablation) | BC on Maze · Maze | performance: 2.06% | No clear effect |
| replace conditional policy with joint-probability DM-only policy (ablation) | policy learning on spiral maze · spiral maze | success rate | Hurt |
| use DBC rather than BC | imitation policy · CarRacing | return: 766.4 -> 791.5 | Helped |
Surprising findings
- Diffusion model loss alone (pi_DM) performs poorly on FetchPick (14.2% success), showing that joint-probability guidance alone is insufficient and BC is needed.
- Diffusion Policy underperforms BC on single-mode locomotion tasks (Cheetah, Walker), which the authors attribute to its multimodal design.
Limitations
- Cannot learn from agent (non-expert) trajectories, only expert demonstrations without environment interaction.
- Optimal lambda varies per task and must be tuned so BC and DM losses have similar magnitude.
- Gains on some tasks (Maze, HandRotate, AntReach normalization ablation) are within standard deviation of Diffusion Policy.
Experimental setup
Datasets: maze2d-medium-v2 expert demonstrations FetchPick demonstrations HandRotate expert demonstrations (SAC) Cheetah and Walker demonstrations AntReach demonstrations CarRacing PPO expert demonstrations
Benchmarks: maze2d-medium-v2 (Maze) FetchPick HandRotate Cheetah Walker AntReach CarRacing
Metrics: success rate return episodes per second
Baselines: BC Implicit BC Diffusion Policy
Builds on: behavioral cloning DDPM Implicit BC Diffusion Policy
Closest papers by meaning
- Diffusion Imitation from Observation (NeurIPS 2024 · similarity 0.90)
- Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.90)
- Imitation Learning from Purified Demonstrations (ICML 2024 · similarity 0.90)
- Latent Diffusion Planning for Imitation Learning (ICML 2025 · similarity 0.90)
- Improving Generative Behavior Cloning via Self-Guidance and Adaptive Chunking (NeurIPS 2025 · similarity 0.89)
- A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.