Research map / Reinforcement learning / Offline RL
Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a conditional diffusion policy with Q-learning guidance | Offline RL · Gym average | normalized score: 77.6 -> 88.0 | Helped |
| Use a conditional diffusion policy with Q-learning guidance | Offline RL · AntMaze average | normalized score: 63.0 -> 69.6 | Helped |
| Use a conditional diffusion policy with Q-learning guidance | Offline RL · Adroit average | normalized score: 56.5 -> 65.1 | Helped |
| Use a conditional diffusion policy with Q-learning guidance | Offline RL · Kitchen average | normalized score: 53.3 -> 69.0 | Helped |
| Replace CVAE behavior cloning with diffusion behavior cloning (ablation) | Behavior cloning · Gym average | normalized score: 79.9 -> 85.9 | Helped |
| Replace CVAE with diffusion within the BCQ framework (ablation) | BCQ policy improvement · Gym average | normalized score: 105.1 -> 106.0 | Helped |
| Replace free Q-learning guidance using CVAE with diffusion policy learning (ablation) | Q-learning-guided policy · Gym average | normalized score: 76.6 -> 106.9 | Helped |
| Use Diffusion-QL rather than BCQ-Diffusion's hard value constraints (ablation) | Diffusion policy · Gym average | normalized score: 106.0 -> 106.9 | Helped |
| Use online rather than offline checkpoint selection (ablation) | Diffusion-QL · AntMaze average | normalized score: 69.6 -> 75.4 | Helped |
| Use online rather than offline checkpoint selection (ablation) | Diffusion-QL · Gym average | normalized score: 88.0 -> 89.3 | Helped |
Closest papers by meaning
- Diffusion Policies Creating a Trust Region for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.94)
- Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning (ICLR 2025 · similarity 0.93)
- Learning a Diffusion Model Policy from Rewards via Q-Score Matching (ICML 2024 · similarity 0.92)
- Score Regularized Policy Optimization through Diffusion Behavior (ICLR 2024 · similarity 0.92)
- Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.92)
- Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimization (NeurIPS 2024 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.