Research map / Reinforcement learning / Policy gradient and actor-critic
Trust-Region Diffusion Policies for Massively Parallel On-Policy RL
Huy Le, Onur Celik, Denis Blessing, Tai Hoang, Claas A Voelcker, Axel Brunnbauer, Felix Richter, Michael Volpp, Gerhard Neumann
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a single diffusion step (T=1) instead of 8 (ablation) | TruDi · MuJoCo Playground Humanoid | aggregated return | Hurt |
| increase diffusion steps from 8 to 32 (ablation) | TruDi · MuJoCo Playground Humanoid | aggregated return | No clear effect |
| set trust-region epsilon too large (50) versus 0.1 to 0.4 (ablation) | TruDi · IsaacLab | normalized return | Hurt |
| use a very small trust-region epsilon (0.01) (ablation) | TruDi · IsaacLab | normalized return | Hurt |
| evaluate with probability-flow ODE (c=1.0) instead of SDE sampling (ablation) | TruDi · MuJoCo Playground Humanoid | aggregated return | Improved |
| evaluate with ODE instead of Best-of-K (K=10, 20) selection (ablation) | TruDi · MuJoCo Playground Humanoid | aggregated return | Improved |
| extend REPPO training from 50M to 150M steps | REPPO · MuJoCo Playground Humanoid | episode return | No clear effect |
| use diffusion policy TruDi instead of Gaussian REPPO | REPPO · StackCube (multimodality) | behavior entropy: 0 -> 0.87 | Improved |
| use diffusion policy TruDi instead of Gaussian REPPO | REPPO · MuJoCo Playground Humanoid (50M steps) | episode return (IQM): 29.6 -> 34.8 | Improved |
| use TruDi instead of DIME | DIME · MuJoCo Playground Humanoid (50M steps) | episode return (IQM): 15.6 -> 34.8 | Improved |
| use TruDi instead of DIME on PushT | DIME · PushT (multimodality) | episode return: 171.92 -> 167.32 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- DIME — DIME: Diffusion-Based Maximum Entropy Reinforcement Learning (ICML 2025) · better on 4 of 6 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Diffusion Policies Creating a Trust Region for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.90)
- Diffusion Policy Policy Optimization (ICLR 2025 · similarity 0.89)
- GenPO: Generative Diffusion Models Meet On-Policy Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- Efficient Online Reinforcement Learning for Diffusion Policy (ICML 2025 · similarity 0.88)
- Score Regularized Policy Optimization through Diffusion Behavior (ICLR 2024 · similarity 0.88)
- Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradient (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
