Research map / Reinforcement learning / Policy gradient and actor-critic
Flow Matching Policy Gradients
David McAllister, Songwei Ge, Brent Yi, Chung Min Kim, Ethan Weber, Hongsuk Choi, Haiwen Feng, Angjoo Kanazawa
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Gaussian PPO with FPO flow policy | MuJoCo Playground policies · MuJoCo Playground (10 tasks) | average evaluation reward: 667.8 -> 759.3 | Improved |
| use FPO ratio instead of denoising-MDP DPPO for diffusion policy | diffusion policy · MuJoCo Playground (10 tasks) | average evaluation reward: 652.5 -> 759.3 | Improved |
| increase number of (tau, eps) samples from 1 to 8 (ablation) | FPO · MuJoCo Playground (10 tasks) | average evaluation reward: 691.6 -> 759.3 | Improved |
| use 4 instead of 8 (tau, eps) pairs (ablation) | FPO · MuJoCo Playground (10 tasks) | average evaluation reward: 759.3 -> 731.2 | Hurt |
| use epsilon-MSE instead of velocity (u-MSE) flow matching loss (ablation) | FPO · MuJoCo Playground (10 tasks) | average evaluation reward: 664.6 -> 759.3 | Improved |
| raise clipping epsilon from 0.05 to 0.2 (ablation) | FPO · MuJoCo Playground (10 tasks) | average evaluation reward: 759.3 -> 526.4 | Hurt |
| raise clipping epsilon from 0.05 to 0.1 (ablation) | FPO · MuJoCo Playground (10 tasks) | average evaluation reward: 759.3 -> 623.3 | Hurt |
| tune Gaussian PPO hyperparameters (lr 3e-4, clip 0.1) versus Playground default (ablation) | Gaussian PPO · MuJoCo Playground (10 tasks) | average evaluation reward: 577.2 -> 667.8 | Improved |
| use FPO instead of Gaussian PPO with root+hands goal conditioning | humanoid tracking policy · Humanoid tracking, root + hands goals | success rate: 46.5% -> 70.6% | Improved |
| use FPO instead of Gaussian PPO with root-only goal conditioning | humanoid tracking policy · Humanoid tracking, root goals | success rate: 29.8% -> 54.3% | Improved |
| use FPO instead of Gaussian PPO with full all-joint conditioning | humanoid tracking policy · Humanoid tracking, all-joint goals | success rate: 98.7% -> 96.4% | Hurt |
| use FPO instead of Gaussian PPO with all-joint conditioning | humanoid tracking policy · Humanoid tracking, all-joint goals | MPJPE: 31.62 -> 41.98 | Hurt |
Closest papers by meaning
- Flow-GRPO: Training Flow Matching Models via Online RL (NeurIPS 2025 · similarity 0.91)
- Flow-DPPO: Divergence Proximal Policy Optimization for Flow Matching Models (NeurIPS 2026 · similarity 0.91)
- Flow Matching for Offline Reinforcement Learning with Discrete Actions (NeurIPS 2026 · similarity 0.90)
- GenPO: Generative Diffusion Models Meet On-Policy Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Aligning Flow Map Policies with Optimal $Q$-Guidance (NeurIPS 2026 · similarity 0.89)
- Flow-Based Policy for Online Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
