Research map / Reinforcement learning / Policy gradient and actor-critic
Mean Flow Policy Optimization
Xiaoyi Dong, Xi Sheryl Zhang, Jian Cheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MeanFlow policy (average velocity) instead of standard flow-matching policy (ablation) | MFPO · HalfCheetah-v3 | episode return | Improved |
| use MeanFlow model for the divergence integral instead of instantaneous divergence network (ablation) | MFPO · HalfCheetah-v3 | episode return | Improved |
| add average divergence network for action likelihood instead of zeroing the divergence integral (ablation) | MFPO · HalfCheetah-v3 | episode return | Improved |
| combine policy proposal and Gaussian proposal SNIS estimators instead of Gaussian only (ablation) | MFPO · HalfCheetah-v3 | episode return | Improved |
| use only the policy proposal for velocity estimation (ablation) | MFPO · HalfCheetah-v3 | episode return | Hurt |
| sample ratio K1:K2 = 1:2 favoring Gaussian proposal (ablation) | MFPO · HalfCheetah-v3 | episode return | Improved |
| combine two SNIS estimators with ESS weights (ablation) | velocity estimator · HalfCheetah-v3 after 120k iterations | estimation variance | Improved |
| auto-tune temperature with target entropy rho=0.5 instead of fixed temperature (ablation) | MFPO · HalfCheetah-v3 | episode return | Improved |
| vary target entropy coefficient rho (0.4, 0.5, 0.6) (ablation) | MFPO · HalfCheetah-v3 | episode return | Mixed |
| add average divergence network to likelihood estimation | MFPO · training time | training time: 5% increase in training time | Hurt |
| replace multi-step diffusion policies with 2-step MeanFlow policy | diffusion-based online RL · MuJoCo benchmarks | training time: approximately 50% reduction | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 2 on most shared results.
- FlowRL — FlowRL: Matching Reward Distributions for LLM Reasoning (ICLR 2026) · better on 1 of 2 shared results
- DIME — DIME: Diffusion-Based Maximum Entropy Reinforcement Learning (ICML 2025) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Flow-Based Policy for Online Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
- Generative Actor-Critic with Soft Bridge Policies (NeurIPS 2026 · similarity 0.90)
- Random Policy Evaluation Uncovers Policies of Generative Flow Networks (ICML 2025 · similarity 0.89)
- Scalable Maximum Entropy Reinforcement Learning for Diffusion Policies via Adjoint Matching (NeurIPS 2026 · similarity 0.89)
- Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints (NeurIPS 2026 · similarity 0.88)
- Aligning Flow Map Policies with Optimal $Q$-Guidance (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
