Research map / Reinforcement learning / Policy gradient and actor-critic
Maximum a Posteriori Policy Optimisation
Abbas Abdolmaleki, Jost Tobias Springenberg, Yuval Tassa, Remi Munos, Nicolas Heess, Martin Riedmiller
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MPO instead of PPO | continuous control tasks in the Control Suite · Control Suite | sample efficiency | Improved |
| Use MPO instead of DDPG | continuous control tasks in the Control Suite · Control Suite | sample efficiency | Improved |
| Replace non-parametric variational distribution with a parametric one (ablation) | MPO · Acrobot, Walker-2D, Hopper | sample efficiency | Hurt |
| Replace the automatically tuned KL constraint with a manually set entropy regulariser (ablation) | MPO / EPG + Retrace · Acrobot, Walker-2D, Hopper | hyperparameter robustness | Hurt |
| Use a single action sample to estimate the action integral (ablation) | off-policy actor-critic with Retrace · Acrobot, Walker-2D, Hopper | performance | Hurt |
| Use MPO instead of PPO | Walker-2D parkour · Walker-2D parkour | trajectories to find a good policy: approximately 1M trajectories -> about 70k trajectories | Improved |
| Use MPO instead of PPO | Walker-2D parkour · Walker-2D parkour | samples to find a good policy: 60M samples | Improved |
| Use MPO instead of C51 | MPO · Breakout | score: 748 -> 360.5 | Hurt |
| Use MPO instead of C51 | MPO · Q*bert | score: 23,784 -> 10,317.0 | Hurt |
| Use MPO instead of C51 | MPO · Tennis | score: 23.1 -> 22.2 | Hurt |
| Use MPO instead of Prior. Dueling | MPO · Boxing | score: 98.9 -> 82.0 | Hurt |
Closest papers by meaning
- V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Control (ICLR 2020 · similarity 0.92)
- Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approach (NeurIPS 2025 · similarity 0.86)
- Oracle-Efficient Reinforcement Learning for Max Value Ensembles (NeurIPS 2024 · similarity 0.86)
- PPO in the Fisher-Rao geometry (NeurIPS 2026 · similarity 0.86)
- Absolute Policy Optimization: Enhancing Lower Probability Bound of Performance with High Confidence (ICML 2024 · similarity 0.86)
- DIME: Diffusion-Based Maximum Entropy Reinforcement Learning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
