Research map / Reinforcement learning / Policy gradient and actor-critic
Bidirectional Model-based Policy Optimization
Hang Lai, Jian Shen, Weinan Zhang, Yong Yu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use bidirectional model rollouts instead of forward-only rollouts | BMPO · Pendulum and Hopper-NT | multi-step prediction error | Improved |
| Use BMPO rather than forward-model-only methods | continuous control tasks · Six OpenAI Gym environments | sample efficiency | Improved |
| Use BMPO rather than forward-model-only methods | continuous control tasks · Six OpenAI Gym environments | asymptotic performance | Improved |
| Replace backward-policy MLE loss with GAN loss (ablation) | BMPO · Hopper-NT | average return | Hurt |
| Remove the backward model and use forward-only trajectory sampling (ablation) | BMPO · Hopper-NT | average return | Hurt |
| Remove the forward model and use backward-only trajectory sampling (ablation) | BMPO · Hopper-NT | average return | Hurt |
| Remove MPC during environment interaction (ablation) | BMPO · Hopper-NT | average return | Hurt |
| Increase the high-value-state sampling parameter beyond a moderate level (ablation) | BMPO · Hopper-NT | average return | Hurt |
| Set backward rollout length equal to forward rollout length (ablation) | BMPO · Hopper-NT | average return | Improved |
| Use a backward rollout length that is too short or too long (ablation) | BMPO · Hopper-NT | average return | Hurt |
| Add MPC to BMPO | BMPO · all six environments | computation time (hours) | Hurt |
Closest papers by meaning
- When to Trust Your Model: Model-Based Policy Optimization (NeurIPS 2019 · similarity 0.90)
- Optimistic Model Rollouts for Pessimistic Offline Policy Optimization (AAAI 2024 · similarity 0.88)
- Any-step Dynamics Model Improves Future Predictions for Online and Offline Reinforcement Learning (ICLR 2025 · similarity 0.88)
- The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
- Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol (NeurIPS 2025 · similarity 0.87)
- Distributional Successor Features Enable Zero-Shot Policy Optimization (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
