Research map / Reinforcement learning / Policy gradient and actor-critic
On Effective Scheduling of Model-based Reinforcement Learning
Hang Lai, Jian Shen, Weinan Zhang, Yimin Huang, Xing Zhang, Ruiming Tang, Yong Yu, Zhenguo Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AutoMBPO to jointly schedule the real-data ratio, model-training frequency, policy-training iteration, and rollout length | MBPO · Hopper, Ant, Humanoid, HopperBullet, Walker2dBullet, HalfcheetahBullet | average return | Improved |
| Use AutoMBPO scheduling instead of PBT or RoR | MBPO · Continuous control environments | average return | Improved |
| Increase the real-data ratio over training | MBPO · Six continuous control environments | average return | Improved |
| Schedule only the real-data ratio while keeping other hyperparameters at the original MBPO settings (ablation) | MBPO · Almost all evaluated tasks | average return | Improved |
| Schedule only policy-training iterations (ablation) | MBPO · Hopper | average return | Improved |
| Schedule only policy-training iterations (ablation) | MBPO · Ant | average return | Hurt |
| Schedule only model-training frequency (ablation) | MBPO · Ant | average return | Improved |
| Schedule only model-training frequency (ablation) | MBPO · Hopper | average return | Hurt |
| Use SAC with policy training iteration matched to MBPO (SAC(20)) instead of SAC(1) (ablation) | SAC · Ant and Walker2dBullet | average return | Hurt |
| Increase the initial real-data ratio or policy-training iteration for AutoMBPO (ablation) | AutoMBPO · Three PyBullet environments | average return | No clear effect |
| Use a larger initial policy-training iteration and wider iteration limit (ablation) | AutoMBPO · Three PyBullet environments | average return | No clear effect |
| Transfer a hyper-controller trained on another PyBullet task without fine-tuning (ablation) | AutoMBPO · PyBullet environments | average return | Improved |
Closest papers by meaning
- When to Trust Your Model: Model-Based Policy Optimization (NeurIPS 2019 · similarity 0.88)
- Ready Policy One: World Building Through Active Learning (ICML 2020 · similarity 0.88)
- On Rollouts in Model-Based Reinforcement Learning (ICLR 2025 · similarity 0.87)
- Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol (NeurIPS 2025 · similarity 0.87)
- Optimistic Model Rollouts for Pessimistic Offline Policy Optimization (AAAI 2024 · similarity 0.86)
- Parallelizing Model-based Reinforcement Learning Over the Sequence Length (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
