Research map / Reinforcement learning / Visual and representation RL
Exploring Model-based Planning with Policy Networks
Tingwu Wang, Jimmy Ba
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| plan by perturbing policy parameters instead of action sequences | POPLIN · Ant | return: 1165.5 ± 226.9 -> 2330.1 ± 320.9 | Improved |
| plan by perturbing policy parameters instead of action sequences | POPLIN · Cheetah | return: 4204.5 ± 789.0 -> 12227.9 ± 5652.8 | Improved |
| use POPLIN-A action-space planning rather than PETS | model-based planning · Cheetah | return: 4204.5 ± 789.0 -> 4651.1 ± 1088.5 | No clear effect |
| use POPLIN-P parameter-space planning rather than POPLIN-A | POPLIN · Hopper | return: 202.5 ± 962.5 -> 2055.2 ± 613.8 | Improved |
| use POPLIN-P parameter-space planning rather than POPLIN-A | POPLIN · Pusher | return: -56.0 ± 24.3 -> -55.8 ± 23.1 | No clear effect |
| use POPLIN-P parameter-space planning rather than POPLIN-A | POPLIN · Pendulum | return: 178.3 ± 19.3 -> 167.9 ± 45.9 | Hurt |
| replan the policy throughout each candidate trajectory instead of using the initialized reference sequence (ablation) | POPLIN-A · MuJoCo benchmark environments | return | Hurt |
| train POPLIN-A with imaginary planned data instead of only real data (ablation) | POPLIN-A · MuJoCo benchmark environments | return | No clear effect |
| use separate parameter noise at each time-step instead of shared noise (ablation) | POPLIN-P · MuJoCo benchmark environments | return | Improved |
| distill the planned behavior with behavior cloning rather than parameter averaging (ablation) | POPLIN-P policy control · Cheetah | return | Improved |
| distill planned behavior with GAN rather than behavior cloning (ablation) | POPLIN-P policy control · Cheetah and Cheetah-v0 | return | Mixed |
| increase initial parameter-noise variance (ablation) | POPLIN-P-Avg · MuJoCo benchmark environments | return | Improved |
Closest papers by meaning
- Evaluating Model-Based Planning and Planner Amortization for Continuous Control (ICLR 2022 · similarity 0.89)
- Bootstrapped Model Predictive Control (ICLR 2025 · similarity 0.88)
- PWM: Policy Learning with Multi-Task World Models (ICLR 2025 · similarity 0.87)
- Improving planning and MBRL with temporally-extended actions (NeurIPS 2025 · similarity 0.87)
- Adaptive Horizon Actor-Critic for Policy Learning in Contact-Rich Differentiable Simulation (ICML 2024 · similarity 0.86)
- Efficient Multi-agent Reinforcement Learning by Planning (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
