Research map / Reinforcement learning / Policy gradient and actor-critic
Policy Optimization with Model-Based Explorations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the target-discrepancy exploration bonus to PPO advantages, with α decayed from 0.1 to 0 | PPO · Atari 2600 (49 games) | games outperforming PPO: 32 games | Helped |
| Add the target-discrepancy exploration bonus to PPO advantages and keep α at 0.1 (ablation) | PPO · Atari 2600 (49 games) | games outperforming PPO: 33 games | Helped |
| Replace the model-free target with the learned model-based target (ablation) | PPO · Atari 2600 games | games outperforming PPO: Only in one game | Hurt |
| Keep the exploration coefficient at 0.1 rather than annealing it (ablation) | POME · Atari 2600 (49 games) | games outperforming PPO: 33 games out of 49 | Helped |
| Add the target-discrepancy bonus with decaying α | PPO · Amidar | average score of last 100 episodes: 674.6 -> 943.9 | Helped |
| Add the target-discrepancy bonus with decaying α | PPO · Asteroids | average score of last 100 episodes: 2097.5 -> 1737.6 | Hurt |
| Add the target-discrepancy bonus with decaying α | PPO · RoadRunner | average score of last 100 episodes: 25076.0 -> 44520.0 | Helped |
| Keep α at 0.1 rather than decaying it (ablation) | POME · Atari 2600 games | average score | Mixed |
| Use a learned model-based target in place of the model-free target (ablation) | PPO · Amidar | average score of last 100 episodes: 674.6 -> 27.7 | Hurt |
| Use a learned model-based target in place of the model-free target (ablation) | PPO · Assault | average score of last 100 episodes: 4971.9 -> 872.2 | Hurt |
Closest papers by meaning
- Model Based Reinforcement Learning for Atari (ICLR 2020 · similarity 0.90)
- Model-Free Active Exploration in Reinforcement Learning (NeurIPS 2023 · similarity 0.88)
- When to Trust Your Model: Model-Based Policy Optimization (NeurIPS 2019 · similarity 0.88)
- Multi-step Greedy Reinforcement Learning Algorithms (ICML 2020 · similarity 0.88)
- Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol (NeurIPS 2025 · similarity 0.86)
- Absolute Policy Optimization: Enhancing Lower Probability Bound of Performance with High Confidence (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.