Research map / Reinforcement learning / Visual and representation RL
Guided Exploration with Proximal Policy Optimization using a Single Demonstration
Gabriele Libardi, Gianni De Fabritiis, Sebastian Dittert
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PPO+D with one demonstration instead of the tested imitation-learning and PPO baselines | Animal-AI Olympics hard-exploration tasks · One box easy, One box hard, Two boxes easy, and Two boxes hard | reward | Improved |
| Add value-prioritized replay of unsuccessful trajectories (ablation) | PPO+D · Two boxes easy | reward | Improved |
| Remove value-buffer replay (ablation) | PPO+D · One box easy | learning speed | Mixed |
| Remove value-buffer replay (ablation) | PPO+D · One box easy | training reliability | Hurt |
| Use PPO+D instead of vanilla PPO | Sparse ReacherPyBulletEnv-v0 · Sparse ReacherPyBulletEnv-v0 | reward | Improved |
| Use PPO+D instead of PPO+BC, with or without a successful-trajectory buffer | Sparse ReacherPyBulletEnv-v0 · Sparse ReacherPyBulletEnv-v0 | reward | Improved |
| Use PPO+D instead of PPO+BC | Sparse LunarLander-v2 · Sparse LunarLander-v2 | reward | Improved |
| Use PPO+D instead of vanilla PPO | Sparse LunarLander-v2 · Sparse LunarLander-v2 | reward | Mixed |
| Apply PPO+D to a dense-reward task instead of vanilla PPO | PPO+D · BreakoutNoFrameskip-v4 | learning speed | Hurt |
| Set the demonstration replay probability to rho=0.3 rather than the other tested values (ablation) | PPO+D with phi=0 · One box easy, fixed agent position | performance | Improved |
| Set demonstration replay probability too low (ablation) | PPO+D · One box easy, fixed agent position | performance | Hurt |
| Set demonstration replay probability too high (ablation) | PPO+D · One box easy, fixed agent position | performance | Hurt |
Closest papers by meaning
- DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- Never Give Up: Learning Directed Exploration Strategies (ICLR 2020 · similarity 0.88)
- Competitive experience replay (ICLR 2019 · similarity 0.88)
- Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying (ICML 2026 · similarity 0.88)
- Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration (ICLR 2025 · similarity 0.87)
- Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Exploration (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
