Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Playing hard exploration games by watching YouTube
Yusuf Aytar, Tobias Pfaff, David Budden, Thomas Paine, Ziyu Wang, Nando de Freitas
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Guide IMPALA with TDC+CMC checkpoints from a YouTube video, without environment reward | IMPALA · Montezuma’s Revenge | score: 37,232.7 | Improved |
| Guide IMPALA with TDC+CMC checkpoints from a YouTube video, without environment reward | IMPALA · Pitfall! | score: 54,912.4 | Improved |
| Guide IMPALA with TDC+CMC checkpoints from a YouTube video, without environment reward | IMPALA · Private Eye | score: 98,212.5 | Improved |
| Add environment reward alongside the TDC+CMC imitation reward (ablation) | YouTube imitation IMPALA · Montezuma’s Revenge | score: 37,232.7 -> 58,175.1 | Improved |
| Add environment reward alongside the TDC+CMC imitation reward (ablation) | YouTube imitation IMPALA · Pitfall! | score: 54,912.4 -> 76,812.5 | Improved |
| Add environment reward alongside the TDC+CMC imitation reward (ablation) | YouTube imitation IMPALA · Private Eye | score: 98,212.5 -> 98,763.2 | Improved |
| Replace pixel-space features with combined TDC+CMC embeddings for imitation reward (ablation) | IMPALA imitation agent · Montezuma’s Revenge, Pitfall!, Private Eye | game score | Improved |
| Use only pixel-space features for imitation reward instead of TDC+CMC embeddings (ablation) | IMPALA imitation agent · Private Eye | score | Improved |
| Use only pixel-space features for imitation reward instead of TDC+CMC embeddings (ablation) | IMPALA imitation agent · Montezuma’s Revenge, Pitfall! | game progress | Hurt |
| Train the policy using frozen pretrained TDC+CMC features instead of the full image-based policy network (ablation) | IMPALA · Montezuma’s Revenge | score | Hurt |
| Condition the feature-based policy on the next checkpoint embedding (ablation) | IMPALA trained from pretrained features · Montezuma’s Revenge | learning speed | Improved |
| Change the YouTube expert trajectory among the three tested Montezuma videos (ablation) | YouTube imitation IMPALA · Montezuma’s Revenge | score | Mixed |
Closest papers by meaning
- Guided Exploration with Proximal Policy Optimization using a Single Demonstration (ICML 2021 · similarity 0.87)
- Never Give Up: Learning Directed Exploration Strategies (ICLR 2020 · similarity 0.87)
- Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration (ICLR 2025 · similarity 0.86)
- Grounding Video Models to Actions through Goal Conditioned Exploration (ICLR 2025 · similarity 0.85)
- Deep Demonstration Tracing: Learning Generalizable Imitator Policy for Runtime Imitation from a Single Demonstration (ICML 2024 · similarity 0.85)
- GROOT: Learning to Follow Instructions by Watching Gameplay Videos (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
