Research map / Reinforcement learning / Imitation learning
Reward learning from human preferences and demonstrations in Atari
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine expert demonstrations with trajectory preferences | Atari reward learning with DQfD · 9 Atari games | game score versus imitation learning | Helped |
| Use demonstrations to guide preference-based learning | Atari reward learning · Hero, Montezuma's Revenge, Private Eye, and Q*bert | game score | Helped |
| Use demonstrations alongside preferences | Atari reward learning · Enduro | game score | Hurt |
| Automatically label demonstration clips as preferable to initial-policy clips (ablation) | Demos + preferences · Private Eye, Hero, and Montezuma's Revenge | game score | Helped |
| Automatically add preference labels from demonstrations (ablation) | Demos + preferences · Most Atari games | game score | No clear effect |
| Automatically add preference labels from demonstrations (ablation) | Demos + preferences · Breakout | game score | Hurt |
| Reduce the demonstration set to 20% (ablation) | Demos + preferences · Atari games | game score | Helped |
| Replace synthetic preferences with human preferences (ablation) | Demos + preferences · Beamrider | game score | Helped |
| Replace synthetic preferences with human preferences (ablation) | Demos + preferences · Breakout, Montezuma's Revenge, and Pong | game score | Hurt |
| Freeze the learned reward model during policy training (ablation) | Reward-learning agent · Hero, Montezuma's Revenge, and Private Eye | true game score | Hurt |
| Use human reward shaping that penalizes death (ablation) | Montezuma's Revenge reward model · Montezuma's Revenge | game score | Hurt |
| Phase out the large-margin demonstration loss during training (ablation) | DQfD with learned rewards · Enduro | game score | Helped |
Closest papers by meaning
- Deep Reinforcement Learning from Human Preferences (NeurIPS 2017 · similarity 0.90)
- Agent57: Outperforming the Atari Human Benchmark (ICML 2020 · similarity 0.89)
- Hybrid Reward Architecture for Reinforcement Learning (NeurIPS 2017 · similarity 0.88)
- Improving Reward Model Generalization from Adversarial Process Enhanced Preferences (ICML 2025 · similarity 0.87)
- A Minimaximalist Approach to Reinforcement Learning from Human Feedback (ICML 2024 · similarity 0.87)
- Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learning (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.