Research map / Reinforcement learning / Visual and representation RL
Unifying Count-Based Exploration and Intrinsic Motivation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a pseudo-count exploration bonus to Double DQN | Double DQN · Montezuma's Revenge | rooms visited after 50 million frames: two -> 15 | Helped |
| Add a pseudo-count exploration bonus to Double DQN | Double DQN · Montezuma's Revenge | average score at 50 million frames: 2461 | Helped |
| Add a pseudo-count exploration bonus to Double DQN | Double DQN · Montezuma's Revenge | average score at 100 million frames: 3439 | Helped |
| Add a pseudo-count exploration bonus to A3C | A3C · Atari 2600, 60 games | games failing to score 50% better than random: 15 -> 10 | Helped |
| Add a pseudo-count exploration bonus to A3C | A3C · Atari 2600, 60 games | median performance | Helped |
| Remove Monte Carlo return mixing from the Double DQN target (ablation) | Pseudo-count DQN · Montezuma's Revenge | performance | Hurt |
| Use a life-loss signal instead of treating play as continuous episodes | A3C+ · Montezuma's Revenge, stochastic setting | average score after 200 million frames: 1127.05 -> 142.50 | Hurt |
| Use a life-loss signal instead of treating play as continuous episodes | A3C+ · Montezuma's Revenge, deterministic setting | average score after 200 million frames: 273.70 -> 273.70 | No clear effect |
| Use optimistic initialization instead of pseudo-count bonuses | DQN · Atari 2600 hard games | training score | Mixed |
| Add a pseudo-count exploration bonus | Double DQN · Atari 2600 hard games, especially Montezuma's Revenge and Venture | training score | Helped |
Closest papers by meaning
- Count-Based Exploration with Neural Density Models (ICML 2017 · similarity 0.90)
- Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learning (ICML 2023 · similarity 0.90)
- Rethinking Exploration in Reinforcement Learning with Effective Metric-Based Exploration Bonus (NeurIPS 2024 · similarity 0.87)
- Beyond Optimism: Exploration With Partially Observable Rewards (NeurIPS 2024 · similarity 0.87)
- Improving Intrinsic Exploration by Creating Stationary Objectives (ICLR 2024 · similarity 0.86)
- Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Exploration (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.