Research map / Learning theory and algorithms / Combinatorial and search optimization
Count-Based Exploration with Neural Density Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PixelCNN pseudo-count exploration bonus | DQN · Atari 57 games | maximum achieved score | Helped |
| add PixelCNN pseudo-count exploration bonus | DQN · Gravitar | maximum score: 473.0 -> 859.1 | Helped |
| add PixelCNN pseudo-count exploration bonus | DQN · Private Eye | maximum score: 146.7 -> 15806.5 | Helped |
| add PixelCNN pseudo-count exploration bonus | DQN · Montezuma's Revenge | maximum score: 0.0 -> 2514.3 | Helped |
| add PixelCNN pseudo-count exploration bonus | Reactor · Atari 57 games | area under the curve | Helped |
| add mixed Monte Carlo update (ablation) | DQN-PixelCNN · hard exploration, sparse-reward Atari games | training progress | Helped |
| add mixed Monte Carlo update (ablation) | DQN · Bank Heist and Time Pilot | training performance | Helped |
| add mixed Monte Carlo update (ablation) | DQN · Ms. Pac-Man and Breakout | training performance | Hurt |
| add both PixelCNN bonus and mixed Monte Carlo update (ablation) | DQN · H.E.R.O. | training performance | Helped |
| replace PixelCNN with convolutional CTS (ablation) | DQN exploration agent · Gravitar, Private Eye, and Venture | training performance | Hurt |
| use convolutional CTS rather than regular CTS (ablation) | DQN exploration agent · Q*Bert and Zaxxon | training performance | Helped |
| increase prediction-gain scale from c=0.1 to c=1 or c=10 (ablation) | DQN-PixelCNN · hard exploration Atari games | peak score: 900; 6,600; 39,000; 1,500 | Mixed |
Closest papers by meaning
- Unifying Count-Based Exploration and Intrinsic Motivation (NeurIPS 2016 · similarity 0.90)
- Unlocking the Power of Representations in Long-term Novelty-based Exploration (ICLR 2024 · similarity 0.87)
- Just Cluster It: An Approach for Exploration in High-Dimensions using Clustering and Pre-Trained Representations (ICML 2024 · similarity 0.86)
- Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo (ICLR 2024 · similarity 0.84)
- Diffusion for World Modeling: Visual Details Matter in Atari (NeurIPS 2024 · similarity 0.84)
- Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.