Research map / Reinforcement learning / Policy gradient and actor-critic
Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace density-based pseudocount (PixelCNN) with Coin Flip Network regression for count bonuses | Rainbow-based exploration agent · Visual Gridworld | bonus prediction accuracy vs ground-truth counts | Helped |
| Add prioritized sampling and random prior (optimistic initialization) to CFN (ablation) | CFN bonus predictor · Visual Gridworld | mean-squared bonus prediction error | Helped |
| Increase transition stochasticity (action noise) in Visual Gridworld: CFN bonus vs RND bonus | CFN relative to RND · Visual Gridworld (stochastic versions) | mean episodic return | Helped |
| Use CFN instead of SAC baseline on sparse continuous control tasks | SAC with CFN bonus · Fetch manipulation tasks (9 tasks) | learning performance | Helped |
| Use CFN bonus on Adroit and Ant tasks | SAC with CFN bonus · D4RL Ant U-Maze and Adroit tasks | learning performance | Helped |
| Use large CFN replay buffer (2e7) instead of smaller buffer (5e6) (ablation) | CFN on Montezuma's Revenge · Montezuma's Revenge | episodic return | Helped |
| Use CFN bonus instead of RND on Montezuma's Revenge with increasing sticky action probability | CFN vs RND · Montezuma's Revenge (sticky actions) | mean cumulative reward | Helped |
| Use CFN bonus on default Montezuma's Revenge (vs Rainbow and RND/PixelCNN) | Rainbow with CFN bonus · Montezuma's Revenge (default) | episodic return | No clear effect |
Closest papers by meaning
- Unifying Count-Based Exploration and Intrinsic Motivation (NeurIPS 2016 · similarity 0.90)
- Unlocking the Power of Representations in Long-term Novelty-based Exploration (ICLR 2024 · similarity 0.86)
- Rethinking Exploration in Reinforcement Learning with Effective Metric-Based Exploration Bonus (NeurIPS 2024 · similarity 0.86)
- Exploring Starts Are Not Enough: Counterexamples and a Fix for Monte Carlo Exploring Starts (NeurIPS 2026 · similarity 0.86)
- Improving Intrinsic Exploration by Creating Stationary Objectives (ICLR 2024 · similarity 0.86)
- Geometry Meets Incentives: Sample-Efficient Incentivized Exploration with Linear Contexts (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.