Research map / Reinforcement learning / MDPs and planning
Count-Based Exploration with the Successor Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the inverse SR norm as an exploration bonus to Sarsa | Sarsa · RiverSwim | average return: 24{,}770 -> 1{,}213{,}544 | Helped |
| add the inverse SR norm as an exploration bonus to Sarsa | Sarsa · SixArms | average return: 247{,}977 -> 1{,}052{,}934 | Helped |
| use the SSR norm as an exploration bonus | ESSR · RiverSwim | average return (millions): 3.0 -> 3.1 (R-Max); 3.0 -> 3.1 (E3); 3.3 -> 3.1 (MBIE) | No clear effect |
| use the SSR norm as an exploration bonus | ESSR · SixArms | average return (millions): 1.8 -> 7.3 (E3); 2.8 -> 7.3 (R-Max); 9.3 -> 7.3 (MBIE) | Mixed |
| add the SR exploration bonus | DQN_e^MMC · Atari hard-exploration games | score: Mont. Rev.: 0.0 -> 1395.4; Gravitar: 1078.3 -> 457.4; Venture: 1220.1 -> 1348.5 | Mixed |
| remove the next-observation reconstruction task (ablation) | DQN_e^MMC+SR · Mont. Rev. | score: 1395.4 -> 100 | Hurt |
| remove the SR module and exploration bonus (ablation) | DQN_e^MMC+SR · Mont. Rev. | score: 1395.4 -> 398.5 | Hurt |
| use the L2 norm instead of the L1 norm for the SR bonus (ablation) | DQN_e^MMC+SR · Atari 2600 games | score: Mont. Rev.: 1395.4 -> 1778.6; Freeway: 29.4 -> 29.5; Gravitar: 457.4 -> 430.3; Private Eye: 104.4 -> 99.1; Solaris: 1890 | No clear effect |
| use the L2 norm instead of the L1 norm for the SR bonus (ablation) | Sarsa+SR · RiverSwim and SixArms | average return: RiverSwim: 1{,}213{,}544 -> 1{,}192{,}052; SixArms: 1{,}052{,}934 -> 819{,}927 | No clear effect |
Closest papers by meaning
- Reward-Aware Proto-Representations in Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
- Learning Successor Features the Simple Way (NeurIPS 2024 · similarity 0.87)
- Rethinking Exploration in Reinforcement Learning with Effective Metric-Based Exploration Bonus (NeurIPS 2024 · similarity 0.87)
- Improving Intrinsic Exploration by Creating Stationary Objectives (ICLR 2024 · similarity 0.87)
- Proto Successor Measure: Representing the Behavior Space of an RL Agent (ICML 2025 · similarity 0.87)
- A Distributional Analogue to the Successor Representation (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.