Research map / Reinforcement learning / Visual and representation RL
NovelD: A Simple yet Effective Exploration Criterion
Tianjun Zhang, Huazhe Xu, Xiaolong Wang, Yi Wu, Kurt Keutzer, Joseph E. Gonzalez, Yuandong Tian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use NovelD (novelty difference) instead of RND as intrinsic reward | PPO agent · MiniGrid hard exploration (MR, KC, OM) | task success | Improved |
| use NovelD instead of RND | PPO agent · Montezuma's Revenge CNN | extrinsic reward: 6700 -> 10000 | Improved |
| use NovelD instead of RND | PPO agent · Montezuma's Revenge RNN | extrinsic reward: 4400 -> 13000 | Improved |
| use NovelD instead of RND | PPO agent · NetHack tasks | reward | Improved |
| use NovelD instead of RND for state coverage | intrinsic reward · Toy corridor | visitation entropy: 0.24 -> 1.96 | Improved |
| add episodic restriction on intrinsic reward (ERIR) to NovelD (ablation) | NovelD · MiniGrid | performance | Improved |
| add ERIR to RND (RND+ERIR) (ablation) | RND · MiniGrid | performance | Improved |
| set scaling alpha=0.5 and clipping beta=0 vs other values (ablation) | NovelD · MiniGrid | performance | Improved |
| replace random embedding with ICM or DBC representation (ablation) | NovelD · MiniGrid | reward | Hurt |
| replace random embedding with successor representation (ablation) | NovelD · MiniGrid KC and MR maps | reward | Mixed |
Later papers that compared with this method
3 later accepted papers report NovelD as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Rethinking Exploration in Reinforcement Learning with Effective Metric-Based Exploration Bonus (NeurIPS 2024 · better on 1 of 2 shared results)
- Improving Intrinsic Exploration with Language Abstractions (NeurIPS 2022 · better on 5 of 5 shared results)
- LECO: Learnable Episodic Count for Task-Specific Intrinsic Reward (NeurIPS 2022 · better on 0 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models (ICLR 2025 · similarity 0.87)
- Improving Intrinsic Exploration by Creating Stationary Objectives (ICLR 2024 · similarity 0.87)
- Rethinking Exploration in Reinforcement Learning with Effective Metric-Based Exploration Bonus (NeurIPS 2024 · similarity 0.87)
- Random Latent Exploration for Deep Reinforcement Learning (ICML 2024 · similarity 0.86)
- Exploration and Anti-Exploration with Distributional Random Network Distillation (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
