Research map / Reinforcement learning / Visual and representation RL
Exploration-Guided Reward Shaping for Reinforcement Learning under Sparse Rewards
Rati Devidze, Parameswaran Kamalaruban, Adish Singla
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add intrinsic bonus (EXPLOB) to intrinsic reward (SELFRS), i.e. full ExploRS (ablation) | ExploRS REINFORCE and Q-learning agents · CHAIN0 and CHAIN+ | expected reward | Improved |
| add intrinsic reward learning to count-based bonus alone (ExploRS vs EXPLOB) (ablation) | count-based exploration bonus · CHAIN, ROOM, LINEK | expected reward | Improved |
| add exploration bonus to self-supervised intrinsic reward (ExploRS vs SELFRS) (ablation) | SELFRS · CHAIN with distractor (CHAIN+) | expected reward | Improved |
| add distractor state with small reward (Rdis=0.01) for SORS', LIRPG', SELFRS | SORS', LIRPG', SELFRS · CHAIN+ vs CHAIN0 | expected reward | Hurt |
| add distractor states (Rdis=0.01) for SORS', LIRPG', SELFRS in neural LINEK | SORS', LIRPG', SELFRS · LINEK+ | expected reward | Hurt |
| use ExploRS instead of all baselines with REINFORCE agent | tabular REINFORCE · CHAIN0, CHAIN+ | expected reward | Improved |
| use ExploRS instead of baselines with Q-learning agent | tabular Q-learning · CHAIN0, CHAIN+ | expected reward | Improved |
| use ExploRS instead of baselines on ROOM | tabular REINFORCE · ROOM0, ROOM+ | expected reward | Improved |
| use ExploRS with neural-network intrinsic reward instead of baselines | neural REINFORCE · LINEK0, LINEK+ | expected reward | Improved |
| combine intrinsic reward and bonus in chain theory (SELFRS=1, EXPLOB=1) vs no shaping (ablation) | stylized TD-style learner · chain environment (theory) | time steps to optimal policy | Improved |
| add only intrinsic bonus (no intrinsic reward) in chain theory (ablation) | stylized TD-style learner · chain environment (theory) | time steps to optimal policy | Improved |
| add only intrinsic reward (no bonus) in chain theory (ablation) | stylized TD-style learner · chain environment (theory) | time steps to optimal policy | No clear effect |
Closest papers by meaning
- Reward Shaping for Reinforcement Learning with An Assistant Reward Agent (ICML 2024 · similarity 0.93)
- Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learning (ICLR 2025 · similarity 0.91)
- DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
- Action-Dependent Optimality-Preserving Reward Shaping (ICML 2025 · similarity 0.91)
- BAMDP Shaping: a Unified Framework for Intrinsic Motivation and Reward Shaping (ICLR 2025 · similarity 0.90)
- Catching Two Birds with One Stone: Reward Shaping with Dual Random Networks for Balancing Exploration and Exploitation (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
