Research map / Reinforcement learning / Policy gradient and actor-critic
Hybrid Reward Architecture for Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| decompose the reward into component rewards and learn a separate value function for each | HRA on Ms. Pac-Man · Ms. Pac-Man fixed start | score: 6,673 -> 25,304 | Helped |
| decompose the reward into component rewards and learn a separate value function for each | HRA on Ms. Pac-Man · Ms. Pac-Man random start | score: 2,251 -> 23,770 | Helped |
| add executive memory that replays successful level-passing action sequences (ablation) | HRA on Ms. Pac-Man · Ms. Pac-Man fixed start | score: 999,990 | Helped |
| remove the diversification head (ablation) | HRA on Ms. Pac-Man · Ms. Pac-Man training | training score | Hurt |
| add a diversification head that randomizes action values during the first 50 steps (ablation) | HRA on Ms. Pac-Man · Ms. Pac-Man training | learning progress | Helped |
| normalize the score-producing heads (ablation) | HRA on Ms. Pac-Man · Ms. Pac-Man training | score | Helped |
| add targeted count-based exploration (ablation) | HRA on Ms. Pac-Man · Ms. Pac-Man training | learning speed | Helped |
| use object-channel preprocessing instead of pixel preprocessing (ablation) | A3C · Ms. Pac-Man | score | No clear effect |
| remove irrelevant features separately for each reward head (ablation) | HRA · Fruit Collection | performance | Helped |
| use the same irrelevant-feature-pruned network (ablation) | DQN · Fruit Collection | performance | Hurt |
| identify terminal states and use the resulting one-hot tabular representation (ablation) | HRA+2 · Fruit Collection | performance | Helped |
| train pseudo-reward GVFs for reaching each fruit location, including when no fruit is present (ablation) | HRA+3 · Fruit Collection | performance | Helped |
Closest papers by meaning
- Reward learning from human preferences and demonstrations in Atari (NeurIPS 2018 · similarity 0.88)
- To the Max: Reinventing Reward in Reinforcement Learning (ICML 2024 · similarity 0.86)
- Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferences (ICML 2024 · similarity 0.86)
- Stop Regressing: Training Value Functions via Classification for Scalable Deep RL (ICML 2024 · similarity 0.86)
- Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback (NeurIPS 2025 · similarity 0.86)
- Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.