Research map / Reinforcement learning / Policy gradient and actor-critic
Munchausen Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add scaled log-policy to the reward (Munchausen term) on top of DQN | DQN · Atari 60 games, human-normalized mean | Human-normalized mean: 228\% -> 340\% | Helped |
| add scaled log-policy to the reward on top of IQN | IQN · Atari 60 games, human-normalized mean | Human-normalized mean: 441\% -> 563\% | Helped |
| add scaled log-policy to the reward on top of IQN | IQN · Atari 60 games, human-normalized median | Human-normalized median: 139\% -> 165\% | Helped |
| add Munchausen term to Soft-DQN (alpha 0 -> 0.9, same tau) (ablation) | Soft-DQN · Atari ablation | Human-normalized mean and median | Helped |
| add maximum entropy (Soft-DQN) to Adam DQN (ablation) | Adam DQN · Atari ablation | Human-normalized mean and median | No clear effect |
| switch optimizer from RMSProp to Adam (ablation) | DQN · Atari ablation | Human-normalized score | Helped |
| use Advantage Learning target on Adam DQN (ablation) | Adam DQN · Atari ablation | Human-normalized score | Helped |
| replace Advantage Learning with Munchausen target on Adam DQN (ablation) | AL · Asterix | final average score: 13k -> 20k | Helped |
| add Munchausen term to Adam DQN | Adam DQN · Asterix | final average score: 15k -> 20k | Helped |
| use 1-step returns in M-IQN | M-IQN · Atari | Human-normalized mean/median | No clear effect |
Closest papers by meaning
- Confounding Robust Deep Reinforcement Learning: A Causal Approach (NeurIPS 2025 · similarity 0.87)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.86)
- Adaptive $Q$-Network: On-the-fly Target Selection for Deep Reinforcement Learning (ICLR 2025 · similarity 0.86)
- Flattening Hierarchies with Policy Bootstrapping (NeurIPS 2025 · similarity 0.86)
- Simplifying Deep Temporal Difference Learning (ICLR 2025 · similarity 0.86)
- Deep Double Q-learning (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.