Research map / Reinforcement learning / Visual and representation RL
Revisiting Rainbow: Promoting more insightful and inclusive deep reinforcement learning research
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine all Rainbow components rather than use DQN alone | DQN · Classic control and MinAtar | performance | Helped |
| Add C51 distributional RL to DQN | DQN · Acrobot and Freeway | performance | Hurt |
| Remove C51 distributional RL (ablation) | Rainbow · MountainCar and Seaquest | performance | Helped |
| Use QR-DQN instead of C51 | Rainbow · Classic control | performance | Hurt |
| Use QR-DQN instead of C51 | Rainbow · MinAtar except Freeway | performance | Helped |
| Use IQN instead of C51 | Rainbow · Classic control | performance | Hurt |
| Use IQN instead of C51 | Rainbow · MinAtar except Freeway | performance | Helped |
| Replace Huber loss with MSE loss | DQN · Classic control and MinAtar | performance | Helped |
| Use Adam with MSE instead of RMSProp with Huber | DQN · Atari 2600 | human normalized score | Helped |
| Use MSE instead of Huber while keeping RMSProp (ablation) | DQN · Atari 2600 | performance | Hurt |
| Add Munchausen RL | DQN · Classic control | performance | No clear effect |
| Add Munchausen RL | DQN · MinAtar | performance | Mixed |
Closest papers by meaning
- Rainbow: Combining Improvements in Deep Reinforcement Learning (AAAI 2018 · similarity 0.94)
- Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC (ICML 2025 · similarity 0.91)
- Faster Deep Reinforcement Learning with Slower Online Network (NeurIPS 2022 · similarity 0.89)
- Confounding Robust Deep Reinforcement Learning: A Causal Approach (NeurIPS 2025 · similarity 0.87)
- Simple Ingredients for Offline Reinforcement Learning (ICML 2024 · similarity 0.87)
- Stop Regressing: Training Value Functions via Classification for Scalable Deep RL (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.