Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Mastering Atari with Discrete World Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DreamerV2 rather than IQN | Atari 55 games · Atari 55 games, 200M steps | Clipped Record Mean: 0.21 -> 0.28 | Helped |
| Use DreamerV2 rather than Rainbow | Atari 55 games · Atari 55 games, 200M steps | Clipped Record Mean: 0.17 -> 0.28 | Helped |
| Anneal actor entropy scale and actor gradient mixing (ablation) | DreamerV2 · Atari 55 games, 200M steps | Gamer Median: 2.15 -> 2.64 | Helped |
| Anneal actor entropy scale and actor gradient mixing (ablation) | DreamerV2 · Atari 55 games, 200M steps | Gamer Mean: 11.33 -> 10.45 | Hurt |
| Add categorical latents instead of Gaussian latents (ablation) | DreamerV2 world model · Atari 55 games, 200M frames (earlier version) | Clipped Record Mean: 0.19 -> 0.25 | Helped |
| Add KL balancing (ablation) | DreamerV2 world model · Atari 55 games, 200M frames (earlier version) | Clipped Record Mean: 0.16 -> 0.25 | Helped |
| Use image-reconstruction gradients to train representations (ablation) | DreamerV2 world model · Atari 55 games, 200M frames (earlier version) | Clipped Record Mean: 0.01 -> 0.25 | Helped |
| Stop reward-prediction gradients from entering model states (ablation) | DreamerV2 world model · Atari 55 games, 200M frames (earlier version) | Clipped Record Mean: 0.25 -> 0.24 | Hurt |
| Retain Reinforce policy gradients (ablation) | DreamerV2 actor · Atari 55 games, 200M frames (earlier version) | Clipped Record Mean: 0.15 -> 0.25 | Helped |
| Use layer normalization in the GRU (ablation) | DreamerV2 world model · Atari 55 games, 200M frames (earlier version) | Clipped Record Mean: 0.25 -> 0.25 | No clear effect |
| Use categorical rather than Gaussian latent variables (ablation) | DreamerV2 world model · Atari tasks | task performance | Mixed |
| Add KL balancing rather than use the standard KL regularizer (ablation) | DreamerV2 world model · Atari tasks | task performance | Helped |
Closest papers by meaning
- Diffusion for World Modeling: Visual Details Matter in Atari (NeurIPS 2024 · similarity 0.90)
- Learning to Play Atari in a World of Tokens (ICML 2024 · similarity 0.88)
- Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining (ICLR 2025 · similarity 0.87)
- Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learning (ICML 2024 · similarity 0.87)
- Zero-shot World Models via Search in Memory (NeurIPS 2025 · similarity 0.86)
- EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modeling (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.