Research map / Reinforcement learning / Policy gradient and actor-critic
The Benefits of Model-Based Generalization in Reinforcement Learning
Kenny John Young, Aditya Ramesh, Louis Kirsch, Jürgen Schmidhuber
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a learned simple model to generate training rollouts instead of relying on experience replay alone | DQN · ProcMaze and ButtonGrid, high data, sufficiently complex instances | greedy policy performance | Improved |
| Use a learned categorical latent model to generate training rollouts instead of relying on experience replay alone | DQN · ProcMaze and ButtonGrid, high data, sufficiently complex instances | greedy policy performance | Improved |
| Use a Gaussian latent model rather than a simpler model or categorical latent model | DQN · ProcMaze and ButtonGrid | greedy policy performance | Hurt |
| Use a learned model rather than the perfect model | DQN · Harder PanFlute instances | greedy policy performance | Improved |
| Use model-generated rollouts rather than experience replay alone | DQN · OpenGrid, low data, increasing complexity | greedy policy performance | Hurt |
| Use model-based learning rather than experience replay alone | DQN · 3x3 grid world, Path-to-Goal coverage | optimal greedy action frequency | Improved |
| Use 10-step rather than single-step model rollouts (ablation) | DQN with a learned model · 3x3 grid world, Single-Cell coverage | optimal greedy action frequency | Improved |
| Use longer rather than 1-step model rollouts (ablation) | DQN with a learned model · Structured online RL environments | greedy policy performance | Improved |
| Use a simple model rather than a categorical latent model | DQN · ProcMaze and ButtonGrid, low data | greedy policy performance | Improved |
| Train the simple model in the low-data rather than high-data regime | DQN, ButtonGrid with 1 or 3 buttons · ButtonGrid | greedy policy performance | Improved |
| Allow learned-model reward and transition predictions to be smoothed rather than exact | DQN with a learned model · PanFlute | greedy policy performance | Improved |
Closest papers by meaning
- Prioritized Generative Replay (ICLR 2025 · similarity 0.88)
- The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Model-Based Transfer Learning for Contextual Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
- Understanding when Dynamics-Invariant Data Augmentations Benefit Model-free Reinforcement Learning Updates (ICLR 2024 · similarity 0.87)
- The Generalization Gap in Offline Reinforcement Learning (ICLR 2024 · similarity 0.87)
- Doubly Mild Generalization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
