AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

The Benefits of Model-Based Generalization in Reinforcement Learning

Kenny John Young, Aditya Ramesh, Louis Kirsch, Jürgen Schmidhuber

ICML 2023 · Reinforcement learning › Policy gradient and actor-critic · approach: Regularized scalable deep value learning · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use a learned simple model to generate training rollouts instead of relying on experience replay aloneDQN · ProcMaze and ButtonGrid, high data, sufficiently complex instancesgreedy policy performanceImproved
Use a learned categorical latent model to generate training rollouts instead of relying on experience replay aloneDQN · ProcMaze and ButtonGrid, high data, sufficiently complex instancesgreedy policy performanceImproved
Use a Gaussian latent model rather than a simpler model or categorical latent modelDQN · ProcMaze and ButtonGridgreedy policy performanceHurt
Use a learned model rather than the perfect modelDQN · Harder PanFlute instancesgreedy policy performanceImproved
Use model-generated rollouts rather than experience replay aloneDQN · OpenGrid, low data, increasing complexitygreedy policy performanceHurt
Use model-based learning rather than experience replay aloneDQN · 3x3 grid world, Path-to-Goal coverageoptimal greedy action frequencyImproved
Use 10-step rather than single-step model rollouts (ablation)DQN with a learned model · 3x3 grid world, Single-Cell coverageoptimal greedy action frequencyImproved
Use longer rather than 1-step model rollouts (ablation)DQN with a learned model · Structured online RL environmentsgreedy policy performanceImproved
Use a simple model rather than a categorical latent modelDQN · ProcMaze and ButtonGrid, low datagreedy policy performanceImproved
Train the simple model in the low-data rather than high-data regimeDQN, ButtonGrid with 1 or 3 buttons · ButtonGridgreedy policy performanceImproved
Allow learned-model reward and transition predictions to be smoothed rather than exactDQN with a learned model · PanFlutegreedy policy performanceImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy