AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

When to use parametric models in reinforcement learning?

Hado P van Hasselt, Matteo Hessel, John Aslanides

NeurIPS 2019 · Reinforcement learning › Policy gradient and actor-critic · approach: Model-based RL with guarantees · 101 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use data-efficient Rainbow DQN instead of SimPLeAtari 26 games · Atari 26 gamesmedian human-normalized episode return: roughly $25\%$ higher performance by 100,000 interactions; 17 out of 26 gamesImproved
Increase multi-step return length from 3 to 20 (ablation)Rainbow DQN · Atari 26 gamesmedian human-normalized episode returnImproved
Use a 5-step rather than 20-step bootstrapping length (ablation)Rainbow DQN · Atari 26 gamesmedian human-normalized episode returnHurt
Use a 10-step rather than 20-step bootstrapping length (ablation)Rainbow DQN · Atari 26 gamesmedian human-normalized episode returnNo clear effect
Perform updates every step rather than every 4 steps (ablation)Rainbow DQN · Atari 26 gamesmedian human-normalized episode returnImproved
Increase replay batch size instead of updating every step, matching the 4× increase in replayed transitions (ablation)Rainbow DQN · Atari 26 gamesmedian human-normalized episode returnHurt
Plan backward rather than forward for credit assignmentTabular Dyna in four-rooms · Four-rooms, deterministic and stochastic variantsepisode lengthImproved
Use replay instead of forward model planning for credit assignmentTabular Q-learning in four-rooms · Four-rooms, deterministic variantepisode length: after 2000 episodesMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy