AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Self-Consistent Models and Values

Gregory Farquhar, Kate Baumli, Zita Marinho, Angelos Filos, Matteo Hessel, Hado van Hasselt, David Silver

NeurIPS 2021 · Reinforcement learning › Policy gradient and actor-critic · approach: Temporal-difference and discounting theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add SC-Direct rather than using only grounded updatesMuesli · Atari games and Sokobansample efficiencyImproved
use SC-Direct instead of Dyna-style value-only planningMuesli with a value-equivalent model · Atari games and SokobanperformanceImproved
use a residual self-consistency objective instead of a semi-gradient objective (ablation)Muesli · Atari games and SokobanperformanceHurt
use SC-Reverse instead of SC-Direct (ablation)Muesli · Atari games and Sokobansample efficiencyMixed
use SC-Residual or SC-Reverse instead of SC-Direct (ablation)tabular Garnet MDP policy evaluation · Garnet MDPvalue-learning speedMixed
use SC-Direct rather than the other tested updates (ablation)tabular Garnet MDP control · Garnet MDPfinal policy valueImproved
replay the environment's original action sequence instead of resampling actions from the policy (ablation)SC-Direct · Atari gamesperformanceHurt
add epsilon exploration to the imagined policy with epsilon = 0.5 (ablation)SC-Direct · Assault and AsteroidsperformanceImproved
add epsilon exploration to the imagined policy with epsilon = 0.5 (ablation)SC-Direct · Most tested environmentsperformanceNo clear effect
perturb starting latent states with multiplicative noise (ablation)SC-Direct · Atari gamesperformanceNo clear effect
combine latent-state perturbations with epsilon = 0.5 exploration (ablation)SC-Direct · Atari gamesperformanceImproved
add self-consistency as an auxiliary task (ablation)policy-gradient agent with a value-equivalent auxiliary model · Atari representation learningepisode returnImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy