AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

A Self-Tuning Actor-Critic Algorithm

Tom Zahavy, Zhongwen Xu, Vivek Veeriah, Matteo Hessel, Junhyuk Oh, Hado P van Hasselt, David Silver, Satinder P. Singh

NeurIPS 2020 · Reinforcement learning › Policy gradient and actor-critic · approach: Actor-critic theory · 37 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add online metagradient self-tuning of differentiable actor-critic loss parametersIMPALA on Atari · Atari (57 games, 200M frames)median human-normalized score: 243% -> 364%Improved
Add online metagradient self-tuningIMPALA on DM Control with features · DM Control, features (30M steps)mean score: 217 to 389Improved
Add online metagradient self-tuningIMPALA on DM Control with pixels · DM Control, pixels (30M steps)mean score: 108 to 202Improved
Add online metagradient self-tuningIMPALA on RWRL · RWRL (30M steps)mean score: 195 to 295Improved
Add auxiliary self-tuned actor-critic tasksSTAC on Atari · Atarimedian human-normalized score: nullImproved
Self-tune more metaparameters (ablation)STAC and STACX on Atari · Atarimedian human-normalized scoreImproved
Add γ embedding while self-tuning γ and λ (ablation)STACX on Atari · Atariscore: 280.6%Improved
Add γ embedding to STACX with all metaparameters self-tuned (ablation)STACX on Atari · Atariscore: 306.8%, 307.7% versus 364%Hurt
Learn both αρ and αc without enforcing αρ ≥ αc (ablation)STACX on Atari · Atari (57 games)median human-normalized score: 353%Hurt
Self-tune α together with a single truncation parameter ρ̄ = c̄ (ablation)STACX on Atari · Atari (57 games)median human-normalized score: 301%Hurt
Use a random ensemble or average-logit ensemble so auxiliary heads can act (ablation)STACX · Experiments reported in the paperperformanceHurt
Use 2, 5, or 8 auxiliary losses instead of 3 (ablation)STACX on Atari · AtariperformanceMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy