AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

DNA: Proximal Policy Optimization with a Dual Network Architecture

Matthew Aitchison, Penny Sweetser

NeurIPS 2022 · Reinforcement learning › Policy gradient and actor-critic · approach: Off-policy actor-critic with expressive policies

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
remove distillation (ablation)DNA · Atari-5score: 252 -> 226Hurt
set policy, value, and distillation mini-batch sizes to 512 (ablation)DNA · Atari-5score: 252 -> 195Hurt
use equal λ values of 0.95 for policy and value (ablation)DNA · Atari-5score: 252 -> 191Hurt
replace DNA's dual-network design and novel components with a basic single-network PPO setup (ablation)DNA · Atari-5score: 252 -> 81Hurt
use a lower λπ of 0.8 while retaining λV=0.95DNA · Atari-3 validationscoreImproved
use fewer than three policy epochs, selecting Eπ=2 (ablation)DNA · Atari-3 validationscoreImproved
increase value training to four epochs (ablation)DNA · Atari-3 validationscoreHurt
remove the distillation phase (ablation)DNA · Atari-3 validationscoreHurt
use random-projection distillation targets (ablation)DNA · Atari-3 validationscoreImproved
use value-estimate distillation targets (ablation)DNA · Atari-3 validationscoreImproved
use feature-matching distillation targets (ablation)DNA · Atari-3 validationscoreHurt
increase PPO policy epochs from one to two (ablation)PPO · Atari-5scoreHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy