AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Understanding the Evolution of Linear Regions in Deep Reinforcement Learning

Setareh Cohan, Nam Hee Gordon Kim, David Rolnick, Michiel van de Panne

NeurIPS 2022 · Reinforcement learning › Policy gradient and actor-critic · approach: Regularized scalable deep value learning · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train longer: from start of training to end (fixed final-policy trajectory)PPO policy network · HalfCheetah-v2region transition count along fixed trajectory: 50%Improved
vary network configuration (width/depth) for normalized region densityPPO policy network · HalfCheetah-v2normalized region density: within ±30%No clear effect
increase policy network depthPPO policy network · HalfCheetah-v2normalized region densityImproved
switch from PPO to SACpolicy network · HalfCheetah-v2region density trajectoryMixed
switch to non-cyclic task LunarLanderPPO policy network · LunarLanderContinuous-v2region density along fixed trajectoryNo clear effect
analyze value network instead of policy networkPPO value network · HalfCheetah-v2linear region evolutionNo clear effect
train with behavior cloning instead of PPOpolicy network · HalfCheetah-v2region density growthHurt
use random-action trajectories instead of final trajectoryPPO policy network · HalfCheetah-v2normalized region densityNo clear effect
lines through mean point instead of origin (ablation)PPO policy network · HalfCheetah-v2region density along random linesNo clear effect
train longer on random lines through originPPO policy network · HalfCheetah-v2normalized region density R_T/NNo clear effect
train longer, evaluate on random-action trajectoriesPPO policy network · HalfCheetah-v2normalized region densityHurt
train longer, evaluate on current trajectoryPPO policy network · HalfCheetah-v2region density along current trajectoryHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy