Research map / Reinforcement learning / Policy gradient and actor-critic
Understanding the Evolution of Linear Regions in Deep Reinforcement Learning
Setareh Cohan, Nam Hee Gordon Kim, David Rolnick, Michiel van de Panne
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train longer: from start of training to end (fixed final-policy trajectory) | PPO policy network · HalfCheetah-v2 | region transition count along fixed trajectory: 50% | Improved |
| vary network configuration (width/depth) for normalized region density | PPO policy network · HalfCheetah-v2 | normalized region density: within ±30% | No clear effect |
| increase policy network depth | PPO policy network · HalfCheetah-v2 | normalized region density | Improved |
| switch from PPO to SAC | policy network · HalfCheetah-v2 | region density trajectory | Mixed |
| switch to non-cyclic task LunarLander | PPO policy network · LunarLanderContinuous-v2 | region density along fixed trajectory | No clear effect |
| analyze value network instead of policy network | PPO value network · HalfCheetah-v2 | linear region evolution | No clear effect |
| train with behavior cloning instead of PPO | policy network · HalfCheetah-v2 | region density growth | Hurt |
| use random-action trajectories instead of final trajectory | PPO policy network · HalfCheetah-v2 | normalized region density | No clear effect |
| lines through mean point instead of origin (ablation) | PPO policy network · HalfCheetah-v2 | region density along random lines | No clear effect |
| train longer on random lines through origin | PPO policy network · HalfCheetah-v2 | normalized region density R_T/N | No clear effect |
| train longer, evaluate on random-action trajectories | PPO policy network · HalfCheetah-v2 | normalized region density | Hurt |
| train longer, evaluate on current trajectory | PPO policy network · HalfCheetah-v2 | region density along current trajectory | Hurt |
Closest papers by meaning
- The Ladder in Chaos: Improving Policy Learning by Harnessing the Parameter Evolving Path in A Low-dimensional Space (NeurIPS 2024 · similarity 0.87)
- Analyzing Generalization in Policy Networks: A Case Study with the Double-Integrator System (AAAI 2024 · similarity 0.86)
- Understanding and Diagnosing Deep Reinforcement Learning (ICML 2024 · similarity 0.86)
- Efficient Reinforcement Learning by Discovering Neural Pathways (NeurIPS 2024 · similarity 0.85)
- Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churn (NeurIPS 2024 · similarity 0.85)
- Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
