Research map / Reinforcement learning / Policy gradient and actor-critic
Learning Dynamics and Generalization in Deep Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train deep value-based agents with temporal-difference updates on dense-reward tasks | Deep RL agents · Atari environments | update rank | Helped |
| Train with policy-gradient objectives instead of TD-only updates | PPO and DAAC agents · ProcGen | update rank | Helped |
| Distill a trained policy into a freshly initialized network | PPO agents · ProcGen test environments | test performance | Helped |
| Distill a trained policy into a freshly initialized network | Deep RL policies · Atari environments | robustness to input perturbations | Helped |
| Use behavior cloning rather than value distillation | Distilled Atari agents · Atari environments | robustness to perturbations | Helped |
| Use behavior cloning or value distillation rather than the online-trained teacher | Atari agents · Atari environments | policy performance | Helped |
| Distill a PPO policy after training | PPO agents · ProcGen test environments | test performance | Mixed |
| Use a shorter RBF-kernel lengthscale | Kernel TD learning · Circle MDP | training-set value approximation | Helped |
| Use a shorter RBF-kernel lengthscale instead of a longer one | Kernel TD learning · Circle MDP test states | test-state value approximation | Hurt |
| Use a larger-lengthscale RBF kernel under TD dynamics | Kernel TD learning · Circle MDP with γ = 0.99 | stability | Hurt |
Closest papers by meaning
- An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks (ICML 2024 · similarity 0.89)
- Discerning Temporal Difference Learning (AAAI 2024 · similarity 0.89)
- The Ladder in Chaos: Improving Policy Learning by Harnessing the Parameter Evolving Path in A Low-dimensional Space (NeurIPS 2024 · similarity 0.89)
- On Double Descent in Reinforcement Learning with LSTD and Random Features (ICLR 2024 · similarity 0.88)
- Transformers Can Learn Temporal Difference Methods for In-Context Reinforcement Learning (ICLR 2025 · similarity 0.88)
- Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churn (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.