Research map / Reinforcement learning / Policy gradient and actor-critic
Expected Policy Gradients for Reinforcement Learning
Kamil Ciosek, Shimon Whiteson
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use EPG covariance-based exploration instead of Ornstein-Uhlenbeck noise in DPG | DPG · MuJoCo continuous control (5 tasks) | learning curve return | Improved |
| replace exponential of Hessian with 1-step approximation max(0,1+lambda) (ablation) | EPG exploration · MuJoCo continuous control (5 tasks) | learning curve return | No clear effect |
| use full Hessian instead of diagonal Hessian (ablation) | EPG · HalfCheetah, Reacher, Walker2d | learning curve return | No clear effect |
| use state-dependent covariance instead of one global averaged Hessian (ablation) | EPG · MuJoCo continuous control (5 tasks) | learning curve return | Improved |
| use numerical quadrature EPG actor update instead of DPG/SPG update | DPG with OU noise 0.2 · MuJoCo continuous control (5 tasks) | learning curve return | Mixed |
| increase quadrature samples from 1 to 4 and 8 (ablation) | NQ-EPG · MuJoCo continuous control (5 tasks) | learning curve return | No clear effect |
| use EPG sum over actions instead of A2C stochastic policy gradient | A2C · Atari Pong | learning curve score | No clear effect |
| compare EPG with PPO | EPG · HalfCheetah, Walker2d | learning curve return and stability | Mixed |
Closest papers by meaning
- Expected Policy Gradients (AAAI 2018 · similarity 0.98)
- Fourier Policy Gradients (ICML 2018 · similarity 0.92)
- Marginal Policy Gradients: A Unified Family of Estimators for Bounded Action Spaces with Applications (ICLR 2019 · similarity 0.92)
- Learning Optimal Deterministic Policies with Stochastic Policy Gradients (ICML 2024 · similarity 0.91)
- Convergence Analysis of Policy Gradient Methods with Dynamic Stochasticity (ICML 2025 · similarity 0.90)
- Accelerated Policy Gradient: On the Convergence Rates of the Nesterov Momentum for Reinforcement Learning (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
