Research map / Reinforcement learning / Policy gradient and actor-critic
Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Mix on-policy policy gradients with actor-critic updates using ν=0.2 | IPG · OpenAI Gym MuJoCo continuous-control benchmarks | maximum average test reward | Mixed |
| Use random replay samples for the critic gradient rather than on-policy samples (ablation) | IPG with ν=0 and control variate · HalfCheetah-v1 | learning speed | Helped |
| Increase random replay critic-gradient batch to 30000 samples (ablation) | IPG with ν=0 and control variate · HalfCheetah-v1 | learning speed | Helped |
| Use the 30000 most recent samples rather than random replay samples (ablation) | IPG with ν=0 and control variate · HalfCheetah-v1 | learning performance | Hurt |
| Use on-policy exploration instead of DDPG's OU noise exploration (ablation) | IPG with ν=1 · Ant-v1 | mean reward | Helped |
| Use OU-process exploration noise (ablation) | IPG with ν=1 · Ant-v1 | learning performance | Hurt |
| Use off-policy samples (β≠π) for the deterministic policy gradient (ablation) | IPG with ν=0.2 · HalfCheetah-v1 | maximum average test reward: 3356 -> 3458 | Helped |
| Use off-policy samples (β≠π) for the deterministic policy gradient (ablation) | IPG with ν=0.2 · Ant-v1 | maximum average test reward: 4237 -> 4415 | Helped |
| Use off-policy samples (β≠π) for the deterministic policy gradient (ablation) | IPG with ν=0.2 · Walker-v1 | maximum average test reward: 3047 -> 1932 | Hurt |
| Use off-policy samples (β≠π) for the deterministic policy gradient (ablation) | IPG with ν=0.2 · Humanoid-v1 | maximum average test reward: 1231 -> 920 | Hurt |
| Add the control variate to IPG with ν=0.2 (ablation) | IPG · HalfCheetah-v1 | maximum average test reward: 3356 -> 4216 | Helped |
| Add the control variate to IPG with ν=0.2 (ablation) | IPG · Ant-v1 | maximum average test reward: 4237 -> 3943 | Hurt |
Closest papers by meaning
- Statistically Efficient Off-Policy Policy Gradients (ICML 2020 · similarity 0.92)
- Trajectory-Based Off-Policy Deep Reinforcement Learning (ICML 2019 · similarity 0.91)
- Off-Policy Evaluation via Off-Policy Classification (NeurIPS 2019 · similarity 0.91)
- Anatomy of Off-Policy Policy Gradient: Importance Sampling, KL Regularization, and Baselines (NeurIPS 2026 · similarity 0.90)
- Efficient Off-Policy Learning for High-Dimensional Action Spaces (ICLR 2025 · similarity 0.90)
- Deep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Buffers (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.