Research map / Reinforcement learning / Policy gradient and actor-critic
Deterministic Value-Policy Gradients
Qingpeng Cai, Ling Pan, Pingzhong Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Ensemble model-based deterministic value gradients with model-free deterministic policy gradients using geometric-decay weights | DVPG · MuJoCo continuous-control benchmarks | episode reward | Improved |
| Use infinite-horizon value gradients rather than finite-horizon gradients | DVG · HumanoidStandup-v2 | episode reward | Improved |
| Use model-based value-gradient rollouts instead of no model rollout | DVG · Hopper-v2 | sample efficiency | Improved |
| Use one model rollout step rather than DDPG's zero-step model-free gradient | DVG · Hopper-v2 | episode reward | Improved |
| Increase model rollout steps to 5 (ablation) | DVG · Hopper-v2 | training stability | Hurt |
| Use a discount factor of 0.95 rather than 0.8 or 0.6 (ablation) | DVG with 2 rollout steps · InvertedPendulum-v2 | episode reward and stability | Improved |
| Use discount factors 0.85 or 0.99 instead of 0.95 (ablation) | DVG with 2 rollout steps · InvertedPendulum-v2 | episode reward and stability | No clear effect |
| Use discount factors 0.8 or 0.6 instead of 0.95 (ablation) | DVG with 2 rollout steps · InvertedPendulum-v2 | episode reward | Hurt |
| Increase the bootstrapping weight λ (ablation) | DVPG with 4 rollout steps · MuJoCo continuous-control benchmarks | sample efficiency and episode reward | Hurt |
| Set the bootstrapping weight λ to 0.1 (ablation) | DVPG with 4 rollout steps · MuJoCo continuous-control benchmarks | sample efficiency and episode reward | Improved |
| Use 1 rollout step rather than other tested rollout-step counts (ablation) | DVPG · MuJoCo continuous-control benchmarks | sample efficiency and episode reward | Improved |
| Use 2 rollout steps rather than other tested rollout-step counts (ablation) | DVPG · HumanoidStandup-v2, Hopper-v2, Ant-v2 | episode reward | Improved |
Closest papers by meaning
- Softmax Deep Double Deterministic Policy Gradients (NeurIPS 2020 · similarity 0.89)
- Value Gradient weighted Model-Based Reinforcement Learning (ICLR 2022 · similarity 0.88)
- Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning (NeurIPS 2017 · similarity 0.88)
- Learning Optimal Deterministic Policies with Stochastic Policy Gradients (ICML 2024 · similarity 0.87)
- On the Second-Order Convergence of Biased Policy Gradient Algorithms (ICML 2024 · similarity 0.87)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
