Research map / Reinforcement learning / Policy gradient and actor-critic
Policy Search by Target Distribution Learning for Continuous Control
Chuheng Zhang, Yuanqi Li, Jian Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TDL instead of PPO | Continuous control policy learning · MuJoCo tasks | cumulative reward | Mixed |
| Use TDL-ESr instead of the other compared methods | Continuous control policy learning · MuJoCo tasks | cumulative reward | Improved |
| Set target distributions rather than directly updating the policy with stochastic gradient ascent (ablation) | TDL-ES · MuJoCo tasks | cumulative reward | Improved |
| Increase on-policy sample reuse (ablation) | TDL-direct, TDL-ES, and TDL-ESr · Hopper-v2 | performance | No clear effect |
| Increase on-policy sample reuse (ablation) | PPO · Hopper-v2 | performance | Hurt |
| Use an entropy term (ablation) | PPO · Simple one-step cost environment | asymptotic performance | Hurt |
| Impose a minimum variance threshold (ablation) | PPO · Simple one-step cost environment | cost | Improved |
| Impose a minimum variance threshold (ablation) | PPO · Simple one-step cost environment | training stability | No clear effect |
| Use a smaller clipping constant (ablation) | PPO · Simple one-step cost environment | training stability | No clear effect |
| Move the target mean in the negative direction for negative-advantage samples (ablation) | TDL-direct · MuJoCo tasks | exploration | Improved |
| Move the target mean in the negative direction for negative-advantage samples (ablation) | TDL-direct · Humanoid | cumulative reward | Hurt |
| Move the target mean in the negative direction for negative-advantage samples (ablation) | TDL-ES · MuJoCo tasks | training stability | Hurt |
Closest papers by meaning
- Nonlinear Distributional Gradient Temporal-Difference Learning (ICML 2019 · similarity 0.88)
- Diffusion Policy Policy Optimization (ICLR 2025 · similarity 0.87)
- TD-MPC2: Scalable, Robust World Models for Continuous Control (ICLR 2024 · similarity 0.87)
- Identifying Policy Gradient Subspaces (ICLR 2024 · similarity 0.86)
- PWM: Policy Learning with Multi-Task World Models (ICLR 2025 · similarity 0.86)
- Learning Optimal Deterministic Policies with Stochastic Policy Gradients (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
