Research map / Reinforcement learning / Policy gradient and actor-critic
Distributed Distributional Deterministic Policy Gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a distributional critic instead of a scalar critic (ablation) | D4PG / D3PG · Continuous control tasks | performance | Helped |
| Use N=5 returns instead of N=1 returns (ablation) | D4PG and its ablations · Control suite, manipulation, and parkour | performance | Helped |
| Add prioritized replay (ablation) | D4PG · Continuous control tasks | performance | No clear effect |
| Add prioritized replay (ablation) | D3PG · Control suite | performance | Helped |
| Add prioritized replay (ablation) | D3PG · Manipulation tasks | performance | Hurt |
| Replace the categorical critic distribution with a mixture of Gaussians (ablation) | D4PG · Selected control suite tasks | performance | Hurt |
| Use Ornstein-Uhlenbeck exploration noise instead of fixed Gaussian noise (ablation) | D4PG · Continuous control tasks | performance | No clear effect |
| Use the full D4PG enhancements instead of canonical DDPG (ablation) | DDPG · Control suite | performance | Helped |
| Use D4PG instead of PPO | PPO · Two-dimensional walker parkour | performance | Helped |
| Use D4PG instead of PPO | PPO · Three-dimensional humanoid parkour | performance | Helped |
Closest papers by meaning
- Bayesian Distributional Policy Gradients (AAAI 2021 · similarity 0.89)
- Score Regularized Policy Optimization through Diffusion Behavior (ICLR 2024 · similarity 0.88)
- Distributions as Actions: A Unified Framework for Diverse Action Spaces (ICLR 2026 · similarity 0.88)
- Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning (ICML 2025 · similarity 0.86)
- ComaDICE: Offline Cooperative Multi-Agent Reinforcement Learning with Stationary Distribution Shift Regularization (ICLR 2025 · similarity 0.86)
- Delightful Distributed Policy Gradient (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.