Research map / Reinforcement learning / Policy gradient and actor-critic
Implicit Distributional Reinforcement Learning
Yuguang Yue, Zhendong Wang, Mingyuan Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| IDAC vs SAC | IDAC · Walker2d | max average return: 4328.95 -> 5107.07 | Improved |
| IDAC vs SAC | IDAC · HalfCheetah | max average return: 10626.34 -> 12222.80 | Improved |
| IDAC vs SAC | IDAC · Ant | max average return: 3732.23 -> 4930.73 | Improved |
| IDAC vs TD3 | IDAC · Ant | max average return: 4905.74 -> 4930.73 | No clear effect |
| IDAC vs SAC | IDAC · Humanoid | max average return: 5055.64 -> 5233.43 | Improved |
| replace Gaussian policy with semi-implicit policy (ablation) | IDAC-Gaussian · HalfCheetah, Walker2d, Ant | training curves | Improved |
| use implicit policy without tractable entropy instead of semi-implicit (ablation) | IDAC · HalfCheetah, Walker2d, Ant | training curves | Hurt |
| replace action-value critic with distributional DGN critic (ablation) | SAC (IDAC-Gaussian vs SAC) · HalfCheetah, Walker2d, Ant | training curves | Improved |
| twin-delayed instead of single DGN critic (ablation) | IDAC-Single · HalfCheetah, Walker2d, Ant | training curves | Improved |
| twin-delayed instead of single critic (ablation) | SDPG · HalfCheetah, Walker2d, Ant | training curves | Improved |
| estimate objective with J sampled actions (ablation) | SAC (SAC-J) · HalfCheetah, Walker2d, Ant | training curves | No clear effect |
| make J, K, L too small | IDAC · some tasks | performance | Hurt |
Closest papers by meaning
- Distributions as Actions: A Unified Framework for Diverse Action Spaces (ICLR 2026 · similarity 0.87)
- CTD4 – a Deep Continuous Distributional Actor-Critic Agent with a Kalman Fusion of Multiple Critics (AAAI 2025 · similarity 0.85)
- Deep Implicit Imitation Reinforcement Learning in Heterogeneous Action Settings (AAAI 2025 · similarity 0.84)
- Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning (ICLR 2025 · similarity 0.84)
- ACE: Off-Policy Actor-Critic with Causality-Aware Entropy Regularization (ICML 2024 · similarity 0.84)
- Diverse Projection Ensembles for Distributional Reinforcement Learning (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
