Research map / Reinforcement learning / Policy gradient and actor-critic
Action Candidate Based Clipped Double Q-learning for Discrete and Continuous Action Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use action-candidate clipped double estimation in place of TD3's target estimator | TD3 · Pusher | average reward: -31.8 -> -22.7 ± 0.39 | Helped |
| Use action-candidate clipped double estimation in place of TD3's target estimator | TD3 · Reacher | average reward: -3.6 -> -3.5 ± 0.06 | Helped |
| Use action-candidate clipped double estimation in place of TD3's target estimator | TD3 · Walker2d | average reward: 3530.4 -> 3800.3 ± 130.95 | Helped |
| Use action-candidate clipped double estimation in place of TD3's target estimator | TD3 · Hopper | average reward: 2974.8 -> 2827.2 ± 83.2 | Hurt |
| Use action-candidate clipped double estimation in place of TD3's target estimator | TD3 · Swimmer | average reward: 63.2 -> 116.2 ± 3.63 | Helped |
| Use action-candidate clipped double estimation in place of TD3's target estimator | TD3 · Ant | average reward: 3044.6 -> 4391 ± 205.6 | Helped |
| Use 15% of actions as candidates rather than the full action set (ablation) | AC-CDE · Internet ads multi-armed bandits | bias^2 | Mixed |
| Set candidate count K to 2 rather than K to 3 (ablation) | AC-CDQ · Grid World | underestimation bias | Helped |
| Use AC-CDDQN instead of clipped Double DQN | Clipped Double DQN · Asterix | averaged reward: +36.3% | Helped |
| Use AC-CDDQN instead of clipped Double DQN | Clipped Double DQN · Seaquest | averaged reward: +74.4% | Helped |
| Use AC-CDDQN instead of clipped Double DQN | Clipped Double DQN · Space Invaders | averaged reward: +19.8% | Helped |
| Vary the number of action candidates across K=2, K=3, and K=4 (ablation) | AC-CDDQN · MinAtar games | averaged reward | No clear effect |
Closest papers by meaning
- Finite-Time Analysis for Double Q-learning (NeurIPS 2020 · similarity 0.89)
- Maxmin Q-learning: Controlling the Estimation Bias of Q-learning (ICLR 2020 · similarity 0.88)
- Deep Double Q-learning (NeurIPS 2026 · similarity 0.87)
- Stochastic Q-learning for Large Discrete Action Spaces (ICML 2024 · similarity 0.86)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.85)
- Learning from Suboptimal Data in Continuous Control via Auto-Regressive Soft Q-Network (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.