Research map / Learning theory and algorithms / Prediction and decision losses
On the Estimation Bias in Double Q-Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add an abstracted dynamic-programming lower bound to double Q-learning targets | Double Q-learning · Atari benchmark tasks | sample efficiency | Helped |
| Add an abstracted dynamic-programming lower bound to double Q-learning targets | Double Q-learning · Atari benchmark tasks | final performance | Helped |
| Add an abstracted dynamic-programming lower bound to clipped double Q-learning targets | Clipped double Q-learning · Atari benchmark tasks | performance | Helped |
| Use clipped double Q-learning instead of double Q-learning within the doubly bounded approach | Doubly bounded Q-learning · Atari benchmark tasks | performance | Helped |
| Use clipped double Q-learning instead of double Q-learning | Deep Q-learning · Atari benchmark tasks | performance | No clear effect |
| Remove the bootstrapping estimator and train only from the abstracted dynamic-programming targets (ablation) | Doubly bounded Q-learning · Atari benchmark tasks | performance | Hurt |
| Add a dynamic-programming lower bound slightly above the non-optimal fixed point | Double Q-learning · Two-state MDP | escape from non-optimal stationary solutions | Helped |
| Add a dynamic-programming lower bound equal to the non-optimal fixed point | Double Q-learning · Two-state MDP | escape from non-optimal stationary solutions | Helped |
| Use double Q-learning without a dynamic-programming lower bound | Double Q-learning · Two-state MDP | convergence to the optimal solution | Hurt |
| Add a deterministic dynamic-programming cutoff to the bootstrapping target | Doubly bounded estimator · Target-value variance experiment | standard deviation of target values | Helped |
| Replace the combined bootstrapping and dynamic-programming target with dynamic-programming targets alone (ablation) | Doubly bounded Q-learning · Atari benchmark tasks | performance | Hurt |
Closest papers by meaning
- Self-correcting Q-learning (AAAI 2021 · similarity 0.91)
- Faster Non-asymptotic Convergence for Double Q-learning (NeurIPS 2021 · similarity 0.88)
- Deep Double Q-learning (NeurIPS 2026 · similarity 0.88)
- Regularized Q-learning through Robust Averaging (ICML 2024 · similarity 0.86)
- ADDQ: Adaptive distributional double Q-learning (ICML 2025 · similarity 0.86)
- Symmetric Q-learning: Reducing Skewness of Bellman Error in Online Reinforcement Learning (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.