Research map / Learning theory and algorithms / Sample complexity
Maxmin Q-learning: Controlling the Estimation Bias of Q-learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Maxmin Q-learning instead of Q-learning | Tabular Q-learning · Toy MDP, μ = +1 | policy optimality | Helped |
| Use Double Q-learning instead of Q-learning | Tabular Q-learning · Toy MDP, μ = +1 | policy optimality | Hurt |
| Use Double Q-learning instead of Q-learning | Tabular Q-learning · Toy MDP, μ = -1 | speed to optimal policy | Helped |
| Increase the number of estimators N in Maxmin Q-learning (ablation) | Maxmin Q-learning · Toy MDP, μ = -1 | speed to optimal policy | Hurt |
| Use Maxmin Q-learning instead of Q-learning, Double Q-learning, or Averaged Q-learning | Mountain Car · Mountain Car, increasing reward variance | training stability | Helped |
| Use Maxmin DQN instead of the compared algorithms | Maxmin DQN · Pixelcopter | return | Helped |
| Use Maxmin DQN instead of the compared algorithms | Maxmin DQN · Lunarlander | return | Helped |
| Use Maxmin DQN instead of the compared algorithms | Maxmin DQN · Asterix | return | Helped |
| Increase N in Maxmin DQN (ablation) | Maxmin DQN · Pixelcopter | final return | Helped |
| Increase N in Maxmin DQN (ablation) | Maxmin DQN · Asterix | final return | Helped |
| Increase N in Maxmin DQN (ablation) | Maxmin DQN · Pixelcopter | early learning speed | Hurt |
| Increase N in Maxmin Q-learning (ablation) | Maxmin Q-learning under the uniform random noise assumption · Action-value estimation | variance: Var[Q_{s a}^{min}] = \frac{12N^2}{(N+1)^{2}(N+2)} Var[Q_{s a}]; for N \geq 8, Var[Q_{s a}^{min}] < Var[Q_{s a}] | Helped |
Closest papers by meaning
- Self-correcting Q-learning (AAAI 2021 · similarity 0.91)
- Adaptive Ensemble Q-learning: Minimizing Estimation Bias via Error Feedback (NeurIPS 2021 · similarity 0.89)
- Ensemble Bootstrapping for Q-Learning (ICML 2021 · similarity 0.88)
- Regularized Q-learning through Robust Averaging (ICML 2024 · similarity 0.88)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.87)
- Exclusively Penalized Q-learning for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.