Research map / Learning theory and algorithms / Prediction and decision losses
Double Gumbel Q-Learning
David Yu-Tung Hui, Aaron Courville, Pierre-Luc Bacon
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a heteroscedastic logistic noise model (moment-matched hetero-normal) instead of homoscedastic normal for TD errors (ablation) | DoubleGum noise model · CartPole-v1 TD-error distribution | NLL of TD errors | Improved |
| Replace homoscedastic Gumbel (XQL assumption) with the heteroscedastic logistic noise fit (ablation) | Noise model · CartPole-v1 TD errors | fit to empirical distribution | Improved |
| Moment-match heteroscedastic normal as approximation to heteroscedastic logistic (ablation) | DoubleGum loss · CartPole-v1 training | NLL throughout training | No clear effect |
| Increase pessimism factor c from -0.5 to 0.5 (ablation) | DoubleGum continuous control · DMC, MuJoCo, Box2D, MetaWorld tasks | average magnitude of target Q-value | Improved |
| Use DoubleGum with default pessimism c = -0.1 versus baselines | DoubleGum · 33 continuous control tasks, default settings | IQM normalized score | Improved |
| Tune pessimism factor per suite instead of default c = -0.1 (ablation) | DoubleGum · 33 continuous control tasks | IQM normalized score | Improved |
| Use DoubleGum versus baselines with pessimism tuned per suite | DoubleGum · 33 continuous control tasks, tuned | IQM normalized score | Improved |
| Use DoubleGum instead of DQN and Dueling DDQN | DoubleGum discrete · CartPole-v1, Acrobot-v1, MountainCar-v0 | IQM of return | No clear effect |
| Use Dueling DQN architecture to learn the Q-function in DoubleGum | DoubleGum discrete · discrete control tasks | training stability | Improved |
| Use GroupNorm with 16 groups without shift and scale instead of LayerNorm | DoubleGum networks · continuous control | performance | Improved |
| Use a scalar pessimism factor instead of five-network ensemble quantile selection (FinerTD3) | DoubleGum · 33 continuous control tasks, tuned per suite | IQM normalized score | Improved |
Closest papers by meaning
- Self-correcting Q-learning (AAAI 2021 · similarity 0.85)
- Deep Double Q-learning (NeurIPS 2026 · similarity 0.85)
- The Mean-Squared Error of Double Q-Learning (NeurIPS 2020 · similarity 0.85)
- On the Global Convergence of Fitted Q-Iteration with Two-layer Neural Network Parametrization (ICML 2023 · similarity 0.85)
- QLABGrad: A Hyperparameter-Free and Convergence-Guaranteed Scheme for Deep Learning (AAAI 2024 · similarity 0.83)
- Universal Approximation Theorem of Deep Q-Networks (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
