AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

Double Gumbel Q-Learning

David Yu-Tung Hui, Aaron Courville, Pierre-Luc Bacon

NeurIPS 2023 · Learning theory and algorithms › Prediction and decision losses · approach: Loss landscape analysis · 0 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add a heteroscedastic logistic noise model (moment-matched hetero-normal) instead of homoscedastic normal for TD errors (ablation)DoubleGum noise model · CartPole-v1 TD-error distributionNLL of TD errorsImproved
Replace homoscedastic Gumbel (XQL assumption) with the heteroscedastic logistic noise fit (ablation)Noise model · CartPole-v1 TD errorsfit to empirical distributionImproved
Moment-match heteroscedastic normal as approximation to heteroscedastic logistic (ablation)DoubleGum loss · CartPole-v1 trainingNLL throughout trainingNo clear effect
Increase pessimism factor c from -0.5 to 0.5 (ablation)DoubleGum continuous control · DMC, MuJoCo, Box2D, MetaWorld tasksaverage magnitude of target Q-valueImproved
Use DoubleGum with default pessimism c = -0.1 versus baselinesDoubleGum · 33 continuous control tasks, default settingsIQM normalized scoreImproved
Tune pessimism factor per suite instead of default c = -0.1 (ablation)DoubleGum · 33 continuous control tasksIQM normalized scoreImproved
Use DoubleGum versus baselines with pessimism tuned per suiteDoubleGum · 33 continuous control tasks, tunedIQM normalized scoreImproved
Use DoubleGum instead of DQN and Dueling DDQNDoubleGum discrete · CartPole-v1, Acrobot-v1, MountainCar-v0IQM of returnNo clear effect
Use Dueling DQN architecture to learn the Q-function in DoubleGumDoubleGum discrete · discrete control taskstraining stabilityImproved
Use GroupNorm with 16 groups without shift and scale instead of LayerNormDoubleGum networks · continuous controlperformanceImproved
Use a scalar pessimism factor instead of five-network ensemble quantile selection (FinerTD3)DoubleGum · 33 continuous control tasks, tuned per suiteIQM normalized scoreImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy