AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

ConQUR: Mitigating Delusional Bias in Deep Q-Learning

Dijia Su, Jayden Ooi, Tyler Lu, Dale Schuurmans, Craig Boutilier

ICML 2020 · Reinforcement learning › Policy gradient and actor-critic · approach: Regularized scalable deep value learning · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use ConQUR's consistency-penalized search instead of the pre-trained DQN checkpointDQN with frozen representation and retrained final layer · Atari 59 gamesaverage score: 189%Improved
Add a soft consistency penalty to DQN (ablation)DQN · Atari 19 gamesscore: DQN with λbest outperforms DQN in all but 3 gamesImproved
Add a soft consistency penalty to DDQN (ablation)DDQN · Atari 19 gamesscore: DDQN with λbest outperforms DDQN in all but 2 gamesImproved
Set consistency penalty to λ=0.5 (ablation)DQN and DDQN · Atari 19 gamesscore: each beat their respective baseline in 11 gamesImproved
Use consistency penalization instead of unaugmented DQN (ablation)DQN · Gravitarscore: 475.00 -> 685.76Improved
Use consistency penalization with λ=0.5 instead of unaugmented DQN (ablation)DQN · SpaceInvadersscore: 2389.22 -> 2707.83Improved
Increase the consistency penalty from λ=1 to λ=10 (ablation)ConQUR · AlienscoreImproved
Increase consistency penalty strength (ablation)ConQUR · AtlantisscoreHurt
Use a very high consistency penalty (λ=100 or λ=1000) rather than λ=10 (ablation)ConQUR · AlienscoreHurt
Replace rollout-based node scoring with Bellman-error-plus-consistency scoring (ablation)ConQUR · Atari selected gamesscore: 33796.30 vs 32618.18; 9914.00 vs 10341.20; 83.34 vs 83.03Mixed
Increase ConQUR frontier from 8 to 16 nodes (ablation)ConQUR versus corresponding multi-DQN baseline · Atari gamesaverage improvement over baseline: 19.6% -> 24.5%Improved
Add multiple-hypothesis search and consistency penalty rather than training independent multi-DQNsDQN · Atari 59 gamesscore relative to multi-DQN: 16 games at least 10%; 19 games 1--10%; 16 games ±1%; 8 games decline greater than 1%Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy