Research map / Reinforcement learning / Policy gradient and actor-critic
ConQUR: Mitigating Delusional Bias in Deep Q-Learning
Dijia Su, Jayden Ooi, Tyler Lu, Dale Schuurmans, Craig Boutilier
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ConQUR's consistency-penalized search instead of the pre-trained DQN checkpoint | DQN with frozen representation and retrained final layer · Atari 59 games | average score: 189% | Improved |
| Add a soft consistency penalty to DQN (ablation) | DQN · Atari 19 games | score: DQN with λbest outperforms DQN in all but 3 games | Improved |
| Add a soft consistency penalty to DDQN (ablation) | DDQN · Atari 19 games | score: DDQN with λbest outperforms DDQN in all but 2 games | Improved |
| Set consistency penalty to λ=0.5 (ablation) | DQN and DDQN · Atari 19 games | score: each beat their respective baseline in 11 games | Improved |
| Use consistency penalization instead of unaugmented DQN (ablation) | DQN · Gravitar | score: 475.00 -> 685.76 | Improved |
| Use consistency penalization with λ=0.5 instead of unaugmented DQN (ablation) | DQN · SpaceInvaders | score: 2389.22 -> 2707.83 | Improved |
| Increase the consistency penalty from λ=1 to λ=10 (ablation) | ConQUR · Alien | score | Improved |
| Increase consistency penalty strength (ablation) | ConQUR · Atlantis | score | Hurt |
| Use a very high consistency penalty (λ=100 or λ=1000) rather than λ=10 (ablation) | ConQUR · Alien | score | Hurt |
| Replace rollout-based node scoring with Bellman-error-plus-consistency scoring (ablation) | ConQUR · Atari selected games | score: 33796.30 vs 32618.18; 9914.00 vs 10341.20; 83.34 vs 83.03 | Mixed |
| Increase ConQUR frontier from 8 to 16 nodes (ablation) | ConQUR versus corresponding multi-DQN baseline · Atari games | average improvement over baseline: 19.6% -> 24.5% | Improved |
| Add multiple-hypothesis search and consistency penalty rather than training independent multi-DQNs | DQN · Atari 59 games | score relative to multi-DQN: 16 games at least 10%; 19 games 1--10%; 16 games ±1%; 8 games decline greater than 1% | Mixed |
Closest papers by meaning
- Confounding Robust Deep Reinforcement Learning: A Causal Approach (NeurIPS 2025 · similarity 0.86)
- Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Models (NeurIPS 2025 · similarity 0.83)
- ADDQ: Adaptive distributional double Q-learning (ICML 2025 · similarity 0.82)
- Deep Double Q-learning (NeurIPS 2026 · similarity 0.82)
- BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation (NeurIPS 2026 · similarity 0.82)
- Diffusion Policies Creating a Trust Region for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
