Research map / Learning theory and algorithms / Prediction and decision losses
Non-delusional Q-learning and value-iteration
Tyler Lu, Dale Schuurmans, Craig Boutilier
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace standard Q-update with policy-consistent backup (PCQL) | Q-learning with linear approximator · Fig.1 toy MDP | policy value: 0.3 -> 0.5 | Improved |
| use PCVI/PCQL instead of classical VI/Q-learning | grid world with random linear features · 4x4 and 5x5 grid world | greedy policy expected value: classical zero value in 9 of 10 (4x4) and 10 of 10 (5x5) trials | Improved |
| PCQL sample backups instead of PCVI full backups | policy-class method · grid world | greedy policy value | No clear effect |
Closest papers by meaning
- ConQUR: Mitigating Delusional Bias in Deep Q-Learning (ICML 2020 · similarity 0.88)
- Greedification Operators for Policy Optimization: Investigating Forward and Reverse KL Divergences (JMLR 2022 · similarity 0.88)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.87)
- Inverting the Bellman Equation: From $Q$-Values to World Models (NeurIPS 2026 · similarity 0.87)
- Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic (ICML 2024 · similarity 0.86)
- Computational Hardness of Reinforcement Learning with Partial $q^{\pi}$-Realizability (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
