Research map / Bandits and online learning / RL theory and MDPs
UCB Momentum Q-learning: Correcting the bias without forgetting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add momentum to Q-learning updates to correct target bias while retaining samples | UCBMQ · Unknown episodic non-stationary tabular MDP | regret upper bound: \tcO(\sqrt{H^3SAT}+ H^4SA) | Helped |
| Use UCBMQ instead of OptQL | Grid-world, H=100, transition noise ε=0.15 · Grid-world | regret | Helped |
| Use UCBMQ instead of UCBVI or UCBVIgreedy | Grid-world, H=100, transition noise ε=0.15 · Grid-world | regret | Hurt |
| Use momentum-based Q-learning rather than model-based UCBVI | UCBMQ · Unknown episodic non-stationary tabular MDP | second-order regret dependence on states: S^2 -> S | Helped |
Closest papers by meaning
- From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses (ICML 2022 · similarity 0.91)
- Q-learning with UCB Exploration is Sample Efficient for Infinite-Horizon MDP (ICLR 2020 · similarity 0.90)
- Sample Efficient Reinforcement Learning with Partial Dynamics Knowledge (AAAI 2024 · similarity 0.87)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.86)
- DARLING: Detection Augmented Reinforcement Learning with Non-Stationary Guarantees (NeurIPS 2026 · similarity 0.86)
- A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.