Research map / Bandits and online learning / RL theory and MDPs
From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Q-value posterior quantiles as upper confidence bounds instead of adding reward bonuses | BayesUCBVI · Episodic tabular MDP | regret upper bound: $\tcO(\sqrt{H^3SAT})$ | Helped |
| Use posterior quantiles rather than posterior samples for exploration | BayesUCBVI vs. PSRL · Episodic tabular MDP | regret upper bound: $\tcO(\sqrt{H^3SAT})$ vs. $\tcO(H^2 S \sqrt{A T})$ | Helped |
| Use incremental Bayesian-bootstrap implementation | BayesUCBVI · 5-room grid world | regret | Mixed |
| Use Bayesian-bootstrap optimism with DQN instead of DoubleDQN | DQN · Atari-57 | median human normalized score | Helped |
| Use Bayesian-bootstrap optimism instead of the BootDQN exploration procedure | DQN · Atari-57 | median human normalized score | No clear effect |
Closest papers by meaning
- UCB Momentum Q-learning: Correcting the bias without forgetting (ICML 2021 · similarity 0.91)
- Optimistic Posterior Sampling for Reinforcement Learning with Few Samples and Tight Guarantees (NeurIPS 2022 · similarity 0.90)
- Minimax Optimal Reinforcement Learning with Quasi-Optimism (ICLR 2025 · similarity 0.88)
- Beyond Optimism: Exploration With Partially Observable Rewards (NeurIPS 2024 · similarity 0.88)
- Bayesian Exploration Networks (ICML 2024 · similarity 0.88)
- Achieving Constant Regret in Linear Markov Decision Processes (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.