Research map / Bandits and online learning / RL theory and MDPs
Efficient Bias-Span-Constrained Exploration-Exploitation in Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Constrain optimistic planning using an upper bound on the optimal bias span | SCAL vs. UCRL · Three-state communicating MDP with large diameter and small bias span | cumulative regret | Helped |
| Use bias-span-constrained exploration rather than unconstrained optimism | SCAL vs. UCRL · Weakly communicating MDP with infinite diameter | regret | Helped |
| Reduce the bias-span constraint c (ablation) | SCAL · Three-state communicating MDP | cumulative regret | Helped |
| Set the transition-probability perturbation η to 0 (ablation) | SCAL · Numerical experiments | convergence of REGOPT | No clear effect |
| Use a constraint c smaller than the true optimal bias span (ablation) | SCAL · MDPs where c < SP(h*_{M*}) | policy performance | Hurt |
| Use the span-constrained algorithm without a valid span bound (ablation) | SCAL · MDPs where c < SP(h*_{M*}) | regret guarantee | Hurt |
Closest papers by meaning
- Near Optimal Exploration-Exploitation in Non-Communicating Markov Decision Processes (NeurIPS 2018 · similarity 0.92)
- Exploration Bonus for Regret Minimization in Discrete and Continuous Average Reward MDPs (NeurIPS 2019 · similarity 0.91)
- Minimax Regret Bounds for Reinforcement Learning (ICML 2017 · similarity 0.90)
- Near-Optimal Randomized Exploration for Tabular Markov Decision Processes (NeurIPS 2022 · similarity 0.90)
- Efficient Exploration in Average-Reward Constrained Reinforcement Learning: Achieving Near-Optimal Regret With Posterior Sampling (ICML 2024 · similarity 0.89)
- NeoRL: Efficient Exploration for Nonepisodic RL (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.