Research map / Bandits and online learning / RL theory and MDPs
Online Reinforcement Learning with Uncertain Episode Lengths
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use generalized value iteration with the episode-length distribution and an effective horizon instead of fixed-horizon UCB-VI | UCB-VI on Taxi · Taxi, geometric discounting (γ = 0.95 and 0.975) and quasi-hyperbolic discounting (all tested β) | regret | Helped |
| Use generalized value iteration with the episode-length distribution and an effective horizon instead of fixed-horizon UCB-VI | UCB-VI on Taxi · Taxi, polynomial discounting | regret | No clear effect |
| Estimate the unknown episode-length distribution over time | UCB-VI Generalized · Geometric discounting | regret upper bound: \widetilde{O}\left({\sqrt{SAT}}/{(1-\gamma)^{1.5}} \right) | No clear effect |
| Estimate the unknown episode-length distribution over time | UCB-VI Generalized · Polynomial discounting, p ≥ 2 | regret upper bound: \widetilde{O}\left(\sqrt{SA} T^{\frac{p+1}{2p}} \right) versus \widetilde{O}\left( S^{1/2} A^{1/2} T^{\frac{1}{2-1/p}} \ | Hurt |
Closest papers by meaning
- Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Bounds (ICML 2019 · similarity 0.89)
- The Regret of Exploration and the Control of Bad Episodes in Reinforcement Learning (ICML 2023 · similarity 0.89)
- Breaking the Sample Complexity Barrier to Regret-Optimal Model-Free Reinforcement Learning (NeurIPS 2021 · similarity 0.89)
- Online Learning in Unknown Markov Games (ICML 2021 · similarity 0.89)
- Online Episodic Convex Reinforcement Learning (ICML 2025 · similarity 0.88)
- Beyond Value-Function Gaps: Improved Instance-Dependent Regret Bounds for Episodic Reinforcement Learning (NeurIPS 2021 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.