Research map / Bandits and online learning / RL theory and MDPs
Model-Free Reinforcement Learning: from Clipped Pseudo-Regret to Sample Complexity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use clipped pseudo-regret to bound the number of ε-suboptimal steps | UCB-MultiStage · Discounted tabular MDP PAC-RL | sample complexity: \tilde{O}\left(\frac{SA\ln(1/p)}{\epsilon^2(1-\gamma)^{5.5}}\right) | Helped |
| Use two types of stages, with shorter type-I stages and longer type-II stages | UCB-MultiStage · Discounted tabular MDP PAC-RL | sample complexity: \tilde{O}\left(\frac{SA\ln(1/p)}{\epsilon^2(1-\gamma)^{5.5}}\right) | Helped |
| Add reference-advantage decomposition with a Bernstein-type exploration bonus | UCB-MultiStage-Advantage · Discounted tabular MDP PAC-RL for small ε | sample complexity: \tilde{O}\left(\frac{SA\ln(1/p)}{\epsilon^2(1-\gamma)^3}\right) | Helped |
| Increase B to shorten type-I stages | UCB-MultiStage · Discounted tabular MDP PAC-RL | clipped pseudo-regret contribution from value changes: \tilde{O}(SAH^5\ln (1/p)/(\epsilon B)) | Helped |
| Increase B, reducing samples per type-I stage | UCB-MultiStage · Discounted tabular MDP PAC-RL | exploration-bonus contribution to clipped pseudo-regret: \tilde{O}(SAB\ln (1/p)/(\epsilon(1-\gamma)^4)) | Hurt |
Closest papers by meaning
- Near-Optimal Sample Complexity for MDPs via Anchoring (ICML 2025 · similarity 0.92)
- Near-Optimal Time and Sample Complexities for Solving Markov Decision Processes with a Generative Model (NeurIPS 2018 · similarity 0.92)
- Regret-Optimal Model-Free Reinforcement Learning for Discounted MDPs with Short Burn-In Time (NeurIPS 2023 · similarity 0.92)
- Achieving $\tilde{O}(1/\epsilon)$ Sample Complexity for Constrained Markov Decision Process (NeurIPS 2024 · similarity 0.91)
- Optimal Sample Complexity for Average Reward Markov Decision Processes (ICLR 2024 · similarity 0.90)
- Finding good policies in average-reward Markov Decision Processes without prior knowledge (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.