Research map / Reinforcement learning / Offline RL
The Role of Coverage in Online Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GOLF with a Bellman-complete value class in an MDP with low coverability | Online RL · Online RL | regret: Reg \leq O ( H \sqrt{\Ccov T \log(\nicefrac{TH |\Fcal|}{\delta}) \log(T) } ) | Helped |
| Apply the coverability result to exogenous block MDPs | GOLF · Exogenous Block MDPs | regret: \Reg\leq O\prn[\big]{ H\sqrt{|\cS||\cA|T \log(\nicefrac{TH |\Fcal|}{\delta})\log(T)} } | Helped |
| Rely only on single-policy concentrability rather than uniform coverage over policies | Online RL · Online RL | sample efficiency | Hurt |
| Use generalized coverability based on Bellman residuals | Online RL · Online RL | trajectories required: \Omega\left( \min\left\{ X, 2^{\Omega(H)}, 2^{\Omega(C)} \right\} \right) | Hurt |
| Measure exploration complexity using average Bellman error | Bellman-Eluder dimension · Online RL with low coverability | complexity: \bedim(\Fcal, \Pi, \varepsilon) = \Omega(\min\{ |\Fcal|, |\Pi| \})=\Omega(d) | Hurt |
| Use the sequential extrapolation coefficient to bound GOLF regret | GOLF · Online RL | regret: \Reg \leq O\big( H \sqrt{\cdimrl(\Fcal,\Pi,T)\cdot{}T\cdot\log(\nicefrac{TH |\Fcal|}{\delta})} \big) | Helped |
| Add decodable rich observations to an MDP | Coverability · MDPs with decodable emissions | coverability: \Ccov(M') \leq \Ccov(M) | No clear effect |
| Add temporally correlated exogenous noise whose evolution is action-independent | Coverability · MDPs with exogenous noise | coverability: \Ccov(M') \leq \Ccov(M) | No clear effect |
Closest papers by meaning
- Scalable Online Exploration via Coverability (ICML 2024 · similarity 0.92)
- On the Complexity of Offline Reinforcement Learning with Q*-Approximation and Partial Coverage (NeurIPS 2026 · similarity 0.91)
- Optimal Single-Policy Sample Complexity and Transient Coverage for Average-Reward Offline RL (NeurIPS 2025 · similarity 0.91)
- Worst-Case Offline Reinforcement Learning with Arbitrary Data Support (NeurIPS 2024 · similarity 0.90)
- Harnessing Density Ratios for Online Reinforcement Learning (ICLR 2024 · similarity 0.89)
- Offline-Online Reinforcement Learning for Linear Mixture MDPs (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.