Research map / Reinforcement learning / Offline RL
On the Sample Complexity of Vanilla Model-Based Offline Reinforcement Learning with Dependent Samples
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use empirical transition frequencies for model-based OPE with a concentration analysis that permits dependent samples | Vanilla model-based OPE · Infinite-horizon discounted OPE with dependent sample paths | sample complexity guarantee: N \geq \tilde{\mathcal{O}} \left( \underset{\beta \in [0,1]}{\min} \ \underset{(s, a) \in D}{\max} \left( \frac{S^{1+2\b | Helped |
| Use vanilla importance sampling instead of the model-based estimator | Infinite-horizon discounted OPE · Infinite-horizon discounted OPE | sample complexity dependence on policy shift: \left(\max_{\substack{s \in \mathcal{S} \setminus s_{\circ} \\ a \in \mathcal{A}}} \frac{\pi^{t}(s, a)}{\pi^{b}(s, a)}\r | Hurt |
| Apply vanilla model-based optimization under uniform coverage | Vanilla model-based optimization · Infinite-horizon discounted off-policy optimization | sample complexity guarantee: \tilde{\mathcal{O}} \left( \underset{\substack{s \in \mathcal{S} \setminus \lbrace s_{\circ} \rbrace \\ a \in \mathcal{A | Helped |
| Boost the estimated probability of the most frequent outcome using the deterministic-favored estimator | Categorical transition probability estimation · Almost-deterministic random variables with low sample counts | mean squared error | Helped |
| Use the deterministic-favored estimator rather than the sample mean when the dominant outcome is sufficiently probable | Categorical transition probability estimation · Almost-deterministic random variables | mean squared error: 1-\nicefrac{N^2}{(N+\sqrt{N})^2} = 0.826 for N = 100 | Helped |
Closest papers by meaning
- Worst-Case Offline Reinforcement Learning with Arbitrary Data Support (NeurIPS 2024 · similarity 0.92)
- On the Complexity of Offline Reinforcement Learning with Q*-Approximation and Partial Coverage (NeurIPS 2026 · similarity 0.92)
- Optimal Single-Policy Sample Complexity and Transient Coverage for Average-Reward Offline RL (NeurIPS 2025 · similarity 0.91)
- A Primal-Dual Algorithm for Offline Constrained Reinforcement Learning with Linear MDPs (ICML 2024 · similarity 0.91)
- Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol (NeurIPS 2025 · similarity 0.91)
- Selective Uncertainty Propagation in Offline RL (AAAI 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.