Research map / Reinforcement learning / Offline RL
What are the Statistical Limits of Offline RL with Linear Function Approximation?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Rely only on realizability and strong feature coverage | Offline policy evaluation with linear function approximation · Offline policy evaluation | required samples: \Omega((d / 2)^H) | Hurt |
| Use low distribution shift, with \overline{\Lambda_h} \preceq C_h\Lambda_h | LSPE under realizability · Offline policy evaluation | squared value estimation error: c \cdot \left(\prod_{h = 1}^H C_h\right) \cdot dH^5 \cdot \sqrt{\frac{d \log (d H/ \delta)}{N}} | Helped |
| Assume policy completeness in addition to realizability and feature coverage | LSPE · Offline policy evaluation | required samples: N = \mathrm{poly}(H, d, 1 / \varepsilon, 1 / \lambda_0) | Helped |
| Use data induced by a fixed policy, with stochastic transitions and sparse deterministic rewards | Offline policy evaluation under realizability · Offline policy evaluation | required samples: \Omega((d / 2-1)^{H / 2}) | Hurt |
| Use offline Bellman backups instead of on-policy rollout data | LSVI versus LSPI · Offline policy optimization | sample complexity | Hurt |
| Estimate policy value with LSPE under realizability without low distribution shift | LSPE · Offline policy evaluation | variance | Hurt |
Closest papers by meaning
- On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling and Beyond (NeurIPS 2023 · similarity 0.94)
- Revisiting the Linear-Programming Framework for Offline RL with General Function Approximation (ICML 2023 · similarity 0.94)
- Instabilities of Offline RL with Pre-Trained Neural Representation (ICML 2021 · similarity 0.93)
- On the Complexity of Offline Reinforcement Learning with Q*-Approximation and Partial Coverage (NeurIPS 2026 · similarity 0.91)
- Worst-Case Offline Reinforcement Learning with Arbitrary Data Support (NeurIPS 2024 · similarity 0.91)
- Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.