Research map / Reinforcement learning / Policy gradient and actor-critic
Information-Theoretic Considerations in Batch Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Require bounded concentratability between the data distribution and every admissible distribution | Batch value-function approximation · Batch value-function approximation | sample complexity: O\left(\frac{C\ln\frac{|\mathcal{F}|}{\delta}}{\epsilon^2(1-\gamma)^4}\right) | Helped |
| Remove restrictions on MDP dynamics while retaining realizability and completeness | Batch value-function approximation · Worst-case MDP family | sample complexity: cannot be poly(|\mathcal{A}|, \tfrac{1}{1-\gamma}, \ln|\mathcal{F}|, \ln|\mathcal{G}|, \tfrac{1}{\epsilon}, \tfrac{1}{\d | Hurt |
| Choose data as a mixture of admissible distributions in reactive POMDPs | Batch learning in reactive POMDPs · Reactive POMDP | concentratability coefficient: C \le |\mathcal{Z} \times \mathcal{A}| | Helped |
| Use a realizable model class and a model-based learning algorithm | Batch model-based RL · Batch model-based RL | sample complexity: O\left(\frac{C\ln\frac{|\mathcal{M}|}{\delta}}{\epsilon^2(1-\gamma)^4}\right) | Helped |
| Use Bernstein's inequality for least-squares regression | FQI and the minimax algorithm · Exact realizability and completeness | error rate: n^{-1/2} | Helped |
| Use the minimax variant instead of FQI | Batch value-function approximation · Batch value-function approximation | sample complexity: O\left(\frac{C\ln\frac{|\mathcal{F}||\mathcal{G}|}{\delta}}{\epsilon^2(1-\gamma)^4}\right) | No clear effect |
| Require completeness for FQI | FQI · Batch value-function approximation | sample complexity: O\left(\frac{C\ln\frac{|\mathcal{F}|}{\delta}}{\epsilon^2(1-\gamma)^4}\right) | Helped |
| Rely on realizability alone for value-function approximation | Value-function approximation · Batch value-function approximation | sample complexity | Mixed |
Closest papers by meaning
- Batch Value-function Approximation with Only Realizability (ICML 2021 · similarity 0.92)
- Provably Good Batch Off-Policy Reinforcement Learning Without Great Exploration (NeurIPS 2020 · similarity 0.90)
- Exponential Lower Bounds for Batch Reinforcement Learning: Batch RL can be Exponentially Harder than Online RL (ICML 2021 · similarity 0.90)
- Sample-Efficiency in Multi-Batch Reinforcement Learning: The Need for Dimension-Dependent Adaptivity (ICLR 2024 · similarity 0.87)
- Switching the Loss Reduces the Cost in Batch Reinforcement Learning (ICML 2024 · similarity 0.87)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.