Research map / Reinforcement learning / MDPs and planning
Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Reweight Bellman residuals using estimated conditional value variance | Pessimistic value iteration for linear MDPs · Time-inhomogeneous episodic linear MDPs | suboptimality bound: improves PEVI over O(\sqrt{d}) on the feature dimension; enjoys O(\sqrt{H}) improvement over PEVI in the tabular case | Helped |
| Use the tighter penalty \Gamma_h^I instead of the VAPVI penalty (ablation) | VAPVI · Tabular MDPs under single-policy concentrability | suboptimality bound: enjoys a $S$ state improvement over Theorem~[ref] | Helped |
| Use variance-aware weighting and the stated pessimistic design in deterministic systems | VAPVI · Fully deterministic systems | convergence rate: O(\frac{1}{K}) | Helped |
Closest papers by meaning
- Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning (ICLR 2024 · similarity 0.92)
- Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
- Neural Network Approximation for Pessimistic Offline Reinforcement Learning (AAAI 2024 · similarity 0.90)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.90)
- A Unified Principle of Pessimism for Offline Reinforcement Learning under Model Mismatch (NeurIPS 2024 · similarity 0.90)
- Deterministic Uncertainty Propagation for Improved Model-Based Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.