Research map / Reinforcement learning / MDPs and planning
Near-Optimal Offline Reinforcement Learning via Double Variance Reduction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Aggregate state-action observations across time steps by using stationary-transition estimators (ablation) | OPDVR · Finite-horizon stationary MDP | sample complexity (episodes): \widetilde{O}(H^3/d_m\epsilon^2) -> \widetilde{O}\left(\frac{H^2}{d_m\epsilon^2}\right) | Helped |
| Use a two-stage doubling procedure rather than a single variance-reduction procedure (ablation) | OPDVR · Finite-horizon non-stationary MDP | sample complexity (episodes): \tilde{O}(H^4/d_m\epsilon^2) -> \tilde{O}(H^3/d_m\epsilon^2) | Helped |
| Use a single variance-reduction procedure when the initial optimality gap exceeds √H (ablation) | OPDVR · Finite-horizon non-stationary MDP | sample complexity (episodes): \tilde{O}(H^3/d_m\epsilon^2) -> \tilde{O}(H^4/d_m\epsilon^2) | Hurt |
| Use non-stationary estimators rather than aggregating observations across time steps (ablation) | OPDVR · Finite-horizon stationary MDP | sample complexity (episodes): \widetilde{O}\left(\frac{H^2}{d_m\epsilon^2}\right) -> \widetilde{O}(H^3/d_m\epsilon^2) | Hurt |
| Apply OPDVR under weak coverage in place of the prior OPEMA guarantee | Offline policy optimization · Finite-horizon non-stationary MDP | sample complexity (episodes): \widetilde{O}(H^3/d_m \epsilon^2) -> \widetilde{O}(H^3/d_m\epsilon^2) | No clear effect |
Closest papers by meaning
- Nearly Horizon-Free Offline Reinforcement Learning (NeurIPS 2021 · similarity 0.93)
- A Primal-Dual Algorithm for Offline Constrained Reinforcement Learning with Linear MDPs (ICML 2024 · similarity 0.92)
- Worst-Case Offline Reinforcement Learning with Arbitrary Data Support (NeurIPS 2024 · similarity 0.91)
- Offline-Online Reinforcement Learning for Linear Mixture MDPs (NeurIPS 2026 · similarity 0.90)
- Fewer May Be Better: Enhancing Offline Reinforcement Learning with Reduced Dataset (ICLR 2025 · similarity 0.90)
- Selective Uncertainty Propagation in Offline RL (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.