Research map / Reinforcement learning / Policy gradient and actor-critic
Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine state-visitation importance sampling with a value-function correction | Infinite-horizon doubly robust estimator · Taxi | MSE | Helped |
| Combine state-visitation importance sampling with a value-function correction | Infinite-horizon doubly robust estimator · Puck-Mountain | MSE | Helped |
| Combine state-visitation importance sampling with a value-function correction | Infinite-horizon doubly robust estimator · InvertedPendulum-v2 | MSE | Helped |
| Use a more accurate value function in the estimator (ablation) | Infinite-horizon doubly robust estimator · Taxi | bias | Helped |
| Use a more accurate density ratio in the estimator (ablation) | Infinite-horizon doubly robust estimator · Taxi | bias | Helped |
| Increase horizon length while keeping the total number of samples fixed (ablation) | Infinite-horizon doubly robust estimator · Taxi | MSE | Helped |
| Increase horizon length while keeping the total number of samples fixed (ablation) | Finite weighted horizon doubly robust estimator · Puck-Mountain | variance | Hurt |
| Increase horizon length while keeping the total number of samples fixed (ablation) | Infinite-horizon estimators · Puck-Mountain | MSE | No clear effect |
| Use a value function close to the true value function (ablation) | Doubly robust estimator · Infinite-horizon off-policy evaluation | variance | Helped |
Closest papers by meaning
- Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation (NeurIPS 2018 · similarity 0.90)
- Understanding the Curse of Horizon in Off-Policy Evaluation via Conditional Importance Sampling (ICML 2020 · similarity 0.90)
- From Importance Sampling to Doubly Robust Policy Gradient (ICML 2020 · similarity 0.89)
- Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning (ICML 2022 · similarity 0.88)
- On the Curses of Future and History in Future-dependent Value Functions for Off-policy Evaluation (NeurIPS 2024 · similarity 0.85)
- Off-policy Evaluation Beyond Overlap: Sharp Partial Identification Under Smoothness (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.