Research map / Reinforcement learning / MDPs and planning
Understanding the Curse of Horizon in Off-Policy Evaluation via Conditional Importance Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PDIS instead of crude IS | Off-policy evaluation in the two-step MDP · Example 1 | variance: 0.12 -> 0.2448 | Hurt |
| Use SIS instead of PDIS | Off-policy evaluation in the two-step MDP · Example 1 | variance: 0.2448 -> 0.2 | Helped |
| Use PDIS instead of crude IS | Off-policy evaluation in the two-step MDP · Example 2 | variance: 0.5424 -> 0.4528 | Helped |
| Use SIS instead of PDIS | Off-policy evaluation in the two-step MDP · Example 2 | variance: 0.4528 -> 0.52 | Hurt |
| Use PDIS instead of crude IS | Off-policy evaluation in the two-step MDP · Example 3 | variance: 0.2304 -> 0.2688 | Hurt |
| Use SIS instead of PDIS | Off-policy evaluation in the two-step MDP · Example 3 | variance: 0.2688 -> 0.32 | Hurt |
| Use SIS with the stated bounded-ratio conditions in the long-horizon setting | Off-policy evaluation · Asymptotic horizon T | variance growth: from exp(T) to O(T^2) | Helped |
| Use PDIS under the stated sufficient conditions | Off-policy evaluation · Asymptotic horizon T | variance growth: O(T^2) | Helped |
| Replace crude IS with PDIS or SIS without checking covariance conditions | Off-policy evaluation in finite-horizon MDPs · Finite horizon | variance | Mixed |
Closest papers by meaning
- Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation (NeurIPS 2018 · similarity 0.94)
- State Relevance for Off-Policy Evaluation (ICML 2021 · similarity 0.93)
- Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling (NeurIPS 2019 · similarity 0.92)
- Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation (ICML 2025 · similarity 0.89)
- On the Curses of Future and History in Future-dependent Value Functions for Off-policy Evaluation (NeurIPS 2024 · similarity 0.89)
- Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.