Research map / Reinforcement learning / Policy gradient and actor-critic
Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use stationary state-visitation density ratios and per-step policy ratios instead of cumulative trajectory importance weights | Stationary density-ratio method · Taxi, average reward | MSE | Helped |
| Use stationary state-visitation density ratios and per-step policy ratios instead of cumulative trajectory importance weights | Stationary density-ratio method · Taxi, discounted reward | MSE | Helped |
| Use stationary state-visitation density ratios and per-step policy ratios instead of cumulative trajectory importance weights | Stationary density-ratio method · Pendulum | MSE | Helped |
| Use stationary state-visitation density ratios and per-step policy ratios instead of cumulative trajectory importance weights | Stationary density-ratio method · SUMO traffic simulator | MSE | Helped |
| Increase trajectory length | Stationary density-ratio method · Taxi, average reward | MSE | Helped |
| Increase trajectory length | Trajectory-wise and step-wise IS/WIS · Taxi | MSE | Hurt |
| Increase discount factor | Stationary density-ratio method · Taxi, discounted reward | MSE | No clear effect |
| Increase discount factor | Trajectory-wise and step-wise IS/WIS · Taxi, discounted reward | MSE | Hurt |
Closest papers by meaning
- Understanding the Curse of Horizon in Off-Policy Evaluation via Conditional Importance Sampling (ICML 2020 · similarity 0.94)
- Black-box Off-policy Estimation for Infinite-Horizon Reinforcement Learning (ICLR 2020 · similarity 0.93)
- Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling (NeurIPS 2019 · similarity 0.93)
- Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation (ICML 2025 · similarity 0.90)
- Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning (NeurIPS 2024 · similarity 0.89)
- Multiple-policy Evaluation via Density Estimation (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.