Research map / Reinforcement learning / MDPs and planning
Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MIS instead of trajectory-wise IS | off-policy evaluation · ModelFail | Relative-RMSE | Helped |
| Use MIS instead of the stationary state-distribution method | off-policy evaluation · time-varying MDPs with continuous actions | Relative-RMSE | Helped |
| Use MIS instead of IS or WIS as the horizon increases | off-policy evaluation · time-varying MDPs with continuous actions | Relative-RMSE | Helped |
| Use MIS instead of DM | off-policy evaluation · ModelWin | Relative-RMSE | No clear effect |
| Use a stationary state distribution rather than step-specific marginal state distributions | SSD-IS · time-varying MDPs with continuous actions | Relative-RMSE | Hurt |
Closest papers by meaning
- Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation (ICML 2025 · similarity 0.94)
- Scaling Marginalized Importance Sampling to High-Dimensional State-Spaces via State Abstraction (AAAI 2023 · similarity 0.94)
- Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation (NeurIPS 2018 · similarity 0.93)
- Black-box Off-policy Estimation for Infinite-Horizon Reinforcement Learning (ICLR 2020 · similarity 0.93)
- Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPs (ICLR 2025 · similarity 0.91)
- Kernel Metric Learning for In-Sample Off-Policy Evaluation of Deterministic RL Policies (ICLR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.