Research map / Reinforcement learning / MDPs and planning
Off-Policy Evaluation for Missingness-Aware Policies in MDPs with Rewards Missing Not at Random
Ziheng Wei, Annie Qu, Rui Miao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Prox bridge-function estimator instead of IPW-FQE | FQE under MNAR rewards · MIMIC-III sepsis, 80% MNAR | absolute bias vs oracle FQE: 9.59 -> 2.66 | Improved |
| use Prox instead of naive FQE | FQE under MNAR rewards · MIMIC-III sepsis, 20% MNAR | absolute bias vs oracle FQE: 1.59 -> 0.05 | Improved |
| use Prox instead of Impute-FQE | FQE under MNAR rewards · MIMIC-III sepsis, 80% MNAR | absolute bias vs oracle FQE: 17.70 -> 2.66 | Improved |
| increase MNAR missing rate from 20% to 80% | Prox · MIMIC-III sepsis | absolute bias vs oracle FQE: 0.05 -> 2.66 | Hurt |
| increase MNAR missing rate from 20% to 80% | IPW-FQE · MIMIC-III sepsis | absolute bias vs oracle FQE: 1.20 -> 9.59 | Hurt |
| replace unobserved rewards with zero in SCOPE | SCOPE · MIMIC-III sepsis | estimated policy value: 10408.2 ± 1991.0 | Hurt |
| increase horizon T up to 32 | Impute-FQE · simulation, ~20% missingness | MSE: MSE exceeding 10^8 at T=32 | Hurt |
| use sigmoid instead of linear reward generation | Prox vs baselines · simulation | MSE: gap narrows under linear reward | Mixed |
Closest papers by meaning
- Abstract Reward Processes: Leveraging State Abstraction for Consistent Off-Policy Evaluation (NeurIPS 2024 · similarity 0.89)
- A General Framework for Off-Policy Learning with Partially-Observed Reward (ICLR 2025 · similarity 0.88)
- Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPs (ICLR 2025 · similarity 0.88)
- Exploiting Action Impact Regularity and Exogenous State Variables for Offline Reinforcement Learning (Abstract Reprint) (AAAI 2024 · similarity 0.88)
- Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation (ICML 2025 · similarity 0.87)
- Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
