Research map / Reinforcement learning / MDPs and planning
Model-Free and Model-Based Policy Evaluation when Causality is Uncertain
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the robust MDP bound instead of NKYB under the paper's respective sensitivity models | Policy evaluation with confounding · toy | policy value lower bound: 0.0436 -> 0.25372 | Helped |
| Use the robust MDP bound instead of NKYB under the paper's respective sensitivity models | Policy evaluation with confounding · ope-graph | policy value lower bound: 0.0280 -> 0.3994 | Helped |
| Use the robust MDP bound instead of NKYB under the paper's respective sensitivity models | Policy evaluation with confounding · ope-mc | policy value lower bound: -64.5040 -> -15.9647 | Helped |
| Use the robust MDP bound instead of NKYB under the paper's respective sensitivity models | Policy evaluation with confounding · ope-gridworld | policy value lower bound: -2.3914 -> -0.4112 | Helped |
| Limit confounding of transitions using Δ | Robust MDP bounds · toy, ope-graph, ope-mc, and ope-gridworld | policy value lower bound | Helped |
| Increase transition-confounding strength Δ | Robust MDP bounds · toy, ope-graph, ope-mc, and ope-gridworld | policy value lower bound | Hurt |
| Use iid confounders rather than persistent confounders | Off-policy policy evaluation · toy, ope-graph, ope-mc, and ope-gridworld | policy value lower bound | Helped |
| Use the model-based robust MDP bound rather than confounded FQE | Policy evaluation with iid confounders · toy, ope-graph, ope-mc, and ope-gridworld | policy value lower bound | Helped |
| Increase the horizon | Robust MDP value iteration · ope-gridworld | robust MDP bound gap: 2.03e-3 -> 2.65e-5; 3.06e-2 -> 2.97e-4 | Mixed |
| Use KZ's infinite-horizon bound instead of the finite-horizon robust MDP bound | Policy evaluation with iid confounders · ope-graph and ope-gridworld | policy value lower bound | Hurt |
Closest papers by meaning
- Two-way Deconfounder for Off-policy Evaluation in Causal Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
- Towards Estimating Bounds on the Effect of Policies under Unobserved Confounding (NeurIPS 2024 · similarity 0.91)
- Off-policy Evaluation Beyond Overlap: Sharp Partial Identification Under Smoothness (ICML 2024 · similarity 0.88)
- Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPs (ICLR 2025 · similarity 0.88)
- Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes (NeurIPS 2024 · similarity 0.88)
- Off-policy estimation with adaptively collected data: the power of online learning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.