Research map / Reinforcement learning / MDPs and planning
Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models
Michael Oberst, David Sontag
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use weighted importance sampling to evaluate the target policy on training trajectories | Target RL policy · Sepsis simulator, hidden state | reward: 0.61 (-0.42, 0.99) vs -0.27 (-0.59, 0.05) | Mixed |
| Use model-based policy evaluation with the learned MDP | Target RL policy · Sepsis simulator, hidden state | reward: 0.81 (0.57, 0.96) vs -0.27 (-0.59, 0.05) | Mixed |
| Use counterfactual off-policy evaluation with the Gumbel-Max SCM | Target RL policy · Sepsis simulator | reward | No clear effect |
| Hide glucose and diabetes state from the learned dynamics model (ablation) | Target RL policy and learned MDP · Sepsis simulator | True RL Reward: -0.19 (-0.41, 0.00) vs -0.27 (-0.59, 0.05) | No clear effect |
Closest papers by meaning
- Two-way Deconfounder for Off-policy Evaluation in Causal Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPs (ICLR 2025 · similarity 0.87)
- Learning Counterfactual Outcomes Under Rank Preservation (NeurIPS 2025 · similarity 0.87)
- Cyclic Counterfactuals under Shift–Scale Interventions (NeurIPS 2025 · similarity 0.87)
- Counterfactual Effect Decomposition in Multi-Agent Sequential Decision Making (ICML 2025 · similarity 0.87)
- Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
