Research map / Reinforcement learning / MDPs and planning
Finding Counterfactually Optimal Action Sequences in Continuous State Spaces
Stratis Tsirtsis, Manuel Gomez Rodriguez
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use A* guided by a Lipschitz-consistent heuristic and Monte Carlo anchor states | Counterfactual action-sequence search on MIMIC-III sepsis episodes · MIMIC-III patient, T=12, L_h=1.0, k=3 | search effort (effective branching factor): 2.1^{12}\approx7{,}355 leaves; more than 3 million action sequences | Improved |
| Increase the number of Monte Carlo samples M used to generate the anchor set (ablation) | A* counterfactual search · MIMIC-III patients, horizon T=12 | runtime | Improved |
| Increase the location-network Lipschitz constant L_h (ablation) | A* counterfactual search · MIMIC-III patients, horizon T=12 | runtime | Hurt |
| Increase the allowed number of action changes k (ablation) | A* counterfactual search · MIMIC-III patients, horizon T=12 | runtime | Hurt |
| Allow more action changes k (ablation) | Counterfactual action sequences for sepsis treatment · MIMIC-III sepsis patients | average counterfactual improvement | Improved |
| Use the learned counterfactual action sequence instead of the observed treatment sequence | MIMIC-III sepsis patients · MIMIC-III sepsis patients | median counterfactual improvement: 5% | Improved |
| Use the suggested action sequence for a selected patient, with earlier higher IV-fluid dosage and later fluids replaced by vasopressors | A* counterfactual search · MIMIC-III sepsis patient case study | counterfactual improvement: 19.9% | Improved |
Closest papers by meaning
- Counterfactual Explanations in Sequential Decision Making Under Uncertainty (NeurIPS 2021 · similarity 0.90)
- Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models (ICML 2019 · similarity 0.89)
- EpiCare: A Reinforcement Learning Benchmark for Dynamic Treatment Regimes (NeurIPS 2024 · similarity 0.86)
- Variational Counterfactual Intervention Planning to Achieve Target Outcomes (ICML 2025 · similarity 0.86)
- Observable Neural ODEs for Identifiable Causal Forecasting in Continuous Time (NeurIPS 2026 · similarity 0.86)
- Position: Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
