Research map / Learning theory and algorithms / Prediction and decision losses
Evaluating and Learning Optimal Dynamic Treatment Regimes under Truncation by Death
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a principal-stratification estimator of the always-survivor value function for multi-stage dynamic treatment regimes under truncation by death, with a multiply robust, locally efficient estimator and off-policy learning, evaluated on simulations and MIMIC-III sepsis data.
Problem
Evaluating and optimizing dynamic treatment regimes when death truncates outcomes, making potential outcomes and the population value function ill-defined.
Approach
Defines the always-survivor value function (latent stratum U=1111) for two-stage DTRs, identifies it from observed data under monotonicity and principal ignorability, and derives its efficient influence function to build a multiply robust estimator and a policy learner based on it.
Key results
- Across simulated scenarios M1-M5 (n=2000) the MR estimator had bias between -0.20 and 0.22 and standard error 0.63 under M1, while misspecified M6 showed bias of -1.74.
- EIF-based 95% confidence intervals for the M1 always-survivor value attained coverage of 94.2% (n=2000) and 95.2% (n=5000).
- MR policy learning achieved average PCD-AS of 0.992 (n=2000) and 0.994 (n=5000), versus 0.987 and 0.988 for AIPW.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the MR estimator under nuisance-model scenarios M1-M5 rather than the misspecification scenario M6 | Always-survivor value estimation · Simulation | Bias: M1-M5: -0.20, -0.10, -0.15, -0.13, 0.22 (n=2000); -0.07, -0.02, -0.04, -0.06, 0.18 (n=5000); M6: -1.74 (n=2000), -1.39 ( | Helped |
| Use MR instead of OR or IPW identification | Always-survivor value estimation · Simulation | S.E.: MR: 0.631 (N=2000), 0.369 (N=5000); OR: 0.475 (N=2000), 0.280 (N=5000); IPW: 2.250 (N=2000), 1.346 (N=5000) | Mixed |
| Learn the policy by maximizing the MR value estimator instead of AIPW | Linear treatment policy · Simulation | PCD-AS: MR: 0.992 and 0.994; AIPW: 0.987 and 0.988 | Helped |
| Learn the policy using MR rather than AIPW | Linear treatment policy · MIMIC-III high-risk sepsis group | distance to true optimum | Helped |
Limitations
- Theory and experiments focus on K=2 decision points; extension to K>=3 is presented without a full proof and is said to be challenging.
- Relies on monotonicity and principal ignorability (mean ignorability) assumptions that are untestable.
- Clinical utility of MIMIC-III-derived sepsis policies is debated, as the paper itself notes.
Experimental setup
Datasets: MIMIC-III v1.4 simulated trajectories with monotone censoring and truncation by death
Benchmarks: MIMIC-III sepsis (high-risk, baseline SOFA>8) policy learning
Metrics: bias standard error 95% confidence interval coverage percentage of correct decisions on always-survivors (PCD-AS)
Baselines: augmented inverse probability weighting (AIPW) treating death as censoring outcome regression (OR) inverse probability weighting (IPW) principal Q-learning
Builds on: principal stratification augmented inverse probability weighting cross-fitting differential evolution
Closest papers by meaning
- SAFER: A Calibrated Risk-Aware Multimodal Recommendation Model for Dynamic Treatment Regimes (ICML 2025 · similarity 0.87)
- Position: Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination (ICML 2024 · similarity 0.86)
- Conservative Continuous-Time Treatment Optimization (NeurIPS 2026 · similarity 0.84)
- EpiCare: A Reinforcement Learning Benchmark for Dynamic Treatment Regimes (NeurIPS 2024 · similarity 0.84)
- Assessing the robustness of heterogeneous treatment effects in survival analysis under informative censoring (NeurIPS 2026 · similarity 0.83)
- Offline Guarded Safe Reinforcement Learning for Medical Treatment Optimization Strategies (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.