Research map / Probabilistic and scientific modeling / Treatment effects
Inferring the Long-Term Causal Effects of Long-Term Treatments from Short-Term Experiments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Estimates the long-term average treatment effect of a continual (long-term) treatment from short-term experiments by casting it as off-policy evaluation in an MDP, using a doubly-robust efficient influence function estimator with stationary equivalent policies.
Problem
Estimating long-term causal effects of long-term (continual) treatments when only short-term experimental data are available; surrogate methods fail because the treatment directly affects long-term outcomes beyond the surrogate measurement window.
Approach
Reframe the long-term ATE as the difference in discounted outcomes of two policies in an MDP, construct a stationary stochastic policy (treatment probability 1-gamma^T) equivalent in occupancy to a T-duration treatment, and estimate it with a doubly-robust, cross-fitted efficient-influence-function estimator using learned Q and density-ratio functions.
Key results
- On the simple MDP, the ORL estimator matches the true ATE across treatment durations from 1 to infinite periods using only two periods of experimental data, while the surrogate index with two periods underestimates for T>1.
- On the sepsis simulator, ORL estimates for ventilation plus antibiotics increase with treatment duration and track the true ATE, while the surrogate index is accurate only at duration 1.
- Variance of the ORL estimator increases as state coverage in experimental data decreases, noticeable only when under-represented data is downweighted 1:100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ORL instead of the surrogate index method with surrogates observed for two periods | ORL · Simple MDP, treatment durations greater than 1 | ATE estimation accuracy | Helped |
| Measure surrogates through the target treatment duration | Surrogate index method · Simple MDP, varying treatment durations | ATE estimation accuracy | Helped |
| Decrease experimental-state coverage | ORL estimator · Simple MDP, coverage sensitivity | Variance of estimates | Hurt |
| Increase the relevance of an unobserved hidden state | ORL estimator · Simple MDP, hidden-state sensitivity | ATE estimation bias | Hurt |
| Use a state-independent stationary policy approximation instead of the exact state-dependent equivalent policy | ORL · Sepsis simulator, short treatment durations | ATE estimation accuracy | Hurt |
| Estimate treatment effects for shorter treatment durations | ORL · Sepsis simulator | ATE estimation accuracy | Hurt |
| Use the surrogate index method instead of ORL | Sepsis simulator, ventilation and antibiotics · Sepsis simulator, treatment durations with sizeable effects | ATE estimation accuracy | Hurt |
| Vary treatment duration for ventilation | Ventilation treatment · Sepsis simulator | Long-term ATE | No clear effect |
| Measure the surrogate at t=1 rather than through the target treatment duration | Surrogate index method · Sepsis simulator, treatment duration 1 | ATE estimation accuracy | Helped |
Surprising findings
- Surrogate index estimates are accurate only for the one-period treatment duration and biased for longer durations when treatment has a sizeable long-term effect.
- A hidden state that strongly drives reward-relevant transitions introduces severe downward bias in the ORL estimate.
Limitations
- Stationary state-independent policy is only an approximation to the exact equivalent policy and degrades for short treatment durations in the sepsis experiments.
- Requires the state space to be rich enough to satisfy the Markov property; hidden states cause bias and variance.
- Strong overlap assumption over the entire state space; poor coverage inflates variance via the density ratio.
Experimental setup
Datasets: simulated streaming video on demand MDP sepsis simulator
Benchmarks: simulated streaming MDP sepsis simulator
Metrics: estimated average treatment effect vs true ATE confidence interval coverage/width estimator variance
Baselines: surrogate index method naive 1-period extrapolation method
Builds on: surrogate index method minimax density ratio estimation for off-policy evaluation double machine learning
Closest papers by meaning
- Continuous Treatment Effects with Surrogate Outcomes (ICML 2024 · similarity 0.87)
- Estimation of Treatment Effects in Extreme and Unobserved Data (NeurIPS 2025 · similarity 0.86)
- Causal Attribution Analysis for Continuous Outcomes (ICML 2025 · similarity 0.85)
- End-To-End Causal Effect Estimation from Unstructured Natural Language Data (NeurIPS 2024 · similarity 0.85)
- LLM-Driven Treatment Effect Estimation Under Inference Time Text Confounding (NeurIPS 2025 · similarity 0.85)
- Causal Inference in the Closed-Loop: Marginal Structural Models for Sequential Excursion Effects (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.