Research map / Probabilistic and scientific modeling / Treatment effects
Constructing Confidence Intervals for Average Treatment Effects from Multiple Datasets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a prediction-powered inference method that combines a small unconfounded observational dataset with a large confounded one to build valid, narrower confidence intervals for the ATE, with an RCT+observational extension.
Problem
Constructing valid confidence intervals for the average treatment effect when combining a small unconfounded observational dataset with a large, possibly confounded observational dataset from different sources (e.g., hospitals).
Approach
Uses prediction-powered inference: a CATE estimator trained on the large dataset gives a measure of fit, and a rectifier built from AIPW influence-function scores on the small unconfounded dataset corrects its bias, yielding asymptotically valid and narrower CIs than using the small dataset alone.
Key results
- On synthetic data, the proposed CIs are on average 49.99%-55.37% narrower than the naive small-dataset AIPW baseline while remaining faithful (covering the oracle ATE).
- On MIMIC-III, the method achieves RMSE 0.057 with CI width 0.023 versus 0.077 for the naive D-only AIPW baseline (about 3.5x narrower).
- On COVID-19 by region, RMSE 7.131 and width 2.341 versus 7.591 and 8.479 for the naive baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PPI instead of AIPW on D only | ATE estimation · MIMIC-III | RMSE: 0.057 -> 0.057 | No clear effect |
| Use PPI instead of AIPW on D only | ATE estimation · MIMIC-III | CI width: 0.077 -> 0.023 | Helped |
| Use PPI instead of AIPW on D only | ATE estimation · COVID-19 (by region) | RMSE: 7.591 -> 7.131 | Helped |
| Use PPI instead of AIPW on D only | ATE estimation · COVID-19 (by region) | CI width: 8.479 -> 2.341 | Helped |
| Use PPI instead of AIPW on D only | ATE estimation · COVID-19 (by ethnicity) | RMSE: 39.970 -> 39.968 | Helped |
| Use PPI instead of AIPW on D only | ATE estimation · COVID-19 (by ethnicity) | CI width: 0.081 -> 0.026 | Helped |
| Use PPI instead of AIPW on D only | ATE estimation · Synthetic data, Scenario 1 | CI width reduction: 49.99% | Helped |
| Use PPI instead of AIPW on D only | ATE estimation · Synthetic data, Scenario 2 | CI width reduction: 55.37% | Helped |
| Use PPI instead of AIPW on D only | ATE estimation · Synthetic data, Scenario 3 | CI width reduction: 55.35% | Helped |
Surprising findings
- AIPW on the large confounded dataset alone yields CIs that rarely cover the oracle ATE, so it is unfaithful despite its very narrow intervals.
Limitations
- Relies on standard causal assumptions (unconfoundedness in the small dataset, overlap, consistency) and nuisance-function convergence rates.
- Gives asymptotic validity only; finite-sample guarantees are limited to an extension with bounded rectifier terms.
- The A-TMLE comparison is unstable due to matrix singularity, so baseline results are noisy.
Experimental setup
Datasets: MIMIC-III COVID-19 Brazil dataset synthetic Gaussian process data
Benchmarks: synthetic Gaussian process data MIMIC-III COVID-19 Brazil (split by region and ethnicity)
Metrics: RMSE CI width coverage/faithfulness
Baselines: AIPW on small dataset only (naive baseline) AIPW on large confounded dataset only A-TMLE
Builds on: prediction-powered inference DR-learner AIPW estimator A-TMLE
Closest papers by meaning
- Asymptotically Optimal and Computationally Efficient Average Treatment Effect Estimation in A/B testing (ICML 2024 · similarity 0.85)
- Meta-Learners for Partially-Identified Treatment Effects Across Multiple Environments (ICML 2024 · similarity 0.85)
- Estimating Heterogeneous Treatment Effects by Combining Weak Instruments and Observational Data (NeurIPS 2024 · similarity 0.85)
- Learning Representations of Instruments for Partial Identification of Treatment Effects (ICML 2025 · similarity 0.85)
- Assessing the robustness of heterogeneous treatment effects in survival analysis under informative censoring (NeurIPS 2026 · similarity 0.84)
- Doubly robust identification of treatment effects from multiple environments (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.