Research map / Probabilistic and scientific modeling / Treatment effects
Improving the Generation and Evaluation of Synthetic Data for Downstream Medical Causal Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes desiderata, metrics (P_alpha/R_beta for covariates, JSD_pi for treatment assignment, U_PEHE for outcomes), and STEAM, a three-step generator that models covariates, propensity, and outcomes separately to produce synthetic data for treatment effect analysis.
Problem
Generic and causal generative models produce synthetic medical data with treatments that is poor for downstream causal inference, and existing evaluation metrics do not distinguish covariates, treatments, and outcomes.
Approach
Factorise the data-generating process as P(X) P(W|X) P(Y|W,X), generating covariates with any generic generator and treatments and outcomes with a logistic regression propensity model and S-learner PO estimators, plus three metrics that separately measure preservation of each factor.
Key results
- STEAM with TabDDPM covariate model raises JSD_pi on ACTG from 0.812 to 0.952 and lowers U_PEHE from 1.761 to 0.468 relative to TabDDPM.
- On ACIC, STEAM_TVAE outperforms the causal generative models DCM and ANM in JSD_pi (0.972 vs 0.957 and 0.872) and U_PEHE (2.422 vs 4.249 and 4.193), while CGMs score higher P_alpha.
- On simulated data, the STEAM vs TabDDPM gap in JSD_pi and U_PEHE grows with covariate dimensionality, treatment complexity, and outcome heterogeneity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply STEAM instead of direct joint generation with the same generic generator | Generic tabular generators · ACTG | JSDπ: 0.934 -> 0.959 | Helped |
| Apply STEAM instead of direct joint generation with the same generic generator | Generic tabular generators · ACTG | U_PEHE: 0.586 -> 0.436 | Helped |
| Apply STEAM instead of direct joint generation with the same generic generator | Generic tabular generators · IHDP | JSDπ: 0.888 -> 0.928 | Helped |
| Apply STEAM instead of direct joint generation with the same generic generator | Generic tabular generators · IHDP | U_PEHE: 2.521 -> 1.709 | Helped |
| Apply STEAM instead of direct joint generation with the same generic generator | Generic tabular generators · ACIC | JSDπ: 0.929 -> 0.972 | Helped |
| Apply STEAM instead of direct joint generation with the same generic generator | Generic tabular generators · ACIC | U_PEHE: 4.223 -> 2.422 | Helped |
| Apply STEAM instead of direct joint generation with the same generic generator | Generic tabular generators · ACTG | Pα,X: 0.926 -> 0.929 | Mixed |
| Apply STEAM instead of direct joint generation with the same generic generator | Generic tabular generators · ACIC | Pα,X: 0.901 -> 0.900 | Mixed |
| Increase covariate dimensionality | STEAM_TabDDPM vs TabDDPM · Simulated data, varying d | JSDπ | Helped |
| Increase treatment assignment complexity | STEAM_TabDDPM vs TabDDPM · Simulated data, varying K | JSDπ | Helped |
| Increase outcome heterogeneity | STEAM_TabDDPM vs TabDDPM · Simulated data, varying K | U_PEHE | Helped |
| Use the T-Learner rather than the S-Learner for the outcome-generation component | Synthetic outcome mechanism · Simulated outcome-generation experiment | Oracle PEHE: 1.279 -> 0.525 | Helped |
Surprising findings
- Joint-distribution metrics (KS, inverse KL, WD, P_alpha, R_beta) fail to rank synthetic datasets that differ only in outcome or treatment mechanisms, and in some cases give rankings anti-correlated with the oracle.
Limitations
- Assumes the DGP factorises as X, W|X, Y|W,X and that causal assumptions (unconfoundedness, overlap) hold in the real data.
- JSD_pi validity depends on the accuracy of the propensity classifier, and U_PEHE averages over a family of CATE learners rather than selecting one.
- Results are demonstrated on binary treatments only; continuous or multiple treatments are left to future work.
Experimental setup
Datasets: ACTG 175 IHDP ACIC 2016 Jobs simulated data
Benchmarks: ACTG 175 IHDP ACIC 2016 Jobs
Metrics: alpha-precision (P_alpha) beta-recall (R_beta) JSD_pi U_PEHE
Baselines: CTGAN TVAE ARF NFlow TabDDPM ANM DCM
Builds on: TabDDPM S-learner alpha-precision and beta-recall PC algorithm DoWhy-GCM
Closest papers by meaning
- Breaking the Barrier of Hard Samples: A Data-Centric Approach to Synthetic Data for Medical Tasks (ICML 2025 · similarity 0.89)
- Controllable Generative Sandbox for Causal Inference (NeurIPS 2026 · similarity 0.88)
- Coupling Generative Modeling and an Autoencoder with the Causal Bridge (NeurIPS 2025 · similarity 0.88)
- Marginal Causal Flows for Validation and Inference (NeurIPS 2024 · similarity 0.87)
- Generative Intervention Models for Causal Perturbation Modeling (ICML 2025 · similarity 0.87)
- Doubly robust identification of treatment effects from multiple environments (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.