Research map / Probabilistic and scientific modeling / Treatment effects
Measuring Model Performance in the Presence of an Intervention
Winston Chen, Michael W. Sjoding, Jenna Wiens
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace control-only AUROC with NPW augmented AUROC (reweight treatment data) | standard evaluation · Synthetic data, ATE=0.2 | MAE of AUROC estimate | Improved |
| increase nuisance parameter estimate variance in NPW (ablation) | NPW · Synthetic data | MAE / C-index: v 0.01 -> 1.0 | Hurt |
| use naive augmentation instead of control-only | standard evaluation · Synthetic data, high ATE or high true AUROC | MAE | Mixed |
| use NPW for model ranking | standard evaluation · Synthetic data, varying ATE | C-index | Improved |
| use NPW for model ranking | standard evaluation · AMR-UTI, n<=1000 | C-index | Improved |
| use naive augmentation for model ranking | standard evaluation · AMR-UTI | C-index | Hurt |
| use NPW for hypothesis testing Epic > LACE | standard evaluation · Readmission RCT | sample size for power 0.8: over 1000 -> 200 | Improved |
| use naive augmentation for hypothesis testing | standard evaluation · Readmission RCT | sample size for power 0.8: over 1000 -> close to 500 | Improved |
| increase sample size (ablation) | NPW vs standard · AMR-UTI | C-index gap | No clear effect |
Closest papers by meaning
- Benchmarking Estimators for Natural Experiments: A Novel Dataset and a Doubly Robust Algorithm (NeurIPS 2024 · similarity 0.85)
- Inverse-Variance Weighting for Estimation of Heterogeneous Treatment Effects (ICML 2024 · similarity 0.85)
- Comparing Targeting Strategies for Maximizing Social Welfare with Limited Resources (ICLR 2025 · similarity 0.85)
- Do Contemporary Causal Inference Models Capture Real-World Heterogeneity? Findings from a Large-Scale Benchmark (ICLR 2025 · similarity 0.85)
- On Efficient Estimation of Distributional Treatment Effects under Covariate-Adaptive Randomization (ICML 2025 · similarity 0.84)
- An Analysis of Causal Effect Estimation using Outcome Invariant Data Augmentation (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
