Research map / Probabilistic and scientific modeling / Treatment effects
Prediction-Powered Causal Inferences
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Prediction-Powered Causal Inference (PPCI) and Deconfounded Empirical Risk Minimization (DERM), a reweighting-based training objective that enforces a causal lifting (conditional independence) constraint so a model fine-tuned on a similar annotated experiment yields valid ATE estimates on an unlabeled target experiment (ISTAnt).
Problem
Estimating causal treatment effects from unlabeled experiments whose outcomes must be imputed from complex measurements (e.g., video), using a predictor trained on similar but different annotated experiments; standard ERM and invariance methods can yield invalid causal conclusions.
Approach
Conditional calibration with respect to treatment and adjustment set guarantees valid causal estimation; a causal lifting constraint (encoder representation independent of experimental settings given outcome) transfers this validity across similar experiments, enforced in practice by reweighting training data so outcome and experimental settings are decorrelated (DERM).
Key results
- On CausalMNIST, DERM reduces ATE bias under soft shift (linear effect) to 0.14 ± 0.14 versus 0.86 ± 0.14 for ERM, and under hard shift to 0.75 ± 0.05 versus 1.05 ± 0.15 for ERM.
- On ISTAnt, DERM is the only method giving valid ATE confidence intervals across all five pre-trained encoders, never estimating the wrong sign, while ERM, vREx and IRM are sometimes invalid.
- ERM consistently underestimates or ignores the treatment effect on ISTAnt when fine-tuned on the authors' similar experiment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DERM instead of ERM | CausalMNIST outcome model · CausalMNIST OoD linear effect, soft shift | ATE bias: 0.86 $\pm$ 0.14 -> 0.14 $\pm$ 0.14 | Helped |
| Use DERM instead of ERM | CausalMNIST outcome model · CausalMNIST OoD linear effect, hard shift | ATE bias: 1.05 $\pm$ 0.15 -> 0.75 $\pm$ 0.05 | Helped |
| Use DERM instead of ERM | CausalMNIST outcome model · CausalMNIST OoD non-linear effect, soft shift | ATE bias: 0.64 $\pm$ 0.18 -> 0.08 $\pm$ 0.27 | Helped |
| Use DERM instead of ERM | CausalMNIST outcome model · CausalMNIST OoD non-linear effect, hard shift | ATE bias: 0.85 $\pm$ 0.17 -> 0.45 $\pm$ 0.12 | Helped |
| Use DERM instead of ERM | CausalMNIST outcome model · CausalMNIST in-distribution | ATE bias: 0.00 $\pm$ 0.02 -> 0.10 $\pm$ 0.07 | Hurt |
| Use DERM rather than ERM for fine-tuning | ISTAnt ATE inference · ISTAnt | ATE inference validity | Helped |
Surprising findings
- Vanilla ERM fine-tuning consistently underestimates or ignores the grooming treatment effect on ISTAnt, and invariance baselines (IRM, vREx) are occasionally invalid even when used with standard invariance constraints.
- On CausalMNIST, the predictor trained by ERM is perfectly in-distribution valid yet produces biases of roughly 0.6-1.0 under distribution shift, showing that good in-distribution accuracy does not transfer causal validity.
Limitations
- Causal lifting constraint and conditional calibration cannot be verified on the target population without annotations.
- Finite-sample convergence of the model is not discussed; the representation transfer assumption is idealized.
- Settings with non-overlapping effect modifiers or violated full-support assumptions are not handled; DERM may reduce sample size and still learn spurious correlations in those cases.
Experimental setup
Datasets: ISTAnt own annotated similar experiment (44 videos) CausalMNIST (MNIST-based)
Benchmarks: ISTAnt CausalMNIST
Metrics: ATE estimation bias standard deviation of ATE 95% confidence interval coverage
Baselines: ERM vREx IRM
Builds on: Prediction-Powered Inference Augmented Inverse Propensity Weighting Invariant Risk Minimization Variance Risk Extrapolation
Closest papers by meaning
- Towards Causal Foundation Model: on Duality between Optimal Balancing and Attention (ICML 2024 · similarity 0.87)
- Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption (ICML 2025 · similarity 0.87)
- Prediction-powered Generalization of Causal Inferences (ICML 2024 · similarity 0.87)
- Smoke and Mirrors in Causal Downstream Tasks (NeurIPS 2024 · similarity 0.87)
- ABC3: Active Bayesian Causal Inference with Cohn Criteria in Randomized Experiments (AAAI 2025 · similarity 0.87)
- Proximal Causal Inference With Text Data (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.