Research map / Probabilistic and scientific modeling / Treatment effects
Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DRM, a nuisance-free distributionally robust metric (KL ambiguity set) for selecting CATE estimators that are robust to covariate shift and hidden confounding, evaluated on semi-synthetic ACIC2016-based data against 13 baseline selectors.
Problem
Selecting a CATE estimator from observational data without counterfactual outcomes or ground-truth CATE; existing plug-in and pseudo-outcome selectors need nuisance models and do not target robustness to distribution shift.
Approach
Bounds the unobservable counterfactual PEHE terms with a KL-divergence ambiguity set centered on the factual distribution, derives a dual (log-sum-exp) form solved by Newton-Raphson, and sets the radius from covariate-distribution KL divergence, so no outcome, propensity or plug-in models are fit.
Key results
- DRM achieves the lowest regret in Setting A (0.96, 0.84, 1.25 across rho values) and Setting C (1.56, 1.40, 1.26 for m=0.1, 0.5, 0.9), where most baselines degrade.
- In Setting B with xi=2 (strong selection bias), DRM regret is 15.51 versus 5.91 for Plug-S/Plug-PS, so DRM underperforms there.
- DRM's rank correlation (about 0.80) is below Plug-S, Plug-PS, Plug-X and Plug-R (about 0.91-0.96), which the authors attribute to worst-case versus average-performance selection.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DRM instead of the strongest reported baseline selector | CATE estimator selection · Setting A (ρ=0, ρ=0.1, ρ=0.3) | Regret: 1.92 -> 0.96; 2.62 -> 0.84; 1.47 -> 1.25 | Helped |
| Use DRM instead of Plug-S | CATE estimator selection · Setting B (ξ=0) | Regret: 0.22 -> 0.38 | Hurt |
| Use DRM instead of Plug-S | CATE estimator selection · Setting B (ξ=2) | Regret: 5.91 -> 15.51 | Hurt |
| Use DRM instead of Plug-R | CATE estimator selection · Setting C (m=0.1, m=0.5, m=0.9) | Regret: 1.91 -> 1.56; 2.53 -> 1.40; 2.12 -> 1.26 | Helped |
| Use DRM for estimator ranking instead of Plug-R | CATE estimator selection · Settings A, B, and C | Spearman rank correlation: 0.96 -> 0.81; 0.96 -> 0.80; 0.96 -> 0.80; 0.95 -> 0.85; 0.93 -> 0.77; 0.96 -> 0.79; 0.96 -> 0.81; 0.96 -> 0.80 | Hurt |
Surprising findings
- DRM's ranking ability is weaker than several plug-in selectors, since worst-case robustness does not equal average ranking performance.
- The factual selection criterion performs better than DRM when there is no selection bias (xi=0).
Limitations
- Ambiguity radius is set heuristically from approximated covariate KL divergence and may not be optimal.
- Experiments are semi-synthetic with a limited CATE function complexity and a single data source (ACIC2016 covariates).
- Poor selection at strong selection bias (xi=2) and weaker rank correlation than several plug-in selectors.
Experimental setup
Datasets: ACIC2016 covariates (semi-synthetic)
Benchmarks: semi-synthetic ACIC2016-based settings A, B, C
Metrics: Regret Spearman rank correlation PEHE
Baselines: Plug-S Plug-T Plug-PS Plug-X Plug-U Plug-IPW Plug-DR Plug-R Plug-RA Pseudo-DR Pseudo-R Pseudo-IF
Builds on: Rubin causal model R-learner DR-learner X-learner Kullback-Leibler distributionally robust optimization
Closest papers by meaning
- Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation (ICLR 2024 · similarity 0.90)
- A Non-parametric Direct Learning Approach to Heterogeneous Treatment Effect Estimation under Unmeasured Confounding (NeurIPS 2024 · similarity 0.90)
- Treatment Effect Estimation for Optimal Decision-Making (NeurIPS 2025 · similarity 0.89)
- Estimating Heterogeneous Treatment Effects by Combining Weak Instruments and Observational Data (NeurIPS 2024 · similarity 0.89)
- Do Contemporary Causal Inference Models Capture Real-World Heterogeneity? Findings from a Large-Scale Benchmark (ICLR 2025 · similarity 0.88)
- Assessing the robustness of heterogeneous treatment effects in survival analysis under informative censoring (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.