Research map / Probabilistic and scientific modeling / Treatment effects
Measuring Variable Importance in Heterogeneous Treatment Effects with Confidence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PermuCATE, a conditional-permutation-importance method for global variable importance in CATE estimation, and compares its finite-sample variance and statistical power against LOCO on simulated and IHDP data.
Problem
Measuring which covariates drive heterogeneity of treatment effects (CATE) with statistical guarantees is hard: refitting-based LOCO is costly and high-variance in finite samples, and existing CPI variants were not adapted to CATE.
Approach
Adapts Conditional Permutation Importance to CATE by permuting conditional residuals of each covariate using a covariate predictor, reusing a single fitted CATE estimator rather than refitting per covariate; theory shows its finite-sample bias/variance avoids the CATE estimation error that LOCO incurs.
Key results
- On IHDP, PermuCATE achieved significantly better AUC and statistical power than LOCO for all three CATE learners (Causal Forest, CATENets, TabPFN DR-learner), with both methods controlling type-1 error except LOCO with CATENets.
- In non-linear high-dimensional simulations, PermuCATE showed significantly lower variance than LOCO, sometimes by orders of magnitude, especially at small sample sizes.
- In the low-dimensional LD benchmark, both methods converged to analytical importance and kept false positive rate below 0.05, but PermuCATE needed smaller sample sizes to identify truly important variables.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PermuCATE instead of LOCO | CATE variable-importance estimation · Low Dimensional (LD) | variance | Helped |
| Use PermuCATE instead of LOCO | CATE variable-importance estimation · Low Dimensional (LD) | statistical power | Helped |
| Use PermuCATE instead of LOCO | CATE variable-importance estimation · High Dimensional Linear (HL), small sample sizes | variance | Helped |
| Use PermuCATE instead of LOCO | CATE variable-importance estimation with CateNet · High Dimensional Linear (HL) | variance | Helped |
| Use PermuCATE instead of LOCO | CATE variable-importance estimation · High Dimensional Polynomial (HP) | variance | Helped |
| Use PermuCATE instead of LOCO | CATE variable-importance estimation · High Dimensional Polynomial (HP) | statistical power | Helped |
| Use PermuCATE instead of LOCO | CATE variable-importance estimation · High Dimensional Linear (HL), small sample sizes | statistical power | Helped |
| Use PermuCATE instead of LOCO | CATE variable-importance estimation · High Dimensional Linear (HL), increasing dimension | statistical power | Helped |
| Use PermuCATE instead of LOCO | Causal Forest, CATENet, and TabPFN · IHDP | AUC | Helped |
| Use PermuCATE instead of LOCO | Causal Forest, CATENet, and TabPFN · IHDP | statistical power | Helped |
| Use LOCO with CATENet instead of LOCO with Causal Forest | LOCO · IHDP | AUC | Hurt |
| Use LOCO with CATENet | LOCO · IHDP | type-1 error control | Hurt |
Surprising findings
- LOCO combined with CATENets had lower AUC than with Causal Forest despite better PEHE, attributed to optimization noise from Adam.
Limitations
- Validation on real data is limited by lack of ground truth; relies on semi-synthetic IHDP outcomes.
- Assumes covariates are predictable from others (conditional density estimable more easily than CATE); high-dimensional settings with strong correlation lose power.
- Scenarios beyond a few hundred variables not addressed; grouping only in appendix.
Experimental setup
Datasets: LD simulation (Nie & Wager-style DGP) HL high-dimensional linear simulation HP high-dimensional polynomial simulation IHDP
Benchmarks: IHDP LD simulation HL simulation HP simulation
Metrics: PEHE AUC for variable importance statistical power type-1 error variance of importance estimates
Baselines: LOCO Causal Forest CATENets TabPFN-based DR-learner
Builds on: Conditional Permutation Importance LOCO DR-learner model-X knockoffs
Closest papers by meaning
- AutoCATE: End-to-End, Automated Treatment Effect Estimation (ICML 2025 · similarity 0.88)
- Do Contemporary Causal Inference Models Capture Real-World Heterogeneity? Findings from a Large-Scale Benchmark (ICLR 2025 · similarity 0.87)
- A Non-parametric Direct Learning Approach to Heterogeneous Treatment Effect Estimation under Unmeasured Confounding (NeurIPS 2024 · similarity 0.86)
- Data Fusion for Partial Identification of Causal Effects (NeurIPS 2025 · similarity 0.86)
- Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation (ICLR 2024 · similarity 0.86)
- Hierarchical and Density-based Causal Clustering (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.