Research map / Probabilistic and scientific modeling / Treatment effects
Defining Expertise: Applications to Treatment Effect Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Defines predictive and prognostic expertise of decision-makers via entropy of actions, shows expertise affects which treatment effect estimator works best, and proposes an expertise-informed pipeline that picks between Action-predictive and Balancing methods.
Problem
Treatment effect (CATE) estimation from observational data usually assumes only overlap and treats the decision-maker's policy as a nuisance, ignoring that decision-makers are experts whose expertise type could inform method choice.
Approach
Formally defines predictive expertise (actions informed by treatment effect) and prognostic expertise (actions informed by potential outcomes) as normalized conditional entropy reductions of actions; shows high expertise implies low in-context action variability (poor overlap); and uses an estimated expertise ratio to select between action-predictive and balancing representation methods.
Key results
- Action-predictive improves over Baseline as predictive expertise increases, while Balancing degrades relative to Baseline under predictive expertise.
- Under prognostic expertise, Balancing tends to outperform Baseline while Action-predictive degrades.
- Expertise-informed pipeline achieves best-of-both-worlds performance across averaged simulated settings in the main TCGA environment, but fails to do so in the Non-linear TCGA setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use action-predictive representations as predictive expertise increases | DragonNet (Action-predictive) · TCGA simulation, predictive expertise | PEHE | Helped |
| use balancing representations as predictive expertise increases | CFRNet (Balancing) · TCGA simulation, predictive expertise | PEHE | Hurt |
| use propensity weighting as expertise increases from Best→Expert | TARNet with IPW (Propensity) · TCGA simulation, predictive expertise | PEHE | Hurt |
| use action-predictive representations when actions reflect prognostic or irrelevant variables | DragonNet (Action-predictive) · TCGA simulation, prognostic expertise | PEHE | Hurt |
| use balancing representations when actions depend on non-predictive variables, especially irrelevant variables | CFRNet (Balancing) · TCGA simulation, prognostic expertise | PEHE | Helped |
| use balancing representations when the policy depends only on prognostic variables | CFRNet (Balancing) · TCGA simulation, prognostic expertise | PEHE | Hurt |
| select Action-predictive or Balancing based on estimated expertise type | Expertise-informed pipeline · TCGA simulation, datasets generated by the considered policies | PEHE | Helped |
| select Action-predictive or Balancing based on estimated expertise type | Expertise-informed pipeline · Non-linear TCGA | PEHE | Hurt |
Surprising findings
- Balancing representations can hurt performance under predictive expertise but help under purely prognostic expertise.
- Action-predictive representations, not balancing, estimate expertise best across configurations.
Limitations
- Expertise is an oracle measure that must be estimated from potential outcome predictions; estimation relies on histogram discretization of outcomes.
- Experiments are limited to synthetic simulators built on TCGA and News covariates with linear outcomes for most results.
- Expertise-informed selection relies on an ad hoc threshold of 1/2 on the ratio and fails in the non-linear TCGA setting.
Experimental setup
Datasets: TCGA gene expression covariates (simulated outcomes) News dataset (simulated outcomes)
Benchmarks: Simulated TCGA environment Simulated News environment (linear and non-linear outcomes)
Metrics: PEHE PEHE improvement over Baseline estimated predictive and prognostic expertise
Baselines: TARNet (Baseline) TARNet with IPW (Propensity) CFRNet (Balancing) DragonNet (Action-predictive)
Builds on: TARNet CFRNet DragonNet
Closest papers by meaning
- Treatment Effect Estimation for Optimal Decision-Making (NeurIPS 2025 · similarity 0.85)
- PairNet: Training with Observed Pairs to Estimate Individual Treatment Effect (ICML 2024 · similarity 0.85)
- KG-TREAT: Pre-training for Treatment Effect Estimation by Synergizing Patient Data with Knowledge Graphs (AAAI 2024 · similarity 0.85)
- On the Actionability of Outcome Prediction (AAAI 2024 · similarity 0.85)
- Comparing Targeting Strategies for Maximizing Social Welfare with Limited Resources (ICLR 2025 · similarity 0.84)
- Meta-Learners for Partially-Identified Treatment Effects Across Multiple Environments (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.