Research map / Probabilistic and scientific modeling / Treatment effects
Doubly-Robust Estimation of Counterfactual Policy Mean Embeddings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Counterfactual Policy Mean Embedding (CPME) for distributional off-policy evaluation in RKHS, with a plug-in and an efficient-influence-function doubly robust estimator, a DR kernel policy test (DR-KPT), and kernel-herding sampling.
Problem
Estimating the full outcome distribution under a counterfactual target policy from logged bandit-style data, in a form that supports nonparametric testing, confidence intervals and sampling, including for structured outcomes.
Approach
Represent the counterfactual outcome distribution as a kernel mean embedding under stochastic policies, derive its Hilbert-space-valued efficient influence function, and use the resulting doubly robust estimator to build an asymptotically normal kernel two-sample test over policies.
Key results
- DR-KPT-rbf attains 0.02 rejection rate under the null on Warfarin versus 0.00-0.00 for baseline KPTs only in the sense of calibration, and 0.80 in mean-shift scenario II versus 0.77 for KPT-linear.
- On dSprites under the null, KPT baselines reject 37-40% of the time while DR-KPT-rbf rejects 2.4%; in shifted mixture DR-KPT-rbf reaches 0.656 power versus at most 0.081 for baselines.
- The paper claims the DR estimator improves convergence to O(n^-1/2) plus nuisance product term versus O(n^-1/4)-type rate for the plug-in estimator, with asymptotic normality of the test statistic.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DR-KPT-rbf instead of KPT-rbf | Kernel policy test on Warfarin · Scenario II (Mean Shift) | rejection rate: 0.01 -> 0.80 | Helped |
| Use DR-KPT-poly instead of KPT-poly | Kernel policy test on Warfarin · Scenario II (Mean Shift) | rejection rate: 0.29 -> 0.66 | Helped |
| Use DR-KPT-rbf instead of KPT-rbf | Kernel policy test on Warfarin · Scenario III (Mixture) | rejection rate: 0.00 -> 0.99 | Helped |
| Use DR-KPT-poly instead of KPT-poly | Kernel policy test on Warfarin · Scenario III (Mixture) | rejection rate: 0.66 -> 0.95 | Helped |
| Use DR-KPT-rbf instead of KPT-rbf | Kernel policy test on Warfarin · Scenario IV (Shifted Mixture) | rejection rate: 0.00 -> 0.76 | Helped |
| Use DR-KPT-poly instead of KPT-poly | Kernel policy test on Warfarin · Scenario IV (Shifted Mixture) | rejection rate: 0.11 -> 0.55 | Helped |
| Use DR-KPT-rbf instead of KPT-rbf | Kernel policy test on dSprites · Scenario I (Null) | rejection rate: 0.401 -> 0.024 | Helped |
| Use DR-KPT-poly instead of KPT-poly | Kernel policy test on dSprites · Scenario I (Null) | rejection rate: 0.375 -> 0.000 | Helped |
| Use DR-KPT-rbf instead of KPT-rbf | Kernel policy test on dSprites · Scenario IV (Shifted Mixture) | rejection rate: 0.054 -> 0.656 | Helped |
| Use DR-KPT-poly instead of KPT-poly | Kernel policy test on dSprites · Scenario IV (Shifted Mixture) | rejection rate: 0.073 -> 0.502 | Helped |
Surprising findings
- Baseline KPT-rbf has essentially zero rejection in Warfarin mean-shift and mixture scenarios, showing poor power for plug-in kernel tests.
Limitations
- Relies on identification assumptions (exchangeability, positivity) that may fail in practice
- Asymptotic normality requires nuisance rate product condition and non-degeneracy
- Experiments are limited to synthetic, Warfarin and dSprites settings
Experimental setup
Datasets: Warfarin dosing dataset dSprites synthetic simulations
Benchmarks: Warfarin dataset (simulated bandit) dSprites (structured image outcomes) synthetic simulations
Metrics: rejection rate true positive rate false positive rate MMD Wasserstein distance
Baselines: KPT-linear KPT-rbf KPT-poly PT-linear KTE-based kernel test
Builds on: counterfactual mean embedding kernel treatment effects doubly robust estimation kernel herding kernel policy test
Closest papers by meaning
- Conditional Counterfactual Mean Embeddings: Doubly Robust Estimation and Learning Rates (NeurIPS 2026 · similarity 0.91)
- Learning Counterfactual Outcomes Under Rank Preservation (NeurIPS 2025 · similarity 0.89)
- Counterfactual Density Estimation using Kernel Stein Discrepancies (ICLR 2024 · similarity 0.87)
- Exogenous Matching: Learning Good Proposals for Tractable Counterfactual Estimation (NeurIPS 2024 · similarity 0.86)
- Counterfactual Contrastive Learning with Normalizing Flows for Robust Treatment Effect Estimation (ICML 2025 · similarity 0.85)
- Towards Characterizing Domain Counterfactuals for Invertible Latent Causal Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.