Research map / Probabilistic and scientific modeling / Variational and generative sampling
Counterfactual Density Estimation using Kernel Stein Discrepancies
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DR-MKSD, a doubly robust minimum kernel Stein discrepancy estimator for modelling counterfactual distributions given unnormalized parametric density families, with consistency and asymptotic normality results and small simulation studies.
Problem
Modelling counterfactual (potential outcome) distributions as densities known only up to normalizing constants, which existing counterfactual density estimators (f-divergence/Lp projections) cannot handle.
Approach
Extends minimum kernel Stein discrepancy estimators to the counterfactual setting by using an efficient-influence-function (doubly robust) debiased V-statistic built from propensity and kernel conditional mean embedding nuisances, with cross-fitting, so that only the score function is needed and nuisances are fitted once.
Key results
- In the Gaussian experiment, DR-MKSD shows consistency whenever either the IPW or the plug-in nuisance is consistent, even if the other is not.
- Empirical coverage of the 95% confidence intervals is 0.94 for n=200 and 0.96 for n=300.
- For the RBM, minimizing g_n recovers the direction of theta (ratio between theta_1 and theta_2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the doubly robust estimator instead of relying only on the inverse-probability weighting or plug-in estimator | DR-MKSD · Normal counterfactual simulation | consistency | Helped |
| Estimate the counterfactual distribution with DR-MKSD | DR-MKSD · Normal counterfactual simulation, n=200 | confidence interval coverage: 0.94 | Helped |
| Estimate the counterfactual distribution with DR-MKSD | DR-MKSD · Normal counterfactual simulation, n=300 | confidence interval coverage: 0.96 | Helped |
| Minimize the DR-MKSD statistic over the RBM parameters | DR-MKSD · Counterfactual RBM | recovery of parameter direction | Helped |
| Use DR-MKSD to estimate the intractable model parameters | DR-MKSD · Intractable model with a Gaussian counterfactual | approximate normality of parameter estimates | Helped |
Limitations
- g_n is generally non-convex in theta, so global minimization is not guaranteed.
- Consistency of theta_n itself is not established (only of the KSD value), and may fail with non-unique minimizers.
- Evaluating g_n costs O(n^2).
Experimental setup
Datasets: MNIST synthetic Gaussian data synthetic restricted Boltzmann machine data synthetic intractable exponential-family model data
Benchmarks: synthetic normal counterfactual intractable exponential family (RBM) MNIST counterfactual digits
Metrics: mean squared error empirical coverage of 95% confidence intervals normal QQ plot
Baselines: IPW version of estimator plug-in (PI) version of estimator
Builds on: minimum kernel Stein discrepancy estimators kernel Stein discrepancy doubly robust estimation cross-fitting conditional mean embeddings
Closest papers by meaning
- Doubly-Robust Estimation of Counterfactual Policy Mean Embeddings (NeurIPS 2025 · similarity 0.87)
- Learning Counterfactual Outcomes Under Rank Preservation (NeurIPS 2025 · similarity 0.86)
- Conditional Counterfactual Mean Embeddings: Doubly Robust Estimation and Learning Rates (NeurIPS 2026 · similarity 0.86)
- Towards Characterizing Domain Counterfactuals for Invertible Latent Causal Models (ICLR 2024 · similarity 0.85)
- Counterfactual Realizability (ICLR 2025 · similarity 0.85)
- Exogenous Matching: Learning Good Proposals for Tractable Counterfactual Estimation (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.