Research map / Probabilistic and scientific modeling / Treatment effects
ABC3: Active Bayesian Causal Inference with Cohn Criteria in Randomized Experiments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ABC3, a Gaussian-process-based Bayesian active learning policy that selects which subject and treatment arm to sample in randomized experiments by minimizing integrated posterior variance (Cohn criterion), with theoretical links to group imbalance (MMD) and type 1 error.
Problem
Efficient selection of subjects and treatment assignments in randomized experiments for CATE estimation, without violating causal assumptions (e.g., not using outcomes when querying), while keeping treatment and control groups balanced and controlling type 1 error.
Approach
Shows that minimizing expected PEHE under a Bayesian GP view equals minimizing the integrated posterior variance over the covariate distribution (Cohn criterion), and that this same criterion also upper-bounds the MMD imbalance between arms and the integrated type 1 error, while selection ignores observed outcomes.
Key results
- ABC3 achieves the lowest mean PEHE for most time steps across IHDP, Boston, ACIC, and Lalonde; on Boston, after 20% observed it beats Naive even at full observation.
- ABC3 yields substantially lower MMD between treatment and control groups than Naive on all datasets, with the largest gap on ACIC and Lalonde early on.
- ABC3 achieves a lower type 1 error rate than Naive over time on Boston and Lalonde, while Mackay has the highest error on Lalonde.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ABC3 instead of other evaluated policies for active sampling | CATE estimation with Gaussian process regression · IHDP, Boston, ACIC, Lalonde | PEHE | Helped |
| Use ACE instead of ABC3 for active sampling | CATE estimation · Lalonde, 20-40% observation interval | PEHE | Helped |
| Use Mackay instead of Naive sampling | CATE estimation · IHDP, Boston, ACIC, Lalonde | PEHE | Hurt |
| Use ABC3 instead of Naive sampling | Treatment-control group balance · IHDP, Boston, ACIC, Lalonde | MMD | Helped |
| Use ABC3 instead of Naive sampling | Null-effect randomized experiments · Boston, Lalonde | Type 1 error rate | Helped |
| Use Mackay instead of ABC3 sampling | Null-effect randomized experiments · Lalonde | Type 1 error rate | Hurt |
| Use Leverage instead of ABC3 sampling | Null-effect randomized experiments · Boston | Type 1 error rate | Helped |
| Replace the RBF kernel with the Sine kernel (ablation) | ABC3 · IHDP, ACIC, Boston | PEHE | Hurt |
| Use the Sine kernel with length scale 0.1 instead of the RBF kernel (ablation) | ABC3 · Lalonde | PEHE | Helped |
| Vary kernel among RBF, Matern, and Sine (ablation) | ABC3 · IHDP, ACIC, Boston, Lalonde | PEHE | Mixed |
| Set control-to-treatment observation noise to 1.0:0.1 instead of the equal-noise setting (ablation) | ABC3 · IHDP | PEHE | Helped |
| Set control-to-treatment observation noise to 1.0:0.1 instead of the equal-noise setting (ablation) | ABC3 · Lalonde | PEHE | Hurt |
Surprising findings
- Mackay, which uses the same posterior uncertainty information, underperforms even the Naive random policy in most cases.
- Leverage significantly underperforms other policies on ACIC despite outperforming early on IHDP and late on Lalonde.
Limitations
- Theoretical MMD bound relies on an empirically checked but unproven assumption (epsilon* <= 2 delta*).
- Relies on Gaussian process regression, so scalability to large datasets is limited without extensions; large-data extension only sketched in appendix.
- Performance depends on kernel and noise hyperparameters; extreme noise ratios degrade performance.
Experimental setup
Datasets: IHDP Boston ACIC Lalonde Weather
Benchmarks: IHDP Boston ACIC Lalonde
Metrics: PEHE MMD type 1 error rate computation time
Baselines: Naive Mackay Leverage ACE
Builds on: Cohn active learning Mackay active learning ACE-UCB Gaussian process regression
Closest papers by meaning
- A Meta-Learning Approach to Bayesian Causal Discovery (ICLR 2025 · similarity 0.87)
- Bayesian Active Learning for Bivariate Causal Discovery (ICML 2025 · similarity 0.87)
- Data-Driven Covariate Selection for Nonparametric and Cycle-Agnostic Causal Effect Estimation (NeurIPS 2026 · similarity 0.87)
- Generalized Bayes for Causal Inference (NeurIPS 2026 · similarity 0.87)
- Prediction-Powered Causal Inferences (NeurIPS 2025 · similarity 0.87)
- Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effect (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.