Research map / Probabilistic and scientific modeling / Causal discovery
Estimating Interventional Distributions with Uncertain Causal Graphs through Meta-Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MACE-TNP, a transformer neural process meta-learned on synthetic Bayesian causal models to directly predict Bayesian model-averaged interventional posteriors, avoiding explicit graph sampling and functional posterior inference.
Problem
Estimating interventional distributions when the causal graph and functional mechanisms are uncertain from observational data; exact Bayesian model averaging over super-exponentially many DAGs and functional posteriors is intractable.
Approach
Meta-train a transformer neural process end-to-end on datasets sampled from a Bayesian causal model so it maps observational data (plus intervention queries) directly to the Bayesian model-averaged posterior interventional distribution, with an alternating sample/node attention encoder and a mixture-of-Gaussians decoder.
Key results
- On three-node experiments, MACE-TNP achieves NLPID 563.9 on GP data vs 628.5 (BCI-GPN) and 632.0 (DECI), and 527.9 on NN data vs 588.0 for DECI.
- On 20-40 node experiments, MACE-TNP achieves NLPID 660.4/653.3/665.8 vs DECI 686.3/675.6/683.0 and DiBS-GP around 700-713.
- On Sachs, MACE-TNP gets NLPID 998.9 ± 104.9 vs DECI 1000.9 ± 133.5 without retraining; in the two-node linear Gaussian case KL to the analytic posterior goes to zero with more data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MACE-TNP instead of the strongest listed baseline | Three-node experiments · Three-node GP and NN test data | NLPID: 563.9 \pm 23.4 vs 628.5 \pm 27.5; 527.9 \pm 19.8 vs 588.0 \pm 23.6 | Helped |
| Use MACE-TNP instead of the strongest listed baseline | Higher-dimensional experiment · 20, 30, and 40 variables | NLPID: 660.4 \pm 5.2 vs 686.3 \pm 6.7; 653.3 \pm 5.7 vs 675.6 \pm 6.1; 665.8 \pm 4.8 vs 683.0 \pm 5.1 | Helped |
| Use the synthetic-trained model on a different functional mechanism | MACE-TNP (GP) · Three-node NN test data | NLPID: 563.9 \pm 23.4 vs 608.3 \pm 17.3 | Hurt |
| Use the synthetic-trained model on a different functional mechanism | MACE-TNP (NN) · Three-node GP test data | NLPID: 563.9 \pm 23.4 vs 678.0 \pm 10.0 | Hurt |
| Train on combined GP+NN data rather than only the mismatched mechanism | MACE-TNP · Three-node NN test data | NLPID: 608.3 \pm 17.3 vs 531.0 \pm 19.4 | Helped |
| Train on combined GP+NN data rather than only the mismatched mechanism | MACE-TNP · Three-node GP test data | NLPID: 678.0 \pm 10.0 vs 583.9 \pm 21.5 | Helped |
| Provide a small number of true interventional samples | MACE-TNP · Two-node non-identifiable linear Gaussian model | KL divergence | Helped |
| Use MHSA + MHCA attention among samples rather than masked-MHSA (ablation) | MACE-TNP · MACE-TNP architecture experiments | NLPID | Helped |
Surprising findings
- MACE-TNP trained on GP mechanisms transfers poorly to NN data (608.3 vs 527.9 in-distribution), showing lack of built-in inductive bias for unseen mechanisms.
- On the Sachs dataset, the advantage over DECI is negligible, with overlapping error bars.
Limitations
- Substantial training-time compute and data required to cover diverse causal mechanisms.
- Generalisation degrades when the test distribution is not covered by the training prior.
- Attention scales quadratically with number of variables and samples.
Experimental setup
Datasets: Synthetic Bayesian causal model datasets (GP and NN functional mechanisms) Sachs proteomics dataset
Benchmarks: Synthetic three-node GP/NN datasets Synthetic 20/30/40-node datasets Sachs proteomics
Metrics: NLPID KL divergence
Baselines: DiBS-GP ARCO-GP BCI-GPN DECI NOGAM+GP
Builds on: Transformer Neural Process Conditional neural processes DiBS DECI
Closest papers by meaning
- A Meta-Learning Approach to Bayesian Causal Discovery (ICLR 2025 · similarity 0.92)
- Sample Efficient Bayesian Learning of Causal Graphs from Interventions (NeurIPS 2024 · similarity 0.90)
- Learning Mixtures of Unknown Causal Interventions (NeurIPS 2024 · similarity 0.89)
- Characterization and Learning of Causal Graphs from Hard Interventions (NeurIPS 2025 · similarity 0.89)
- Intervention and Conditioning in Causal Bayesian Networks (NeurIPS 2024 · similarity 0.88)
- Bayesian Active Learning for Bivariate Causal Discovery (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.