Research map / Probabilistic and scientific modeling / Causal discovery

Since Faithfulness Fails: The Performance Limits of Neural Causal Discovery

ICML 2025 · Probabilistic and scientific modeling › Causal discovery · approach: Constraint-based structure learning with latents · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Systematically benchmarks neural causal discovery methods (DCDI, DiBS, BayesDAG, SDCD) and argues that finite-sample neural likelihood score errors and lambda-strong faithfulness violations fundamentally limit their accuracy, calling for a paradigm shift.

Problem

Neural causal discovery methods are scalable but recover ground-truth causal structures inaccurately; it is unclear whether the bottleneck is estimation error or faithfulness violation in finite samples.

Approach

Introduces a practical data-driven estimate of lambda-strong faithfulness for nonlinear data (threshold on partial Spearman correlation maximizing F1) and links it to the sample size needed and performance of neural score-based and differentiable methods; uses an exhaustive-search NN score evaluator as an upper bound to isolate estimation error.

Key results

• On ER(10,2), DCDI, DiBS and others predict more than half of edges incorrectly; best E-SHD on ER(30,2) is DCDI at 45.9 and on ER(5,1) DiBS at 2.6.
• Across 90 ER(10,2) distributions, average method performance correlates with log lambda-hat (Spearman rho=-0.46, p=4e-6).
• Only about half of 6-node sparse ER distributions are lambda-strong faithful at lambda=0.03 and 20% at lambda=0.1; in a 5-node NN-score study, 800 samples yield up to 398 structures with significantly better scores than the target structure.

Measured effects reported in the paper

Surprising findings

• Performance of neural causal discovery methods does not consistently improve with more observational data (20 to 8,000 samples), with plateaus for DiBS and DCDI.
• Even an exhaustive-search NN score evaluator with the true architecture yields many structures scoring significantly better than the ground-truth structure at moderate sample sizes.

Limitations

• Results limited to Erdos-Renyi graphs and synthetic MLP-generated data; no real-world data.
• Lambda-hat approximation is noisy and only partially consistent (std/lambda-hat up to 0.7 for small values).
• Evaluated only observational data; interventional data and other function approximators not studied.

Experimental setup

Datasets: synthetic Erdos-Renyi graphs with MLP-generated nonlinear SCMs (ER(5,1), ER(10,2), ER(30,2))

Benchmarks: ER(5,1) ER(10,2) ER(30,2)

Metrics: expected SHD between CPDAGs expected F1 between CPDAGs Spearman correlation

Baselines: DCDI DiBS BayesDAG SDCD

Builds on: DCDI DiBS BayesDAG SDCD lambda-strong faithfulness

Closest papers by meaning

• THE ROBUSTNESS OF DIFFERENTIABLE CAUSAL DISCOVERY IN MISSPECIFIED SCENARIOS (ICLR 2025 · similarity 0.88)
• Differentiable Causal Discovery for Latent Hierarchical Causal Models (ICLR 2025 · similarity 0.88)
• CausalTab: Pretraining Across Causal Environments for Tabular Causal Discovery (NeurIPS 2026 · similarity 0.88)
• Causality can systematically address the monsters under the bench(marks) (NeurIPS 2026 · similarity 0.88)
• The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability? (NeurIPS 2025 · similarity 0.88)
• Causal learning with the invariance principle (NeurIPS 2026 · similarity 0.87)