Research map / Large language models / Language, safety and interpretability

Can Large Language Models Infer Causation from Correlation?

ICLR 2024 · Large language models › Language, safety and interpretability · approach: Faithfulness of LLM explanations · 22 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Corr2Cause, a 200K-sample benchmark testing whether LLMs can infer causal relations from correlational statements using causal discovery rules, finding near-random zero-shot LLM performance and poor out-of-distribution generalization of finetuned models.

Problem

Existing causal NLP datasets test causal knowledge from empirical/commonsense sources; it is unclear whether LLMs can perform pure formal causal inference, i.e., deriving valid causal conclusions from correlational statements.

Approach

Formulates correlation-to-causation inference as a validity task: given a set of d-separation-derived correlational statements over N variables and a causal hypothesis, decide whether the hypothesis holds for all DAGs in the Markov equivalence class; data is generated programmatically from all unique DAGs with up to 6 nodes.

Key results

• Zero-shot LLMs perform near random: best is BART MNLI at 33.38 F1, above GPT-4 at 29.08 F1.
• Finetuned RoBERTa-Large MNLI reaches 94.74 F1 on the original test set.
• Under perturbations (paraphrase and variable refactorization), finetuned models drop by up to 39.29 F1 (paraphrase) and up to 62.30 F1 (variable refactorization).

Measured effects reported in the paper

Surprising findings

• Zero-shot BART MNLI (33.38 F1) outperforms GPT-4 (29.08 F1) on the task.
• The best finetuned model, RoBERTa-Large MNLI, is highly sensitive to paraphrasing of hypotheses, with the largest drop among models.

Limitations

• Graphs limited to 2-6 nodes.
• No hidden confounders assumed; FCI-style latent settings are not covered.
• Dataset is synthetic and abstract, with natural-story extension only a small GPT-4-generated case study.

Experimental setup

Datasets: Corr2Cause

Benchmarks: Corr2Cause

Metrics: F1 precision recall accuracy

Baselines: BART MNLI RoBERTa MNLI GPT-4 GPT-3.5 LLaMA-7B Alpaca-7B GPT-3 Davinci

Builds on: PC algorithm d-separation MNLI

Closest papers by meaning

• Unveiling Causal Reasoning in Large Language Models: Reality or Mirage? (NeurIPS 2024 · similarity 0.91)
• Rethinking the Development of Large Language Models from the Causal Perspective: A Legal Text Prediction Case Study (AAAI 2024 · similarity 0.90)
• COLD: Causal reasOning in cLosed Daily activities (NeurIPS 2024 · similarity 0.89)
• Causality can systematically address the monsters under the bench(marks) (NeurIPS 2026 · similarity 0.89)
• Mitigating Hallucinations in Large Language Models via Causal Reasoning (AAAI 2026 · similarity 0.88)
• Scalable Influence and Fact Tracing for Large Language Model Pretraining (ICLR 2025 · similarity 0.88)