Research map / Large language models / Agents, code and math
Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Argues LLMs only do shallow level-1 causal reasoning, introduces the fresh CausalProbe-2024 Q&A benchmark, and proposes G2-Reasoner (RAG general knowledge plus goal-oriented prompt) that modestly improves causal QA.
Problem
Whether LLMs perform genuine human-like causal reasoning or only reuse causal knowledge memorized from training corpora; existing causal QA benchmarks may be in pretraining data.
Approach
Combines a fresh, post-cutoff news-derived causal Q&A benchmark (CausalProbe-2024) with a training-free framework that retrieves general knowledge via RAG and adds a goal-oriented prompt to steer causal reasoning.
Key results
- LLMs drop sharply on CausalProbe-H (e.g., Claude 3 Opus ~0.69 EM and LLaMA 2 7B ~0.57 EM vanilla) versus COPA (0.99 and 0.75).
- G2-Reasoner gives modest gains over vanilla, e.g., GPT 3.5 turbo on CausalProbe-H rises from 0.670 to 0.690 EM and LLaMA 2 on CausalProbe-E from 0.616 to 0.642.
- Authors state G2-Reasoner cannot significantly outperform baselines given the small (~16 MB) knowledge base.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add general-knowledge retrieval and a goal-oriented prompt (G²-Reasoner) instead of vanilla inference | LLMs · CausalProbe-H | EM: 0.565 -> 0.582 (LLaMA 2); 0.652 -> 0.658 (LLaMA 3); 0.671 -> 0.693 (GPT 3.5); 0.692 -> 0.696 (Claude 3) | Helped |
| Remove the goal-oriented prompt, retaining RAG only (ablation) | G²-Reasoner · CausalProbe-H | EM: 0.582 -> 0.575 (LLaMA 2); 0.658 -> 0.655 (LLaMA 3); 0.693 -> 0.678 (GPT 3.5) | Hurt |
| Use chain-of-thought prompting instead of vanilla inference | LLMs · CausalNet | EM: 0.673 -> 0.666 (LLaMA 2); 0.857 -> 0.767 (LLaMA 3); 0.897 -> 0.874 (GPT 3.5); 0.933 -> 0.910 (Claude 3) | Hurt |
| Evaluate on fresh CausalProbe-E rather than COPA | studied LLMs · EM | EM: COPA: 0.752, 0.937, 0.948, 0.991; C-E: 0.616, 0.715, 0.732, 0.758 | Hurt |
Surprising findings
- Chain-of-thought often performs slightly worse than vanilla on the earlier benchmarks (COPA, e-CARE, CausalNet).
- RAG alone usually fails to reach the vanilla baseline, which the authors take as evidence the goal-oriented prompt is the key component.
Limitations
- G2-Reasoner does not achieve genuine level-2 causal reasoning and gains are limited with a small knowledge base.
- Unseen status of CausalProbe-2024 cannot be fully confirmed; Min-K% Prob only run on open-source models.
- Benchmark generated by GPT-3.5 turbo introduces model, generation, and language/cultural biases; licensed for non-profit use only.
Experimental setup
Datasets: COPA e-CARE CausalNet CausalProbe-2024 general knowledge Q&A dataset (HuggingFace MuskumPillerum/General-Knowledge)
Benchmarks: COPA e-CARE CausalNet CausalProbe-E CausalProbe-H CausalProbe-M
Metrics: exact match partial match average negative log-likelihood (Min-K% Prob)
Baselines: vanilla CoT RAG
Builds on: chain-of-thought retrieval-augmented generation Min-K% Prob CausalQA
Closest papers by meaning
- COLD: Causal reasOning in cLosed Daily activities (NeurIPS 2024 · similarity 0.93)
- Language Agents Meet Causality -- Bridging LLMs and Causal World Models (ICLR 2025 · similarity 0.92)
- Nuance Matters: Probing Epistemic Consistency in Causal Reasoning (AAAI 2025 · similarity 0.91)
- Can Large Language Models Infer Causation from Correlation? (ICLR 2024 · similarity 0.91)
- Mitigating Hallucinations in Large Language Models via Causal Reasoning (AAAI 2026 · similarity 0.90)
- Eliciting Causal Abilities in Large Language Models for Reasoning Tasks (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.