Research map / Large language models / Agents, code and math
CauSciBench: Can LLMs Automate Causal Inference in Real-World Scientific Research?
Sawal Acharya, Terry Jingchen Zhang, Andrew Kim, Rahul Babu Shrestha, Xianlin Sun, Pepijn Cobben, Maximilian Mordig, Jacob T. Emmerson, Anahita Haghighat, Furkan Danisman, Yuen Chen, Clijo Jose and 8 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| evaluate on real-world datasets instead of synthetic | LLM causal inference agents · CauSciBench | Method correctness: 69.9% -> 45.8% | Hurt |
| evaluate on real-world datasets instead of synthetic | LLM causal inference agents · CauSciBench | Effect accuracy: 53% -> 14.4% | Hurt |
| select the correct causal method (ablation) | OpenAI-o3 · CauSciBench Real | Effect accuracy: 3.40 -> 25.70 | Improved |
| select the correct causal method (ablation) | GPT-5-mini · CauSciBench Real | Effect accuracy: 4.70 -> 26.63 | Improved |
| use Pass@3 instead of Pass@1 (ablation) | GPT-4o and GPT-4o-mini · CauSciBench | Method selection accuracy: +7.3 pp on average | Improved |
| use a larger model instead of GPT-4o-mini | GPT-4o · CauSciBench Real | Method correctness (Direct): 12.57 -> 51.12 | Improved |
| choose CoT over Direct prompting (ablation) | GPT-4o-mini · QRData-CI | Method correctness: 13.56 -> 48.15 | Improved |
| choose PoT prompting over CoT (ablation) | Grok-4-Fast · CauSciBench Real | Method correctness: 62.51 -> 41.08 | Hurt |
| use CoT or ReAct prompting for coding (ablation) | OpenAI-o3 · CauSciBench Real | Coding success rate: 37.3% (CoT), 22.2% (ReAct) | Hurt |
| choose any prompting strategy (ablation) | all models · CauSciBench | Method and effect accuracy | No clear effect |
Closest papers by meaning
- Can Large Language Models Infer Causation from Correlation? (ICLR 2024 · similarity 0.88)
- Exploiting LLMs for Automatic Hypothesis Assessment via a Logit-Based Calibrated Prior (NeurIPS 2025 · similarity 0.88)
- Causality can systematically address the monsters under the bench(marks) (NeurIPS 2026 · similarity 0.87)
- CausalRivers - Scaling up benchmarking of causal discovery for real-world time-series (ICLR 2025 · similarity 0.87)
- Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference (ICLR 2025 · similarity 0.87)
- ACED-Bench: Evaluating How LLM Agents Acquire and Act on Causal Evidence (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
