Research map / Large language models / Agents, code and math
LLMs Struggle to Balance Reasoning and World Knowledge in Causal Narrative Understanding
Khurram Yamin, Shantanu Gupta, Gaurav Rohit Ghosal, Zachary Chase Lipton, Bryan Wilder
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| extract explicit causal graph then answer, instead of standard QA | GPT-4o · CauseNet semi-synthetic, parametric conflict | accuracy (%): 67.2 -> 98.7 | Improved |
| extract explicit causal graph then answer, instead of standard QA | GPT-4o · CauseNet real-world, parametric conflict | accuracy (%): 52.1 -> 93.2 | Improved |
| add chain-of-thought to standard prompting | GPT-4o · CauseNet semi-synthetic, parametric conflict | accuracy (%): 67.2 -> 73.1 | Improved |
| add chain-of-thought to standard prompting | GPT-4o · CauseNet real-world, parametric conflict | accuracy (%): 52.1 -> 57.6 | Improved |
| add chain-of-thought to standard prompting | GPT-4o · CauseNet real-world, no conflict | accuracy (%): 90.9 -> 89.2 | No clear effect |
| narrative conflicts with parametric knowledge instead of agreeing | GPT-4o standard prompting · CauseNet real-world | accuracy (%): 90.9 -> 52.1 | Hurt |
| narrative conflicts with parametric knowledge instead of agreeing | GPT-4o standard prompting · CauseNet semi-synthetic | accuracy (%): 99.8 -> 67.2 | Hurt |
| narrate events in reverse topological order instead of forward | GPT-4o standard, CoT, in-context · synthetic chains | accuracy (%): ~50 % better | Hurt |
| provide narrative together with extracted graph instead of graph alone | GPT-4o · synthetic reverse narratives | accuracy (%) | Hurt |
| complex graphs with forks and colliders instead of simple chains | GPT-4o · synthetic and CauseNet narratives | accuracy (%) | Hurt |
Closest papers by meaning
- Unveiling Causal Reasoning in Large Language Models: Reality or Mirage? (NeurIPS 2024 · similarity 0.92)
- Language Agents Meet Causality -- Bridging LLMs and Causal World Models (ICLR 2025 · similarity 0.92)
- COLD: Causal reasOning in cLosed Daily activities (NeurIPS 2024 · similarity 0.89)
- Nuance Matters: Probing Epistemic Consistency in Causal Reasoning (AAAI 2025 · similarity 0.88)
- ACED-Bench: Evaluating How LLM Agents Acquire and Act on Causal Evidence (NeurIPS 2026 · similarity 0.88)
- Thought Branches: Interpreting LLM Reasoning Requires Resampling (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
