Research map / Large language models / Language, safety and interpretability

Compositional Causal Reasoning Evaluation in Language Models

ICML 2025 · Large language models › Language, safety and interpretability · approach: Mechanistic analysis of attention and representations

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes compositional causal reasoning (CCR) evaluation for LMs, using compositional consistency of PNS/ATE over causal graphs with cutpoints, and demonstrates it on a CandyParty math word problem across seven LMs.

Problem

Existing causal reasoning benchmarks do not systematically test whether LMs compose and decompose causal measures consistently across graph paths; current evaluations focus on single-quantity accuracy.

Approach

Defines compositional causal reasoning and compositional consistency (external validity and internal consistency), uses commutative cut trees to enumerate causal paths, and tests PNS composition (product over biconnected components) by treating LM yes/no answers as samples from counterfactual data simulations.

Key results

• Only o1 achieved the valid-consistent profile; all other models were 0% valid on the global quantity PNS_XY with and without CoT.
• For all models except GPT-4o with CoT and o1, mean RAE increased monotonically with causal path length and number of mediators.
• GPT-4o with CoT reached 75.1% near-valid estimates, but its mean mediator-path error was more than 10x higher for six mediators than three.

Measured effects reported in the paper

Surprising findings

• Lower-level local causal estimates can be externally valid while the global composed estimate fails, revealing internal inconsistency that external validity alone misses.
• Errors did not decrease monotonically with model size or Llama version.

Limitations

• Empirical demonstration limited to one illustrative toy task.
• Only the ATE and PNS under monotonicity and linearity assumptions are covered.
• Assumes DAGs with a single root, single leaf, at least one cutpoint, and no unobserved confounders.

Experimental setup

Datasets: CandyParty math word problem

Benchmarks: CCR task on CandyParty (PNS composition)

Metrics: relative absolute error external validity internal consistency percent of estimates within RAE threshold

Baselines: o1 GPT-4o Llama 3.1 Llama 3.1 Math Llama 3 Llama 2 Phi-3-Mini

Builds on: CLadder probabilities of causation Corr2Cause

Closest papers by meaning

• Compositional Reasoning in Language Models under Reinforcement Learning Post-Training (NeurIPS 2026 · similarity 0.90)
• AlgoTrace: Algorithmic Primitives and Compositional Geometry of Reasoning in Language Models (ICML 2026 · similarity 0.88)
• COLD: Causal reasOning in cLosed Daily activities (NeurIPS 2024 · similarity 0.88)
• Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning (NeurIPS 2025 · similarity 0.88)
• Nuance Matters: Probing Epistemic Consistency in Causal Reasoning (AAAI 2025 · similarity 0.88)
• Unveiling Causal Reasoning in Large Language Models: Reality or Mirage? (NeurIPS 2024 · similarity 0.88)