AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

LLMs Must Think Thrice to Solve Executable Counterfactuals

Aniket Vashishtha, Qirun Dai, Hongyuan Mei, Amit Sharma, Chenhao Tan, Hao Peng

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies LLM counterfactual reasoning through executable programs covering abduction, intervention, and prediction steps; the abstract is truncated so the proposed method is not stated.

Problem

LLMs need counterfactual reasoning (abduction, intervention, prediction) for causal understanding in high-stakes domains such as science and healthcare; the abstract is truncated, so the specific gap is only partly known.

Approach

Not stated in the provided abstract (truncated). The title suggests that LLMs must reason across the three counterfactual steps, i.e., think thrice, to solve executable counterfactuals.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train with RLVR (GRPO) on counterfactual codeQwen2.5-7B-Instruct · if_else (ID)exact match: 13.9 -> 67.8Improved
train with RLVR (GRPO)Qwen2.5-7B-Instruct · while (OOD)exact match: 3.3 -> 8.1Improved
train with RLVR (GRPO)Qwen2.5-7B-Instruct · multi_r (OOD)exact match: 17.9 -> 36.3Improved
train with RLVR (GRPO)Qwen2.5-7B-Instruct · if_else-long (OOD)exact match: 28.2 -> 58.3Improved
distillation SFT on R1-Distill-Qwen-32B tracesQwen2.5-7B-Instruct · if_else (ID)exact match: 13.9 -> 59.0Improved
distillation SFT on R1-Distill-Qwen-32B tracesQwen2.5-7B-Instruct · while (OOD)exact match: 3.3 -> 2.1Hurt
distillation SFT on R1-Distill-Qwen-32B tracesQwen2.5-7B-Instruct · multi_r (OOD)exact match: 17.9 -> 23.3Mixed
distillation SFT on code tracesQwen2.5-7B-Instruct · GSM counterfactual (code-to-math)accuracy: 39.0 -> 12.9Hurt
RLVR on code tracesQwen2.5-7B-Instruct · GSM counterfactual (code-to-math)accuracy: 39.0 -> 46.3Improved
reveal the latent variable (interventional instead of counterfactual)Qwen2.5-7B-Instruct · GSMaccuracy: 39.0 -> 60.4Improved
scale model size from 7B to 72BQwen2.5-Instruct · while counterfactualexact match: 3.3 -> 9.0Mixed

Limitations

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy