Research map / Large language models / Reasoning and chain of thought
Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a causal PNS (probability of necessity and sufficiency) framework that prunes and reconstructs chain-of-thought steps, then uses the filtered CoTs for in-context learning and SFT to improve efficiency while keeping accuracy.
Problem
Chain-of-thought reasoning often contains redundant steps (overthinking) and may omit necessary ones; existing efficiency and step-selection methods rely on correlation-based metrics rather than causal sufficiency and necessity of steps.
Approach
Applies the counterfactual probability of necessity and sufficiency (PNS) at the step level: a chain is checked for sufficiency (correct answer), and each step is tested for necessity by replacing it with an alternative and rolling out the continuation with an LLM, then pruning steps whose removal does not change the outcome. The retained chains serve as demonstrations for ICL and SFT.
Key results
- On GSM-8k with DeepSeek-V3, Ours-ICL raises accuracy from 97.6% to 99.9% while cutting tokens by about 67%.
- SFT of DeepSeek-R1-Distill-Qwen-1.5B with causal CoTs raises CommonsenseQA accuracy from 37.6% to 47.2% while roughly halving steps (21.4 to 11.4 steps... see table).
- Human evaluation of 50 optimized CoTs found 84% judged both sufficient and necessary.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PNS-optimized CoT exemplars instead of standard CoT | DeepSeek-V3 · GSM-8k | accuracy: 97.6% to 99.9% | Helped |
| Use PNS-optimized CoT exemplars instead of Standard CoT | Qwen-2.5-72B-Instruct · GSM-8k | accuracy: 93.6% to 99.5% | Helped |
| Use PNS-optimized CoT exemplars instead of Standard CoT | DeepSeek-V3 · MATH-500 | accuracy: 93.2% to 96.2% | Helped |
| Use PNS-optimized CoT exemplars instead of Standard CoT | Qwen-2.5-72B-Instruct · MATH-500 | accuracy: 84.0% to 81.2% | Hurt |
| Use PNS-optimized CoT exemplars instead of Standard CoT | Llama-3.1-8B-Instruct · MATH-500 | accuracy: 47.6% to 54.8% | Helped |
| Fine-tune on PNS-pruned rather than raw CoTs | DeepSeek-R1-Distill-Qwen-1.5B · CommonsenseQA | accuracy: 37.6% to 47.2% | Helped |
| Fine-tune on PNS-pruned rather than raw CoTs | DeepScaleR-1.5B-Preview · GSM-8k | accuracy: 86.1% to 86.2% | No clear effect |
| Fine-tune on PNS-pruned CoTs rather than the original model | DeepSeek-R1-Distill-Qwen-1.5B · MATH-500 | accuracy: 76.4% to 78.2% | Helped |
| Fine-tune on PNS-pruned CoTs rather than the original model | Phi-4-mini-reasoning · AIME25 | accuracy: 30.0% to 30.0% | No clear effect |
Surprising findings
- Pruned CoTs sometimes improve accuracy as well as reducing length, e.g., DeepSeek-R1 on AIME rises from 79.2% to 92.6% after optimization.
Limitations
- Performance may decline on highly complex tasks.
- Choosing pruning thresholds and managing PNS estimation cost remain challenges.
- Counterfactual generation quality is not guaranteed.
Experimental setup
Datasets: GSM-8k MATH-500 AIME CommonsenseQA MATH MMLU ZebraLogicBench
Benchmarks: GSM-8k MATH-500 AIME AIME25 CommonsenseQA
Metrics: accuracy token length step count PNS
Baselines: Standard CoT Fast-Solve Reduction Chain-of-Draft Noncausal SFT
Builds on: chain-of-thought prompting probability of necessity and sufficiency Chain-of-Draft
Closest papers by meaning
- Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thought (NeurIPS 2024 · similarity 0.92)
- To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning (ICLR 2025 · similarity 0.91)
- Chain-of-Thought Reasoning Without Prompting (NeurIPS 2024 · similarity 0.91)
- Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns (AAAI 2025 · similarity 0.91)
- Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning (NeurIPS 2025 · similarity 0.91)
- How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding (AAAI 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.