Research map / Large language models / Reasoning and chain of thought

When More is Less: Understanding Chain-of-Thought Length in LLMs

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that accuracy vs. CoT length is inverted-U shaped with an optimal length that grows with task difficulty and shrinks with model capability (simplicity bias), proves this with a simple theory, and proposes Length-Filtered Vote for inference.

Problem

Whether longer chain-of-thought always improves LLM reasoning; understanding how optimal CoT length depends on task difficulty, model capability, and RL training.

Approach

Combines real-world measurements on Qwen2.5 models, controlled synthetic addition tasks with GPT-2 models of varying depth, and a theory where error accumulation over steps trades off against per-step difficulty, yielding a closed-form optimal CoT length (Lambert W) and an RL simplicity-bias corollary; applies this to training data selection and a length-filtered majority vote.

Key results

• Optimal CoT length shifts from 14 steps for the 1.5B model to 4 steps for the 72B model on MATH Level 5; for a 72B model, the gap between optimal and longest CoT accuracy can reach 40%.
• Length-Filtered Vote consistently outperforms vanilla majority vote on a 100-question GPQA subset across sample counts.
• In synthetic RL with PPO, the GPT-2 model converges toward the optimal length N*=5 with 96% peak accuracy at T=24.

Surprising findings

• Stronger models reach peak accuracy with shorter CoTs, so more capable models prefer fewer, more complex steps (simplicity bias).
• A smaller 6-layer model trained on optimal-length CoT data outperforms a larger 9-layer model trained on mixed-length CoTs.

Limitations

• Estimating the optimal CoT length in real-world settings is difficult; the authors offer only coarse heuristics.
• The theory uses simplified linear error assumptions and does not model cognitive behaviors such as reflection or backtracking explicitly.
• Real-world experiments use a limited number of questions (e.g., 30 per model, 100 for difficulty analysis) and step-count proxies for CoT length.

Experimental setup

Datasets: MATH MMLU STEM LeetCode-2K GPQA synthetic arithmetic

Benchmarks: MATH MMLU STEM GPQA LeetCode-2K

Metrics: accuracy optimal CoT length

Baselines: self-consistency (majority vote) mixed-length CoT training

Builds on: chain-of-thought prompting grpo self-consistency

Closest papers by meaning

• Demystifying Long Chain-of-Thought Reasoning (ICML 2025 · similarity 0.95)
• Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.93)
• Long Chain-of-Thought Reasoning Across Languages (ICLR 2026 · similarity 0.93)
• Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning (NeurIPS 2025 · similarity 0.92)
• Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Models (NeurIPS 2025 · similarity 0.92)
• Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropy (ICLR 2026 · similarity 0.92)