Research map / Large language models / Reasoning and chain of thought

How does Chain of Thought decompose complex tasks?

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows classification error scales as a power law in number of classes, models CoT as a tree-structured decomposition, derives an optimal degree e^{d/2} and threshold below which extra thinking hurts, supported by a ViT CIFAR-100 experiment, a synthetic transformer task, and Qwen/DeepSeek reasoning-length experiments.

Problem

Why does chain-of-thought help on some tasks but hurt on others, and how should reasoning length be set? The paper asks when and why decomposing a classification-style prediction into multi-step reasoning reduces error.

Approach

Derives a power-law scaling of classification error with number of classes (m^{2/d} D^{-1/d}) and treats CoT as a tree of classification sub-tasks whose degree determines whether decomposition reduces error; predicts an optimal degree m*=e^{d/2} and a critical degree threshold beyond which an optimal reasoning depth exists.

Key results

• Fine-tuned ViT on CIFAR-100 superclass groupings shows test error following a power law in number of classes m (fit 0.036 m^0.31 + 0.02, implied intrinsic dimension d≈6.45), while random groupings stay roughly constant.
• Transformers trained on the synthetic tree task achieve smallest error when the tree degree is constant across layers (structure = 1), supporting the balanced-tree prediction.
• Test error of Qwen2.5-7B-Instruct on GSM8k and MATH-500, and DeepSeek-V3 on AIME, is minimized at intermediate reasoning lengths, i.e., longer reasoning eventually increases error.

Surprising findings

• Thinking (longer CoT) is detrimental when the tree degree is below a critical threshold, and only beneficial above it, so more reasoning is not uniformly helpful.
• Difficulty of a task, under this theory, is not determined by reasoning depth; the reasoning gain depends only on task size N and intrinsic dimension d.

Limitations

• Experiments on real LLMs (Qwen, DeepSeek) do not definitively prove an optimal reasoning length, as the paper itself acknowledges.
• Theory assumes transition dynamics between reasoning steps are easy to learn; does not cover open-ended or divergent tasks like creative writing.
• Intrinsic dimension is estimated via PCA capturing 80% variance and is a rough proxy; the scaling derivation relies on assumptions such as Lipschitz smoothness and uniform class sizes.

Experimental setup

Datasets: CIFAR-100 GSM8k WikiText-2 MATH-500 AIME 2022-2024 ImageNet-21k

Benchmarks: CIFAR-100 GSM8k MATH-500 AIME 2022-2024 WikiText-2 synthetic logical deduction

Metrics: test error classification error mean entropy of next-token distribution intrinsic dimension (PCA 80% variance)

Baselines: direct prediction without CoT Qwen2.5-7B-Instruct DeepSeek-V3

Builds on: chain-of-thought prompting nucleus sampling neural scaling laws

Closest papers by meaning

• How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding (AAAI 2026 · similarity 0.88)
• A Formal Comparison Between Chain of Thought and Latent Thought (ICML 2026 · similarity 0.87)
• Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis (ICLR 2025 · similarity 0.87)
• When More is Less: Understanding Chain-of-Thought Length in LLMs (ICLR 2026 · similarity 0.87)
• Effective Reasoning Chains Reduce Intrinsic Dimensionality (ICML 2026 · similarity 0.87)
• Bounds of Chain-of-Thought Robustness: Reasoning Steps, Embed Norms, and Beyond (ICLR 2026 · similarity 0.87)