Research map / Large language models / Reasoning and chain of thought

A Formal Comparison Between Chain of Thought and Latent Thought

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Formal complexity-theoretic comparison of chain-of-thought (CoT) and latent thought (Coconut and looped transformers), showing latent thought wins on parallel (TC^k) decision problems while CoT wins on approximate counting and sampling via stochastic decoding, with small experiments on algorithmic tasks.

Problem

Which reasoning paradigm is more expressive—explicit chain-of-thought token generation or latent thought in continuous hidden space (Coconut, looped transformers)—and how do their separations depend on iteration count?

Approach

Shows latent thought with log^k n iterations captures TC^k (parallel depth), while CoT with log^k n steps is bounded by TC^{k-1}; and shows stochastic CoT admits FPRAS for counting/sampling that deterministic latent thought with polynomial iterations cannot match, under FPTAS ≠ FPRAS.

Key results

• On parallelizable tasks, looped TF reaches 99.0% on graph connectivity and 99.5% on arithmetic evaluation with 6 loops, while CoT reaches 88.2% and 48.2% with 32 steps.
• On word problems, looped TF reaches 100% at 6 loops, while CoT reaches 100% only at 32 steps.
• Theory: CoT with log^k n steps is contained in TC^{k-1}, and latent thought with log^k n iterations matches TC^k, giving separations under standard complexity assumptions.

Surprising findings

• Coconut failed (0% accuracy) in the preliminary continuous-thought variant, so the authors used looped TF instead.
• Looped model concentrates probability mass on a smaller subset of colorings, while CoT yields a more uniform distribution.

Limitations

• Non-uniform circuit model with embedding dimension scaling; separations rely on complexity assumptions (TC^{k-1} ≠ NC^k, FPTAS ≠ FPRAS).
• Experiments are small synthetic algorithmic tasks, not realistic downstream reasoning.
• Latency comparison argued qualitatively, not measured.

Experimental setup

Datasets: synthetic algorithmic tasks (word problems, graph connectivity, arithmetic evaluation, edit distance, DNF counting, grap

Benchmarks: word problem graph connectivity arithmetic evaluation edit distance DNF counting graph coloring sampling

Metrics: accuracy relative error TV distance to uniform

Baselines: chain of thought Coconut looped transformer

Builds on: Coconut looped transformer CoT expressivity (Merrill & Sabharwal) Feng et al. CoT for DAGs

Closest papers by meaning

• Let Me Think! A Long Chain of Thought Can Be Worth Exponentially Many Short Ones (NeurIPS 2025 · similarity 0.91)
• Continuous Chain of Thought Enables Parallel Exploration and Reasoning (ICLR 2026 · similarity 0.91)
• Reasoning with Latent Thoughts: On the Power of Looped Transformers (ICLR 2025 · similarity 0.91)
• Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoning (AAAI 2026 · similarity 0.89)
• How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding (AAAI 2026 · similarity 0.89)
• SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought (NeurIPS 2025 · similarity 0.89)