Research map / Large language models / Reasoning and chain of thought

Continuous Chain of Thought Enables Parallel Exploration and Reasoning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes continuous chain-of-thought (CoT2) with simplex-weighted continuous tokens, a continuous supervision method (CSFT) with hindsight top-B superposition targets, and GRPO-based RL via multi-token sampling (MTS) and Dirichlet sampling, validated on MNNS, ProntoQA and ProsQA with small GPT-2 models trained from scratch.

Problem

Discrete autoregressive chain-of-thought commits to single tokens and cannot track multiple reasoning paths in parallel, limiting exploration on search-heavy logical reasoning tasks.

Approach

Feed the model's softmax-weighted mixture of vocabulary embeddings as continuous thoughts, supervise intermediate steps with soft targets that superpose the states of a budget-B set of top trajectories, and apply GRPO in continuous action space by sampling K discrete tokens and averaging them (MTS) or sampling from a Dirichlet over the simplex.

Key results

• On MNNS (4 digits, 1-9), CoT2 with full budget reaches 98.94% validation accuracy versus 84.92% for discrete CoT and 92.58% for COCONUT.
• On ProsQA, GRPO with CoT2-MTS (K=12) raises CoT2 from 93.37% to 94.21%, while discrete CoT rises from 68.50% to 72.76%.
• On MNNS with 1-layer 1-head GPT-2 at d=24, CoT2-MTS GRPO with K=3 raises validation accuracy from 39.76% to 52.60%.

Surprising findings

• Discrete CoT benefits more from RL than CoT2 on ProsQA and ProntoQA, suggesting CSFT already provides implicit exploration.
• Scaling logits by 1/K instead of the geometric-mean policy ratio causes a distribution shift at inference and degrades performance.

Limitations

• Main experiments use small GPT-2 models trained from scratch on synthetic tasks (MNNS, ProntoQA, ProsQA)
• GSM8K results with Qwen3-0.6B are preliminary and single-run
• Gains on MNNS from RL are small because CSFT targets already align closely with the task

Experimental setup

Datasets: MNNS (synthetic) ProntoQA ProsQA GSM8K

Benchmarks: MNNS ProsQA ProntoQA GSM8K

Metrics: validation accuracy Pass@k Maj@k token-level entropy

Baselines: Discrete CoT COCONUT Discrete no-CoT Self-consistency (Maj@k / Pass@k)

Builds on: COCONUT GRPO CCoT CODI

Closest papers by meaning

• Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought (NeurIPS 2025 · similarity 0.91)
• Metastable Dynamics of Chain-of-Thought Reasoning: Provable Benefits of Search, RL and Distillation (ICML 2025 · similarity 0.91)
• A Formal Comparison Between Chain of Thought and Latent Thought (ICML 2026 · similarity 0.91)
• Emergence of Superposition: Unveiling the Training Dynamics of Chain of Continuous Thought (ICLR 2026 · similarity 0.90)
• Let Me Think! A Long Chain of Thought Can Be Worth Exponentially Many Short Ones (NeurIPS 2025 · similarity 0.90)
• Mind the Gap: Bridging Thought Leap for Improved Chain-of-Thought Tuning (NeurIPS 2025 · similarity 0.90)