Research map / Large language models / Reasoning and chain of thought

State of Thought Enables Endogenous Reasoning

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes State of Thought (SoT), which reads a 4-coordinate dynamics-geometric state from frozen LLM internals and uses a 582-parameter controller to select historical reasoning evidence and decide when to stop, improving accuracy and cutting tokens and latency across 3 LLMs, 16 datasets and 2 VLMs.

Problem

Test-time reasoning in LLMs relies on externally imposed control (fixed reasoning programs or costly search), limiting generalization and efficiency; the model's own internal state is unused to govern reasoning.

Approach

Read out a compact dynamics-geometric state (dispersion, step displacement, directional consistency, predictive entropy) from hidden states of each reasoning sentence, and use a tiny learned controller on this state to select which historical reasoning steps stay in context and when to stop, with the backbone frozen.

Key results

• On Llama-3.1-8B, SoT reaches 58.4 average over S&C and 65.8 on QS-related average across domains, improving the strongest non-SoT baseline by about 10.8 points on average across domains, with the best or tied-best on 13 of 16 datasets.
• Averaged over four domains on Llama-3.1-8B, SoT uses 223.9 tokens and 9.8s latency versus 472.7 tokens and 29.0s for sampling/search methods (52.6% fewer tokens, 66.4% lower latency).
• A 582-parameter controller on frozen backbones yields a selector AUC of 0.779 ± 0.005 across 5 problem-held-out refits; training-free and embedding-only variants retain 38.2% and 36.5% accuracy gains.

Surprising findings

• Removing the stopping operator slightly improves S&C accuracy (54.5 to 57.1) but at a large cost in tokens and latency.
• Trajectory-only SoT-Judge reaches 84.1% average agreement with reference correctness across three API models, while the self-consistency and LLM-as-a-judge baselines drop to 5-17% on long-context reasoning.

Limitations

• Gains are weaker on general understanding and S&C for the trajectory-only judge than standard judges.
• Controller requires white-box access to hidden states for the main method; limited-access variants lose accuracy relative to full SoT (about 17-18%).

Experimental setup

Datasets: GSM8K MATH DROP CommonsenseQA StrategyQA BoolQ MMLU RACE FOLIO ProofWriter BBH-Temporal HumanEval

Benchmarks: GSM8K MATH DROP CommonsenseQA StrategyQA BoolQ MMLU RACE FOLIO ProofWriter BBH-Temporal HumanEval

Metrics: exact match F1 pass@1 generated tokens end-to-end latency agreement with reference correctness

Baselines: Vanilla greedy CoT Plan-and-Solve Self-Refine Self-Consistency Constrained Beam MCTS H2O SnapKV StreamingLLM COCONUT GRPO-SP

Builds on: chain-of-thought self-refine COCONUT SnapKV Monte Carlo Tree Search

Closest papers by meaning

• Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization (ICLR 2026 · similarity 0.90)
• Modeling Hierarchical Thinking in Large Reasoning Models (ICML 2026 · similarity 0.90)
• The Tell-Tale Norm: $\ell_2$ Magnitude as a Signal for Reasoning Dynamics in Large Language Models (ICML 2026 · similarity 0.90)
• A State-Transition Framework for Efficient LLM Reasoning (ICLR 2026 · similarity 0.90)
• Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning (NeurIPS 2025 · similarity 0.89)
• Eliciting Reasoning in Language Models with Cognitive Tools (NeurIPS 2025 · similarity 0.89)