Research map / Large language models / Reasoning and chain of thought

A State-Transition Framework for Efficient LLM Reasoning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Models long CoT as a state-transition process by replacing most softmax attention with a linear-attention reasoning-state module (mixed attention) plus a momentum-based state correction, cutting attention cost and KV-cache while keeping accuracy.

Problem

Long chain-of-thought generation is costly in computation and KV-cache memory, and existing CoT compression methods trade away reasoning accuracy.

Approach

Each reasoning step is generated attending only to the prompt and its own step via softmax attention, while a linear-attention state matrix carries compressed information from completed steps; a momentum-averaged global reasoning direction corrects noisy step updates. Attention cost becomes linear and KV-cache constant per step, without shortening the CoT.

Key results

• On Qwen2.5-1.5B, the method averages 56.2 accuracy across five math benchmarks versus 51.5 for the base model, with average reasoning latency 253.0 vs 300.0 (ms-scale per sample as reported).
• On Qwen2.5-7B, average accuracy rises to 68.3 from 65.9 for the base model, with latency 263.3 vs 314.2.
• Ablation removing the model reasoning state (replaced by a sliding window of 64) yields the largest drop, to 46.8 average accuracy.

Surprising findings

• The method improves accuracy over the full-attention base model while also reducing latency, despite replacing most attention with linear attention.
• Removing low-process-reward steps (under 20%) by a cosine-based score improved performance, suggesting the state carries step-quality signal.

Limitations

• Results reported with greedy decoding; gains shrink or vary under sampling and the authors note hyperparameters tuned mainly on 1.5B.
• Reasoning-step segmentation and thinking-type annotation rely on heuristic high-entropy tokens and K-means clustering rather than human or LLM annotation.
• Efficiency gains are reported only for FlashAttention-2 setting at batch size 1 on a single A100.

Experimental setup

Datasets: OpenR1-Math-220K

Benchmarks: GSM8K MATH-500 AMC23 AIME24 AIME25 GPQA_Diamond HumanEval

Metrics: accuracy token number reasoning latency

Baselines: H2O SepLLM LightThinker INFTYTHINK TAMM SBT-D L1-Max AR

Builds on: Qwen2.5 DeepSeek-R1 linear attention test-time training view of linear attention LightThinker INFTYTHINK LoRA

Closest papers by meaning

• Dynamic Early Exit in Reasoning Models (ICLR 2026 · similarity 0.93)
• Training Language Models to Reason Efficiently (NeurIPS 2025 · similarity 0.91)
• Bag of Tricks for Inference-time Computation of LLM Reasoning (NeurIPS 2025 · similarity 0.91)
• Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning (ICML 2025 · similarity 0.91)
• Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning (NeurIPS 2025 · similarity 0.90)
• Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens (ICML 2026 · similarity 0.90)