Research map / Large language models / Reasoning and chain of thought

Controlling Thinking Speed in Reasoning Models

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows reasoning models (DeepSeek-R1-Distill, QwQ, Qwen3) have an intrinsic fast/slow switch and steers thinking speed via a PCA-derived representation vector, plus a logit-divergence difficulty signal that adapts steering online, giving +1.26% accuracy and -8.56% tokens without training.

Problem

Large reasoning models think slowly and verbosely, incurring high computational cost and latency, and existing test-time control (budget forcing, thought extrapolation) is prompt-based and suboptimal; it is unclear how to switch between fast and slow thinking and when to do so.

Approach

Identify a fast-slow steering vector in the hidden representations of LRMs via unsupervised PCA over contrastive stimulus pairs, inject it during decoding to control thinking speed, and modulate the steering intensity per token using a JS-divergence between early-layer and final-layer logits as a real-time difficulty signal.

Key results

• Adaptive control yields on average +1.26% accuracy and -8.56% token usage across models and benchmarks versus the original LRMs (e.g., DeepSeek-R1-Distill-Qwen-7B MATH-500 92.9→93.7 with tokens 3403.9→3122.8).
• Constant fast steering (alpha>0) reduces tokens substantially with accuracy drops, e.g., DeepSeek-R1-Distill-Qwen-32B alpha=4 gives -1680.6 tokens on average but -3.27 Pass@1 delta.
• Slow steering (alpha=-4) improves Qwen3-8B average Pass@1 by +1.15 while adding about 726 tokens, outperforming Wait-based thought extrapolation which degrades accuracy.

Measured effects reported in the paper

Surprising findings

• Starting the reasoning with the single opening word "To" cuts response length about 4.4x to 5.4x while retaining 60% (7B) and 68% (32B) of original accuracy.
• Prompt-based thought extrapolation ("Wait") can degrade accuracy as reflection time increases.

Limitations

• Different LRMs show varying sensitivity to steering intensity, so a single alpha range does not give uniform length control.
• Adaptive control uses a heuristic sliding-window rule rather than an optimized steering schedule.
• The reason why opening words trigger fast thinking remains unexplained.

Experimental setup

Datasets: MATH training set AMC

Benchmarks: MATH-500 AIME24 AIME25 GPQA Diamond LiveCodeBench

Metrics: Pass@1 num tokens

Baselines: Budget Forcing Thought Extrapolation (Wait) constant steering

Builds on: representation engineering (RepE) budget forcing DoLa contrastive decoding

Closest papers by meaning

• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.91)
• On Reasoning Strength Planning in Large Reasoning Models (NeurIPS 2025 · similarity 0.89)
• Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training (NeurIPS 2025 · similarity 0.89)
• Does Thinking More Always Help? Mirage of Test-Time Scaling in Reasoning Models (NeurIPS 2025 · similarity 0.89)
• Modeling Hierarchical Thinking in Large Reasoning Models (ICML 2026 · similarity 0.89)
• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity (NeurIPS 2025 · similarity 0.89)