Research map / Large language models / Reasoning and chain of thought

Efficient Reasoning via Constrained Optimization in Latent Space

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a training-free inference-time method that estimates an efficient-reasoning region in PCA latent space from Direct vs. Vanilla CoT hidden states and projects deviating step-delimiter hidden states back into it via a QP (accelerated with offline mp-QP), reducing tokens while improving accuracy on reasoning benchmarks.

Problem

Large reasoning models overthink, generating redundant tokens, while existing efficiency fixes (early exit, shortening, reactive steering) cause underthinking and accuracy loss.

Approach

Efficient reasoning steps cluster in a concentrated region of latent space (built from Direct CoT hidden states via K-Means + SVM half-spaces forming a polytope); steering step-delimiter hidden states back into this region with a closest-point QP, preserving the orthogonal residual, reduces length and keeps accuracy.

Key results

• On DeepSeek-R1-Distill-Qwen-1.5B, the method improves AMC2023 accuracy from 64.4 to 75.2 while reducing tokens by 37.5% (7708 to 4821).
• Across four models and six benchmarks, token reduction ranges from 11.8% to 52.8% with accuracy gains up to 12.1 points (AIME2025 on DeepSeek-R1-Distill-Qwen-7B).
• End-to-end vLLM inference latency (TPR) drops 24.9% to 38.4% on MATH500 relative to the native baseline.

Surprising findings

• Steering only at step-delimiter tokens with a closest-point projection improves accuracy while cutting tokens, rather than trading accuracy for brevity as early-exit methods do.

Limitations

• Efficient reasoning set is built from Direct CoT generated with ground-truth answers and conciseness prompts, requiring offline data construction per model.
• Hyperparameters such as steering layer, PCA dimension, cluster count and boundary proportion require sensitivity analysis.

Experimental setup

Datasets: MATH GSM8K GPQA-Diamond AMC 2023 AIME 2025

Benchmarks: GSM8K MATH500 AMC 2023 AIME 2025 LiveCodeBench GPQA-Diamond

Metrics: accuracy average generated tokens time per request tokens per second

Baselines: Vanilla CCoT DEER SEAL STIR ReBalance

Builds on: SEAL vLLM EasySteer PDAQP

Closest papers by meaning

• Geometric Latent Reasoning Induces Shorter Generations in LLMs (NeurIPS 2026 · similarity 0.91)
• TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning (NeurIPS 2026 · similarity 0.90)
• How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.90)
• Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space (NeurIPS 2026 · similarity 0.90)
• In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedback (AAAI 2026 · similarity 0.90)
• Beyond What Seems Necessary: Hidden Gains from Scaling Training-Time Reasoning Length under Outcome Supervision (NeurIPS 2026 · similarity 0.90)