Research map / Large language models / Reasoning and chain of thought

Efficient Reasoning with Balanced Thinking

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ReBalance, a training-free method that uses stepwise confidence and its variance to detect overthinking and underthinking, then steers hidden states along a prototype-derived vector with a confidence-based dynamic gate.

Problem

Large reasoning models overthink simple problems, and existing anti-overthinking methods (keyword suppression, length control) often induce underthinking and hurt accuracy.

Approach

Uses step confidence and confidence variance as continuous signals of overthinking (high variance, low confidence) and underthinking (persistent high confidence); builds a steering vector between overthinking and underthinking hidden-state prototypes and modulates its sign and strength per step with a confidence-based gate.

Key results

• On DeepSeek-R1-Distill-Qwen-1.5B, ReBalance raises MATH-500 Pass@1 from 79.6 to 83.0 while cutting tokens from 4516 to 3474 (-23.1%).
• On QwQ-32B, ReBalance improves AIME25 Pass@1 from 53.3 to 63.3 and reduces AIME24 tokens by 27.8%.
• Across nine benchmarks and four models, the paper reports reductions in generated tokens of up to 35.4% alongside accuracy gains up to 10.0 points.

Surprising findings

• Prior anti-overthinking methods reduce reasoning length on correct and incorrect samples alike, indicating underthinking is induced by suppression methods.
• Vocabulary-based methods like NoWait and SEAL mostly act on low-confidence steps, suggesting keyword lists are incomplete proxies for confidence.

Limitations

• Steering vector and control function are fit per backbone using a seen dataset, so transfer to new models requires re-fitting.
• Gains on saturated tasks such as StrategyQA are small because the method behaves conservatively there.
• Larger confidence-variance windows increase token usage, so window size trades off responsiveness and length.

Experimental setup

Datasets: MATH MATH-500 GSM8K AIME24 AIME25 AMC23 OlympiadBench GPQA Diamond StrategyQA LiveCodeBench

Benchmarks: MATH-500 AIME24 AIME25 GSM8K AMC23 OlympiadBench GPQA Diamond StrategyQA LiveCodeBench

Metrics: Pass@1 #Tokens

Baselines: Baseline (original reasoning model) CoD DEER NoThinking NoWait Dynasor-CoT SEAL Manifold Steering FlashThink TrimR

Builds on: NoWait SEAL DEER Dynasor-CoT Manifold Steering

Closest papers by meaning

• Do NOT Think That Much for 2+3=? On the Overthinking of Long Reasoning Models (ICML 2025 · similarity 0.92)
• How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.91)
• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.91)
• Let LRMs Break Free from Overthinking via Self-Braking Tuning (NeurIPS 2025 · similarity 0.90)
• On Reasoning Strength Planning in Large Reasoning Models (NeurIPS 2025 · similarity 0.90)
• Mitigating Overthinking in Large Reasoning Models via Manifold Steering (NeurIPS 2025 · similarity 0.90)