Research map / Large language models / Reasoning and chain of thought

Does Your Reasoning Model Implicitly Know When to Stop Thinking?

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that reasoning models implicitly know when to stop thinking and proposes SAGE, a self-confidence-guided step-wise sampling method, plus SAGE-RL, which mixes SAGE rollouts into GRPO/GSPO to improve both accuracy and token efficiency on math benchmarks.

Problem

Long chains of thought in large reasoning models are redundant and often uncorrelated with correctness, yet standard pass@1 sampling fails to terminate reasoning at the right moment, obscuring efficient short correct chains.

Approach

Use cumulative length-normalized log-probability to beam-search over reasoning steps and accept candidates that end with the </think> token, since the model assigns high confidence to stopping on high-confidence paths; then mix these SAGE rollouts (2 of 8) into the RLVR rollout group so the policy learns the efficient patterns under standard pass@1 inference.

Key results

• SAGE-GRPO on DS-1.5B raises MATH-500 pass@1 from 83.2 to 84.8 while cutting length from 4882 to 2915 tokens.
• SAGE-GSPO on DeepScaleR reaches 36.1 pass@1 on AIME 2024 (vs 31.4 base) with length 8094 vs 9370.
• SAGE-GRPO on DS-7B reaches 93.0 pass@1 on MATH-500 with 2141 tokens, versus 91.6 and 3871 for the base model.

Surprising findings

• Beam search with cumulative log-probability performs comparably to random sampling, while the proposed step-wise confidence-based search improves accuracy and shortens chains.
• Replacing only 2 of 8 rollouts with SAGE sampling yields larger gains than standard RLVR, and most baselines that compress length lose accuracy.

Limitations

• SAGE step-wise inference requires repeated generate calls, reducing efficiency relative to standard random sampling
• Evaluation focuses on mathematical reasoning benchmarks; generalization is only claimed without broad detailed evidence
• Relies on step segmentation by double newline, which is a heuristic

Experimental setup

Datasets: DAPO MATH MATH-500 AIME 2024 AIME 2025 AMC23 OlympiadBench Minerva

Benchmarks: MATH-500 AIME 2024 AIME 2025 AMC23 OlympiadBench Minerva

Metrics: pass@1 response length token efficiency (pass@1/length)

Baselines: LC-R1 ThinkPrune-2k AdaptThink Efficient Reasoning GRPO-LEAD GRPO GSPO

Builds on: GRPO GSPO beam search RLVR verl

Closest papers by meaning

• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.90)
• How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.90)
• Let LRMs Break Free from Overthinking via Self-Braking Tuning (NeurIPS 2025 · similarity 0.89)
• Thinkless: LLM Learns When to Think (NeurIPS 2025 · similarity 0.89)
• Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities (AAAI 2026 · similarity 0.89)
• On Reasoning Strength Planning in Large Reasoning Models (NeurIPS 2025 · similarity 0.89)