Research map / Large language models / Reasoning and chain of thought

ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ShorterBetter, an RL method (GRPO) that rewards responses by their distance from the Sample Optimal Length (shortest correct response in a rollout group), cutting reasoning-model output length by roughly 60-80% with near-stable accuracy.

Problem

Reasoning models like DeepSeek-R1 produce overly long chain-of-thought traces (overthinking), with redundant verification and exploration that waste tokens; prior length-control methods need user budgets or reference models.

Approach

Define Sample Optimal Length (SOL) as the length of the shortest correct sampled response (or the mean length if none is correct) and use a reward combining correctness with a penalty on the absolute deviation of each response's length from SOL, optimized with GRPO, so the model learns its own optimal length without manual budgets.

Key results

• ShorterBetter-1.5B reduces in-domain output length by 77.6% (alpha=1) and 79.2% (alpha=2) with accuracy changes of +2.5% and +3.2%.
• ShorterBetter-7B reduces in-domain length by 62.1% with +7.1% accuracy versus Distill-7B; out-of-domain length drops 62.3% with -0.48% accuracy.
• Naively using the shortest response length as the target leads to training collapse within 100 steps in ablations.

Measured effects reported in the paper

Surprising findings

• Setting the correctness reward to zero (alpha=0) still trains stably, suggesting SOL implicitly encodes correctness.
• Naively using the shortest response length as the target causes training collapse within 100 steps.

Limitations

• Evaluated only on verifiable math and coding tasks; open-ended extension is not tested.
• Out-of-domain accuracy drops for some benchmarks (e.g., BBH -5.2 points for 7B) and the 8B Llama variant shows larger out-of-domain degradation.
• Training caps max length at 6K (1.5B) and 5K (7B) tokens due to compute constraints.

Experimental setup

Datasets: DeepScaleR-Preview AIME AMC Omni-MATH Still dataset MATH Minerva Olympiad

Benchmarks: AIME AMC Olympiad Minerva MathQA BBH MMLU LiveCodeBench MBPP HumanEval

Metrics: accuracy output length AE score

Baselines: DeepSeek-R1-Distill-Qwen-1.5B DeepSeek-R1-Distill-Qwen-7B Qwen2.5-1.5B-Instruct Qwen2.5-7B-Instruct Training Efficient O1-Pruner

Builds on: GRPO DeepSeek-R1 L1 O1-Pruner Training Efficient

Closest papers by meaning

• SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning (ICML 2026 · similarity 0.91)
• LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models (NeurIPS 2026 · similarity 0.91)
• Learn to Reason Efficiently with Adaptive Length-based Reward Shaping (ICLR 2026 · similarity 0.90)
• TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning (NeurIPS 2026 · similarity 0.90)
• DeepCompress: A Dual Reward Strategy for Dynamically Exploring and Compressing Reasoning Chains (ICLR 2026 · similarity 0.90)
• When More is Less: Understanding Chain-of-Thought Length in LLMs (ICLR 2026 · similarity 0.90)