Research map / Large language models / Reasoning and chain of thought
Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that longer CoTs can hurt math reasoning in SFT-trained models and proposes Thinking-Optimal Scaling (TOPS), which trains a reasoning-effort-conditioned tag model and self-improves on the shortest correct responses.
Problem
Whether excessively scaling chain-of-thought length at test time harms LLM reasoning performance, and how to scale test-time compute optimally rather than simply longer.
Approach
Train a tag model on seed data with low/medium/high reasoning-effort CoTs, sample under each effort on extra problems, keep the shortest correct response per problem as the thinking-optimal SFT target, and iterate with DPO using shortest-correct vs. longest/shortest-wrong pairs.
Key results
- Qwen2.5-32B-TOPS reaches 95.82 on GSM8K, 91.48 on MATH500 and 43.33 on AIME2024 with fewer tokens than QwQ-32B-Preview on GSM8K and MATH500 (412 vs 761 tokens on GSM8K).
- Qwen2.5-32B-TOPS-Iter-DPO reaches 46.00 on AIME2024 and 91.60 on MATH500, comparable to QwQ-32B-Preview (92.02 / 45.33).
- Training on shortest-correct responses (TOPS) outperforms random-correct responses (Qwen2.5-32B-Random) on all three math benchmarks, e.g., 91.48 vs 90.16 on MATH500.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Select the shortest correct response rather than a random correct response (ablation) | Qwen2.5-32B-TOPS vs Qwen2.5-32B-Random · GSM8K | accuracy: 95.00 -> 95.82 | Helped |
| Select the shortest correct response rather than a random correct response (ablation) | Qwen2.5-32B-TOPS vs Qwen2.5-32B-Random · MATH500 | accuracy: 90.16 -> 91.48 | Helped |
| Select the shortest correct response rather than a random correct response (ablation) | Qwen2.5-32B-TOPS vs Qwen2.5-32B-Random · AIME2024 | accuracy: 39.33 -> 43.33 | Helped |
| Use low instead of high reasoning effort (ablation) | Qwen2.5-32B-Tag · GSM8K | accuracy: 93.31 -> 95.53 | Helped |
| Use medium instead of low reasoning effort (ablation) | Qwen2.5-32B-Tag · MATH500 | accuracy: 90.60 -> 91.48 | Helped |
| Increase reasoning effort from medium to high (ablation) | Qwen2.5-32B-Tag · MATH500 | accuracy: 91.48 -> 90.56 | Hurt |
| Use medium instead of low reasoning effort (ablation) | Qwen2.5-32B-Tag · AIME2024 | accuracy: 34.67 -> 42.00 | Helped |
| Increase reasoning effort from medium to high (ablation) | Qwen2.5-32B-Tag · AIME2024 | accuracy: 42.00 -> 41.33 | Hurt |
| Further self-improve TOPS with SFT (ablation) | Qwen2.5-32B-TOPS · MATH500 | accuracy: 91.48 -> 90.76 | Hurt |
| Further self-improve TOPS with DPO (ablation) | Qwen2.5-32B-TOPS · AIME2024 | accuracy: 43.33 -> 46.00 | Helped |
| Filter training samples containing erroneous steps (ablation) | Qwen2.5-32B-Tag-High · GSM8K | accuracy: 93.31 -> 94.87 | Helped |
| Filter training samples containing erroneous steps (ablation) | Qwen2.5-32B-Tag-High · MATH500 | accuracy: 90.56 -> 90.00 | Hurt |
Surprising findings
- Training on longer correct CoTs under high reasoning effort reduced accuracy on easy tasks (e.g., GSM8K 95.53 low vs 93.31 high for Qwen2.5-32B-Tag), indicating longer CoTs can hurt performance.
- Erroneous reasoning rounds increase with reasoning effort, and loss masking on erroneous steps improved performance over training on all tokens.
Limitations
- Analysis focuses mainly on math; general-reasoning results are preliminary and limited to a 7B model.
- Work is limited to SFT and DPO; the effect of CoT length under RL is not tested.
- AIME2024 results are noisy (30 problems) and TOPS slightly underperforms STILL-2-32B there before DPO.
Experimental setup
Datasets: NuminaMath MATH GSM8K AIME1983-2023 WebInstruct-verified MMLU-Pro GPQA-Diamond
Benchmarks: GSM8K MATH500 AIME2024 MMLU-Pro GPQA-Diamond
Metrics: accuracy average number of tokens average number of distinct answers
Baselines: QwQ-32B-Preview STILL-2-32B Sky-T1-32B-Preview Qwen2.5-32B-Random Qwen2.5-32B-Instruct LLaMA3.1-8B-Random-SFT
Builds on: Chain-of-Thought Direct Preference Optimization STILL-2 Sky-T1
Closest papers by meaning
- Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens (ICML 2026 · similarity 0.92)
- When More is Less: Understanding Chain-of-Thought Length in LLMs (ICLR 2026 · similarity 0.92)
- Dynamic Early Exit in Reasoning Models (ICLR 2026 · similarity 0.92)
- Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (NeurIPS 2025 · similarity 0.92)
- Demystifying Long Chain-of-Thought Reasoning (ICML 2025 · similarity 0.91)
- SCALE: Selective Resource Allocation for Overcoming Performance Bottlenecks in Mathematical Test-time Scaling (AAAI 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.