Research map / Large language models / Reasoning and chain of thought
SteadyThought: Mitigating LLM Under-Thinking via Thought-Level Preference Optimization
Zhenyue Peng, Lin Yang, Jie Wang, Xiaoqi Ni, Hanzhu Chen, Xize Liang, Yanjun Wu, RujingWang, Kaiwen Zhou, Jianye Hao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SteadyThought, a thought-level preference optimization method that targets LRM under-thinking (excessive thought switching that abandons promising reasoning trajectories).
Problem
Large Reasoning Models switch between reasoning thoughts excessively and fail to sustain promising ones (under-thinking); prior fixes that suppress switching may discard valuable trajectories.
Approach
Mitigates under-thinking via thought-level preference optimization rather than simply suppressing thought switching, aiming to keep valuable reasoning trajectories while reducing premature switching. Full method details are not available from the abstract.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply Steady Thought preference optimization to the reasoning model | Qwen3-8B · 4-benchmark overall | accuracy and tokens: 80.23 -> 84.23 accuracy; tokens 6122 -> 4011 | Improved |
| suppress switching tokens with NOWAIT instead of ST | DeepSeek-R1-Distill-Qwen-14B · 4-benchmark overall | accuracy: 79.73 -> 75.88 | Hurt |
| use NoThink baseline | all four models · 4-benchmark overall | accuracy: 55.43 -> 37.20 (1.5B) | Hurt |
| apply SEAL steering | Qwen3-8B · MuSR / StrategyQA / MedQA-USMLE overall | accuracy: 72.83 -> 35.50 | Hurt |
| train only on math, test on LiveCode coding | Steady Thought, Qwen3-8B · LiveCode | accuracy: 71.8 -> 83.2 | Improved |
| train on English math, test on Chinese GAOKAO-bench | Steady Thought, Qwen3-8B · GAOKAO-bench | accuracy: 86.0 -> 88.4 | Improved |
| evaluate from incorrect first-thought prefixes after ST | Steady Thought, 1.5B · Controlled recovery set | accuracy: 13.2 -> 20.3 | Improved |
| use thought-level STPO instead of whole-trajectory SimPO (ablation) | Qwen3-8B · AIME 2024 | accuracy: 60.8 -> 64.6 | Improved |
| use STPO instead of DPO at thought level (ablation) | DeepSeek-R1-Distill-Qwen-1.5B · MATH-500 tokens | tokens: 4428 -> 2691 | Improved |
| use SFT on chosen continuations instead of preference optimization (ablation) | DeepSeek-R1-Distill-Qwen-1.5B · AIME 2024 | accuracy: 27.5 -> 20.8 (SFT-thought) | Hurt |
| add explicit linguistic markers to switching-point detection (ablation) | Steady Thought · AIME 2024 | accuracy: 29.2 -> 30.8 (1.5B); 63.3 -> 64.6 (Qwen3-8B) | Improved |
| add boundary entropy spikes to switching-point detection (ablation) | Steady Thought · MATH-500 tokens | tokens: 3554 -> 2691 (1.5B); 3642 -> 2139 (Qwen3-8B) | Improved |
Closest papers by meaning
- Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.91)
- Pruning Long Chain-of-Thought of Large Reasoning Models via Small-Scale Preference Optimization (ICLR 2026 · similarity 0.90)
- Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.90)
- Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization (ICLR 2026 · similarity 0.90)
- Thinkless: LLM Learns When to Think (NeurIPS 2025 · similarity 0.90)
- RATT: A Thought Structure for Coherent and Correct LLM Reasoning (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
