Research map / Large language models / Reasoning and chain of thought
Thinkless: LLM Learns When to Think
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Thinkless trains a reasoning LLM to choose between short and long reasoning via control tokens, using SFT warm-up followed by Decoupled GRPO (DeGRPO) that separately normalizes the control-token and response-token losses.
Problem
Reasoning LLMs apply long chain-of-thought to all queries, wasting compute on easy problems; existing hybrid methods rely on manual heuristics or prompt switches rather than learned mode selection.
Approach
Learn a per-query mode decision by making the first generated token a control token (<think> or <short>) trained with RL, and fix vanilla GRPO's mode collapse by decoupling and independently normalizing the control-token loss and the response-token loss with a small weight alpha on the control token.
Key results
- On Minerva Algebra, Thinkless reaches 0.9459 Pass@1 with 1144 tokens and only 25.88% think mode, versus 0.9577 Pass@1 and 3029 tokens for DeepSeek-R1-1.5B.
- On GSM8K, Thinkless achieves 0.8418 Pass@1 with 624 tokens (13.31% think), compared with 0.8347 and 1919 tokens for DeepSeek-R1-1.5B.
- On MATH-500, Thinkless achieves 0.8184 Pass@1 with 2555 tokens (51.56% think), outperforming the Router Q-7B baseline (0.7948) and the R1 model's 0.8608 with 5675 tokens.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Thinkless hybrid reasoning instead of DeepSeek-R1-1.5B | DeepSeek-R1-Distill-Qwen-1.5B · AIME 2024 | Pass@1: 0.2800 -> 0.2733 | Hurt |
| Use Thinkless hybrid reasoning instead of DeepSeek-R1-1.5B | DeepSeek-R1-Distill-Qwen-1.5B · Minerva Algebra | Pass@1: 0.9577 -> 0.9459 | No clear effect |
| Use Thinkless hybrid reasoning instead of DeepSeek-R1-1.5B | DeepSeek-R1-Distill-Qwen-1.5B · Math-500 | Pass@1: 0.8608 -> 0.8184 | Hurt |
| Use Thinkless hybrid reasoning instead of QMath-1.5B | DeepSeek-R1-Distill-Qwen-1.5B · GSM8K | Pass@1: 0.8572 -> 0.8418 | Hurt |
| Use Thinkless adaptive mode selection | DeepSeek-R1-Distill-Qwen-1.5B · Minerva Algebra | thinking-mode query proportion: Thinkless ... 25.88%; DeepSeek-R1-1.5B ... 3029 | Helped |
| Replace DeGRPO with vanilla GRPO | Thinkless · RL training | long-chain responses generated: below 10 within just 120 update steps | Hurt |
| Increase the control-token weight from 0.001 to 0.5 (ablation) | Thinkless DeGRPO · RL training | emergence of all-correct short samples: 0.001; 0.5 | Helped |
| Use OpenR1-97K for warm-up distillation (ablation) | R1-1.5B hybrid model · Math-500 | Pass@1: 0.8608 -> 0.8336 | Hurt |
| Increase warm-up data from OpenThoughts-114K to OpenThoughts-1M (ablation) | R1-1.5B hybrid model · Math-500 | long-chain Pass@1: 0.8280 -> 0.8360 | Helped |
Surprising findings
- Vanilla GRPO collapses quickly: the number of long-chain responses drops below 10 within 120 update steps.
- On AIME 2024 Thinkless uses think mode for essentially all queries, and its Pass@1 (0.2733) is close to DeepSeek-R1-1.5B's 0.2800, despite the low accuracy of the short mode.
Limitations
- Warm-up SFT is simple and produces a slightly degraded initial hybrid model (e.g., lower MATH-500 accuracy after distillation).
- Validated only on math datasets, with RL training limited to DeepScaleR (~40K math problems).
- Results are reported for a single 1.5B base model.
Experimental setup
Datasets: OpenR1-97K OpenThoughts-114K OpenThoughts-1M DeepScaleR AIME 2024 Minerva Algebra MATH-500 GSM8K
Benchmarks: AIME 2024 Minerva Algebra MATH-500 GSM8K
Metrics: Pass@1 average tokens generated think-mode proportion (%)
Baselines: Merging CoT-Valve Router Random Router Q-7B Qwen2.5-Math-1.5B-Instruct Qwen2.5-1.5B-Instruct DeepSeek-R1-1.5B vanilla GRPO
Builds on: GRPO DeepSeek-R1-Distill-Qwen-1.5B Megatron VeRL
Closest papers by meaning
- RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning (ICLR 2026 · similarity 0.91)
- Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.91)
- Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.91)
- Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models (NeurIPS 2025 · similarity 0.91)
- In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedback (AAAI 2026 · similarity 0.91)
- Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization (ICLR 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.