Research map / Large language models / Reasoning and chain of thought
Think Only When You Need with Large Hybrid-Reasoning Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Large Hybrid-Reasoning Models (LHRMs) that adaptively choose between thinking and no-thinking modes via Hybrid Fine-Tuning followed by Hybrid Group Policy Optimization (HGPO), and a Hybrid Accuracy metric.
Problem
Large reasoning models overthink simple queries, wasting tokens and latency, while conventional LLMs underperform on complex reasoning; there is no mechanism for choosing when to think and no metric to evaluate that choice.
Approach
A two-stage pipeline: hybrid-formatted SFT (think and no_think tags) as cold start, then GRPO-style RL with inter-group (mode-level) and intra-group (sample-level) binary rewards so the policy learns which thinking mode to use per query; plus a Hybrid Accuracy metric comparing chosen mode to a reward-model-derived preferred mode.
Key results
- LHRMs-7B reaches 63.1 average vs 56.8 for DeepSeek-R1-Distill-Qwen-7B and 58.9 for HFT-DPO-7B, with AIME24 at 66.7.
- LHRMs-7B achieves Hybrid Accuracy of 71.9 versus 34.2 for HFT-7B, a 110.2% improvement from HGPO.
- LHRMs-1.5B averages 45.3 vs 38.3 for DeepSeek-R1-Distill-Qwen-1.5B, and gains 11.1% on MBPP despite RL only on math and general data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add HGPO after HFT | Qwen2.5-Math-1.5B · MATH500 | accuracy: 87.8 -> 87.8 | Helped |
| Add HGPO after HFT | Qwen2.5-Math-1.5B · Hybrid Accuracy | accuracy: 41.4 -> 54.4 | Helped |
| Add HGPO after HFT | Qwen2.5-Math-7B · Hybrid Accuracy | accuracy: 34.2 -> 71.9 | Helped |
| Add HGPO after HFT | Qwen2.5-Math-7B · Arena-Hard | score: 14.0 -> 26.0 | Helped |
| Use HGPO instead of DPO after HFT | Qwen2.5-Math-1.5B · MBPP | pass@1: 50.5 -> 61.1 | Helped |
| Use HGPO instead of DPO after HFT | Qwen2.5-Math-1.5B · Average across evaluation tasks | average score: 41.5 -> 45.3 | Helped |
| Use HGPO instead of DPO after HFT | Qwen2.5-Math-7B · AIME24 | accuracy: 58.7 -> 66.7 | Helped |
| Use HGPO instead of DPO after HFT | Qwen2.5-Math-7B · AlpacaEval 2.0 | score: 23.3 -> 35.0 | Helped |
| Use HGPO instead of DPO after HFT | Qwen2.5-Math-7B · Hybrid Accuracy | accuracy: 37.1 -> 71.9 | Helped |
| Replace GRPO with RLOO or REINFORCE++ as the HGPO advantage estimator (ablation) | HGPO · HGPO training | performance | No clear effect |
| Increase margin δ (ablation) | HGPO · Hybrid thinking behavior | No-Thinking mode preference | Helped |
| Apply HGPO after HFT | Qwen2.5-Math-7B · MATH500 | thinking ratio on easier difficulty levels | Helped |
Surprising findings
- The 7B model's thinking ratio decreases during RL while the 1.5B model's thinking ratio increases, suggesting larger models shift toward no-thinking on solvable problems.
- HFT-RFT and HFT-DPO show performance drops on code tasks, whereas LHRMs generalizes hybrid thinking to code despite RL on math and general data.
Limitations
- Hybrid Accuracy depends on a reward model to define the preferred mode, so the metric inherits reward model bias.
- Experiments limited to Qwen-2.5 models at 1.5B and 7B scales.
- Rule-based rewards only for definitive-answer queries; open-ended queries rely on a parametric reward model.
Experimental setup
Datasets: WildChat-1M OpenAssistant oasst1 SYNTHETIC-1 OpenR1-Math OpenThoughts AoPS AIME 1983-2024 OpenR1-Codeforces KodCode DeepScaleR Tulu 3 preference mixture
Benchmarks: MATH500 AIME24 AMC23 OlympiadBench LiveCodeBench MBPP MBPP+ AlpacaEval 2.0 Arena-Hard Hybrid Accuracy on MATH500
Metrics: pass@1 length-controlled win rate / win rate Hybrid Accuracy average score
Baselines: Qwen2.5-Math-1.5B Qwen2.5-1.5B-Instruct Qwen2.5-Math-1.5B-Instruct DeepSeek-R1-Distill-Qwen-1.5B Qwen2.5-7B-Instruct Qwen2.5-Math-7B-Instruct DeepSeek-R1-Distill-Qwen-7B HFT-DPO HFT-RFT
Builds on: GRPO REINFORCE++ RLOO DPO DeepSeek-R1
Closest papers by meaning
- Hybrid Latent Reasoning via Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Training Language Models to Reason Efficiently (NeurIPS 2025 · similarity 0.89)
- Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.89)
- A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning (ICLR 2026 · similarity 0.89)
- Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities (AAAI 2026 · similarity 0.89)
- How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.