Research map / Large language models / Reasoning and chain of thought
Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Ada-R1 merges a Long-CoT and a Short-CoT DeepSeek-R1-Distill model into a hybrid model, then applies bi-level DPO (group-level style selection plus instance-level length preference) to adaptively choose reasoning depth and cut length by about 50% on math benchmarks.
Problem
Long-CoT reasoning models incur high inference cost, and the benefit of long reasoning varies across problems, sometimes degrading accuracy on easy ones. Existing efficiency methods mostly compress within the Long-CoT distribution and do not adaptively choose reasoning style per input.
Approach
Linearly merge a Long-CoT and a Short-CoT model into one hybrid model, then train with bi-level DPO: group-level preferences choose which reasoning style (long or short) is better for each problem based on sampled accuracy, and instance-level preferences prefer the shortest correct response within the chosen style.
Key results
- Ada-R1 (7B) reduces average length by 50.93% with -1.65% average accuracy change relative to the Long model, versus -3.56% and -33.26% length for DPO.
- Ada-R1 (1.5B) reduces length by 43.28% with -1.21% accuracy change; O1-Pruner gives -33.75% length with +2.18% accuracy.
- Ablation on 7B: Merge + bi-level achieves 52.08% length reduction with -0.51% accuracy change, versus -12.83% accuracy for naive merge.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Ada-R1 instead of Long-CoT | DeepSeek-R1-Distill-Qwen-7B · MATH500 | output length: 3534 -> 1468 | Helped |
| Apply Ada-R1 instead of Long-CoT | DeepSeek-R1-Distill-Qwen-7B · MATH500 | accuracy: 90.2 -> 90.2 | No clear effect |
| Apply Ada-R1 instead of Long-CoT | DeepSeek-R1-Distill-Qwen-7B · GSM8K | accuracy: 88.9 -> 90.3 | Helped |
| Apply Ada-R1 instead of Long-CoT | DeepSeek-R1-Distill-Qwen-7B · GSM8K | output length: 1014 -> 260 | Helped |
| Apply Ada-R1 instead of Long-CoT | DeepSeek-R1-Distill-Qwen-7B · AIME25 | accuracy: 38.3 -> 35.8 | Hurt |
| Apply Ada-R1 instead of Long-CoT | DeepSeek-R1-Distill-Qwen-7B · Olympiad | accuracy: 54.4 -> 52.4 | Hurt |
| Apply Ada-R1 instead of Long-CoT | DeepSeek-R1-Distill-Qwen-7B · Minerva | accuracy: 35.7 -> 34.1 | Hurt |
| Add SFT to the merged model (ablation) | Ada-R1 · AIME25 | accuracy: 21.7 -> 35.8 | Helped |
| Add group-level preference training to the merged model (ablation) | Ada-R1 · GSM8K | accuracy: 88.4 -> 91.6 | Helped |
| Add instance-level preference training to group-level preference training (ablation) | Ada-R1 · GSM8K | output length: 359 -> 260 | Helped |
| Add instance-level preference training to group-level preference training (ablation) | Ada-R1 · GSM8K | accuracy: 91.6 -> 90.3 | Hurt |
| Replace bi-level preference training with SFT (ablation) | Merged model · GSM8K | output length: 260 -> 375 | Hurt |
Surprising findings
- Naive merging of long and short models causes over 10-point accuracy drops on AIME25 and MATH500 despite large length reductions.
- On GSM8K, Ada-R1 improves accuracy over the Long model (90.3 vs 88.9 for 7B) while cutting length by roughly 74%.
Limitations
- Assumes problem complexity can be estimated from the input; may be harder on real-world tasks with diverse distributions.
- Evaluated only on math reasoning benchmarks.
- Code is only promised ('coming soon'), with no released artifacts at submission.
Experimental setup
Datasets: GSM8K MATH AIME Olympiad Minerva
Benchmarks: AIME25 MATH500 GSM8K Olympiad Minerva
Metrics: accuracy average response length average accuracy change average length reduction thinking ratio
Baselines: DPO O1-Pruner CoT-Valve Naive Merge Short-CoT model
Builds on: DPO DeepSeek-R1-Distill O1-Pruner CoT-Valve model merging
Closest papers by meaning
- SLAT: Segment-Level Adaptive Trimming for Efficient CoT Reasoning (ICML 2026 · similarity 0.90)
- The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think (ICLR 2026 · similarity 0.90)
- QFFT, Question-Free Fine-Tuning for Adaptive Reasoning (NeurIPS 2025 · similarity 0.90)
- Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning (NeurIPS 2025 · similarity 0.90)
- The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation (ICLR 2026 · similarity 0.90)
- SuCo: Sufficiency-guided Continuous Adaptive Reasoning (ICML 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.