Research map / Large language models / Reasoning and chain of thought
AdaReason: Progressive Training of Multi-LoRA Adapters for Budget-Adaptive Language Reasoning Models
Jiacheng Wang, Tianle Chen, Pengyu Cheng, Xiaofeng Hou, Jiacheng Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdaReason, a progressive training scheme for multiple LoRA adapters that lets a single reasoning model adapt its chain-of-thought length to a user-specified compute budget.
Problem
Large reasoning models face a trade-off between efficiency and accuracy; existing methods either lack user-specified computational budget support or require maintaining multiple independent models.
Approach
Progressively trains multiple LoRA adapters on a shared base reasoning model so that a single deployed model supports budget-adaptive reasoning, avoiding separate models per budget level.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use multi-LoRA adapters with progressive training instead of L1-Max | DeepScaleR-1.5B · GPQA+HumanEval avg | accuracy: 57.3 -> 58.5 | Improved |
| budget-weighted adapter merging vs naive pairwise merging at shortest budget (ablation) | AdaReason · Math avg | accuracy: 53.9 -> 55.5 | Improved |
| budget-weighted merging vs naive pairwise merging at longest budget (ablation) | AdaReason · Math avg | accuracy: 61.3 -> 61.0 | No clear effect |
| budget-weighted merging vs naive pairwise merging at medium budgets (ablation) | AdaReason · Math avg | accuracy: 59.9 -> 59.6 (T2/T3), 57.6 -> 58.0 (T3/T4) | Mixed |
| tighten budget progressively, shorter adapter (T5 vs T1) (ablation) | AdaReason · GPQA+HumanEval avg | accuracy: 59.3 -> 57.9 | Hurt |
| compare AdaReason adapters against unconstrained model | DeepScaleR-1.5B · GPQA+HumanEval avg | generation length: 5047.3 -> 4196.3 (T1) | Improved |
Closest papers by meaning
- Training Language Models to Reason Efficiently (NeurIPS 2025 · similarity 0.89)
- LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models (NeurIPS 2026 · similarity 0.89)
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.89)
- Tina: Tiny Reasoning Models via LoRA (ICLR 2026 · similarity 0.89)
- AdaReasoner: Adaptive Reasoning Enables More Flexible Thinking (NeurIPS 2025 · similarity 0.88)
- Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
