Research map / Large language models / Reasoning and chain of thought
ARM: Adaptive Reasoning Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Adaptive Reasoning Model (ARM), which chooses among Direct Answer, Short CoT, Code, and Long CoT formats per task, trained with SFT then Ada-GRPO (a format-diversity-reweighted GRPO) to cut tokens ~30% with comparable accuracy.
Problem
Reasoning models apply long chain-of-thought uniformly regardless of task difficulty, causing overthinking and wasted tokens; standard GRPO also collapses onto the highest-accuracy format (Long CoT).
Approach
Train the model to select among four reasoning formats adaptively via special tokens, and modify GRPO advantages with a format-diversity scaling factor G/F(o_i) multiplied by a cosine decay so rare, cheaper formats are not extinguished early, while accuracy dominates later.
Key results
- ARM-7B reaches 75.9 avg accuracy vs 76.1 for Qwen2.5-7B SFT+GRPO (maj@8) while using ~33% fewer tokens on average (786 vs 1164).
- Tokens reduced by up to ~70-75% on easy tasks (e.g., CSQA -73.0% for 7B, OBQA -75.4%).
- Ada-GRPO reaches comparable final accuracy with ~2x training speedup over GRPO, with roughly half the average response length.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Ada-GRPO with adaptive reasoning formats instead of GRPO | Qwen2.5-7B · Evaluation average (k=8) | tokens: 1164 -> 786 | Helped |
| Use Ada-GRPO with adaptive reasoning formats instead of GRPO | Qwen2.5-7B · Evaluation average (k=8) | accuracy: 76.1 -> 75.9 | No clear effect |
| Add function-calling reasoning format (ablation) | ARM-7B · Evaluation average | accuracy: +0.4 | Helped |
| Add Direct Answer format (ablation) | ARM-7B · CSQA | tokens: +29.4% tokens when removed | Helped |
| Add Short CoT format (ablation) | ARM-7B · GSM8K | tokens: +26.2% tokens when removed | Helped |
| Add Long CoT format (ablation) | ARM-7B · AIME'25 | accuracy: 16.7 -> 8.3 (−8.4) when removed | Helped |
| Use the diverse full ARM training recipe instead of AIME-only training (ablation) | Qwen2.5-7B · CSQA | accuracy: 78.6 -> 86.1 | Helped |
| Use the diverse full ARM training recipe instead of AIME-only training (ablation) | Qwen2.5-7B · OBQA | accuracy: 82.2 -> 84.4 | Helped |
| Include a decay mechanism in Ada-GRPO (ablation) | Ada-GRPO · CSQA, GSM8K, and MATH test sets | accuracy stability | Helped |
| Use Consensus-Guided Mode instead of Adaptive Mode | ARM-7B · Evaluation average | accuracy: 72.0 -> 75.2 | Helped |
| Use Consensus-Guided Mode instead of Adaptive Mode | ARM-7B · Evaluation average | tokens: 766 -> 1856 | Hurt |
| Use Adaptive Mode rather than a fixed specified reasoning format | ARM-7B · Evaluation datasets | accuracy: +4.7% Direct Answer; +2.7% Short CoT and Code | Helped |
Surprising findings
- Standard GRPO collapses onto Long CoT within about 10 training steps, even though Long CoT is not needed for easy tasks.
- SFT alone produces near-uniform format choices regardless of difficulty, and its use of Direct Answer on medium tasks yields only 35.2% accuracy, dropping overall performance.
Limitations
- Format selection relies on a fixed set of four predefined formats; adding function-calling gave only marginal gains (+0.4 avg), and extra formats add implementation cost.
- Evaluation is on relatively small sets (e.g., AIME'25 with 30 problems, reported in 3.3%-steps), so differences of a few points are noisy.
- Consensus-Guided Mode costs substantially more tokens than Adaptive Mode and is not a reduction in cost.
Experimental setup
Datasets: AQuA-Rat CommonsenseQA GSM8K MATH AIME 1983-2024 (ablation only)
Benchmarks: CommonsenseQA OpenBookQA GSM8K MATH SVAMP BBH AIME'25 GPQA StrategyQA
Metrics: accuracy (pass@1) majority@8 accuracy average tokens
Baselines: Qwen2.5 SFT Qwen2.5 SFT+GRPO DeepSeek-R1 DeepSeek-V3 GPT-4o o1-preview o4-mini-high L1 ThinkPrune
Builds on: GRPO DeepSeek-R1 LoRA VeRL
Closest papers by meaning
- Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization (NeurIPS 2025 · similarity 0.89)
- SuCo: Sufficiency-guided Continuous Adaptive Reasoning (ICML 2026 · similarity 0.88)
- A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning (ICLR 2026 · similarity 0.88)
- Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models (NeurIPS 2025 · similarity 0.88)
- How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.88)
- Optimizing Anytime Reasoning via Budget Relative Policy Optimization (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.