Research map / Large language models / Reasoning and chain of thought
AdaReasoner: Adaptive Reasoning Enables More Flexible Thinking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdaReasoner, an LLM-agnostic RL-trained plugin that adaptively selects reasoning instruction, temperature, and step count per question, using a DeBERTa reward model and Boltzmann exploration, and reports gains over CoT and other prompting baselines across six LLMs.
Problem
LLM reasoning performance is highly sensitive to configuration choices (temperature, reasoning steps, instruction format), but existing prompting methods use fixed configurations that are not task-specific.
Approach
Treat per-question configuration selection as a multi-armed bandit solved by a factorized policy (three heads for instruction, temperature, steps over a shared layer) trained with REINFORCE and Boltzmann exploration, with reward from a pretrained DeBERTa-based judge of answer correctness.
Key results
- AdaReasoner reaches 80.42% average accuracy on GPT-4o versus 74.42% for the best baseline (In-context CoT) across four datasets.
- With Llama-3.3-70B-Instruct, AdaReasoner averages 80.74%, and with DeepSeek-R1 it averages 86.28%, both highest in their model groups.
- Replacing Boltzmann exploration with Thompson sampling drops GPT-4o average accuracy to 75.89%; transferring a Qwen-trained policy to GPT-4o drops it to 72.31%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AdaReasoner instead of the strongest listed baseline | GPT-4o · Four-dataset average | accuracy (%): 74.42 -> 80.42 | Helped |
| Use AdaReasoner instead of the strongest listed baseline | Llama-3.3-70B-Ins. · Four-dataset average | accuracy (%): 76.51 -> 80.74 | Helped |
| Use AdaReasoner instead of the strongest listed baseline | Qwen-2.5-72B-Ins. · Four-dataset average | accuracy (%): 76.17 -> 77.49 | Helped |
| Use AdaReasoner instead of the strongest listed baseline | Claude-3.5-sonnet · Four-dataset average | accuracy (%): 79.28 -> 81.43 | Helped |
| Use AdaReasoner instead of the strongest listed baseline | Deepseek-R1 · Four-dataset average | accuracy (%): 82.85 -> 86.28 | Helped |
| Use AdaReasoner instead of the strongest listed baseline | GPT-o3-mini · Four-dataset average | accuracy (%): 79.24 -> 84.39 | Helped |
| Apply AdaReasoner rather than the strongest listed baseline | Qwen-2.5-72B · BRIGHTER | accuracy (%): 55.17 -> 55.36 | Helped |
| Apply AdaReasoner rather than the strongest listed baseline | Qwen-2.5-72B · StepGame | accuracy (%): 77.15 -> 78.00 | Helped |
| Apply AdaReasoner rather than the strongest listed baseline | Qwen-2.5-72B · CRoW | accuracy (%): 93.97 -> 95.56 | Helped |
| Adapt reasoning instructions only instead of full AdaReasoner (ablation) | AdaReasoner on GPT-4o · Four-dataset average | accuracy (%): 80.42 -> 78.62 | Hurt |
| Replace Boltzmann exploration with Thompson sampling (ablation) | AdaReasoner on GPT-4o · Four-dataset average | accuracy (%): 80.42 -> 75.89 | Hurt |
| Add Gaussian noise ($\sigma=0.01$) to the reward (ablation) | AdaReasoner on GPT-4o · Four-dataset average | accuracy (%): 80.42 -> 78.26 | Hurt |
Surprising findings
- Random Action performs well on MMLU, which the authors attribute to a reward landscape with multiple local optima.
- Random seed adaptation hurts performance (69.19% average on GPT-4o), the authors interpret this as seed tuning not being beneficial.
Limitations
- Requires per-task few-shot RL training and adds computational overhead.
- Policy does not transfer across models (Qwen-trained policy degrades GPT-4o performance).
- Operates over a manually defined discrete action space that may limit expressiveness.
Experimental setup
Datasets: MMLU (Math) Metaphor TruthfulQA LogiQA BRIGHTER StepGame CRoW GPQA MMLUChem MedExQA
Benchmarks: MMLU (Math) Metaphor TruthfulQA LogiQA BRIGHTER StepGame CRoW GPQA MMLUChem MedExQA
Metrics: accuracy
Baselines: CoT Think Short ToT Best-of-N Auto-CoT In-context CoT Thompson sampling bandit adapter
Builds on: Chain-of-Thought Tree-of-Thoughts Auto-CoT REINFORCE Thompson sampling
Closest papers by meaning
- QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation (ICLR 2026 · similarity 0.90)
- SuCo: Sufficiency-guided Continuous Adaptive Reasoning (ICML 2026 · similarity 0.89)
- RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning (ICLR 2026 · similarity 0.89)
- General-Reasoner: Advancing LLM Reasoning Across All Domains (NeurIPS 2025 · similarity 0.89)
- Thinker: Learning to Think Fast and Slow (NeurIPS 2025 · similarity 0.89)
- Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.