Research map / Large language models / Agents, code and math
DNR Bench: Benchmarking Over-Reasoning in Reasoning LLMs
Oluwanifemi Bamgbose, Masoud Hashemi, Sathwik Tejaswi Madhusudhan, Jishnu Sethumadhavan Nair, Aman Tiwari, Vikas Yadav
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use reasoning models (o3-mini-high, DeepSeek-R1) instead of GPT-4o without instructions | DNR Bench prompts · DNR Bench all categories | accuracy: Unanswerable 0.42 (GPT-4o) -> 0.00 (O3-mini, DS-R1) | Hurt |
| add explicit instructions to abstain on unanswerable problems (ablation) | reasoning models · DNR Bench Unanswerable | accuracy: Claude-3.7-Thinking 0.12 -> 0.82; GPT-4o 0.42 -> 0.98 | Improved |
| add explicit instructions (ablation) | GPT-4o and Claude-3.7-Thinking · DNR Bench Indifferent | accuracy: GPT-4o 0.84 -> 0.08; Claude 0.80 -> 0.04 | Hurt |
| add explicit instructions (ablation) | DeepSeek-R1 · DNR Bench Imaginary Reference | accuracy: 0.00 -> 1.00 | Improved |
| add explicit instructions (ablation) | O3-mini-high · DNR Bench Unanswerable | accuracy: 0.00 -> 0.02 | No clear effect |
| add explicit instructions (ablation) | reasoning models · DNR Bench | token count: up to 70x tokens vs GPT-4o -> 2x to 20x | Improved |
| allow longer reasoning (more tokens) | reasoning models · DNR Bench | accuracy | Hurt |
| use controlled reasoning budget (GPT-OSS low/medium/high) (ablation) | GPT-OSS-120B · DNR Bench Unanswerable | accuracy: low 0.14, medium 0.16, high 0.12 | No clear effect |
| use Gemini-2.5-Pro adaptive reasoning | Gemini-2.5-Pro · DNR Bench | tokens and accuracy: over 2000 tokens avg, ~10x GPT-4o | Hurt |
Closest papers by meaning
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.89)
- Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens (ICML 2026 · similarity 0.89)
- AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions (NeurIPS 2025 · similarity 0.88)
- Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignment (ICML 2025 · similarity 0.88)
- General-Reasoner: Advancing LLM Reasoning Across All Domains (NeurIPS 2025 · similarity 0.88)
- Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
