Research map / Large language models / Reasoning and chain of thought

Answering the Unanswerable Is to Err Knowingly: Analyzing and Mitigating Abstention Failures in Large Reasoning Models

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyzes why large reasoning models fail to abstain on unanswerable math problems, finds they internally encode answerability, and proposes a training-free two-stage method (linear probe monitoring plus inference-time prompt intervention with early exit) that raises abstention rates.

Problem

Large reasoning models often hallucinate answers or fall into prolonged reasoning instead of abstaining when math questions are unanswerable due to missing or ill-posed conditions.

Approach

Linear probes on hidden activations detect answerability during reasoning; when unanswerability is detected, an instructional prompt restating the possibility of unanswerability is injected and reasoning is exited early, converting internal recognition into explicit abstention without retraining.

Key results

• On SUM with R1-Distill-Qwen-7B, the method raises abstention rate from 21.13% (vanilla) to 73.94%, versus 56.34% for Dynasor-CoT.
• On UMWP with Qwen3-14B, abstention rises from 82.33% to 92.67% while answer accuracy on answerable questions stays near 92.5% (vanilla 94.33%).
• Linear probes on hidden states reach AUROC 0.879–0.936 on SUM across five LRMs, indicating answerability is linearly encoded during reasoning.

Surprising findings

• Most LRMs fail to abstain on more than half of unanswerable questions, and larger capacity does not fix this.
• Early-exit baselines (Dynasor-CoT, DEER) without guidance increase hallucinated answers, since the model makes up missing conditions to give a definite answer.

Limitations

• Evaluated only on math word-problem style unanswerability (SUM and UMWP), not other domains.
• Answer accuracy on answerable questions drops for some models (e.g., Qwen3-8B 66.55 to 61.62 on SUM-type metric; R1-Distill-Qwen-7B on SUM 69.72 to 67.25), and token usage on answerable questions sometimes rises.
• Probe thresholds are set per dataset (0.6 for SUM, 0.5 for UMWP), requiring tuning.

Experimental setup

Datasets: SUM UMWP SVAMP MultiArith GSM8K ASDiv Omni-MATH Still AIME AMC

Benchmarks: SUM UMWP

Metrics: abstention rate reason accuracy token usage answer accuracy AUROC F1

Baselines: Vanilla Dynasor-CoT DEER

Builds on: Dynasor-CoT DEER

Closest papers by meaning

• AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions (NeurIPS 2025 · similarity 0.91)
• An Enigma of Artificial Reason: Investigating the Production-Evaluation Gap in Large Reasoning Models (NeurIPS 2026 · similarity 0.89)
• Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities (AAAI 2026 · similarity 0.89)
• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity (NeurIPS 2025 · similarity 0.89)
• Are Large Reasoning Models Interruptible? (ICML 2026 · similarity 0.88)
• RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models (ICLR 2026 · similarity 0.88)