Research map / Large language models / Reasoning and chain of thought

BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMs

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Identifies factual overthinking in large reasoning models and proposes BARREL, an SFT-plus-GRPO pipeline with a three-level reward that teaches LRMs to refuse ("I don't know") on unknown questions and stay confident on known ones.

Problem

Large reasoning models rarely admit ignorance and often give confident incorrect answers on factual questions, with incorrect answers associated with longer, overthinking reasoning traces.

Approach

Diagnoses two overthinking patterns (last-minute guessing and second-thought spiraling), builds evidence-grounded SFT traces for known and unknown questions, then applies GRPO with a three-level reward (correct > truthful refusal > incorrect) so refusal is rewarded rather than penalized equally with wrong answers.

Key results

• On DeepSeek-R1-Distill-Llama-8B, BARREL raises factual reliability from 39.33% (Distill) to 61.48% (abstract; 61.58% in table) while keeping accuracy at 40.7% vs 38.43% for Distill.
• On Qwen3-8B, BARREL reaches 71.46% reliability versus 58.94% for the Probing baseline.
• Removing the refusal reward in GRPO leads the model to almost never admit uncertainty, showing refusal reward is necessary.

Surprising findings

• Removing the medium refusal reward makes GRPO almost never refuse, so current RL paradigms implicitly do not reward admitting ignorance.
• Post-hoc verbal-confidence and probing baselines are severely overconfident, with probing AUC near 0.53.

Limitations

• Due to limited compute, experiments are restricted to 7-8B distilled and Qwen3-8B models.
• Accuracy on known questions can drop relative to vanilla GRPO because more refusals are issued.
• Evaluation uses string matching with about 88% agreement with human labels.

Experimental setup

Datasets: TriviaQA SciQ NQ-Open MATH

Benchmarks: TriviaQA SciQ NQ-Open MATH500 SimpleQA (OOD unknown) SelfAware (unanswerable)

Metrics: accuracy truthfulness reliability abstention rate pass@k

Baselines: ICL ICL-IDK Distill Vanilla GRPO Vanilla GRPO w/ Verbal Confidence Vanilla GRPO w/ Probing

Builds on: GRPO R-Tuning Reliability metric from prior factuality work

Closest papers by meaning

• Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression (AAAI 2026 · similarity 0.87)
• Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring (NeurIPS 2026 · similarity 0.87)
• Stop When Further Reasoning Won’t Help: Attention-State Adaptive Generation in Reasoning Models (ICML 2026 · similarity 0.87)
• How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.87)
• VeriThinker: Learning to Verify Makes Reasoning Model Efficient (NeurIPS 2025 · similarity 0.87)
• Reasoning Structure of Large Language Models (ICML 2026 · similarity 0.86)