Research map / Large language models / Retrieval and knowledge

Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answer

NeurIPS 2026 · Large language models › Retrieval and knowledge · approach: Knowledge editing in LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that many LLM hallucinations occur despite substantial probability mass on the correct answer concept at the commitment step, with the rate rising with scale under instruction tuning; correct vs. wrong outcomes differ by how that mass is spread across surface forms.

Problem

Hallucination is usually attributed to missing knowledge, but it is unclear whether the correct answer is absent from the model's distribution at the moment it commits to an answer.

Approach

Defines concepts as equivalence classes of answer surface forms and a per-step semantic probability mass Pmass(t;c) that aggregates probability over alias tokens; uses it to identify "commitment failures" where the correct concept has substantial mass yet a wrong answer is emitted, and attributes them to instruction-tuning-induced sharpening of the distribution.

Key results

• Commitment-failure rate among Instruct hallucinations rises from 16% (Qwen3.5-0.8B) to 47% (Llama-3.1-70B Instruct), with Pmass(t_c;c*) >= 0.2.
• Within first-token selection failures on Qwen3.5-9B Instruct, max single-alias probability averages 0.26 vs 0.78 for matched correct samples (Cohen's d = 2.98).
• Wrong-token probability in selection failures rises from 0.31 (0.8B) to 0.57 (72B) across Qwen Instruct models but stays near 0.30 for Base models.

Surprising findings

• Larger instruction-tuned models shift their errors toward commitment failures, i.e. more hallucinations occur despite substantial correct-concept mass.
• Output-level Instruct-Base gap (+0.29 AUROC) is much larger than the hidden-state gap (+0.08).

Limitations

• Pmass requires ground-truth alias sets, so it is an analytical probe rather than a deployable detector.
• Experiments use short answers (1-3 tokens) and greedy decoding; long-form and sampling settings are left to future work.
• Analyses cover only Qwen and Llama families.

Experimental setup

Datasets: TriviaQA NQ-Open MMLU ARC-Challenge

Benchmarks: TriviaQA NQ-Open MMLU ARC-Challenge

Metrics: commitment-failure rate accuracy AUROC Cohen's d ECE Brier score

Baselines: semantic entropy SEP token-level entropy P(y_t) generated-token probability

Builds on: semantic entropy forking tokens TBG probing (SEP)

Closest papers by meaning

• Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models (NeurIPS 2025 · similarity 0.91)
• In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024 · similarity 0.90)
• The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstaining (NeurIPS 2026 · similarity 0.90)
• Attributive Reasoning for Hallucination Diagnosis of Large Language Models (AAAI 2025 · similarity 0.89)
• BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation (NeurIPS 2026 · similarity 0.89)
• Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models (AAAI 2026 · similarity 0.89)