Research map / Large language models / Reasoning and chain of thought
Mind the Gap: Catching Hallucinations via Evidence Drop on the Reasoning Manifold
Qunjie Chen, Yufei Chen, Xiaodong Yue, Linye Li
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a step-wise, dynamics-based hallucination detector for LLM reasoning that tracks evidence change along the reasoning trajectory rather than averaging sequence-level uncertainty.
Problem
Hallucinations in LLM reasoning, where fluent chains of thought become factually or logically wrong; existing uncertainty-based detectors use sequence-level averaging that ignores step-wise reasoning dynamics and misclassifies hard-but-correct or easy-but-wrong samples.
Approach
Models reasoning as a trajectory on a reasoning manifold and detects hallucinations via the "evidence drop" across steps, replacing sequence-level averaging with a dynamic, step-wise signal. Full method details are not available from the abstract.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace sequence-average with worst-case Evidence Drop aggregation (ablation) | Shannon entropy, Qwen3-8B · MATH (alpha=0.05) | selective accuracy (%): 70.24 -> 88.26 | Improved |
| replace average with Evidence Drop (ablation) | Shannon entropy, Qwen3-8B · MATH | AURC (x1000): 288.8 -> 190.0 | Improved |
| replace average with Evidence Drop (ablation) | LogTokU, Qwen3-4B · MATH | AURC (x1000): 481.3 -> 299.1 | Improved |
| replace average with Evidence Drop (ablation) | LogTokU, Qwen3-4B · GSM8K (alpha=0.05) | selective accuracy (%): 0 -> 85.57 | Improved |
| replace average with Evidence Drop (ablation) | LN-S, Qwen3-4B · GSM8K (alpha=0.05) | selective accuracy (%): 95.45 -> 83.33 | Hurt |
| replace average with Evidence Drop (ablation) | LN-S, Qwen3-8B · GSM8K (alpha=0.05) | selective accuracy (%): 98.68 -> 94.44 | Hurt |
| replace average with Evidence Drop (ablation) | LN-S, Qwen3-8B · MATH (alpha=0.05) | selective accuracy (%): 69.79 -> 86.94 | Improved |
| replace average with Evidence Drop (ablation) | Shannon entropy, Qwen3-8B · GSM8K (alpha=0.50) | selective accuracy (%): 97.06 -> 93.72 | Hurt |
| use Shannon entropy instead of LN-S or LogTokU as the evidence proxy (ablation) | Evidence Drop · GSM8K Qwen3-4B (alpha=0.05) | selective accuracy (%): 83.33 -> 100 | Improved |
| replace average with Evidence Drop (ablation) | Shannon entropy, Qwen3-8B · GSM8K step localization | step-level localization accuracy (%): 27.66 -> 46.11 | Improved |
| replace average with Evidence Drop (ablation) | Shannon entropy, Qwen3-8B · MATH step localization | step-level localization accuracy (%): 24.62 -> 32.90 | Improved |
| replace average with Evidence Drop (ablation) | Shannon entropy, Qwen3.5-27B · MATH (alpha=0.05) | selective accuracy (%): 62.50 -> 92.92 | Improved |
Closest papers by meaning
- Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models (NeurIPS 2025 · similarity 0.92)
- Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models (AAAI 2026 · similarity 0.91)
- Scalable Token-Level Hallucination Detection in Large Language Models (NeurIPS 2026 · similarity 0.91)
- FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees (ICML 2025 · similarity 0.91)
- MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM (NeurIPS 2025 · similarity 0.91)
- Mitigating Hallucinations in Large Language Models via Causal Reasoning (AAAI 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
