AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

DOUBT: Decoupled Object-level Understanding and Bridging via vMF-based Trustworthiness for Hallucination Detection in MLLMs

Kaiqi Chen, Yang Qin, Changhao He, Xi Peng, Peng Hu

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Decoding-time hallucination mitigation

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DOUBT, a hallucination detection method for MLLMs that decouples object-level visual understanding from language and uses vMF-based trustworthiness scores.

Problem

Hallucinations in multimodal LLMs (assertions contradicting the image or facts) undermine reliability; existing detectors feed image and text jointly and measure output consistency, but the visual module often lags behind the language module in understanding.

Approach

Decouples object-level understanding from bridging with the language side and estimates trustworthiness using von Mises-Fisher (vMF)-based modeling, rather than jointly feeding image and text and measuring output consistency.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add object-level understanding and bridging (OUB) (ablation)DOUBT · MMMU (avg over 10 MLLMs)detection accuracy (%): 59.77 -> 60.22Improved
add object-level understanding and bridging (OUB) (ablation)DOUBT · ScienceQA (avg over 10 MLLMs)detection accuracy (%): 75.93 -> 76.07No clear effect
add object-level understanding and bridging (OUB) (ablation)DOUBT · MMMU (Qwen2VL-2B)detection accuracy (%): 60.85 vs 61.94Hurt
add vMF-based trustworthiness in place of semantic entropy (ablation)DOUBT · MMMU (avg over 10 MLLMs)detection accuracy (%): 60.01 -> 60.22Improved
add vMF-based trustworthiness in place of semantic entropy (ablation)DOUBT · ScienceQA (avg over 10 MLLMs)detection accuracy (%): 67.10 -> 76.07Improved
add both OUB and vMF trustworthiness (ablation)DOUBT · MMMU (avg over 10 MLLMs)detection accuracy (%): 56.70 -> 60.22Improved
add both OUB and vMF trustworthiness (ablation)DOUBT · ScienceQA (avg over 10 MLLMs)detection accuracy (%): 62.03 -> 76.07Improved
increase sampling number K beyond 10 (ablation)DOUBT · MMVet (Qwen2VL-2B)detection accuracy (%)No clear effect
increase top-p (ablation)DOUBT · MMVet (Qwen2VL-2B)detection accuracy (%)Improved
two-stage OUB design versus VL-UncertaintyDOUBT · LLaVABench (Qwen2VL-2B)time per sample (s): 9.82 -> 34.01Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy