AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

VCGD: Visual Clue Guided Decoding with Caption Model for Mitigating Hallucination in Multimodal Large Language Models

Guoqing Chen, Fu Zhang, Bingqian Liu, Chenglong Lu, Jingwei Cheng

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Decoding-time hallucination mitigation · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VCGD, a visual-clue-guided decoding method that uses a caption model to mitigate hallucinations in multimodal LLMs.

Problem

Hallucinations in multimodal large language models, where generated outputs contain erroneous or fabricated information; existing contrastive-decoding work relies on manually perturbed inputs as references.

Approach

Instead of manually perturbing visual or instruction inputs to induce hallucinations, the method uses a caption model to derive visual clues that guide decoding of the MLLM. Only the abstract was available, so the exact mechanism is not described here.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train caption model (Qwen2.5-VL-3B) with RA-GRPO (ablation)VCGD on InternVL-26B · MMVPaverage accuracy: 44.8 -> 51.4Improved
train caption model (Qwen2.5-VL-7B) with RA-GRPO (ablation)VCGD on InternVL-26B · MMVPaverage accuracy: 48.0 -> 55.9Improved
train caption model (Qwen2.5-VL-7B) with RA-GRPO (ablation)VCGD on InternVL-26B · LLaVA-Bench QA90overall score: 86.6 -> 94.8Improved
train caption model (Qwen2.5-VL-7B) with RA-GRPO (ablation)VCGD on InternVL-26B · MMHal-Benchhallucination rate: 33.3 -> 31.2Improved
scale caption model from 3B to 7B (no training) (ablation)VCGD on InternVL-26B · MMVPaverage accuracy: 44.8 -> 48.0Improved
scale caption model from 7B to 32B (no training) (ablation)VCGD on InternVL-26B · MMVPaverage accuracy: 48.0 -> 48.2No clear effect
use Qwen2.5-VL-72B instead of 7B caption model (no training) (ablation)VCGD on InternVL-26B · MMVPaverage accuracy: 48.0 -> 52.6Improved
use RA-GRPO-trained 7B instead of OpenAI o3 as caption model (ablation)VCGD on InternVL-26B · MMVPaverage accuracy: 52.9 -> 55.9Improved
add image confidence constraint on alpha_t (ablation)VCGD with injected hallucinated clues · hallucination-injected cluesbenchmark performanceImproved
use all four reward terms in Reward Agent rather than subsets (ablation)VCGD caption model training · MMVP, POPE, MMHal-BenchaverageImproved
replace CODE with VCGD decodingLLaVA-1.5-13B · MMVPaverage accuracy: 34.0 -> 37.2Improved
replace CODE with VCGD decodingLLaVA-NeXT-34B · MMVPaverage accuracy: 42.6 -> 47.5Improved

Experimental setup

Builds on: contrastive decoding

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy