Research map / Vision-language and multimodal / Multimodal LLMs

Learning to Decode Against Compositional Hallucination in Video Multimodal Large Language Models

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Decoding-time hallucination mitigation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces OmniVCHall, a video hallucination benchmark with compositional (multi-type) queries and adversarial options, and TriCD, a triple-pathway contrastive decoding method whose perturbation controller and saliency enhancer are trained with REINFORCE on a frozen VLLM.

Problem

Existing video hallucination benchmarks and mitigation methods target isolated hallucination types, leaving compositional hallucinations (incorrect reasoning over multiple interacting spatial and temporal factors) underexplored in video MLLMs.

Approach

Combines a learned adaptive perturbation controller that selects video corruption tools to build negative samples with a saliency-guided enhancer (DINOv3 attention plus Farneback optical flow) that builds positive samples, and calibrates logits via triple-pathway contrastive decoding, with the selectors trained by policy gradient on the frozen VLLM.

Key results

• TriCD raises Qwen3-VL-Instruct-8B average accuracy on OmniVCHall from 0.67 to 0.73 (+9.61%).
• TriCD raises VideoLLaMA3-7B average accuracy from 0.60 to 0.67 (+12.03%).
• Human baseline reaches 0.95 average accuracy versus 0.76 for the best evaluated model (Qwen3-VL-Thinking-235B), while 39 VLLMs are evaluated.

Surprising findings

• Random perturbation selection drops Qwen3-VL-Instruct-8B accuracy below the unmodified backbone (0.63 vs 0.67).
• Models score lowest on camera-related hallucinations, the newly introduced type.

Limitations

• Only two backbones (Qwen3-VL-8B and VideoLLaMA3-7B) are used for the main method results.
• Compositional hallucination benchmark relies on Gemini-2.5-Pro-generated QA that was manually curated, with a modest-size dataset of 823 videos.

Experimental setup

Datasets: VidHalluc MHBench EventHallusion online-searched YouTube videos AI-generated videos (Sora, Veo, Wan, etc.)

Benchmarks: OmniVCHall

Metrics: accuracy yes/no accuracy macro precision macro recall macro F1

Baselines: MotionCD TCD DINO-HEAL

Builds on: contrastive decoding MotionCD TCD DINOv3 REINFORCE

Closest papers by meaning

• Mitigating Hallucination in VideoLLMs via Temporal-Aware Activation Engineering (NeurIPS 2025 · similarity 0.94)
• VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understanding (CVPR 2025 · similarity 0.93)
• Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation (ICLR 2026 · similarity 0.93)
• Reducing Hallucinations in Large Vision-Language Models via Latent Space Steering (ICLR 2025 · similarity 0.93)
• Hallucination-aware Intermediate Representation Editing in Large Vision-Lanugage Models (ICLR 2026 · similarity 0.92)
• Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs (AAAI 2026 · similarity 0.92)