Research map / Video / Video-language understanding

VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understanding

CVPR 2025 · Video › Video-language understanding · approach: Video LLMs with fine-grained understanding · 16 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces VIDHALLUC, a 5,002-video benchmark for action, temporal-sequence, and scene-transition hallucinations in video MLLMs built from semantically similar but visually dissimilar video pairs, plus DINO-HEAL, a training-free DINOv2-saliency reweighting of visual features.

Problem

Hallucinations in video MLLMs are underexplored; existing video hallucination benchmarks are small and focus on static content rather than dynamic temporal aspects such as action order and scene transitions.

Approach

Mine video pairs that CLIP/SigLIP rates semantically similar but DINOv2 rates visually dissimilar, use them to build adversarial action, temporal-order, and scene-transition questions, and mitigate hallucinations at inference by reweighting CLIP-style visual features with DINOv2 attention saliency.

Key results

• GPT-4o achieves 81.15% binary QA and 90.95% MCQ accuracy on action hallucination, still below human performance (95.14%).
• DINO-HEAL yields an average gain of 3.02% across hallucination categories and five models, e.g. VideoLLaMA2 STH +18.33%.
• Most open-source models score below 50% on temporal sequence hallucination (except several at 64-84%) and below 40% on scene transition for most models.

Measured effects reported in the paper

Surprising findings

• Model size and number of input frames do not correlate directly with performance; LLaVA-NeXT-Video-34B underperforms VILA1.5-13B.
• DINO-HEAL gives little or no gain on scene transition in most models, attributed to DINOv2's foreground bias.

Limitations

• DINOv2 foreground bias limits gains on background scene-transition hallucination.
• Automatically generated QAs required human correction (about 88.76% accuracy before correction of generated QAs).
• Improvements from DINO-HEAL are modest on average (3.02%).

Experimental setup

Datasets: ActivityNet YouCook2 VALOR32K

Benchmarks: VIDHALLUC

Metrics: accuracy MCC-based classification score description similarity score

Baselines: Video-ChatGPT Video-LLaVA ShareGPT4Video Chat-UniVi LLaVA-NeXT-Video PLLaVA VideoLLaMA2 VILA1.5 Gemini-1.5-Pro GPT-4o

Builds on: Video-ChatGPT DINOv2 CLIP SigLIP

Closest papers by meaning

• Learning to Decode Against Compositional Hallucination in Video Multimodal Large Language Models (ICML 2026 · similarity 0.93)
• Mitigating Hallucination in VideoLLMs via Temporal-Aware Activation Engineering (NeurIPS 2025 · similarity 0.92)
• AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models (ICLR 2025 · similarity 0.92)
• Reducing Hallucinations in Large Vision-Language Models via Latent Space Steering (ICLR 2025 · similarity 0.92)
• SVHalluc: Benchmarking Speech-Vision Hallucination in Audio-Visual Large Language Models (CVPR 2026 · similarity 0.92)
• Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Models (CVPR 2026 · similarity 0.92)