Research map / Video / Video segmentation and tracking
MHBench: Demystifying Motion Hallucination in VideoLLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces motion hallucination in VideoLLMs, builds MHBench (1,200 triplet videos over 20 action categories), and proposes training-free MotionCD contrastive decoding using bidirectional motion elimination with reversed video.
Problem
VideoLLMs often answer action-existence questions based on static object appearance rather than true motion understanding, a failure mode (motion hallucination) not covered by existing video benchmarks.
Approach
Defines motion hallucination and evaluates it with adversarial triplets (original, antonym-action, incomplete-action videos); mitigates it via MotionCD, a training-free contrastive decoding that builds an amateur distribution by averaging logits from the original and reversed video to strip motion cues while keeping static appearance.
Key results
- MotionCD raises VideoChat2-Mistral macro F1 on action classification by 15.1 points (50.1 to 65.2) and PairAcc by 16.9 points.
- All 11 evaluated 7B VideoLLMs show severe motion hallucination; most have PairAcc near or below chance, and none answer any adversarial triplet correctly.
- Bidirectional motion elimination (reversed-video averaging) outperforms shuffle, video removal, visual noise, and static-frame amateur models on VideoChat2-Mistral (Macro F1 65.2 vs 50.1 without CD).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply MotionCD | VideoChat2-Mistral · MHBench | PairAcc: 19.4 -> 36.3 | Helped |
| Apply MotionCD | VideoChat2-Mistral · MHBench | AvgAcc: 58.3 -> 65.2 | Helped |
| Apply MotionCD | VideoChat2-Mistral · MHBench | Macro F1: 50.1 -> 65.2 | Helped |
| Apply MotionCD | VideoChat2 · MHBench | PairAcc: 31.5 -> 29.2 | Hurt |
| Apply MotionCD | VideoChat2 · MHBench | AvgAcc: 59.3 -> 59.7 | Helped |
| Apply MotionCD | VideoChat2 · MHBench | Macro F1: 34.9 -> 40.9 | Helped |
| Apply MotionCD | Video-LLaVA · MHBench | PairAcc: 7.5 -> 10.4 | Helped |
| Apply MotionCD | Video-LLaVA · MHBench | AvgAcc: 31.7 -> 45.2 | Helped |
| Apply MotionCD | Video-LLaVA · MHBench | Macro F1: 30.8 -> 33.9 | Helped |
| Add bidirectional motion elimination instead of omitting contrastive decoding (ablation) | VideoChat2-Mistral · MHBench | Macro F1: 50.1 -> 65.2 | Helped |
| Use bidirectional motion elimination rather than random frame shuffling (ablation) | VideoChat2-Mistral · MHBench | Macro F1: 58.1 -> 65.2 | Helped |
| Use bidirectional motion elimination rather than removing the video input (ablation) | VideoChat2-Mistral · MHBench | Macro F1: 48.2 -> 65.2 | Helped |
Surprising findings
- Models with the same LLM backbone family can differ sharply; VideoChat2-Mistral outperforms VideoChat2 (Vicuna) largely due to backbone choice, suggesting LLM backbone strongly affects motion hallucination.
- Several models (e.g., Valley2, VideoLaVIT) answer 'yes' almost always on the binary task, showing that strong MVBench scores do not translate to motion understanding.
Limitations
- MHBench is small (1,200 videos, 20 action categories) and the authors acknowledge limited scale and evaluation granularity.
- MotionCD does not fully solve motion hallucination; gains vary across models (e.g., VideoChat2 binary accuracy drops slightly).
- Hyperparameters (alpha, beta) were tuned by grid search on VideoChat2-Mistral and transferred to other models.
Experimental setup
Datasets: something-something v2 MVBench
Benchmarks: MHBench MVBench
Metrics: AvgAcc YesAcc NoAcc PairAcc PctDiff FPR Macro Precision Macro Recall Macro F1
Baselines: VideoChatGPT Valley2 VideoChat2 Video-LLaVA LLaMA-VID VideoLaVIT Video-LLaMA2 MiniGPT4-Video VideoChat2-Mistral PLLaVA LLaVA-NeXT-Video-DPO VCD
Builds on: Visual Contrastive Decoding MVBench something-something v2
Closest papers by meaning
- Mitigating Hallucination in VideoLLMs via Temporal-Aware Activation Engineering (NeurIPS 2025 · similarity 0.92)
- MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representations (CVPR 2025 · similarity 0.92)
- Learning to Decode Against Compositional Hallucination in Video Multimodal Large Language Models (ICML 2026 · similarity 0.92)
- VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understanding (CVPR 2025 · similarity 0.91)
- One Token, Two Fates: A Unified Framework via Vision Token Manipulation Against MLLMs Hallucination (CVPR 2026 · similarity 0.90)
- Analyzing and Mitigating Object Hallucination: A Training Bias Perspective (AAAI 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.