Research map / Vision-language and multimodal / Multimodal LLMs

EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces EmotionHallucer, an adversarial binary-QA benchmark for emotion hallucinations in LLMs/MLLMs across emotion psychology knowledge and multimodal perception, evaluating 38 models and proposing the training-free PEP-MEK prompting framework.

Problem

MLLMs hallucinate in emotion understanding, but no dedicated benchmark exists to evaluate emotion-related hallucinations across psychology knowledge and multimodal emotion perception.

Approach

Pairs each basic question with an intentionally hallucinated counterpart (adversarial binary QA, scored jointly), split into emotion psychology knowledge (theory, definition, finding) and multimodal perception (category, intensity, reasoning result, reasoning cue) across text, image, audio, and video; plus a predict-explain-predict prompting framework with modality and emotion knowledge extraction

Key results

• Across 38 LLMs and MLLMs, most models show substantial emotion hallucination; open-source models all fall below 25% overall accuracy (random-guess level) on the full benchmark.
• On EmotionHallucer-NoAudio, Gemini-2.5-Pro reaches 51.58% overall accuracy, the best result, versus 43.02% for Qwen2.5-VL-72B.
• PEP-MEK yields an average 9.90% improvement in emotion hallucination detection across three selected models; Emotion-LLaMA gains 16.38 points in overall accuracy.

Surprising findings

• Models perform better on emotion psychology knowledge than on multimodal emotion perception, with accuracy dropping from text to image to audio and video.
• Emotion-specific model Emotion-LLaMA shows significantly worse yes/no bias than general-purpose MLLMs.

Limitations

• Hallucinated questions are manually constructed by annotators, which limits scale.
• Multimodal coverage is uneven: many MLLMs cannot process audio or video, so comparisons use subsets.
• PEP-MEK is evaluated on only three models and its gains are reported as an average across them.

Experimental setup

Datasets: Emotion (Shiota & Kalat textbook) SOUL Twitter15 Twitter17 RAVDESS MER 2023 Social-IQ 2.0

Benchmarks: EmotionHallucer EmotionHallucer-NoAudio EmotionHallucer-P

Metrics: accuracy (basic, hallucinated, overall paired) Yes Percentage Difference False Positive Ratio

Baselines: Qwen2.5-Omni Emotion-LLaMA Gemini-2.5-Flash Gemini-2.5-Pro GPT-4o GPT-4.1 QvQ-Max Qwen2.5-VL

Builds on: POPE CHAIR Chain-of-thought

Closest papers by meaning

• LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.90)
• Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback (AAAI 2025 · similarity 0.89)
• Detecting and Preventing Hallucinations in Large Vision Language Models (AAAI 2024 · similarity 0.89)
• Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning (AAAI 2025 · similarity 0.89)
• MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM (NeurIPS 2025 · similarity 0.89)
• HaDeMiF: Hallucination Detection and Mitigation in Large Language Models (ICLR 2025 · similarity 0.89)