Research map / Vision-language and multimodal / Multimodal LLMs

EEmo-Logic: A Unified Dataset and Multi-Stage Framework for Comprehensive Image-Evoked Emotion Assessment

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds EEmoDB, a ~1.2M-QA plus 36K-assessment dataset for image-evoked emotion, and trains EEmo-Logic (Qwen2.5-VL-7B) with LoRA SFT followed by task-customized GRPO using hybrid regression, ranking, and similarity rewards.

Problem

Existing MLLMs for image-evoked emotion offer only coarse emotion perception or weak reasoning, and they lack a unified treatment of categorical (dominant emotion, emotion ranking) and dimensional (VAD) emotion assessment with intensity nuances.

Approach

Combines a large automatically generated emotion QA corpus (SFT with LoRA) with GRPO on a curated assessment set, using three task-specific rewards: a ranking reward (weighted hit rate plus Kendall's tau), a funnel-shaped hybrid regression reward for VAD scores, and a VAD-plus-embedding similarity reward for dominant emotion classification.

Key results

• EEmo-Logic reaches 68.54% overall perception accuracy and 67.97% emotion ranking score on EEmo-Bench, versus 61.88% for Qwen2.5-VL-7B on ranking.
• On the assessment task, EEmo-Logic (Think) reaches overall SRCC/PLCC of 0.73/0.71, and dominance SRCC of 0.78 versus 0.32 for Qwen2.5-VL-7B under the probability-based protocol.
• On cross-domain tests, EEmo-Logic achieves the best ArtEmis F1/ACC (31.58%/35.10%) and the best AesBench AesE overall (75.68%), though Emotion-Qwen scores higher on UNIAA Sentiment (79.53% vs 75.65%).

Surprising findings

• Full-parameter SFT in Stage 1 gives higher ranking score (64.02% vs LoRA 63.01%) in some settings, and the authors attribute LoRA's advantage in perception to better regularization against overfitting to label noise.
• Emotion-Qwen outperforms EEmo-Logic on the UNIAA Sentiment subset (79.53% vs 75.65%).

Limitations

• Much of the training and evaluation labels are generated by DeepSeek and rule-based pipelines from source annotations, with QA accuracy checked only on a sample of 400 items (82.88%).
• Continuous VAD training labels come largely from IESN lexicon synthesis and are only partially validated by humans (SRCC about 0.62 to 0.71).
• Evaluation of the description task relies on a DeepSeek LLM judge, which may be biased.

Experimental setup

Datasets: Affection Abstract-8 EMOTIC Emotion6 FindingEmo GAPED OASIS ArtEmis Artphoto UNIAA AesBench EEmo-Bench

Benchmarks: EEmo-Bench Artphoto ArtEmis UNIAA AesBench

Metrics: accuracy SRCC PLCC F1 description score

Baselines: EmoVIT AffectGPT EmoCaliber Emotion-Qwen R1-Omni-0.5B GPT-4o Gemini-2.5-Pro Qwen2.5-VL-72B

Builds on: GRPO LoRA IESN EEmo-Bench Qwen2.5-VL

Closest papers by meaning

• Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach (ICLR 2026 · similarity 0.89)
• Nano-EmoX: Unifying Multimodal Emotional Intelligence from Perception to Empathy (CVPR 2026 · similarity 0.88)
• AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models (ICML 2025 · similarity 0.88)
• EmoThinker: Advancing Visual-Acoustic Emotion Analysis via Structural Token Selection and Chain-of-Thought Reasoning (CVPR 2026 · similarity 0.88)
• EmoNet-Face: An Expert-Annotated Benchmark for Synthetic Emotion Recognition (NeurIPS 2025 · similarity 0.87)
• $E^3$: Exploring Embodied Emotion Through A Large-Scale Egocentric Video Dataset (NeurIPS 2024 · similarity 0.87)