Research map / Video / Video-language understanding
Infer Induced Sentiment of Comment Response to Video: A New Task, Dataset and Baseline
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the MSA-CRVI task of inferring viewers' induced opinion and emotion from comments on micro videos, releases the CSMV dataset (107,267 comments, 8,210 TikTok videos), and proposes the VC-CSA baseline.
Problem
Existing video multimodal sentiment analysis focuses on speakers' expressed sentiment and neglects viewers' induced sentiment, which comments on micro videos reveal; comments are responses rather than descriptions, so inferring their sentiment requires grounding relevant video content.
Approach
A video-comment consensus transformer with trainable consensus tokens bridges the semantic gap between comments and video, followed by multi-scale temporal CNN features, LSTM-based golden feature grounding, and token-level fusion with the comment text for opinion and emotion classification.
Key results
- VC-CSA achieves 73.52 micro-F1 on opinion and 62.99 micro-F1 on emotion, outperforming MISA (72.41 and 57.42).
- Text-only RoBERTa reaches 56.95 opinion micro-F1 and 47.27 emotion micro-F1, below all multimodal baselines, supporting the value of video input.
- Ablations drop micro-F1 by about 1-4 points when removing single-layer design, last-layer design, last-token consensus replacement, or direct attention scores.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use video and comment text with VC-CSA instead of text-only RoBERTa | RoBERTa / VC-CSA · CSMV opinion | micro F1-score: 56.95 -> 73.52 | Helped |
| Use video and comment text with VC-CSA instead of text-only RoBERTa | RoBERTa / VC-CSA · CSMV emotion | micro F1-score: 47.27 -> 62.99 | Helped |
| Use VC-CSA instead of the strongest baseline, MISA | VC-CSA · CSMV opinion | micro F1-score: 72.41 -> 73.52 | Helped |
| Use VC-CSA instead of the strongest baseline, MISA | VC-CSA · CSMV emotion | micro F1-score: 57.42 -> 62.99 | Helped |
| Add multi-scale temporal representations instead of using only a single CNN layer (ablation) | VC-CSA · CSMV opinion | micro F1: 72.35 -> 73.52 | Helped |
| Add multi-scale temporal representations instead of using only a single CNN layer (ablation) | VC-CSA · CSMV emotion | micro F1: 62.06 -> 62.99 | Helped |
| Add multi-scale temporal representations instead of using only the last CNN layer (ablation) | VC-CSA · CSMV opinion | micro F1: 69.13 -> 73.52 | Helped |
| Add multi-scale temporal representations instead of using only the last CNN layer (ablation) | VC-CSA · CSMV emotion | micro F1: 59.67 -> 62.99 | Helped |
| Use consensus tokens instead of the last token in the original transformer encoder (ablation) | VC-CSA · CSMV opinion | micro F1: 72.32 -> 73.52 | Helped |
| Use consensus tokens instead of the last token in the original transformer encoder (ablation) | VC-CSA · CSMV emotion | micro F1: 62.52 -> 62.99 | Helped |
| Use second-order grounding rather than the first-order attention score directly (ablation) | VC-CSA · CSMV opinion | micro F1: 71.93 -> 73.52 | Helped |
| Use second-order grounding rather than the first-order attention score directly (ablation) | VC-CSA · CSMV emotion | micro F1: 61.22 -> 62.99 | Helped |
Surprising findings
- Visual modality contributes less than expected in prior multimodal sentiment work, while here the authors argue video is essential for comment sentiment.
- Dataset label distribution is imbalanced, with negative opinions at 11% and sadness, fear, anger at 5%, 2%, 2% for emotion.
Limitations
- Only I3D visual features are released rather than raw videos, and audio is not included.
- Label distribution is strongly imbalanced, with few negative and fear/anger/sadness examples.
- Improvements over baselines are modest (about 1-2 micro-F1 points over MISA on opinion) with no significance testing reported.
Experimental setup
Datasets: CSMV
Benchmarks: CSMV
Metrics: micro F1 macro F1 macro recall macro precision
Baselines: BERT RoBERTa TBJE Self-MM MISA MMIM CubeMLP
Builds on: RoBERTa I3D LSTM MISA
Closest papers by meaning
- Enriching Multimodal Sentiment Analysis Through Textual Emotional Descriptions of Visual-Audio Content (AAAI 2025 · similarity 0.86)
- Enhancing Multimodal Affective Analysis with Learned Live Comment Features (AAAI 2025 · similarity 0.86)
- AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models (ICML 2025 · similarity 0.85)
- Uncertain Multimodal Intention and Emotion Understanding in the Wild (CVPR 2025 · similarity 0.83)
- An Empirical Study on Configuring In-Context Learning Demonstrations for Unleashing MLLMs' Sentimental Perception Capability (ICML 2025 · similarity 0.83)
- MART: Masked Affective RepresenTation Learning via Masked Temporal Distribution Distillation (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.