Research map / Video / Video-language understanding
How Would The Viewer Feel? Estimating Wellbeing From Video Scenarios
Mantas Mazeika, Eric Tang, Andy Zou, Steven Basart, Jun Shern Chan, Dawn Song, David Forsyth, Jacob Steinhardt, Dan Hendrycks
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VideoMAE rather than the strongest listed alternative, STAM | VCE emotion prediction · VCE test | top-3 accuracy: 66.4% -> 68.9% | Improved |
| Use a video Transformer rather than R(2+1)D | VCE emotion prediction · VCE test | top-3 accuracy: 65.6% -> 68.9% | Improved |
| Use zero-shot CLIP rather than the majority-emotion predictor | VCE emotion prediction · VCE test | top-3 accuracy: 35.7% -> 28.4% | Hurt |
| Pretrain STAM-16 on Kinetics (ablation) | STAM-16 wellbeing prediction · V2V test | pairwise accuracy: 63.4% -> 86.7% | Improved |
| Pretrain STAM-8 on Kinetics (ablation) | STAM-8 wellbeing prediction · V2V test | pairwise accuracy: 62.8% -> 84.4% | Improved |
| Pretrain STAM-16 on Kinetics (ablation) | STAM-16 wellbeing prediction · V2V test | listwise accuracy: 26.7% -> 44.6% | Improved |
| Pretrain STAM-8 on Kinetics (ablation) | STAM-8 wellbeing prediction · V2V test | listwise accuracy: 26.9% -> 38.8% | Improved |
| Add the VCE emotion-prediction loss to STAM-8 (ablation) | STAM-8 wellbeing prediction · V2V test | pairwise accuracy: 62.8% -> 63.2% | No clear effect |
| Add the VCE emotion-prediction loss to STAM-16 (ablation) | STAM-16 wellbeing prediction · V2V test | pairwise accuracy: 63.4% -> 63.4% | No clear effect |
| Add the VCE emotion-prediction loss to STAM-8 (ablation) | STAM-8 wellbeing prediction · V2V test | listwise accuracy: 26.9% -> 25.7% | Hurt |
| Add the VCE emotion-prediction loss to STAM-16 (ablation) | STAM-16 wellbeing prediction · V2V test | listwise accuracy: 26.7% -> 26.8% | No clear effect |
| Use all 27 emotion annotations rather than joy alone as a feature (ablation) | Logistic regression for V2V pleasantness · V2V test | pairwise accuracy: 51% -> 89.6% | Improved |
Closest papers by meaning
- $E^3$: Exploring Embodied Emotion Through A Large-Scale Egocentric Video Dataset (NeurIPS 2024 · similarity 0.87)
- VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation Models (NeurIPS 2025 · similarity 0.86)
- FEEL: Quantifying Heterogeneity in Physiological Signals for Generalizable Emotion Recognition (NeurIPS 2025 · similarity 0.85)
- Infer Induced Sentiment of Comment Response to Video: A New Task, Dataset and Baseline (NeurIPS 2024 · similarity 0.85)
- Bridge Then Begin Anew: Generating Target-Relevant Intermediate Model for Source-Free Visual Emotion Adaptation (AAAI 2025 · similarity 0.85)
- EEVR: A Dataset of Paired Physiological Signals and Textual Descriptions for Joint Emotion Representation Learning (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
