AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

CMMA: Benchmarking Multi-Affection Detection in Chinese Multi-Modal Conversations

Yazhou Zhang, Yang Yu, Qing Guo, Benyou Wang, Dongming Zhao, Sagar Uprety, Dawei Song, Qiuchi Li, Jing Qin

NeurIPS 2023 · Vision-language and multimodal › Human and embodied modeling · approach: Multimodal and affective interaction modeling · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add video (ResNet) and audio (VGGish) to BERT text-only model (ablation)BERT multimodal baseline · CMMA sentimentmacro-F1: 54.89 -> 73.29Improved
replace EfficientNet video encoder with ResNet in BERT trimodal model (ablation)BERT+VGGish model · CMMA sentimentmacro-F1: 71.12 -> 73.29Improved
replace BERT text encoder with GPT-2 in trimodal model (ablation)ResNet+VGGish model · CMMA sarcasmmacro-F1: 75.23 -> 70.67Hurt
replace BiLSTM with BERT as text encoder (ablation)text-only model · CMMA sentimentmacro-F1: 50.74 -> 54.89Improved
add audio to BERT+ResNet bimodal model (ablation)BERT+ResNet · CMMA sarcasmmacro-F1: 66.75 -> 75.23Improved
use multi-head attention instead of multiplication for fusion (ablation)trimodal model · CMMA sentimentaccuracy: 72.22 -> 78.48Improved
use multi-head attention instead of concatenation for fusion (ablation)trimodal model · CMMA emotionaccuracy: 76.32 -> 77.09Improved
increase conversational context length from 0 to 2 (ablation)trimodal model · CMMA sentimentmacro-F1: 73.29 -> 76.91Improved
increase conversational context length from 0 to 1 (ablation)trimodal model · CMMA sentimentmacro-F1: 73.29 -> 71.36Hurt
increase conversational context length from 0 to 2 (ablation)trimodal model · CMMA emotionmacro-F1: 61.76 -> 68.79Improved
add speaker information (ablation)trimodal model · CMMA humormacro-F1: 75.91 -> 72.06Hurt
add speaker information (ablation)trimodal model · CMMA emotionmacro-F1: 61.76 -> 64.23Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy