Research map / Label-efficient and robust learning / Recognition and long-tail

Beyond Single Emotion: Multi-label Approach to Conversational Emotion Recognition

AAAI 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Partial and multi-label learning · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ML-ERC, which turns single-label emotion recognition in conversation into multi-label prediction via pseudo multi-labels from emotion shifts and a weighted multi-label supervised contrastive loss (MulWCL).

Problem

Emotion recognition in conversation (ERC) struggles with emotion shifts and confusing labels because each utterance is forced into a single emotion label, discarding co-occurring emotions.

Approach

Generate pseudo multi-emotion labels for utterances at speaker emotion shifts (based on emotion inertia), and train with a multi-label weighted supervised contrastive loss that down-weights repulsion for sub-emotion samples by valence-arousal label similarity and down-weights high-entropy instances, plus entropy-gated soft multi-labeling.

Key results

• Adding the ML-ERC loss improved RoBERTa weighted F1 on EmoryNLP from 33.08 to 35.74, and on MELD from 64.27 to 65.15.
• On IEMOCAP, ML-ERC improved DAG-ERC from 67.07 to 68.12 and CoMPM from 66.47 to 68.00 (single-label weighted F1).
• In multi-label evaluation on IEMOCAP, ML-ERC reached macro-F1 68.58 and hamming loss 0.1312 versus 59.39 and 0.2421 for MPLP.

Measured effects reported in the paper

Surprising findings

• Emotion-shift utterances show a 3-20% weighted-F1 drop relative to the full test set across models, indicating shift utterances are a major weakness of ERC models.
• Ablation shows that the single-label F1 gains are not limited to the multi-label setting; the multi-label objective also improves single-label classification on confusing-label pairs.

Limitations

• Pseudo multi-labels are generated by a heuristic rule from emotion shifts rather than human annotation.
• Multi-label evaluation relies on pseudo labels, and gains on emotion-shift subsets are not uniform across all baselines.

Experimental setup

Datasets: EmoryNLP MELD IEMOCAP

Benchmarks: EmoryNLP MELD IEMOCAP

Metrics: weighted F1 macro F1 AUC hamming loss

Baselines: DialogueRNN DAG-ERC CoMPM MPLP RoBERTa-large SCL JSCL JSPCL SLCL ICL

Builds on: SupCon RoBERTa DialogueRNN DAG-ERC CoMPM

Closest papers by meaning

• Utterance-level Emotion Recognition in Conversation with Conversation-level Supervision (AAAI 2025 · similarity 0.89)
• CARAT: Contrastive Feature Reconstruction and Aggregation for Multi-Modal Multi-Label Emotion Recognition (AAAI 2024 · similarity 0.86)
• Dual-View Learning for Conversational Emotion Recognition Through Context and Emotion-Shift Modeling (AAAI 2025 · similarity 0.85)
• Adaptive Graph Learning for Multimodal Conversational Emotion Detection (AAAI 2024 · similarity 0.85)
• EmoGrowth: Incremental Multi-label Emotion Decoding with Augmented Emotional Relation Graph (ICML 2025 · similarity 0.85)
• Multi-to-Single: Reducing Multimodal Dependency in Emotion Recognition Through Contrastive Learning (AAAI 2025 · similarity 0.84)