Research map / Label-efficient and robust learning / Recognition and long-tail
Dual-View Learning for Conversational Emotion Recognition Through Context and Emotion-Shift Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DVL-CER, which adds an emotion-shift (emotion-view) Transformer to a conversational context (utterance-view) network and couples them with two MSE-trained projection heads that map emotion distributions across views for conversational emotion recognition.
Problem
Conversational emotion recognition models mainly model utterance-level context and ignore intra-speaker emotion shifts, and existing dual-view methods share representations rather than emotion category distributions.
Approach
An emotion-view network (Transformer over the initial emotion distribution and the utterance span up to the speaker's next emotion) is trained jointly with the utterance-view network; two non-linear projection heads map each view's emotion distribution onto the other with MSE losses, and only the utterance-view network is kept at inference.
Key results
- DVL-CER reaches 70.11 weighted-F1 on IEMOCAP (+1.10 over DialogueCRN backbone at 69.01) and 39.92 on EmoryNLP (+0.95).
- Applied to DialogueRNN, COSMIC, RGAT, DAG-ERC-W and SACL-LSTM, DVL-CER improves IEMOCAP scores by 0.41 to 1.74 points.
- Dual-view projection learning outperforms feature aggregation, subspace and alignment dual-view strategies on IEMOCAP (70.11 vs. 69.73, 69.55, 69.19).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DVL-CER dual-view learning to DialogueCRN | DialogueCRN · IEMOCAP | weighted-F1: 69.01 -> 70.11 | Helped |
| Add DVL-CER dual-view learning to DialogueCRN | DialogueCRN · EmoryNLP | weighted-F1: 38.97 -> 39.92 | Helped |
| Add DVL-CER to DialogueRNN, COSMIC, RGAT, DAG-ERC-W, and SACL-LSTM | CER backbones · IEMOCAP and EmoryNLP | weighted-F1: ↑1.74, ↑1.51, ↑0.89, ↑0.41, ↑1.27 on IEMOCAP; ↑0.57, ↑0.53, ↑0.90, ↑0.86, ↑0.27 on EmoryNLP | Helped |
| Replace dual-view projection learning with feature aggregation (ablation) | DVL-CER · IEMOCAP | weighted-F1: 70.11 -> 69.73 | Hurt |
| Replace dual-view projection learning with subspace learning (ablation) | DVL-CER · IEMOCAP | weighted-F1: 70.11 -> 69.55 | Hurt |
| Replace dual-view projection learning with alignment learning (ablation) | DVL-CER · IEMOCAP | weighted-F1: 70.11 -> 69.19 | Hurt |
| Add both cross-view projectors and their projection losses (ablation) | DVL-CER · IEMOCAP and EmoryNLP | weighted-F1: 69.01 -> 70.11; 38.97 -> 39.92 | Helped |
| Add emotion-view supervision loss LψMSE (ablation) | DVL-CER · IEMOCAP and EmoryNLP | weighted-F1: 69.70 -> 70.11; 39.68 -> 39.92 | Helped |
| Add utterance-view projection loss LθMSE (ablation) | DVL-CER · IEMOCAP and EmoryNLP | weighted-F1: 69.30 -> 70.11; 39.42 -> 39.92 | Helped |
| Add emotion-shift classification loss LESM (ablation) | DVL-CER · IEMOCAP and EmoryNLP | weighted-F1: 69.63 -> 70.11; 39.65 -> 39.92 | Helped |
| Add ReLU activation to the projectors (ablation) | DVL-CER · IEMOCAP and EmoryNLP | weighted-F1: 69.48 -> 70.11; 39.66 -> 39.92 | Helped |
| Project emotion distributions instead of emotion representations (ablation) | DVL-CER · IEMOCAP and EmoryNLP | weighted-F1: 68.47 -> 70.11; 38.50 -> 39.92 | Helped |
Surprising findings
- Using the emotion distribution rather than the emotion representation as the interaction object works much better (70.11 vs 68.47 on IEMOCAP), and using a zero initial-emotion vector costs little on EmoryNLP ablation.
Limitations
- Evaluated only on text modality, with multimodal extension left to future work
- Gains over strong baselines are modest (around 1 point weighted-F1)
- Emotion shift span and loss weights are hyperparameters requiring tuning
Experimental setup
Datasets: IEMOCAP EmoryNLP
Benchmarks: IEMOCAP EmoryNLP
Metrics: weighted-F1
Baselines: DialogueGCN RGAT SKAIG DialogXL DAG-ERC-W CoMPM SACL-LSTM DualRAN DialogueRNN COSMIC DialogueCRN HMVDM
Builds on: DialogueCRN COSMIC Multi-view subspace learning Contrastive learning
Closest papers by meaning
- Adaptive Graph Learning for Multimodal Conversational Emotion Detection (AAAI 2024 · similarity 0.87)
- AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models (ICML 2025 · similarity 0.87)
- Utterance-level Emotion Recognition in Conversation with Conversation-level Supervision (AAAI 2025 · similarity 0.87)
- Bridge Then Begin Anew: Generating Target-Relevant Intermediate Model for Source-Free Visual Emotion Adaptation (AAAI 2025 · similarity 0.86)
- Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach (ICLR 2026 · similarity 0.86)
- Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.