Research map / Graph learning / Graph self-supervised learning
BIG-FUSION: Brain-Inspired Global-Local Context Fusion Framework for Multimodal Emotion Recognition in Conversations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BIG-FUSION, a brain-inspired MERC framework with a parallel dual-attention Transformer (global plus sliding-window attention with spiking neuron dynamics) and dual global/local node-importance graph augmentation for graph contrastive learning.
Problem
Multimodal emotion recognition in conversations relies on sequential designs (Transformer for global context, then GNN for local speaker dependencies), which bias local feature extraction, and GCL augmentations that ignore local context when judging node importance.
Approach
Process global and local context in parallel rather than sequentially: a dual-attention Transformer mixing global and sliding-window attention with spiking-neuron attention, plus a dual-evaluation graph augmentation that scores node importance by mutual-information changes under perturbation at both global-graph and local-subgraph levels.
Key results
- BIG-FUSION reaches 72.91 weighted F1 on IEMOCAP and 67.17 on MELD, beating the strongest baseline by 2.65% and 0.57% respectively.
- Replacing the GCL component with graph augmentation baselines (GraphCL, GCA, AD-GCL, etc.) yields lower WF1 than the proposed dual-evaluation augmentation on both datasets (e.g., 66.31 best baseline vs 67.17 on MELD).
- Removing GCL drops WF1 from 72.91 to 68.50 on IEMOCAP and from 67.17 to 64.79 on MELD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add GCL (ablation) | BIG-FUSION · IEMOCAP | WF1: 68.50 -> 72.91 | Helped |
| add GCL (ablation) | BIG-FUSION · MELD | WF1: 64.79 -> 67.17 | Helped |
| add local information to GCL evaluation (ablation) | BIG-FUSION · IEMOCAP | WF1: 71.23 -> 72.91 | Helped |
| add local information to GCL evaluation (ablation) | BIG-FUSION · MELD | WF1: 65.49 -> 67.17 | Helped |
| add global information to GCL evaluation (ablation) | BIG-FUSION · IEMOCAP | WF1: 71.76 -> 72.91 | Helped |
| add global information to GCL evaluation (ablation) | BIG-FUSION · MELD | WF1: 65.99 -> 67.17 | Helped |
| add spiking attention (ablation) | BIG-FUSION · IEMOCAP | WF1: 71.72 -> 72.91 | Helped |
| add spiking attention (ablation) | BIG-FUSION · MELD | WF1: 65.93 -> 67.17 | Helped |
| add local attention to spiking attention (ablation) | BIG-FUSION · IEMOCAP | WF1: 71.90 -> 72.91 | Helped |
| add local attention to spiking attention (ablation) | BIG-FUSION · MELD | WF1: 66.29 -> 67.17 | Helped |
| add global attention to spiking attention (ablation) | BIG-FUSION · IEMOCAP | WF1: 69.82 -> 72.91 | Helped |
| add global attention to spiking attention (ablation) | BIG-FUSION · MELD | WF1: 65.82 -> 67.17 | Helped |
Limitations
- Gains over the strongest baseline are small on MELD (0.57% WF1).
- Fear and Disgust classes excluded on MELD due to insufficient samples.
Experimental setup
Datasets: IEMOCAP MELD
Benchmarks: IEMOCAP MELD
Metrics: weighted F1 accuracy per-class F1
Baselines: DialogueRNN HCL MVN SCMM MMGCN MMDFN GA2MIF CMCF-SRNet COGMEN Joyful AdaIGN GraphCL
Builds on: Joyful AdaIGN RGCN GraphCL AutoGCL
Closest papers by meaning
- Adaptive Graph Learning for Multimodal Conversational Emotion Detection (AAAI 2024 · similarity 0.89)
- Revisiting Multimodal Emotion Recognition in Conversation from the Perspective of Graph Spectrum (AAAI 2025 · similarity 0.89)
- Dynamic Interactive Bimodal Hypergraph Networks for Emotion Recognition in Conversations (AAAI 2025 · similarity 0.86)
- Multi-to-Single: Reducing Multimodal Dependency in Emotion Recognition Through Contrastive Learning (AAAI 2025 · similarity 0.85)
- Dual-View Learning for Conversational Emotion Recognition Through Context and Emotion-Shift Modeling (AAAI 2025 · similarity 0.84)
- Less is More: Local Intrinsic Dimensions of Contextual Language Models (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.