Research map / Vision-language and multimodal / Cross-modal learning

Federated Dialogue-Semantic Diffusion for Emotion Recognition under Incomplete Modalities

NeurIPS 2025 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal sentiment analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedDISC, a federated framework that trains modality-specific conditional diffusion models (DISC-Diffusion, conditioned by a dialogue graph network and semantic conditioning network) to recover missing modalities in multimodal emotion recognition in conversations, with an alternating frozen aggregation strategy.

Problem

Multimodal emotion recognition in conversations degrades severely under unpredictable modality absence, especially fixed missing protocols where a client lacks a modality entirely, so generative recovery models lacking complete training data fail.

Approach

Federated aggregation of modality-specific diffusion models across clients so that clients missing a modality receive a global recovery model, with dialogue-graph and semantic conditioning to keep recovered features consistent in context, speaker identity and semantics, and alternating freezing of recovery and classifier modules during federated training.

Key results

• Under fixed missing protocol on IEMOCAP4, FedDISC (P) achieves the best accuracy in every single- and bi-modal setting, outperforming the best SOTA by 1.3% to 7.5% in accuracy.
• On CMU-MOSI under random missing rates 0.0 to 0.7, FedDISC (P) averages 81.2 ACC versus 79.6 for IMDer and 78.9 for SDR-GNN; it is best at missing rates of 0.4 and above.
• FedDISC's communication cost is 10.57 MB/round per client with conditional DDPM versus 22.64 MB/round for FedAvg.

Measured effects reported in the paper

Surprising findings

• FedDISC (P) slightly trails SDR-GNN on several IEMOCAP6 fixed-missing settings, and on CMU-MOSI/MOSEI it is slightly lower than SOTA at low missing rates (below 0.4).

Limitations

• Iterative denoising makes sampling costly; the paper suggests consistency models as a remedy.
• Assumes synchronous federated training and does not handle asynchronous devices or dynamic modality absence.
• No error bars reported because experiments were too costly to repeat.

Experimental setup

Datasets: IEMOCAP CMU-MOSI CMU-MOSEI

Benchmarks: IEMOCAP4 IEMOCAP6 CMU-MOSI CMU-MOSEI

Metrics: ACC4 ACC6 WAF1 ACC

Baselines: GCNet CIF-MMIN SDR-GNN IMDer DiCMoR DCCAE

Builds on: FedAvg R-GCN DDPM DDIM IMDer

Closest papers by meaning

• Adaptive Graph Learning for Multimodal Conversational Emotion Detection (AAAI 2024 · similarity 0.86)
• Multi-to-Single: Reducing Multimodal Dependency in Emotion Recognition Through Contrastive Learning (AAAI 2025 · similarity 0.85)
• EMOE: Modality-Specific Enhanced Dynamic Emotion Experts (CVPR 2025 · similarity 0.85)
• AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models (ICML 2025 · similarity 0.85)
• A Unified Self-Distillation Framework for Multimodal Sentiment Analysis with Uncertain Missing Modalities (AAAI 2024 · similarity 0.85)
• Gradient-Guided Modality Decoupling for Missing-Modality Robustness (AAAI 2024 · similarity 0.85)