Research map / Vision-language and multimodal / Cross-modal learning
COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion Recognition
Mani Kumar Tellamekala, Shahin Amiriparian, Björn Wolfgang Schuller, Elisabeth André, Timo Giesbrecht, Michel Valstar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add calibration and ordinality constrained latent variance weighting to context fusion | AV context fusion (CNN+GRU) · AVEC 2019 CES val regression | average CCC: 0.599 -> 0.636 | Improved |
| add calibration and ordinality constrained latent variance weighting to context fusion | AV context fusion (CNN+GRU) · IEMOCAP test regression | average CCC: 0.588 -> 0.613 | Improved |
| add COLD uncertainty-weighted fusion when 50% of test faces are masked | AV context fusion (CNN+GRU) · AVEC 2019 CES val with face masking | average CCC: 0.445 -> 0.528 | Improved |
| add COLD constraints to context fusion for 3-way classification | AV context fusion (CNN+GRU) · AVEC 2019 CES val classification | valence F1: 67.0 -> 68.9 | Improved |
| add COLD constraints to context fusion for 3-way classification | AV context fusion (CNN+GRU) · IEMOCAP test classification | arousal F1: 58.6 -> 62.1 | Improved |
| add COLD constraints to three-modality GRU context fusion | AVL context fusion · CMU-MOSEI test | average weighted accuracy: 65.3 -> 71.1 | Improved |
| add COLD constraints to three-modality GRU context fusion | AVL context fusion · IEMOCAP test categorical | average accuracy: 78.8 -> 82.7 | Improved |
| use COLD fusion instead of prior SOTA AMOA | CMU-MOSEI categorical emotion · CMU-MOSEI test | average weighted accuracy: 65.7 -> 71.1 | Improved |
| fuse at context level instead of prediction level | AV fusion baselines · AVEC 2019 CES val regression | average CCC: 0.584 -> 0.599 | Improved |
| early feature fusion instead of the best unimodal branch (audio) | IEMOCAP regression · IEMOCAP test regression | average CCC: 0.573 -> 0.479 | Hurt |
| add COLD fusion to reduce calibration error before temperature scaling | AV context fusion · AVEC 2019 CES val classification calibration | arousal ECE (BTS): 6.3e-2 -> 1.3e-2 | Improved |
| apply temperature scaling to COLD fusion valence | AV COLD Fusion · AVEC 2019 CES val calibration | valence ECE: 3.7e-2 -> 4.3e-2 | Hurt |
Closest papers by meaning
- M3ER: Multiplicative Multimodal Emotion Recognition using Facial, Textual, and Speech Cues (AAAI 2020 · similarity 0.89)
- Noise-Resistant Multimodal Transformer for Emotion Recognition (IJCV 2024 · similarity 0.88)
- EMOE: Modality-Specific Enhanced Dynamic Emotion Experts (CVPR 2025 · similarity 0.88)
- Multi-to-Single: Reducing Multimodal Dependency in Emotion Recognition Through Contrastive Learning (AAAI 2025 · similarity 0.87)
- Embracing Unimodal Aleatoric Uncertainty for Robust Multimodal Fusion (CVPR 2024 · similarity 0.85)
- AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
