AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion Recognition

Mani Kumar Tellamekala, Shahin Amiriparian, Björn Wolfgang Schuller, Elisabeth André, Timo Giesbrecht, Michel Valstar

TPAMI 2023 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal sentiment analysis · 81 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add calibration and ordinality constrained latent variance weighting to context fusionAV context fusion (CNN+GRU) · AVEC 2019 CES val regressionaverage CCC: 0.599 -> 0.636Improved
add calibration and ordinality constrained latent variance weighting to context fusionAV context fusion (CNN+GRU) · IEMOCAP test regressionaverage CCC: 0.588 -> 0.613Improved
add COLD uncertainty-weighted fusion when 50% of test faces are maskedAV context fusion (CNN+GRU) · AVEC 2019 CES val with face maskingaverage CCC: 0.445 -> 0.528Improved
add COLD constraints to context fusion for 3-way classificationAV context fusion (CNN+GRU) · AVEC 2019 CES val classificationvalence F1: 67.0 -> 68.9Improved
add COLD constraints to context fusion for 3-way classificationAV context fusion (CNN+GRU) · IEMOCAP test classificationarousal F1: 58.6 -> 62.1Improved
add COLD constraints to three-modality GRU context fusionAVL context fusion · CMU-MOSEI testaverage weighted accuracy: 65.3 -> 71.1Improved
add COLD constraints to three-modality GRU context fusionAVL context fusion · IEMOCAP test categoricalaverage accuracy: 78.8 -> 82.7Improved
use COLD fusion instead of prior SOTA AMOACMU-MOSEI categorical emotion · CMU-MOSEI testaverage weighted accuracy: 65.7 -> 71.1Improved
fuse at context level instead of prediction levelAV fusion baselines · AVEC 2019 CES val regressionaverage CCC: 0.584 -> 0.599Improved
early feature fusion instead of the best unimodal branch (audio)IEMOCAP regression · IEMOCAP test regressionaverage CCC: 0.573 -> 0.479Hurt
add COLD fusion to reduce calibration error before temperature scalingAV context fusion · AVEC 2019 CES val classification calibrationarousal ECE (BTS): 6.3e-2 -> 1.3e-2Improved
apply temperature scaling to COLD fusion valenceAV COLD Fusion · AVEC 2019 CES val calibrationvalence ECE: 3.7e-2 -> 4.3e-2Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy