AI Paper Atlas

Research map / Video / Video-language understanding

Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysis

Zhongkai Sun, Prathusha Sarma, William Sethares, Yingyu Liang

AAAI 2020 · Video › Video-language understanding · approach: Video-language pretraining · 419 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add text-based audio and text-based video outer-product interactions, rather than applying DCCA directly to audio and video (ablation)ICCN · CMU-MOSIAcc-2: 82.13 -> 83.07Improved
Add text-based audio and text-based video outer-product interactions, rather than applying DCCA directly to audio and video (ablation)ICCN · CMU-MOSEIAcc-2: 83.01 -> 84.18Improved
Use CCA loss instead of cosine-similarity loss (ablation)ICCN · CMU-MOSIAcc-2: 82.32 -> 83.07Improved
Use CCA loss instead of cosine-similarity loss (ablation)ICCN · CMU-MOSEIAcc-2: 82.98 -> 84.18Improved
Add text-based outer products and use CCA loss instead of direct audio-video inputs with cosine-similarity loss (ablation)ICCN · CMU-MOSIAcc-2: 81.49 -> 83.07Improved
Add text-based outer products and use CCA loss instead of direct audio-video inputs with cosine-similarity loss (ablation)ICCN · CMU-MOSEIAcc-2: 82.59 -> 84.18Improved
Use ICCN rather than the strongest reported baselineCMU-MOSI · CMU-MOSIAcc-2: 82.53 -> 83.07Improved
Use ICCN rather than the strongest reported baselineCMU-MOSEI · CMU-MOSEIAcc-2: 84.40 -> 84.18Hurt
Use ICCN rather than the strongest reported baselineIEMOCAP Happy · IEMOCAP HappyAcc-2: 86.67 -> 87.41Improved
Use ICCN rather than the strongest reported baselineIEMOCAP Angry · IEMOCAP AngryAcc-2: 86.99 -> 88.62Improved
Use ICCN rather than the strongest reported baselineIEMOCAP Neutral · IEMOCAP NeutralAcc-2: 70.26 -> 69.73Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy