AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

Distilling Audio-Visual Knowledge by Compositional Contrastive Learning

Yanbei Chen, Yongqin Xian, A. Sophia Koepke, Ying Shan, Zeynep Akata

CVPR 2021 · Vision-language and multimodal › Cross-modal learning · approach: Self-supervised audio-visual representation learning · 72 citations (OpenAlex)

Check your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use CCL for audio distillation instead of the strongest baselineR(2+1)D-18 video student · UCF51Top-1 accuracy (%): 60.3 -> 64.9Improved
Use CCL for image distillation instead of FitNetR(2+1)D-18 video student · UCF51Top-1 accuracy (%): 67.4 -> 69.1Improved
Use CCL for audio-visual distillation instead of CORR(2+1)D-18 video student · ActivityNetTop-1 accuracy (%): 41.7 -> 47.3Improved
Use CCL for audio retrieval instead of CRDR(2+1)D-18 video student · ActivityNetR@1 (%): 31.6 -> 30.6Hurt
Use CCL for audio-visual distillation instead of the baselineR(2+1)D-18 video student · VGGSoundTop-1 accuracy: 19.1 -> 23.6Improved
Add compositional embeddings (ablation)CCL · UCF51 audio distillationTop-1 accuracy (%): 63.2 -> 64.9Improved
Add compositional embeddings (ablation)CCL · UCF51 image distillationTop-1 accuracy (%): 65.8 -> 69.1Improved
Add compositional embeddings (ablation)CCL · UCF51 audio-visual distillationTop-1 accuracy (%): 66.9 -> 70.0Improved
Use multi-class NCE rather than instance-level contrastive loss (ablation)CCL · UCF51 audio distillationTop-1 accuracy (%): 60.4 -> 64.9Improved
Use multi-class NCE rather than instance-level contrastive loss (ablation)CCL · UCF51 audio-visual distillationTop-1 accuracy (%): 67.8 -> 70.0Improved
Add multi-class NCE loss (ablation)CCL · UCF51 audio-visual distillationTop-1 accuracy (%): 66.3 -> 70.0Improved
Add JSD prediction-space loss (ablation)CCL · UCF51 audio-visual distillationTop-1 accuracy (%): 68.2 -> 70.0Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy