Research map / Vision-language and multimodal / Cross-modal learning
Distilling Audio-Visual Knowledge by Compositional Contrastive Learning
Yanbei Chen, Yongqin Xian, A. Sophia Koepke, Ying Shan, Zeynep Akata
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CCL for audio distillation instead of the strongest baseline | R(2+1)D-18 video student · UCF51 | Top-1 accuracy (%): 60.3 -> 64.9 | Improved |
| Use CCL for image distillation instead of FitNet | R(2+1)D-18 video student · UCF51 | Top-1 accuracy (%): 67.4 -> 69.1 | Improved |
| Use CCL for audio-visual distillation instead of COR | R(2+1)D-18 video student · ActivityNet | Top-1 accuracy (%): 41.7 -> 47.3 | Improved |
| Use CCL for audio retrieval instead of CRD | R(2+1)D-18 video student · ActivityNet | R@1 (%): 31.6 -> 30.6 | Hurt |
| Use CCL for audio-visual distillation instead of the baseline | R(2+1)D-18 video student · VGGSound | Top-1 accuracy: 19.1 -> 23.6 | Improved |
| Add compositional embeddings (ablation) | CCL · UCF51 audio distillation | Top-1 accuracy (%): 63.2 -> 64.9 | Improved |
| Add compositional embeddings (ablation) | CCL · UCF51 image distillation | Top-1 accuracy (%): 65.8 -> 69.1 | Improved |
| Add compositional embeddings (ablation) | CCL · UCF51 audio-visual distillation | Top-1 accuracy (%): 66.9 -> 70.0 | Improved |
| Use multi-class NCE rather than instance-level contrastive loss (ablation) | CCL · UCF51 audio distillation | Top-1 accuracy (%): 60.4 -> 64.9 | Improved |
| Use multi-class NCE rather than instance-level contrastive loss (ablation) | CCL · UCF51 audio-visual distillation | Top-1 accuracy (%): 67.8 -> 70.0 | Improved |
| Add multi-class NCE loss (ablation) | CCL · UCF51 audio-visual distillation | Top-1 accuracy (%): 66.3 -> 70.0 | Improved |
| Add JSD prediction-space loss (ablation) | CCL · UCF51 audio-visual distillation | Top-1 accuracy (%): 68.2 -> 70.0 | Improved |
Closest papers by meaning
- From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generation (ICML 2024 · similarity 0.90)
- Asymmetric Hierarchical Anchoring for Robust Audio–Visual Cross-Modal Generalization (NeurIPS 2026 · similarity 0.89)
- XKD: Cross-Modal Knowledge Distillation with Domain Alignment for Video Representation Learning (AAAI 2024 · similarity 0.89)
- Mixtures of Experts for Audio-Visual Learning (NeurIPS 2024 · similarity 0.89)
- Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data (ICLR 2024 · similarity 0.88)
- Looking Similar Sounding Different: Leveraging Counterfactual Cross-Modal Pairs for Audiovisual Representation Learning (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
