Research map / Vision-language and multimodal / Cross-modal learning
ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learning
Sangho Lee, Jiwan Chung, Youngjae Yu, Gunhee Kim, Thomas Breuel, Gal Chechik, Yale Song
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use clustering-based MI estimation instead of NCE contrastive (ablation) | MI estimator · Kinetics-Sounds correspondence retrieval | precision: 73.066 -> 88.705 | Improved |
| use contrastive instead of clustering MI estimator (ablation) | MI estimator · CIFAR10-Rotation correspondence retrieval | precision: 87.292 -> 99.395 | Improved |
| use multiple layers with Combination pairing instead of single layer 5 (ablation) | clustering MI · Kinetics-Sounds correspondence retrieval | precision: 58.418 -> 88.705 | Improved |
| pair all layers across and within modalities (Combination) instead of Diagonal (ablation) | clustering MI · Kinetics-Sounds correspondence retrieval | precision: 71.450 -> 88.705 | Improved |
| pair all layers (Combination) instead of Bipartite audio-visual only (ablation) | clustering MI · Kinetics-Sounds correspondence retrieval | precision: 76.969 -> 88.705 | Improved |
| use mini-batch SGD K-means instead of EM (Lloyd's) (ablation) | clustering MI · Kinetics-Sounds correspondence retrieval | precision: 88.732 -> 88.705 | No clear effect |
| use batch greedy instead of exact greedy selection (ablation) | subset selection · Kinetics-Sounds correspondence retrieval | precision: 98.970 -> 98.020 | No clear effect |
| raise selection-to-batch ratio s/b above 0.25 (ablation) | batch greedy · Kinetics-Sounds correspondence retrieval | precision | Hurt |
| use steep exponential layer weights exp(-10) instead of uniform (ablation) | clustering MI · Kinetics-Sounds correspondence retrieval | precision: 88.705 -> 50.791 | Hurt |
| use 2000 clusters instead of about 500 (ablation) | clustering curation · UCF101, ESC-50, Kinetics-Sounds linear eval | top-1 accuracy | Hurt |
| curate with clustering MI instead of random subset (20K) (ablation) | pretraining data · UCF101 linear eval | top-1 accuracy: 36.34 -> 46.28 | Improved |
| curate with clustering MI instead of NCE contrastive (2M) (ablation) | pretraining data · UCF101 linear eval | top-1 accuracy: 45.87 -> 55.63 | Improved |
Closest papers by meaning
- MJEPA: A Simple and Scalable Joint-Embedding Predictive Architecture for Audio-Visual Learning (ECCV 2026 · similarity 0.89)
- Self-Supervised Learning of Audio-Visual Objects from Video (ECCV 2020 · similarity 0.88)
- From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generation (ICML 2024 · similarity 0.88)
- Audio-Visual Contrastive Learning with Temporal Self-Supervision (AAAI 2023 · similarity 0.88)
- Language-Guided Audio-Visual Source Separation via Trimodal Consistency (CVPR 2023 · similarity 0.88)
- UWAV: Uncertainty-weighted Weakly-supervised Audio-Visual Video Parsing (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
