AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learning

Sangho Lee, Jiwan Chung, Youngjae Yu, Gunhee Kim, Thomas Breuel, Gal Chechik, Yale Song

ICCV 2021 · Vision-language and multimodal › Cross-modal learning · approach: Self-supervised audio-visual representation learning · 35 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use clustering-based MI estimation instead of NCE contrastive (ablation)MI estimator · Kinetics-Sounds correspondence retrievalprecision: 73.066 -> 88.705Improved
use contrastive instead of clustering MI estimator (ablation)MI estimator · CIFAR10-Rotation correspondence retrievalprecision: 87.292 -> 99.395Improved
use multiple layers with Combination pairing instead of single layer 5 (ablation)clustering MI · Kinetics-Sounds correspondence retrievalprecision: 58.418 -> 88.705Improved
pair all layers across and within modalities (Combination) instead of Diagonal (ablation)clustering MI · Kinetics-Sounds correspondence retrievalprecision: 71.450 -> 88.705Improved
pair all layers (Combination) instead of Bipartite audio-visual only (ablation)clustering MI · Kinetics-Sounds correspondence retrievalprecision: 76.969 -> 88.705Improved
use mini-batch SGD K-means instead of EM (Lloyd's) (ablation)clustering MI · Kinetics-Sounds correspondence retrievalprecision: 88.732 -> 88.705No clear effect
use batch greedy instead of exact greedy selection (ablation)subset selection · Kinetics-Sounds correspondence retrievalprecision: 98.970 -> 98.020No clear effect
raise selection-to-batch ratio s/b above 0.25 (ablation)batch greedy · Kinetics-Sounds correspondence retrievalprecisionHurt
use steep exponential layer weights exp(-10) instead of uniform (ablation)clustering MI · Kinetics-Sounds correspondence retrievalprecision: 88.705 -> 50.791Hurt
use 2000 clusters instead of about 500 (ablation)clustering curation · UCF101, ESC-50, Kinetics-Sounds linear evaltop-1 accuracyHurt
curate with clustering MI instead of random subset (20K) (ablation)pretraining data · UCF101 linear evaltop-1 accuracy: 36.34 -> 46.28Improved
curate with clustering MI instead of NCE contrastive (2M) (ablation)pretraining data · UCF101 linear evaltop-1 accuracy: 45.87 -> 55.63Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy