AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

CrossCLR: Cross-Modal Contrastive Learning for Multi-Modal Video Representations

Mohammadreza Zolfaghari, Yi Zhu, Peter Gehler, Thomas Brox

ICCV 2021 · Vision-language and multimodal › Cross-modal learning · approach: Probabilistic embeddings for cross-modal retrieval · 153 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add proximity weighting (ablation)CrossCLR without other components · YouCook2 text-to-videoR@1: 17.9 -> 18.2No clear effect
add intra-modality alignment (ablation)CrossCLR without other components · YouCook2 text-to-videoR@1: 17.9 -> 18.7Improved
add proximity weighting to intra-modality alignment (ablation)CrossCLR · YouCook2 text-to-videoR@1: 18.7 -> 19.0Improved
add negative pruning of influential samples (ablation)CrossCLR with weighting and alignment · YouCook2 text-to-videoR@1: 19.0 -> 19.5Improved
use CrossCLR loss instead of NT-XentCOOT architecture · YouCook2 text-to-videoR@1: 17.5 -> 19.5Improved
use CrossCLR loss instead of MaxMarginCOOT architecture · LSMDC text-to-videoR@1: 8.2 -> 10.9Improved
replace COOT loss with CrossCLR with the same architectureCOOT · YouCook2 text-to-videoR@1: 16.7 -> 19.5Improved
feed CrossCLR clip embeddings to captioning model instead of COOT embeddingsMART captioner · YouCook2 captioningCIDEr-D: 54.07 -> 58.65Improved
add multiple positives (ablation)CrossCLR · YouCook2 text-to-videoR@1: 19.5 -> 19.3No clear effect
add multiple positives (ablation)CrossCLR · LSMDC text-to-videoR@10: 34.7 -> 35.8Improved
raise pruning threshold from low values up to 0.9 (ablation)CrossCLR · YouCook2 and LSMDC retrievalretrieval performanceMixed
use too high weight scale kappa (ablation)CrossCLR · retrievalretrieval performanceHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy