Research map / Vision-language and multimodal / Cross-modal learning
CrossCLR: Cross-Modal Contrastive Learning for Multi-Modal Video Representations
Mohammadreza Zolfaghari, Yi Zhu, Peter Gehler, Thomas Brox
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add proximity weighting (ablation) | CrossCLR without other components · YouCook2 text-to-video | R@1: 17.9 -> 18.2 | No clear effect |
| add intra-modality alignment (ablation) | CrossCLR without other components · YouCook2 text-to-video | R@1: 17.9 -> 18.7 | Improved |
| add proximity weighting to intra-modality alignment (ablation) | CrossCLR · YouCook2 text-to-video | R@1: 18.7 -> 19.0 | Improved |
| add negative pruning of influential samples (ablation) | CrossCLR with weighting and alignment · YouCook2 text-to-video | R@1: 19.0 -> 19.5 | Improved |
| use CrossCLR loss instead of NT-Xent | COOT architecture · YouCook2 text-to-video | R@1: 17.5 -> 19.5 | Improved |
| use CrossCLR loss instead of MaxMargin | COOT architecture · LSMDC text-to-video | R@1: 8.2 -> 10.9 | Improved |
| replace COOT loss with CrossCLR with the same architecture | COOT · YouCook2 text-to-video | R@1: 16.7 -> 19.5 | Improved |
| feed CrossCLR clip embeddings to captioning model instead of COOT embeddings | MART captioner · YouCook2 captioning | CIDEr-D: 54.07 -> 58.65 | Improved |
| add multiple positives (ablation) | CrossCLR · YouCook2 text-to-video | R@1: 19.5 -> 19.3 | No clear effect |
| add multiple positives (ablation) | CrossCLR · LSMDC text-to-video | R@10: 34.7 -> 35.8 | Improved |
| raise pruning threshold from low values up to 0.9 (ablation) | CrossCLR · YouCook2 and LSMDC retrieval | retrieval performance | Mixed |
| use too high weight scale kappa (ablation) | CrossCLR · retrieval | retrieval performance | Hurt |
Closest papers by meaning
- Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data (ICLR 2024 · similarity 0.91)
- Probabilistic Embeddings for Cross-Modal Retrieval (CVPR 2021 · similarity 0.89)
- Learning Shared Representations from Unpaired Data (NeurIPS 2025 · similarity 0.88)
- Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion (ICLR 2025 · similarity 0.88)
- Generalized Contrastive Learning for Universal Multimodal Retrieval (NeurIPS 2025 · similarity 0.87)
- Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrieval (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
