Research map / Vision-language and multimodal / Cross-modal learning
Vision-Language Pre-Training With Triple Contrastive Learning
Jinyu Yang, Jiali Duan, Son Tran, Yi Xu, Sampath Chanda, Liqun Chen, Belinda Zeng, Trishul Chilimbi, Junzhou Huang
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add intra-modal contrastive learning without stronger image augmentation (ablation) | CMA+ITM+MLM · MSCOCO zero-shot text retrieval | R@1: 68.7 -> 71.1 | Improved |
| Use two strongly augmented image views for IMC instead of the no-augmentation variant (ablation) | TCL with IMC · MSCOCO zero-shot image retrieval | R@1: 52.2 -> 53.3 | Improved |
| Add local MI maximization to CMA+IMC (ablation) | TCL with IMC · Flickr30K zero-shot image retrieval | R@1: 78.9 -> 79.6 | Improved |
| Add local MI maximization to CMA+IMC (ablation) | TCL with IMC · MSCOCO zero-shot text retrieval | R@1: 71.4 -> 71.4 | No clear effect |
| Pool last-layer image patch embeddings for LMI (ablation) | TCL with LMI · Flickr30K zero-shot image retrieval | R@1: 79.2 -> 79.6 | Improved |
| Use intermediate-layer local features rather than last-layer features (ablation) | TCL with LMI · Flickr30K zero-shot text retrieval | R@1: 92.4 -> 91.5 | Hurt |
| Increase pre-training data from 4M to 14M (ablation) | TCL with IMC (w/o aug) · Flickr30K fine-tuned image retrieval | R@1: 82.9 -> 86.0 | Improved |
| Set momentum coefficient to 0.5 (ablation) | TCL · Downstream image-text retrieval | performance | Improved |
Closest papers by meaning
- Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment (NeurIPS 2024 · similarity 0.90)
- Extending Multi-modal Contrastive Representations (NeurIPS 2024 · similarity 0.90)
- ITO: Multi-View Alignment and Training-Time Fusion for Image-Text Pretraining (NeurIPS 2026 · similarity 0.90)
- TiMix: Text-Aware Image Mixing for Effective Vision-Language Pre-training (AAAI 2024 · similarity 0.89)
- Improving fine-grained understanding in image-text pre-training (ICML 2024 · similarity 0.89)
- Post-pre-training for Modality Alignment in Vision-Language Foundation Models (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
