Research map / Vision-language and multimodal / Cross-modal learning
Multimodal Aligned Semantic Knowledge for Unpaired Image-text Matching
Laiguo Yin, Yixin Zhang, YUQING SUN, Lizhen Cui
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add OOD words with relation-preserving prototypes (ablation) | MASK · Flickr30K (unpaired) | Rs: 116.2 -> 122.8 | Improved |
| add OOD words with relation-preserving prototypes (ablation) | MASK · MSCOCO (unpaired) | Rs: 193.1 -> 209.5 | Improved |
| add semantic-relation loss Lcm (ablation) | MASK · Flickr30K (unpaired) | Rs: 101.0 -> 122.8 | Improved |
| add semantic-relation loss Lcm (ablation) | MASK · MSCOCO (unpaired) | Rs: 150.8 -> 209.5 | Improved |
| add prototype consistency contrastive loss Lcl (ablation) | MASK · Flickr30K (unpaired) | Rs: 92.4 -> 122.8 | Improved |
| add prototype consistency contrastive loss Lcl (ablation) | MASK · MSCOCO (unpaired) | Rs: 123.5 -> 209.5 | Improved |
| add region prototypes instead of random region representation (ablation) | MASK · MSCOCO (unpaired) | Rs: 148.5 -> 209.5 | Improved |
| add max-mean pooling instead of global mean (ablation) | MASK · MSCOCO (unpaired) | Rs: 112.7 -> 209.5 | Improved |
| use lambda1/lambda2 = 0.3 instead of 3.0 (ablation) | MASK · MSCOCO (unpaired) | Rs: 209.5 -> 174.8 at ratio 3.0; 185.1 at ratio 0.3 | Hurt |
| increase sampling size m from 10 to 50 (ablation) | MASK · MSCOCO (unpaired) | Rs: 209.5 -> 222.6 | Improved |
| increase sampling size m from 50 to 100 (ablation) | MASK · MSCOCO (unpaired) | Rs: 222.6 -> 216.0 | Hurt |
| add MASK re-ranking to CLIP (alpha=0.15) | CLIP · MSCOCO (zero-shot re-ranking) | Rs: 386.0 -> 400.4 | Improved |
Closest papers by meaning
- Unpaired Image-Text Matching via Multimodal Aligned Conceptual Knowledge (TPAMI 2024 · similarity 0.92)
- MACK: Multimodal Aligned Conceptual Knowledge for Unpaired Image-text Matching (NeurIPS 2022 · similarity 0.90)
- AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding (NeurIPS 2025 · similarity 0.88)
- Text as Partial Constraint: Core–Residual Alignment for Robust Vision–Language Learning (NeurIPS 2026 · similarity 0.88)
- BiMAC: Bidirectional Multimodal Alignment in Contrastive Learning (AAAI 2025 · similarity 0.87)
- Linguistic-Aware Patch Slimming Framework for Fine-grained Cross-Modal Alignment (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
