Research map / Vision-language and multimodal / Cross-modal learning
CoV-Align: Efficient Fine-grained Cross-Modal Alignment with Cohesive Visual Semantics Priority
Hengqi Liu, Wanting Zhou, Longteng Kong, Fangxiang Feng, Lei Ren, Wei Chen, Xiaojie Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add coarse visual semantic feature extractor (deformable + consistent assign attention) to query-based baseline (ablation) | query-based baseline · Flickr30K (ViT-B-224) | R@1 (TR/IR): 70.4/59.4 -> 73.9/62.1 | Improved |
| add visual contrastive loss (ablation) | CoV-Align · Flickr30K (ViT-B-224) | R@1 (TR/IR): 73.9 -> 75.4 (TR); 62.1 -> 62.6 (IR) | Improved |
| add spatial concentration loss (ablation) | CoV-Align · Flickr30K (ViT-B-224) | R@1 (TR/IR): 75.4 -> 78.5 (TR); 62.6 -> 63.3 (IR) | Improved |
| use masking threshold 0.7 instead of 0.0 (attend all patches) (ablation) | CoV-Align · Flickr30K (ViT-B-224) | R@1 (TR): 74.5 -> 78.5 | Improved |
| raise masking threshold from 0.7 to 0.9 (ablation) | CoV-Align · Flickr30K (ViT-B-224) | R@1 (TR): 78.5 -> 76.2 | Hurt |
| use 8 region queries instead of 4 (ablation) | CoV-Align · Flickr30K (ViT-B-224) | R@1 (TR): 72.7 -> 78.5 | Improved |
| increase region queries from 8 to 16 (ablation) | CoV-Align · Flickr30K (ViT-B-224) | R@1 (TR): 78.5 -> 75.2 | Hurt |
| use text-free region aggregation instead of text-guided aggregation (LAPS) | CoV-Align · Flickr30K 1K (Swin-B-384) | RSUM: 545.3 -> 557.0 | Improved |
| use text-free aggregation instead of text-guided aggregation (LAPS) | CoV-Align · MS-COCO evaluation time | evaluation time (s): 442.2s -> 114.0s | Improved |
| use text-free aggregation instead of text-guided aggregation (LAPS) | CoV-Align · Flickr30K training | GPU memory: 49G -> 25G | Improved |
| integrate CoV-Align into CLIP ViT-B/16 instead of LG-MGC fine-tuning | CLIP · MSCOCO 5K | RSUM: 465.3 -> 477.0 | Improved |
| use CoV-Align instead of AVSE with Swin-224 on Flickr30K | CoV-Align · Flickr30K 1K (Swin-B-224) | RSUM: 538.7 -> 541.3 | No clear effect |
Closest papers by meaning
- Linguistic-Aware Patch Slimming Framework for Fine-grained Cross-Modal Alignment (CVPR 2024 · similarity 0.92)
- CovMatch: Cross-Covariance Guided Multimodal Dataset Distillation with Trainable Text Encoder (NeurIPS 2025 · similarity 0.91)
- Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment (NeurIPS 2024 · similarity 0.90)
- SmartCLIP: Modular Vision-language Alignment with Identification Guarantees (CVPR 2025 · similarity 0.88)
- AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding (NeurIPS 2025 · similarity 0.88)
- Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
