Research map / Vision-language and multimodal / Human and embodied modeling
Aligning Information Capacity Between Vision and Language via Dense-to-Sparse Feature Distillation for Image-Text Matching
Yang Liu, Wentao Feng, Zhuoyao Liu, Shudong Huang, Jiancheng Lv
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dense-to-sparse distillation to pretrain+align fine-tuning (ablation) | D2S-VSE · Flickr30K 1K test | rSum: 525.7 -> 531.9 | Improved |
| add dense-text pretraining before sparse fine-tuning (ablation) | D2S-VSE · Flickr30K 1K test | rSum: 509.5 -> 525.7 | Improved |
| add sparse-text alignment loss alongside pretrain and distillation (ablation) | D2S-VSE · Flickr30K 1K test | rSum: 512.1 -> 531.9 | Improved |
| fine-tune on sparse text after dense pretraining (ablation) | D2S-VSE · Flickr30K 1K test | rSum: 425.2 -> 525.7 | Improved |
| add distillation without sparse alignment on top of pretraining only (ablation) | D2S-VSE · Flickr30K 1K test | rSum: 425.2 -> 512.1 | Improved |
| use negative cosine similarity rather than l1 distance for distillation (ablation) | D2S-VSE · Flickr30K 1K test | rSum: 528.9 -> 531.9 | Improved |
| use negative cosine similarity rather than l2 distance for distillation (ablation) | D2S-VSE · Flickr30K 1K test | rSum: 530.2 -> 531.9 | Improved |
| place learnable tokens surrounding the sparse text rather than as prefix (ablation) | D2S-VSE · Flickr30K 1K test | rSum: 529.3 -> 531.9 | Improved |
| place learnable tokens surrounding the sparse text rather than as postfix (ablation) | D2S-VSE · Flickr30K 1K test | rSum: 529.5 -> 531.9 | Improved |
| increase the number of learnable tokens from 100 to 120 (ablation) | D2S-VSE · Flickr30K 1K test | R@1 | No clear effect |
| increase decoder attention heads beyond 4 (ablation) | D2S-VSE · Flickr30K 1K test | R@1: within 0.2% | No clear effect |
| increase decoder depth beyond 4 layers (ablation) | D2S-VSE · Flickr30K 1K test | R@1 | Hurt |
Closest papers by meaning
- Image-Text Embedding Learning via Visual and Textual Semantic Reasoning (TPAMI 2022 · similarity 0.90)
- Asymmetric Visual Semantic Embedding Framework for Efficient Vision-Language Alignment (AAAI 2025 · similarity 0.89)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.88)
- Improving fine-grained understanding in image-text pre-training (ICML 2024 · similarity 0.87)
- LensVLM: Selective Context Expansion for Compressed Visual Representation of Text (NeurIPS 2026 · similarity 0.87)
- Composing Object Relations and Attributes for Image-Text Matching (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
