Research map / Vision-language and multimodal / Cross-modal learning
Learning the Best Pooling Strategy for Visual Semantic Embedding
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace complex visual aggregation with the best selected simple pooling | VSE image aggregation · MS-COCO 1K, Region features, T→I | R@1: 60.7 vs 60.7 | Helped |
| Add GPO to both visual and text aggregation instead of AvgPool and CLS (ablation) | VSE∞ · COCO 5-fold 1K Test | R@1 I→T: 67.7 -> 79.7 | Helped |
| Add GPO to visual aggregation while retaining CLS for text (ablation) | VSE∞ · COCO 5-fold 1K Test | R@1 I→T: 67.7 -> 73.8 | Helped |
| Add GPO to text aggregation while retaining AvgPool for visual features (ablation) | VSE∞ · COCO 5-fold 1K Test | R@1 I→T: 67.7 -> 71.7 | Helped |
| Add Size Augmentation by randomly dropping visual and text elements (ablation) | VSE∞ with BUTD Region + BiGRU · COCO 5-fold 1K Test | R@1 I→T: 74.6 -> 78.5 | Helped |
| Add Size Augmentation to visual elements (ablation) | VSE∞ with BUTD Region + BiGRU · COCO 5-fold 1K Test | R@1 I→T: 74.6 -> 75.4 | Helped |
| Replace the GPO BiGRU generator with a Transformer (ablation) | VSE∞ with BUTD Region + BiGRU · COCO 5-fold 1K Test | R@1 I→T: 78.5 -> 77.4 | Hurt |
| Replace the GPO BiGRU generator with a Transformer (ablation) | VSE∞ with BUTD Grid + BiGRU · COCO 5-fold 1K Test | R@1 I→T: 78.0 -> 76.6 | Hurt |
| Use per-dimension rather than shared pooling coefficients (ablation) | VSE∞ with BUTD Region + BiGRU · COCO 5-fold 1K Test | R@1 I→T: 78.5 -> 76.5 | Hurt |
| Use per-dimension rather than shared pooling coefficients (ablation) | VSE∞ with BUTD Region + BERT · COCO 5-fold 1K Test | R@1 I→T: 79.7 -> 79.3 | Hurt |
| Use GPO instead of the best K-MaxPool found by manual search | VSE∞ with BUTD Region + BiGRU · COCO 5-fold 1K Test | rsum: 520.4 -> 520.8 | Helped |
| Use GPO instead of AvgPool for frame and text aggregation | VSE++ · MSR-VTT | rsum: 160.5 -> 174.7 | Helped |
Closest papers by meaning
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.87)
- Bridging Vision and Language Spaces with Assignment Prediction (ICLR 2024 · similarity 0.87)
- Asymmetric Visual Semantic Embedding Framework for Efficient Vision-Language Alignment (AAAI 2025 · similarity 0.87)
- Revisiting the Role of Language Priors in Vision-Language Models (ICML 2024 · similarity 0.86)
- Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models (CVPR 2025 · similarity 0.86)
- Mamba as a Bridge: Where Vision Foundation Models Meet Vision Language Models for Domain-Generalized Semantic Segmentation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.