Research map / Vision-language and multimodal / Human and embodied modeling
Visual Semantic Reasoning for Image-Text Matching
Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, Yun Fu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add region relationship reasoning (GCN) layer before mean-pooling (ablation) | Mean-pool · MS-COCO 1K test | Caption retrieval R@1: 64.3 -> 68.5 | Improved |
| add region relationship reasoning (GCN) layer before mean-pooling (ablation) | Mean-pool · MS-COCO 1K test | Image retrieval R@1: 49.2 -> 56.8 | Improved |
| replace mean-pooling with global semantic reasoning (GRU) (ablation) | Mean-pool · MS-COCO 1K test | Caption retrieval R@1: 64.3 -> 72.3 | Improved |
| replace mean-pooling with global semantic reasoning (GRU) (ablation) | Mean-pool · MS-COCO 1K test | Image retrieval R@1: 49.2 -> 59.6 | Improved |
| add one RRR layer before GSR (ablation) | GSR · MS-COCO 1K test | Caption retrieval R@1: 72.3 -> 75.3 | Improved |
| increase RRR layers from 1 to 4 (ablation) | 1RRR + GSR · MS-COCO 1K test | Caption retrieval R@1: 75.3 -> 76.2 | Improved |
| increase RRR layers from 1 to 4 (ablation) | 1RRR + GSR · MS-COCO 1K test | Image retrieval R@1: 62.1 -> 62.8 | Improved |
| add the text generation loss to joint training (ablation) | 4RRR + GSR · MS-COCO 1K test | Caption retrieval R@1: 74.6 -> 76.2 | Improved |
| add the text generation loss to joint training (ablation) | 4RRR + GSR · MS-COCO 1K test | Image retrieval R@1: 61.2 -> 62.8 | Improved |
| order regions by detection confidence instead of randomly (ablation) | VSRN · MS-COCO 1K test | Caption retrieval R@1: 75.1 -> 76.2 | Improved |
| order regions by confidence instead of bounding box size (ablation) | VSRN · MS-COCO 1K test | Caption retrieval R@1: 75.8 -> 76.2 | No clear effect |
| use VSRN instead of SCAN | SCAN · MS-COCO 1K test | Caption retrieval R@1: 72.7 -> 76.2 | Improved |
Closest papers by meaning
- Image-Text Embedding Learning via Visual and Textual Semantic Reasoning (TPAMI 2022 · similarity 0.94)
- Composing Object Relations and Attributes for Image-Text Matching (CVPR 2024 · similarity 0.91)
- Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition (AAAI 2022 · similarity 0.91)
- Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition (ICCV 2021 · similarity 0.90)
- Identification of Necessary Semantic Undertakers in the Causal View for Image-Text Matching (AAAI 2024 · similarity 0.89)
- RMIR: A Benchmark Dataset for Reasoning-Intensive Multimodal Image Retrieval (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
