Research map / Vision-language and multimodal / Human and embodied modeling
Learning Two-Branch Neural Networks for Image-Text Matching Tasks
Liwei Wang, Yin Li, Jing Li Huang, Svetlana Lazebnik
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add nonlinear (two-layer) branches instead of linear (ablation) | embedding network · Flickr30K Entities phrase localization | R@1: 39.60 -> 43.15 | Improved |
| use bi-directional instead of single-directional loss (ablation) | embedding network · Flickr30K Entities phrase localization | R@1: 43.15 -> 44.53 | Improved |
| add positive region augmentation (IoU > 0.7) (ablation) | embedding network · Flickr30K Entities phrase localization | R@1: 44.53 -> 49.09 | Improved |
| add neighborhood sampling (ablation) | embedding network · Flickr30K Entities phrase localization | R@1: 49.09 -> 51.03 | Improved |
| add neighborhood constraints on top of sampling (ablation) | embedding network · Flickr30K Entities phrase localization | R@1: 51.03 -> 50.69 | No clear effect |
| add positive region augmentation (ablation) | similarity network · Flickr30K Entities phrase localization | R@1: 45.19 -> 51.05 | Improved |
| add nonlinear branches instead of linear (ablation) | similarity network · Flickr30K Entities phrase localization | R@1: 43.19 -> 45.19 | Improved |
| use three FC layers after element-wise product instead of one (ablation) | similarity network · Flickr30K Entities phrase localization | R@1: 36.61 -> 51.05 | Improved |
| use bi-directional instead of single-directional loss (ablation) | embedding network · Flickr30K sentence-to-image | R@1: 22.1 -> 30.7 | Improved |
| add neighborhood sampling (ablation) | embedding network · MSCOCO 1000 image-to-sentence | R@1: 53.0 -> 54.2 | Improved |
| add neighborhood constraints on top of sampling (ablation) | embedding network · MSCOCO 1000 image-to-sentence | R@1: 54.2 -> 54.9 | Improved |
| add neighborhood constraints on top of sampling (ablation) | embedding network · Flickr30K sentence-to-sentence | R@1: 60.5 -> 63.8 | Improved |
Closest papers by meaning
- Learning Deep Structure-Preserving Image-Text Embeddings (CVPR 2016 · similarity 0.90)
- Expressing Objects Just Like Words: Recurrent Visual Embedding for Image-Text Matching (AAAI 2020 · similarity 0.90)
- End-to-End Convolutional Semantic Embeddings (CVPR 2018 · similarity 0.88)
- Conditional Image-Text Embedding Networks (ECCV 2018 · similarity 0.88)
- Stacked Cross Attention for Image-Text Matching (ECCV 2018 · similarity 0.88)
- Composing Object Relations and Attributes for Image-Text Matching (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
