Research map / Vision-language and multimodal / Human and embodied modeling
Conditional Image-Text Embedding Networks
Bryan A. Plummer, Paige Kordas, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu, Svetlana Lazebnik
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add spatial location features | Similarity Network · Flickr30K Entities test (PASCAL-tuned features) | accuracy: 53.45 -> 54.52 | Improved |
| add learned concept weight branch instead of fixed K-means assignment (ablation) | CITE · Flickr30K Entities test (PASCAL-tuned features) | accuracy: 57.89 -> 58.69 | Improved |
| use K-means text-cluster assignments instead of random assignment (ablation) | CITE · Flickr30K Entities test (PASCAL-tuned features) | accuracy: 57.58 -> 57.89 | Improved |
| use random embedding assignment instead of coarse categories (ablation) | CITE · Flickr30K Entities test (PASCAL-tuned features) | accuracy: 55.42 -> 57.58 | Improved |
| share weights across conditional embeddings (CITE) instead of separate networks per coarse category (ablation) | CITE · Flickr30K Entities test (PASCAL-tuned features) | accuracy: 55.32 -> 55.42 | No clear effect |
| increase Edge Box proposals from 200 to 500 | CITE, Learned, K=4 · Flickr30K Entities test (PASCAL-tuned features) | accuracy: 58.69 -> 59.27 | Improved |
| increase Edge Box proposals from 500 to 1000 | CITE, Learned, K=4 · Flickr30K Entities test (PASCAL-tuned features) | accuracy: 59.27 -> 58.63 | Hurt |
| reduce embedding size from 256 to 64 | CITE, Learned, K=4 · Flickr30K Entities test (PASCAL-tuned features) | accuracy: 58.69 -> 57.77 | Hurt |
| use Flickr30K-tuned features instead of PASCAL-tuned features | CITE, Learned, K=4, 500 Edge Boxes · Flickr30K Entities test | accuracy: 59.27 -> 61.89 | Improved |
| use learned concept weights instead of K-means (ablation) | CITE, K=12 · Visual Genome test | accuracy: 23.67 -> 24.43 | Improved |
| use learned concept weights instead of K-means, K=2 vs K=12 on ReferIt (ablation) | CITE · ReferIt Game test | accuracy: 34.01 -> 34.13 | Mixed |
| use CITE K-means instead of Similarity Network baseline | CITE, K-Means, K=12 · Visual Genome test | accuracy: 20.08 -> 23.67 | Improved |
Closest papers by meaning
- Multi-Level Multimodal Common Semantic Space for Image-Phrase Grounding (CVPR 2019 · similarity 0.91)
- Align2Ground: Weakly Supervised Phrase Grounding Guided by Image-Caption Alignment (ICCV 2019 · similarity 0.91)
- Neural Sequential Phrase Grounding (SeqGROUND) (CVPR 2019 · similarity 0.90)
- Improving Zero-Shot Phrase Grounding via Reasoning on External Knowledge and Spatial Relations (AAAI 2022 · similarity 0.90)
- Grounding of Textual Phrases in Images by Reconstruction (ECCV 2016 · similarity 0.89)
- Improved Visual Grounding through Self-Consistent Explanations (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
