Research map / Vision-language and multimodal / Human and embodied modeling
Contrastive Learning for Weakly Supervised Phrase Grounding
Tanmay Gupta, Arash Vahdat, Gal Chechik, Xiaodong Yang, Jan Kautz, Derek Hoiem
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add pretrained BERT word representations instead of random-init BERT (ablation) | Ours, no negatives · Flickr30K Entities test | pointing accuracy: 57.37 -> 66.89 | Improved |
| add pretrained BERT word representations instead of random-init BERT (ablation) | Ours, no negatives · Flickr30K Entities test | R@1: 25.66 -> 35.74 | Improved |
| add random negative captions from training data (ablation) | Ours, pretrained BERT · Flickr30K Entities test | pointing accuracy: 66.89 -> 66.92 | No clear effect |
| add random negative captions from training data (ablation) | Ours, pretrained BERT · Flickr30K Entities test | R@1: 35.74 -> 36.32 | No clear effect |
| use contextually plausible negatives instead of random negatives (ablation) | Ours, pretrained BERT · Flickr30K Entities test | pointing accuracy: 66.92 -> 74.91 | Improved |
| use contextually plausible negatives instead of random negatives (ablation) | Ours, pretrained BERT · Flickr30K Entities test | R@1: 36.32 -> 48.05 | Improved |
| exclude near-synonyms and hypernyms from plausible negatives (ablation) | Ours, pretrained BERT · Flickr30K Entities test | R@1: 48.05 -> 51.67 | Improved |
| exclude near-synonyms and hypernyms from plausible negatives (ablation) | Ours, pretrained BERT · Flickr30K Entities test | pointing accuracy: 74.91 -> 76.74 | Improved |
| train on Flickr30K Entities instead of COCO | Ours · Flickr30K Entities test | pointing accuracy: 76.74 -> 74.94 | Hurt |
| use Ours instead of Align2Ground with same visual features | Ours (COCO train) · Flickr30K Entities test | pointing accuracy: 71.00 -> 76.74 | Improved |
| use one or two layer MLP key-value modules instead of linear (ablation) | Ours · Flickr30K Entities test | grounding performance | Improved |
| maximize InfoNCE bound to convergence rather than stopping early (ablation) | Ours · Flickr30K Entities test | phrase grounding performance | Hurt |
Closest papers by meaning
- Align2Ground: Weakly Supervised Phrase Grounding Guided by Image-Caption Alignment (ICCV 2019 · similarity 0.93)
- Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation (CVPR 2021 · similarity 0.92)
- Weakly-Supervised Visual Grounding of Phrases With Linguistic Structures (CVPR 2017 · similarity 0.91)
- Weakly-Supervised Generation and Grounding of Visual Descriptions With Conditional Generative Models (CVPR 2022 · similarity 0.91)
- Momentum Pseudo-Labeling for Weakly Supervised Phrase Grounding (AAAI 2025 · similarity 0.90)
- Cycle-Consistency Learning for Captioning and Grounding (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
