Research map / Vision-language and multimodal / Human and embodied modeling
Grounding of Textual Phrases in Images by Reconstruction
Anna Rohrbach, Marcus Rohrbach, Ronghang Hu, Trevor Darrell, Bernt Schiele
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add weight decay 0.0005 (ablation) | GroundeR unsupervised · Flickr30k Entities val | accuracy: 20.33% -> 22.96% | Improved |
| add batch normalization to phrase LSTM and visual feature (ablation) | GroundeR supervised · Flickr30k Entities val | accuracy: 37.42% -> 40.93% | Improved |
| use MSRA instead of Xavier init for conv layers (ablation) | GroundeR unsupervised · Flickr30k Entities val | accuracy: 21.04% -> 22.96% | Improved |
| add sentence constraint post-processing (no box reused within a sentence) (ablation) | GroundeR unsupervised · Flickr30k Entities test (VGG-CLS) | accuracy: 24.66% -> 25.01% | Improved |
| add sentence constraint post-processing (no box reused within a sentence) (ablation) | GroundeR unsupervised · Flickr30k Entities test (VGG-DET) | accuracy: 28.94% -> 29.02% | Improved |
| use VGG-DET features instead of VGG-CLS (ablation) | GroundeR unsupervised · Flickr30k Entities test | accuracy: 24.66% -> 28.94% | Improved |
| use VGG-DET features instead of VGG-CLS (ablation) | GroundeR supervised · Flickr30k Entities test | accuracy: 41.56% -> 47.81% | Improved |
| add 3.12% annotated boxes to unsupervised training | GroundeR · Flickr30k Entities test (VGG-CLS) | accuracy: 24.66% -> 33.02% | Improved |
| add 3.12% annotated boxes to unsupervised training | GroundeR · Flickr30k Entities test (VGG-DET) | accuracy: 28.94% -> 42.32% | Improved |
| add reconstruction loss to supervised attention loss with all annotations (ablation) | GroundeR · Flickr30k Entities test (VGG-CLS) | accuracy: 41.56% -> 42.43% | Improved |
| add reconstruction loss to supervised attention loss with all annotations (ablation) | GroundeR · Flickr30k Entities test (VGG-DET) | accuracy: 47.81% -> 48.38% | Improved |
| add spatial features to VGG features (ablation) | GroundeR supervised · ReferItGame test | accuracy: 23.44% -> 26.93% | Improved |
Closest papers by meaning
- Align2Ground: Weakly Supervised Phrase Grounding Guided by Image-Caption Alignment (ICCV 2019 · similarity 0.92)
- Improving Zero-Shot Phrase Grounding via Reasoning on External Knowledge and Spatial Relations (AAAI 2022 · similarity 0.92)
- Learning to Generate Grounded Visual Captions without Localization Supervision (ECCV 2020 · similarity 0.92)
- Improved Visual Grounding through Self-Consistent Explanations (CVPR 2024 · similarity 0.91)
- Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding (CVPR 2025 · similarity 0.90)
- Referencing Where to Focus: Improving Visual Grounding with Referential Query (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
