Research map / Detection and segmentation / Open-set and few-shot detection
Zero-Shot Grounding of Objects From Natural Language Queries
Arka Sadhu, Kan Chen, Ram Nevatia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace softmax with binary cross entropy loss (ablation) | ZSGNet base model · RefClef (ReferIt validation) | accuracy: 48.54 -> 55.20 | Improved |
| replace binary cross entropy with focal loss (ablation) | ZSGNet base model · RefClef (ReferIt validation) | accuracy: 55.20 -> 57.13 | Improved |
| train with 600x600 images instead of 300x300 (ablation) | ZSGNet base model + FL · RefClef (ReferIt validation) | accuracy: 57.13 -> 61.75 | Improved |
| use Res50 (RetinaNet) backbone instead of VGG (SSD) | ZSGNet · Flickr30k Entities | accuracy@IoU0.5: 60.12 -> 63.39 | Improved |
| use Res50 instead of VGG | ZSGNet · ReferIt | accuracy@IoU0.5: 53.31 -> 58.63 | Improved |
| use Res50 instead of VGG | ZSGNet · Flickr-Split-0 | accuracy@IoU0.5: 39.32 -> 43.02 | Improved |
| replace pretrained-detector proposals with dense proposals (ZSGNet vs QRG) | ZSGNet (VGG) · Flickr-Split-1 | accuracy@IoU0.5: 24.42 -> 29.35 | Improved |
| replace pretrained-detector proposals with dense proposals (ZSGNet vs QRG) | ZSGNet (VGG) · VG-Split-2 balanced | accuracy@IoU0.5: 7.64 -> 11.02 | Improved |
| remove query language input (language blind) (ablation) | ZSGNet · Flickr30k Entities | accuracy@IoU0.5: 63.39 -> 0.008 | Hurt |
| remove image (image blind) (ablation) | ZSGNet · Flickr30k Entities | accuracy@IoU0.5: 60.12 -> 28.07 | Hurt |
| add horizontal flip augmentation (ablation) | ZSGNet · grounding datasets | accuracy | Hurt |
| train at 600x600 instead of 300x300 resolution (ablation) | ZSGNet · grounding datasets | accuracy: 2-3% improvement | Improved |
Closest papers by meaning
- Improving Zero-Shot Phrase Grounding via Reasoning on External Knowledge and Spatial Relations (AAAI 2022 · similarity 0.93)
- Revisiting Image-Language Networks for Open-Ended Phrase Detection (TPAMI 2020 · similarity 0.92)
- GroundVLP: Harnessing Zero-Shot Visual Grounding from Vision-Language Pre-training and Open-Vocabulary Object Detection (AAAI 2024 · similarity 0.90)
- DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding (AAAI 2023 · similarity 0.90)
- Grounding of Textual Phrases in Images by Reconstruction (ECCV 2016 · similarity 0.90)
- Using Syntax to Ground Referring Expressions in Natural Images (AAAI 2018 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
