Research map / Vision-language and multimodal / Human and embodied modeling
Modeling Context in Referring Expressions
Licheng Yu, Patrick Poirson, Shan Yang, Alexander C. Berg, Tamara L. Berg
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add global image context (ablation) | Baseline · RefCOCO testB | comprehension accuracy: 66.31% -> 64.21% | Hurt |
| use cropped window around target (scale 4) instead of global context (ablation) | Baseline · RefCOCO testB | comprehension accuracy: 64.21% -> 68.56% | Improved |
| add visual comparison features (visdif) (ablation) | Baseline · RefCOCO testB | comprehension accuracy: 64.21% -> 71.19% | Improved |
| add visual comparison features (visdif) (ablation) | MMI · RefCOCO+ testB | comprehension accuracy: 51.23% -> 55.62% | Improved |
| add visual comparison features (visdif) (ablation) | Baseline · RefCOCOg val | comprehension accuracy: 55.16% -> 59.25% | Improved |
| add visual comparison features (visdif) (ablation) | Baseline(det) · RefCOCO testB (detections) | comprehension accuracy: 48.48% -> 50.80% | Improved |
| compare only to same-category objects instead of different or all objects (ablation) | visdif · RefCOCO/RefCOCO+ | comprehension accuracy | Improved |
| add MMI training (ablation) | Baseline · RefCOCO testA | comprehension accuracy: 63.15% -> 71.72% | Improved |
| add visual comparison (visdif) (ablation) | Baseline · RefCOCO testA | Bleu 1: 0.477 -> 0.505 | Improved |
| add MMI training (ablation) | Baseline · RefCOCO+ testA | Bleu 1: 0.391 -> 0.370 | Hurt |
| add MMI training (ablation) | Baseline · RefCOCO testA | human-judged accuracy: 62.42% -> 65.76% | Improved |
| add LSTM tying across objects (ablation) | Baseline · RefCOCO testB | human-judged accuracy: 64.99% -> 68.34% | Improved |
Closest papers by meaning
- Comprehension-Guided Referring Expressions (CVPR 2017 · similarity 0.94)
- Cops-Ref: A New Dataset and Task on Compositional Referring Expression Comprehension (CVPR 2020 · similarity 0.92)
- Referring Expression Generation and Comprehension via Attributes (ICCV 2017 · similarity 0.92)
- Ref-Adv: Exploring MLLM Visual Reasoning in Referring Expression Tasks (ICLR 2026 · similarity 0.91)
- RefDetector: A Simple Yet Effective Matching-based Method for Referring Expression Comprehension (AAAI 2025 · similarity 0.90)
- A Large-Scale Human-Centric Benchmark for Referring Expression Comprehension in the LMM Era (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
