Research map / Vision-language and multimodal / Human and embodied modeling
Referring Transformer: A One-step Approach to Multi-task Visual Grounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Jointly train REC and RES instead of training with REC loss only (ablation) | Referring Transformer · RefCOCO+ val | REC: 70.02 -> 71.13 | Helped |
| Jointly train REC and RES instead of training with RES loss only (ablation) | Referring Transformer · RefCOCO+ val | RES: 58.39 -> 61.08 | Helped |
| Add Visual Genome pretraining to joint REC+RES training (ablation) | Referring Transformer · RefCOCO+ val | REC: 71.13 -> 76.40 | Helped |
| Add Visual Genome pretraining to joint REC+RES training (ablation) | Referring Transformer · RefCOCO+ val | RES: 61.08 -> 65.30 | Helped |
| Add joint REC+RES training (ablation) | Referring Transformer · RefCOCO+ val | Inconsistency Error: 23.52% -> 4.73% | Helped |
| Add Visual Genome pretraining to REC-only training (ablation) | Referring Transformer · RefCOCO+ val | REC: 70.02 -> 75.07 | Helped |
| Add the query decoder (ablation) | Referring Transformer · Flickr30K test | top-1 accuracy: 49.38 -> 78.13 | Helped |
| Add the context encoder (ablation) | Referring Transformer · Flickr30K test | top-1 accuracy: 73.68 -> 78.13 | Helped |
| Add both context and phrase features to query encoding (ablation) | Referring Transformer · Flickr30K test | top-1 accuracy: 42.05 -> 78.13 | Helped |
| Add the context feature to query encoding (ablation) | Referring Transformer · Flickr30K test | top-1 accuracy: 76.64 -> 78.13 | Helped |
| Add the phrase feature to query encoding (ablation) | Referring Transformer · Flickr30K test | top-1 accuracy: 77.02 -> 78.13 | Helped |
| Add the learnable query embedding (ablation) | Referring Transformer · Flickr30K test | top-1 accuracy: 77.64 -> 78.13 | Helped |
Closest papers by meaning
- Referencing Where to Focus: Improving Visual Grounding with Referential Query (NeurIPS 2024 · similarity 0.93)
- Mask Grounding for Referring Image Segmentation (CVPR 2024 · similarity 0.91)
- Rethinking Two-Stage Referring Expression Comprehension: A Novel Grounding and Segmentation Method Modulated by Point (AAAI 2024 · similarity 0.91)
- Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints (AAAI 2025 · similarity 0.91)
- Task-aware Cross-modal Feature Refinement Transformer with Large Language Models for Visual Grounding (CVPR 2025 · similarity 0.90)
- OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modeling (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.