Research map / Vision-language and multimodal / Human and embodied modeling
Reasoning About Fine-Grained Attribute Phrases Using Reference Games
Jong-Chyi Su, Chenyun Wu, Huaizu Jiang, Subhransu Maji
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train speaker on contrastive pairs (discerning speaker) instead of single image (ablation) | speaker · Reference game (SL listener, top-5) | accuracy (%): 80.6 -> 91.4 | Improved |
| train speaker on contrastive pairs (discerning speaker) instead of single image (ablation) | speaker · Reference game (human listener, top-5, Test*) | accuracy (%): 64.2 -> 80.2 | Improved |
| rerank simple speaker outputs with listener SL_r | simple speaker · Reference game (human listener, top-5) | accuracy (%): 64.2 -> 82.6 | Improved |
| rerank discerning speaker outputs with listener SL_r | discerning speaker · Reference game (human listener, top-5) | accuracy (%): 80.2 -> 90.0 | Improved |
| rerank with contrastive listener SL instead of SL_r (ablation) | pragmatic simple speaker · Reference game (human listener, top-5) | accuracy (%): 82.6 -> 80.8 | Hurt |
| train listener on contrastive data (SL vs SL_r) (ablation) | simple listener · Reference game, human phrases (test) | accuracy (%): 84.2 -> 86.3 | Improved |
| interpret both phrases at once (DL) instead of averaging two simple listeners (ablation) | listener · Reference game, human phrases (test) | accuracy (%): 89.3 -> 88.9 | No clear effect |
| use first-listed attribute instead of later ones (ablation) | simple listener SL · Reference game (test) | accuracy (%): 81.6 -> 92.3 | Improved |
| use attribute phrases (same number, 46) instead of OID attributes | image embedding · FGVC-aircraft classification | accuracy: 12% better | Improved |
| increase to 300 attribute phrases | image embedding · FGVC-aircraft classification | accuracy: 32%, about 20% better than OID | Improved |
| use opponent attribute space (phrase pairs) instead of single phrases | image embedding · FGVC-aircraft classification | accuracy | No clear effect |
| use contrastive SL_r vs SL listener for embedding (ablation) | image embedding · FGVC-aircraft classification | accuracy | No clear effect |
Closest papers by meaning
- VisMin: Visual Minimal-Change Understanding (NeurIPS 2024 · similarity 0.86)
- GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection (NeurIPS 2025 · similarity 0.85)
- Democratizing Fine-grained Visual Recognition with Large Language Models (ICLR 2024 · similarity 0.85)
- Language Model as Visual Explainer (NeurIPS 2024 · similarity 0.85)
- The Devil is in the Fine-Grained Details: Evaluating Open-Vocabulary Object Detectors for Fine-Grained Understanding (CVPR 2024 · similarity 0.85)
- Investigating Compositional Challenges in Vision-Language Models for Visual Grounding (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
