Research map / Vision-language and multimodal / Human and embodied modeling
Who Are You Referring To? Coreference Resolution In Image Narrations
Arushi Goel, Basura Fernando, Frank Keller, Hakan Bilen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add linguistic-prior regularizer to MAF training | MAF · CIN test (coreference) | MUC-F1: 13.21 -> 15.65 | Improved |
| add mouse trace modality (ablation) | Ours · CIN test (coreference) | MUC-F1: 17.58 -> 19.19 | Improved |
| add linguistic regularizer (Frobenius norm) (ablation) | Ours · CIN (narrative grounding) | Overall grounding accuracy (%): 26.75 -> 29.36 | Improved |
| add linguistic regularizer with mouse traces (ablation) | Ours · CIN test (coreference) | MUC-F1: 16.26 -> 19.19 | Improved |
| use cross attention over words instead of averaging word features (ablation) | Ours · CIN test (coreference) | MUC-F1: 17.02 -> 19.19 | Improved |
| use cross attention instead of averaging for grounding (ablation) | Ours · CIN (narrative grounding) | Overall grounding accuracy (%): 28.83 -> 29.36 | Improved |
| Frobenius norm regularizer instead of L1 (ablation) | Ours · CIN test (coreference) | MUC-F1: 16.05 -> 19.19 | Improved |
| Frobenius norm regularizer instead of MSE (ablation) | Ours · CIN test (coreference) | MUC-F1: 17.00 -> 19.19 | Improved |
| transformer encoders and attention over mouse traces instead of MLP (MAF++) | Ours without MT · CIN test (coreference) | MUC-F1: 15.65 -> 17.58 | Improved |
| use off-the-shelf text-only neural coreference model | Neural-Coref · CIN test (coreference) | MUC-F1: 0.13 | Hurt |
Closest papers by meaning
- Visual Coreference Resolution in Visual Dialog using Neural Module Networks (ECCV 2018 · similarity 0.87)
- DetermiNet: A Large-Scale Diagnostic Dataset for Complex Visually-Grounded Referencing using Determiners (ICCV 2023 · similarity 0.86)
- Panoptic Narrative Grounding (ICCV 2021 · similarity 0.85)
- Weakly Supervised Referring Image Segmentation with Intra-Chunk and Inter-Chunk Consistency (ICCV 2023 · similarity 0.85)
- Zero-shot Referring Expression Comprehension via Structural Similarity Between Images and Captions (CVPR 2024 · similarity 0.84)
- Boosting Weakly Supervised Referring Image Segmentation via Progressive Comprehension (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
