Research map / Vision-language and multimodal / Human and embodied modeling
Show, Interpret and Tell: Entity-Aware Contextualised Image Captioning in Wikipedia
Khanh Nguyen, Ali Furkan Biten, Andres Mafla, Lluis Gomez, Dimosthenis Karatzas
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MNEM pre-training to T5_BASE++ (ablation) | T5_BASE++ · WIT test | CIDEr: 276.6 -> 279.09 | Improved |
| Add MNEM pre-training to TaT (ablation) | TaT · WIT test | CIDEr: +12 points in CIDEr | Improved |
| Replace Full Masking with MNEM (ablation) | TaT · WIT test | CIDEr: 260.87 -> 272.46 | Improved |
| Replace MLM with MNEM (ablation) | TaT · WIT test | CIDEr: 258.25 -> 272.46 | Improved |
| Replace Full Masking with MNEM (ablation) | T5_BASE++ · WIT test | CIDEr: 276.58 -> 279.09 | Improved |
| Replace MLM with MNEM (ablation) | T5_BASE++ · WIT test | CIDEr: 275.22 -> 279.09 | Improved |
| Replace Full Masking with MNEM (ablation) | GPT-2_BASE++ · WIT test | CIDEr: 266.4 -> 266.29 | No clear effect |
| Add images to Seq2seq captioning inputs | Seq2seq models · WIT test | CIDEr: 258.84 -> 249.67 | Hurt |
| Add images to TaT captioning inputs | TaT · WIT test | CIDEr: 229.77 -> 267.37 | Improved |
| Add MNEM pre-training on GoodNews (ablation) | T5_BASE++ · GoodNews test | CIDEr: 62.76 -> 63.89 | Improved |
| Add MNEM pre-training on WIT before GoodNews fine-tuning (ablation) | T5_BASE++ · GoodNews test | CIDEr: 60.17 -> 59.07 | Hurt |
| Add WIT pre-training before GoodNews fine-tuning | T5_BASE++ · GoodNews test | CIDEr: 59.19 -> 62.76 | Improved |
Closest papers by meaning
- A Generative Approach for Wikipedia-Scale Visual Entity Recognition (CVPR 2024 · similarity 0.87)
- Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach (NeurIPS 2024 · similarity 0.87)
- A Dual-Way Enhanced Framework from Text Matching Point of View for Multimodal Entity Linking (AAAI 2024 · similarity 0.86)
- Panoptic Captioning: An Equivalence Bridge for Image and Text (NeurIPS 2025 · similarity 0.84)
- CaptionQA: Is Your Caption as Useful as the Image Itself? (CVPR 2026 · similarity 0.84)
- Reverse Region-to-Entity Annotation for Pixel-Level Visual Entity Linking (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
