Research map / Vision-language and multimodal / Human and embodied modeling
From Show to Tell: A Survey on Deep Learning-Based Image Captioning
Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Silvia Cascianelli, Giuseppe Fiameni, Rita Cucchiara
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace global CNN features with region-based visual encoding (Up-Down vs Show, Attend and Tell) | LSTM captioner · COCO Karpathy test | CIDEr: 104.6 -> 122.7 | Improved |
| Add reinforcement learning with CIDEr reward (SCST FC vs Show and Tell) | FC captioner · COCO Karpathy test | CIDEr: 97.2 -> 104.5 | Improved |
| Add inter-object relations via graphs (SGAE) over region-only Up-Down | Up-Down · COCO Karpathy test | CIDEr: 122.7 -> 129.1 | Improved |
| Add vision-and-language pre-training (VinVL vs X-Transformer) | Transformer captioner · COCO Karpathy test | CIDEr: 132.8 -> 140.9 | Improved |
| Add vision-and-language pre-training (VinVL vs X-Transformer) | Transformer captioner · COCO Karpathy test | SPICE: 23.4 -> 25.2 | Improved |
| Increase model size (VinVL 369.6M vs X-LAN 75.2M parameters) | captioner · COCO Karpathy test | CIDEr: 132.0 -> 140.9 | Mixed |
| Use larger model (CPTR 138.5M vs M2 Transformer 38.4M parameters) | Transformer captioner · COCO Karpathy test | CIDEr: 131.2 -> 129.4 | Hurt |
| Use text-only BERT-S metric to discriminate approaches | evaluation · COCO Karpathy test | BERT-S: 88.5 -> 94.4 | No clear effect |
| Add self-attention (X-LAN X-Linear attention vs AoANet) | LSTM captioner · COCO Karpathy test | CIDEr: 129.8 -> 132.0 | Improved |
Closest papers by meaning
- Convolutional Image Captioning (CVPR 2018 · similarity 0.92)
- An Empirical Study of Language CNN for Image Captioning (ICCV 2017 · similarity 0.91)
- Aligning Where to See and What to Tell: Image Captioning with Region-Based Attention and Scene-Specific Contexts (TPAMI 2016 · similarity 0.91)
- Show and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge (TPAMI 2016 · similarity 0.91)
- Generating Diverse and Descriptive Image Captions Using Visual Paraphrases (ICCV 2019 · similarity 0.91)
- Unsupervised Image Captioning (CVPR 2019 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
