Research map / Detection and segmentation / CNN features
Image Captioning: Transforming Objects into Words
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add geometric attention to the Standard Transformer (ablation) | Image captioning Transformer · MS-COCO Karpathy test | CIDEr-D: 113.21 -> 115.37 | Helped |
| Add geometric attention to the Standard Transformer (ablation) | Image captioning Transformer · MS-COCO Karpathy test | SPICE Relation: 5.88 -> 6.31 | Helped |
| Add geometric attention to the Standard Transformer (ablation) | Image captioning Transformer · MS-COCO Karpathy test | SPICE Count: 11.30 -> 17.51 | Helped |
| Replace LSTM with Transformer (ablation) | Up-Down image captioning model · MS-COCO Karpathy validation | CIDEr-D: 105.6 -> 110.5 | Helped |
| Replace LSTM with Transformer (ablation) | Up-Down image captioning model · MS-COCO Karpathy test | CIDEr-D: 106.6 -> 111.0 | Helped |
| Add geometric attention (ablation) | Up-Down + Transformer · MS-COCO Karpathy validation | CIDEr-D: 110.5 -> 113.2 | Helped |
| Add geometric attention (ablation) | Up-Down + Transformer · MS-COCO Karpathy test | CIDEr-D: 111.0 -> 112.6 | Helped |
| Add beam search with beam size 2 (ablation) | Up-Down + Object Relation Transformer · MS-COCO Karpathy validation | CIDEr-D: 113.2 -> 114.7 | Helped |
| Add beam search with beam size 2 (ablation) | Up-Down + Object Relation Transformer · MS-COCO Karpathy test | CIDEr-D: 112.6 -> 115.4 | Helped |
| Order detected objects by bounding-box size instead of leaving them unordered (ablation) | Transformer image encoder · MS-COCO Karpathy validation | CIDEr-D: 111.0 -> 108.7 | Hurt |
| Order detected objects left-to-right instead of leaving them unordered (ablation) | Transformer image encoder · MS-COCO Karpathy validation | CIDEr-D: 111.0 -> 110.2 | Hurt |
| Order detected objects top-to-bottom instead of leaving them unordered (ablation) | Transformer image encoder · MS-COCO Karpathy validation | CIDEr-D: 111.0 -> 109.1 | Hurt |
Closest papers by meaning
- Object Relation Attention for Image Paragraph Captioning (AAAI 2021 · similarity 0.88)
- Improving OCR-Based Image Captioning by Incorporating Geometrical Relationship (CVPR 2021 · similarity 0.88)
- Partially-Supervised Image Captioning (NeurIPS 2018 · similarity 0.87)
- Attention-Aligned Transformer for Image Captioning (AAAI 2022 · similarity 0.87)
- Relational Distant Supervision for Image Captioning without Image-Text Pairs (AAAI 2024 · similarity 0.87)
- Composing Object Relations and Attributes for Image-Text Matching (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.