Research map / Vision-language and multimodal / Human and embodied modeling
Improving Image Captioning by Leveraging Intra- and Inter-layer Global Representation in Transformer Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add GET with MAC instead of the strongest single-model baseline | Image captioning on MS COCO Karpathy test · MS COCO Karpathy test, single model | CIDEr: 131.2 -> 131.6 | Helped |
| Use GET with MAC instead of the strongest listed single-model BLEU-4 baseline | Image captioning on MS COCO Karpathy test · MS COCO Karpathy test, single model | BLEU-4: 39.3 -> 39.5 | Helped |
| Use GET in place of MMT | Two-model caption ensemble on MS COCO Karpathy test · MS COCO Karpathy test, ensemble of 2 | CIDEr: 133.2 -> 133.5 | Helped |
| Use GET in place of MMT | Four-model caption ensemble on MS COCO Karpathy test · MS COCO Karpathy test, ensemble of 4 | CIDEr: 134.5 -> 135.1 | Helped |
| Use GET instead of MMT | Four-model ensemble on MS COCO online test · MS COCO online test, c5 | CIDEr-D: 129.3 -> 130.3 | Helped |
| Use 3 encoder and decoder layers rather than 2 (ablation) | GET · MS COCO Karpathy test, ablation | CIDEr: 129.7 -> 131.6 | Helped |
| Increase the encoder and decoder depth from 3 to 4 layers (ablation) | GET · MS COCO Karpathy test, ablation | CIDEr: 131.6 -> 130.7 | Hurt |
| Increase the encoder and decoder depth from 3 to 5 or 6 layers (ablation) | GET · MS COCO Karpathy test, ablation | CIDEr: 131.6 -> 130.3 | Hurt |
| Add GEA to the plain Transformer (ablation) | Plain Transformer · MS COCO Karpathy test, ablation | CIDEr: 128.1 -> 128.3 | Helped |
| Add GEA to the encoder while using MAC and no inter-layer fusion (ablation) | g0 + MAC variant · MS COCO Karpathy test, ablation | CIDEr: 128.6 -> 128.9 | Helped |
| Use average pooling for inter-layer fusion (ablation) | GEA + MAC · MS COCO Karpathy test, ablation | CIDEr: 128.9 -> 129.4 | Helped |
| Use attention for inter-layer fusion (ablation) | GEA + MAC · MS COCO Karpathy test, ablation | CIDEr: 128.9 -> 129.8 | Helped |
Closest papers by meaning
- End-to-End Transformer Based Model for Image Captioning (AAAI 2022 · similarity 0.88)
- Image Caption with Global-Local Attention (AAAI 2017 · similarity 0.87)
- Scaling Laws for Conditional Emergence of Multilingual Image Captioning via Generalization from Translation (AAAI 2026 · similarity 0.86)
- Modeling Caption Diversity in Contrastive Vision-Language Pretraining (ICML 2024 · similarity 0.85)
- Frozen Transformers in Language Models Are Effective Visual Encoder Layers (ICLR 2024 · similarity 0.84)
- Revisiting Change Captioning from Self-supervised Global-Part Alignment (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.