Research map / Vision-language and multimodal / Human and embodied modeling
Attention Correctness in Neural Image Captioning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add strong attention supervision from region-phrase alignments | Implicit image captioner · Flickr30k test | BLEU-4: 20.1 -> 21.0 | Helped |
| Add strong attention supervision from region-phrase alignments | Implicit image captioner · Flickr30k test | attention correctness, generated captions: 0.5202 -> 0.5787 | Helped |
| Add weak attention supervision from object class labels and segmentation masks | Implicit image captioner · COCO | BLEU-4: 26.9 -> 27.6 | Helped |
| Add weak attention supervision from object class labels and segmentation masks | Implicit image captioner · COCO | METEOR: 24.46 -> 24.78 | Helped |
| Add strong attention supervision from region-phrase alignments | Implicit image captioner · Flickr30k test, small boxes | attention correctness: 0.2484 -> 0.3682 | Helped |
| Use attention correctness to divide generated captions into high, middle, and low groups | Implicit attention captioner · Flickr30k test | BLEU-4: 28.1, 26.1, 25.4 | Mixed |
| Use average rather than maximum word-level correctness to score phrase attention (ablation) | Attention correctness metric · Flickr30k | main conclusion | No clear effect |
| Set scene-category words to uniform attention rather than aligning them to object regions (ablation) | Weakly supervised attention construction · COCO | attention-map quality | Helped |
Closest papers by meaning
- Text-Guided Attention Model for Image Captioning (AAAI 2017 · similarity 0.89)
- Attention-Aligned Transformer for Image Captioning (AAAI 2022 · similarity 0.89)
- Attention Alignment Between Humans and Vision-Language Models (NeurIPS 2026 · similarity 0.88)
- Painting with Words: Elevating Detailed Image Captioning with Benchmark and Alignment Learning (ICLR 2025 · similarity 0.87)
- Visual Attention Never Fades: Selective Progressive Attention ReCalibration for Detailed Image Captioning in Multimodal Large Language Models (ICML 2025 · similarity 0.86)
- Text Embedding is Not All You Need: Attention Control for Text-to-Image Semantic Alignment with Text Self-Attention Maps (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.