Research map / Vision-language and multimodal / Human and embodied modeling
Learning to Guide Decoding for Image Captioning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace fixed MIL guidance with an adaptively learned guiding vector | Soft Attention · MS COCO test, single model | CIDEr: 1.005 -> 1.023 | Helped |
| Replace fixed MIL guidance with an adaptively learned guiding vector | Review Net · MS COCO test, single model | CIDEr: 1.012 -> 1.039 | Helped |
| Add a learned guiding vector to the Review Net decoder | Review Net · MS COCO test, single model | BLEU-4: 0.324 -> 0.336 | Helped |
| Add a learned guiding vector to the Review Net decoder | Review Net · MS COCO test, ensemble | CIDEr: 1.046 -> 1.078 | Helped |
| Use discriminative-loss weight 10 rather than 100, 1, 0.1, or 0.01 (ablation) | LTG-Review-Net · MS COCO validation | CIDEr: 100: 0.963; 10: 0.983; 1: 0.964; 0.1: 0.947; 0.01: 0.939 | Helped |
| Add MIL attributes to the guiding-network input (ablation) | LTG-Review-Net with annotation vectors · MS COCO validation | CIDEr: 0.973 -> 0.983 | Helped |
| Add encoder annotation vectors to the guiding-network input (ablation) | LTG-Review-Net with MIL · MS COCO validation | CIDEr: 0.975 -> 0.983 | Helped |
| Add MIL attributes and annotation vectors rather than neither (ablation) | LTG-Review-Net · MS COCO validation | CIDEr: 0.958 -> 0.983 | Helped |
| Add MIL attributes to the guiding-network input (ablation) | LTG-Soft-Attention with annotation vectors · MS COCO validation | CIDEr: 0.936 -> 0.965 | Helped |
| Add encoder annotation vectors to the guiding-network input (ablation) | LTG-Soft-Attention with MIL · MS COCO validation | CIDEr: 0.931 -> 0.965 | Helped |
| Add MIL attributes and annotation vectors rather than neither (ablation) | LTG-Soft-Attention · MS COCO validation | CIDEr: 0.929 -> 0.965 | Helped |
| Add a learned guiding vector to Review Net | Review Net · MS COCO test, single model | CIDEr: 1.009 -> 1.039 | Helped |
Closest papers by meaning
- Improving Image Captioning by Leveraging Intra- and Inter-layer Global Representation in Transformer Network (AAAI 2021 · similarity 0.86)
- Image Captioning with Context-Aware Auxiliary Guidance (AAAI 2021 · similarity 0.86)
- End-to-End Transformer Based Model for Image Captioning (AAAI 2022 · similarity 0.86)
- Show, Edit and Tell: A Framework for Editing Image Captions (CVPR 2020 · similarity 0.86)
- Review Networks for Caption Generation (NeurIPS 2016 · similarity 0.85)
- Neural Image Compression with Text-guided Encoding for both Pixel-level and Perceptual Fidelity (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.