Research map / Vision-language and multimodal / Human and embodied modeling
Exploring Overall Contextual Information for Image Captioning in Human-Like Cognitive Style
Hongwei Ge, Zehang Yan, Kai Zhang, Mingde Zhao, Liang Sun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add mutual-aid auxiliary structure (lambda 0.01) (ablation) | forward LSTM L_f · MS COCO Karpathy test (frozen encoder) | CIDEr: 99.9 -> 105.7 | Improved |
| add mutual-aid auxiliary structure (lambda 0.01) (ablation) | backward LSTM L_b · MS COCO Karpathy test (frozen encoder) | CIDEr: 99.7 -> 103.6 | Improved |
| add mutual-aid auxiliary structure (lambda 0.01) (ablation) | forward LSTM L_f · MS COCO Karpathy test (frozen encoder) | BLEU-4: 32.1 -> 33.5 | Improved |
| add mutual-aid auxiliary structure (lambda 0.01) (ablation) | backward LSTM L_b · MS COCO Karpathy test (frozen encoder) | BLEU-4: 31.0 -> 31.9 | Improved |
| raise lambda from 0.01 to 0.1 (ablation) | forward LSTM L_f · MS COCO Karpathy test (frozen encoder) | CIDEr: 105.7 -> 100.0 | Hurt |
| raise lambda from 0.01 to 0.1 (ablation) | backward LSTM L_b · MS COCO Karpathy test (frozen encoder) | CIDEr: 103.6 -> 99.5 | Hurt |
| lambda 0.1 versus no auxiliary structure (ablation) | forward LSTM L_f · MS COCO Karpathy test (frozen encoder) | CIDEr: 99.9 -> 100.0 | No clear effect |
| use forward instead of backward generation | MaBi-LSTMs · MS COCO Karpathy test | CIDEr: 108.5 -> 110.4 | Improved |
| add cross-modal attention over image and pre-generated sentences (ablation) | MaBi-LSTMs L_f · MS COCO Karpathy test | CIDEr: 110.4 -> 116.6 | Improved |
| add cross-modal attention over image and pre-generated sentences (ablation) | MaBi-LSTMs L_f · MS COCO Karpathy test | BLEU-4: 36.4 -> 36.8 | Improved |
| add cross-modal attention (vs L_b captions) (ablation) | MaBi-LSTMs L_b · MS COCO Karpathy test | CIDEr: 108.5 -> 116.6 | Improved |
| fine-tune encoder jointly with decoder | MaBi-LSTMs · MS COCO Karpathy test | CIDEr: 105.7 -> 110.4 | Improved |
Closest papers by meaning
- Convolutional Image Captioning (CVPR 2018 · similarity 0.88)
- Look Back and Predict Forward in Image Captioning (CVPR 2019 · similarity 0.88)
- Unsupervised Image Captioning (CVPR 2019 · similarity 0.88)
- Reference Based LSTM for Image Captioning (AAAI 2017 · similarity 0.88)
- Incorporating Copying Mechanism in Image Captioning for Learning Novel Objects (CVPR 2017 · similarity 0.87)
- An Empirical Study of Language CNN for Image Captioning (ICCV 2017 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
