Research map / Vision-language and multimodal / Human and embodied modeling
Show, Tell and Discriminate: Image Captioning by Self-retrieval with Partially Labeled Data
Xihui Liu, Hongsheng Li, Jing Shao, Dapeng Chen, Xiaogang Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add self-retrieval reward (fully labeled) (ablation) | CIDEr-reward baseline · COCO Karpathy test | CIDEr: 112.7 -> 114.6 | Improved |
| add unlabeled images with retrieval-only reward (ablation) | SR-FL · COCO Karpathy test | CIDEr: 114.6 -> 117.1 | Improved |
| add self-retrieval reward (fully labeled) (ablation) | CIDEr-reward baseline · COCO Karpathy test | BLEU-4: 35.0 -> 34.9 | No clear effect |
| add self-retrieval reward and unlabeled data (ablation) | CIDEr-reward baseline · Flickr30k | CIDEr: 57.1 -> 65.0 | Improved |
| add self-retrieval reward (fully labeled) (ablation) | CIDEr-reward baseline · Flickr30k | CIDEr: 57.1 -> 61.7 | Improved |
| use VSE++ instead of VSE0 retrieval loss (ablation) | SR-PL · COCO Karpathy test (ablation) | CIDEr: 116.9 -> 117.1 | Improved |
| use VSE++ instead of softmax retrieval loss (ablation) | SR-PL · COCO Karpathy test (ablation) | CIDEr: 114.5 -> 117.1 | Improved |
| use self-retrieval weight 1 instead of 0 (ablation) | SR-PL · COCO Karpathy test (ablation) | CIDEr: 112.7 -> 117.1 | Improved |
| raise self-retrieval weight from 1 to 4 (ablation) | SR-PL · COCO Karpathy test (ablation) | CIDEr: 117.1 -> 113.7 | Hurt |
| use 1:1 labeled to unlabeled ratio instead of 1:2 (ablation) | SR-PL · COCO Karpathy test (ablation) | CIDEr: 115.4 -> 117.1 | Improved |
| use 1:1 labeled to unlabeled ratio instead of 2:1 (ablation) | SR-PL · COCO Karpathy test (ablation) | CIDEr: 115.0 -> 117.1 | Improved |
| add moderately hard negative mining (range 100-1000) (ablation) | SR-PL · COCO Karpathy test (ablation) | CIDEr: 114.6 -> 117.1 | Improved |
Closest papers by meaning
- Guiding Image Captioning Models Toward More Specific Captions (ICCV 2023 · similarity 0.90)
- Discriminability Objective for Training Descriptive Captions (CVPR 2018 · similarity 0.90)
- Cross-Domain Image Captioning With Discriminative Finetuning (CVPR 2023 · similarity 0.89)
- From Show to Tell: A Survey on Deep Learning-Based Image Captioning (TPAMI 2022 · similarity 0.89)
- Context-Aware Captions From Context-Agnostic Supervision (CVPR 2017 · similarity 0.89)
- CaptionQA: Is Your Caption as Useful as the Image Itself? (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
