AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training

Rakshith Shetty, Marcus Rohrbach, Lisa Anne Hendricks, Mario Fritz, Bernt Schiele

ICCV 2017 · Vision-language and multimodal › Human and embodied modeling · approach: Neural image captioning · 230 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace maximum-likelihood training with adversarial training (beam search)caption generator · MS-COCO test (5000 images)Vocabulary size: 1085 -> 2176Improved
replace maximum-likelihood training with adversarial training (sampling)caption generator · MS-COCO test (5000 images)Vocabulary size: 1460 -> 2671Improved
replace maximum-likelihood training with adversarial training (beam search)caption generator · MS-COCO test (5000 images)% novel sentences: 44.27 -> 72.53Improved
replace maximum-likelihood training with adversarial training (sampling)caption generator · MS-COCO test (5000 images)mBleu-4: 0.68 -> 0.51Improved
replace maximum-likelihood training with adversarial training (beam search)caption generator · MS-COCO test (5000 images)Meteor: 0.272 -> 0.239Hurt
replace maximum-likelihood training with adversarial training (beam search)caption generator · MS-COCO test (5000 images)SPICE: 0.187 -> 0.167Hurt
replace maximum-likelihood training with adversarial training (beam search)caption generator · MS-COCO test (5000 images)SPICE Count: 0.025 -> 0.080Improved
adversarial training instead of ML, judged by humanscaption generator · MS-COCO test (5000 images)human eval: adversarial better vs worse (%), beam search: 36.9 vs 34.8No clear effect
discriminator evaluates 5 captions instead of 1 (ablation)adversarial generator (VGG) · MS-COCO val (5000 images)Meteor: 0.179 -> 0.197Improved
discriminator evaluates 5 captions instead of 1 (ablation)adversarial generator (VGG) · MS-COCO val (5000 images)Vocabulary size: 812 -> 1810Improved
add feature matching loss (ablation)adversarial generator (VGG, p=5) · MS-COCO val (5000 images)Meteor: 0.197 -> 0.207Improved
add feature matching loss (ablation)adversarial generator (VGG, p=5) · MS-COCO val (5000 images)Vocabulary size: 1810 -> 2547Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy