AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Joint Optimization for Cooperative Image Captioning

Gilad Vered, Gal Oren, Yuval Atzmon, Gal Chechik

ICCV 2019 · Vision-language and multimodal › Human and embodied modeling · approach: Neural image captioning · 20 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use partial sampling (PSST) instead of always sampling (ST Multinomial) (ablation)speaker-listener captioner · COCO fixed CIDEr=1.2recall@10 (%): 81.8 -> 86.3Improved
use PSST on Gumbel softmax instead of ST Gumbel softmax (ablation)speaker-listener captioner · COCO fixed CIDEr=1.2recall@10 (%): 80.6 -> 85.7Improved
train speaker and listener jointly instead of freezing speaker (MLE) then training listener (ablation)PSST Multinomial · COCO fixed CIDEr=1.13recall@10 (%): 61.3 -> 89.9Improved
train jointly instead of Luo et al. frozen listener approach (ablation)PSST Multinomial · COCO fixed CIDEr=1.13recall@10 (%): 74.6 -> 89.9Improved
joint training with PSST instead of REINFORCE (ablation)speaker-listener captioner · COCO fixed CIDEr=1.13recall@10 (%): 86.2 -> 89.9Improved
PSST instead of REINFORCE at fixed recall@5=80% (ablation)speaker-listener captioner · COCOCIDEr: 0.902 -> 1.119Improved
PSST instead of REINFORCE at fixed recall@5=90% (ablation)speaker-listener captioner · Flickr30KCIDEr: 0.431 -> 0.488Improved
PSST instead of ST Multinomial at fixed recall@5=90% (ablation)speaker-listener captioner · Flickr30KCIDEr: 0.478 -> 0.488No clear effect
test listener on human-generated captions instead of speaker captionsPSST listener · COCO valrecall@10 (%): 89.9 -> 59.3Hurt
train listener on GT captions instead of jointly (ablation)PSST · COCO valrecall@10 (%): 89.9 -> 66.8Hurt
use partial sampling rate 0<rho<1 instead of rho=0 or 1 (ablation)PSST · COCO fixed CIDEr=1.22recall@10Improved
PSST Multinomial captions vs ST Multinomial in human discrimination (ablation)PSST Multinomial · human 2AFCaccuracy (%): 69% -> 75%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy