AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Guiding Image Captioning Models Toward More Specific Captions

Simon Kornblith, Lala Li, Zirui Wang, Thao Nguyen

ICCV 2023 · Vision-language and multimodal › Human and embodied modeling · approach: Neural image captioning · 19 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
apply classifier-free guidance scale 2 at decoding instead of greedy decodingCoCa-Base bottleneck captioner · MS-COCO Karpathy testCLIPScore: 0.775 -> 0.808Improved
apply classifier-free guidance scale 2CoCa-Base bottleneck captioner · MS-COCO Karpathy testcaption-to-image recall@1: 26.5 -> 44.6Improved
apply classifier-free guidance scale 2CoCa-Base bottleneck captioner · MS-COCO Karpathy testCIDEr: 126.1 -> 78.6Hurt
raise guidance scale from 3.0 to 4.0 (ablation)CoCa-Base bottleneck captioner · MS-COCO Karpathy testcaption-to-image recall@1: 49.4 -> 44.7Hurt
use moderate guidance scale (up to 2.0) instead of none (ablation)CoCa-Base bottleneck captioner · MS-COCO Karpathy testcaption length (words): 9.6 -> 9.3No clear effect
apply classifier-free guidance scale 2 on Stanford DogsCoCa-Base bottleneck captioner · Stanford Dogs test% captions containing breed name: 1.9 -> 42.4Improved
apply classifier-free guidance scale 2 on Stanford DogsCoCa-Base bottleneck captioner · Stanford Dogs test% breeds correct: 61.7 -> 58.5Mixed
prompt LM guidance with 20 MS-COCO captions for web-only modelCoCa-Base pretrained on JFT-5B + ALIGN · MS-COCO Karpathy testCIDEr: 21.8 -> 57.4Improved
increase number of LM prompt captions from 5 to 50 (ablation)LM guidance on web-only model · MS-COCO Karpathy testCIDEr: 48.6 -> 59.7Improved
use LM guidance (alpha=5, beta=-5/2) instead of CFG gamma=1.6CoCa-Base fine-tuned on MS-COCO · MS-COCO Karpathy testrecall@1 and CIDEr: R@1 39.0 -> 39.6, CIDEr 109.3 -> 114.4Improved
use LM guidance (alpha=beta=1) instead of CFG on web-only modelCoCa-Base pretrained on JFT-5B + ALIGN · MS-COCO Karpathy testCLIPScore and recall@1: R@1 27.3 (CFG gamma 4) -> 14.4 (LM 20 captions)Hurt
use attention-pooling CoCa instead of the bottleneck architecture (ablation)classifier-free guidance captioner · MS-COCO Karpathy testcaption-to-image recall@1 (gamma 2.0): 44.6 -> 48.6Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy