AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Exploring Diverse In-Context Configurations for Image Captioning

Xu Yang, Yongliang Wu, Mingzhuo Yang, Haokun Chen, Xin Geng

NeurIPS 2023 · Vision-language and multimodal › Human and embodied modeling · approach: Concept-guided image captioning · 11 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use MGC-TF@135 captions as anchors to select ground-truth captions rather than simply taking the first GTCRS in-context image selection on Open-Flamingo v1 9B · MSCOCO test, 4/8/16/32-shot meanCIDEr: 80.04 -> 82.51Improved
Combine SIIR-CLIP retrieval with MGC-TF@135 anchorsOpen-Flamingo v1 9B · MSCOCO test, 4/8/16/32-shot meanCIDEr: +20.9Improved
Use low-quality MGC-TF@66 captions directly instead of ground-truth captionsSIIR-CLIP in-context image selection on Open-Flamingo v1 9B · MSCOCO testCIDEr: 90.62 -> 66.79Hurt
Use MGC-VLM(0)@63 instead of syntactically error-prone MGC-TF@66Open-Flamingo v1 9B caption assignment · MSCOCO testCIDEr: 64.24 -> 74.05Improved
Use MGC-TF@135 anchors instead of first ground-truth captionsSix image selection strategies on Open-Flamingo v1 9B · MSCOCO testCIDEr: +8.8Improved
Use MGC-TF@66 as anchors to select GTCs rather than use the first GTCRS in-context image selection on Open-Flamingo v1 9B · MSCOCO testCIDEr: 80.04 -> 83.24Improved
Use similarity-based image retrieval rather than random selection with low-quality captionsMGC-TF@66 caption assignment on Open-Flamingo v1 9B · MSCOCO testCIDEr: 64.24 -> 66.79Mixed
Use similar in-context images rather than random images while retaining unrelated captionsOpen-Flamingo v1 9B · MSCOCO test; similarity to in-context captionsCIDEr: 54.8 -> 192. 2Improved
Use similar in-context images rather than random images while retaining unrelated captionsOpen-Flamingo v1 9B · MSCOCO test; similarity to test-image ground truthCIDEr: 58.5 -> 6.7Hurt
Use more iterations of generated-caption prompting after iteration 3 (ablation)MGC-VLM(0) iterative prompting on Open-Flamingo v1 9B · MSCOCO test, successive iterationsCIDEr: 79.9 -> 79.3 -> 77.3Hurt
Use more iterations of generated-caption prompting after iteration 2 (ablation)MGC-VLM(32) iterative prompting on Open-Flamingo v1 9B · MSCOCO test, successive iterationsCIDEr: 80.5 -> 79.4 -> 78.9 -> 77.1Hurt
Use DIIR-TT rather than DIIR-TR for diverse image retrieval (ablation)Open-Flamingo v1 9B with GTC captions · MSCOCO test, 4/8/16/32-shot meanCIDEr: 84.82 -> 86.00Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy