AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Context-Aware Captions From Context-Agnostic Supervision

Ramakrishna Vedantam, Samy Bengio, Kevin Murphy, Devi Parikh, Gal Chechik

CVPR 2017 · Vision-language and multimodal › Human and embodied modeling · approach: Concept-guided image captioning · 112 citations (OpenAlex)

Check your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use introspective speaker inference instead of literal image-and-class captioningCUB captioning model · CUB-Justify testCIDEr-D: 12.3 ± 0.1 -> 18.4 ± 0.2Improved
Use introspective speaker inference instead of sample-and-rerank RSCUB captioning model · CUB-Justify testCIDEr-D: 15.8 ± 0.2 -> 18.4 ± 0.2Improved
Remove image information from the introspector (ablation)IS(λ) on CUB-Justify · CUB-Justify testCIDEr-D: 18.4 ± 0.2 -> 18.5 ± 0.2No clear effect
Remove image information from both speaker and introspector (ablation)IS(λ) on CUB-Justify · CUB-Justify testCIDEr-D: 18.4 ± 0.2 -> 16.1 ± 0.2Hurt
Add distractor-image context through introspective inferenceCOCO image captioning model · COCO easy confusionhuman target-image selection (%): 74.6 -> 89.0Improved
Add distractor-image context through introspective inferenceCOCO image captioning model · COCO hard confusionhuman target-image selection (%): 52.5 -> 74.1Improved
Add emitter-suppressor inference to vis-exp (ablation)vis-exp on CUB-Justify · CUB-Justify testCIDEr-D: 20.36 ± 0.16 -> 21.52 ± 0.17Improved
Lower λ from 1 toward an intermediate value to include distractor context (ablation)Introspective speaker · CUB-Justify validationCIDEr-DMixed
Use a trained listener for RS sample reranking instead of the introspector (ablation)RS(λ) on CUB-Justify · CUB-Justify validationCIDEr-D: 13.9 ± 0.2 -> 14.7 ± 0.2No clear effect
Train the inference objective rather than using it only at inference (ablation)Introspective speaker · Not specifiedperformanceNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy