AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Learning to Generate Grounded Visual Captions without Localization Supervision

Chih-Yao Ma, Yannis Kalantidis, Ghassan AlRegib, Peter Vajda, Marcus Rohrbach, Zsolt Kira

ECCV 2020 · Vision-language and multimodal › Human and embodied modeling · approach: Concept-guided image captioning · 37 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add cyclical localization-reconstruction trainingunsupervised captioning baseline · Flickr30k Entities (test)F1_all: 4.08 -> 4.98Improved
add cyclical localization-reconstruction trainingunsupervised captioning baseline · Flickr30k Entities (test)F1_loc_per_sent: 31.83 -> 35.54Improved
add cyclical localization-reconstruction trainingunsupervised captioning baseline · Flickr30k Entities (test)CIDEr: 59.6 -> 61.4Improved
add cyclical localization-reconstruction trainingunsupervised captioning baseline · ActivityNet-Entities (val)F1_loc: 12.00 -> 15.84Improved
add cyclical localization-reconstruction trainingunsupervised captioning baseline · ActivityNet-Entities (val)F1_loc_per_sent: 31.68 -> 41.56Improved
add cyclical localization-reconstruction trainingunsupervised captioning baseline · ActivityNet-Entities (val)SPICE: 15.1 -> 14.8Hurt
add explicit KL attention consistency between attended and localized regions (ablation)unsupervised baseline · Flickr30k Entities (val)F1_all: 4.18 -> 4.19No clear effect
localize from the generated word embedding only instead of h^A (ablation)cyclical model · Flickr30k Entities (val)F1_all: 4.58 -> 5.63Improved
use a single fully-connected localizer instead of MLP (ablation)cyclical model · Flickr30k Entities (test)F1_all: 4.40 -> 4.98Improved
zero the localized region for non noun/verb words (ablation)cyclical model · Flickr30k Entities (test)F1_all_per_sent: 15.03 -> 16.30Mixed
compute reconstruction loss only on noun/verb words (ablation)cyclical model · Flickr30k Entities (test)F1_all: 4.98 -> 5.14No clear effect
add cyclical training with a perfect object detectorunsupervised baseline · Flickr30k Entities (test, perfect object detector)F1_all: 20.82 -> 25.27Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy