AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Context-Aware Group Captioning via Self-Attention and Contrastive Features

Zhuowan Li, Quan Tran, Long Mai, Zhe Lin, Alan L. Yuille

CVPR 2020 · Vision-language and multimodal › Human and embodied modeling · approach: Neural image captioning · 40 citations (OpenAlex)

Check your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add self-attention for group feature aggregation (ablation)Average feature aggregation · Conceptual and StockWordAcc: 36.7329 -> 37.9916; 37.9428 -> 39.2410Improved
Add contrastive features based on shared group information (ablation)Self-attention group captioning · Conceptual and StockWordAcc: 37.9916 -> 39.4496; 39.2410 -> 40.6113Improved
Add contrastive features to average-pooled representations (ablation)Average feature aggregation · Conceptual and StockWordAcc: 36.7329 -> 37.8450; 37.9428 -> 39.1985Improved
Apply self-attention jointly across target and reference images (ablation)Group captioning · Conceptual and StockWordAcc: 37.9916 -> 36.1231; 39.2410 -> 38.9215Hurt
Add cross-attention between groups after within-group self-attention (ablation)Self-attention group captioning · Conceptual and StockWordAcc: 37.9916 -> 36.2892; 39.2410 -> 38.6316Hurt
Use negative cross-attention to contrast the groups (ablation)Self-attention group captioning · Conceptual and StockWordAcc: 37.9916 -> 37.6046; 39.2410 -> 39.3278Mixed
Represent contrast as target-group features minus reference-group features (ablation)Self-attention group captioning · Conceptual and StockWordAcc: 37.9916 -> 38.2574; 39.2410 -> 39.9114Improved
Use only target-group features minus joint features as contrast (ablation)Self-attention group captioning · Conceptual and StockWordAcc: 37.9916 -> 38.5916; 39.2410 -> 40.2068Improved
Include both target and reference residual features relative to joint information (ablation)SA+Contrast2 · Conceptual and StockWordAcc: 38.5916 -> 39.4496; 40.2068 -> 40.6113Improved
Use five target images instead of three (ablation)SA+Contrast · StockWordAcc: 34.6574 -> 40.6113Improved
Add reference images to the target group (ablation)SA+Contrast · StockWordAcc: 31.8061 -> 37.1283 -> 39.4072 -> 40.6113Improved
Train with more random noise images in the target group (ablation)Group captioning model · ConceptualWordAccMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy