AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

HAAV: Hierarchical Aggregation of Augmented Views for Image Captioning

Chia-Wen Kuo, Zsolt Kira

CVPR 2023 · Vision-language and multimodal › Human and embodied modeling · approach: Neural image captioning · 38 citations (OpenAlex)

This paper tried a common change: Add a contrastive loss

Add contrastive loss to the shared encoder · MS-COCO Karpathy validation, cross-entropy only · CIDEr: 125.6 -> 127.6 Improved

What 1,317 accepted papers report for this change

See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use HAAV instead of Xmodal-CtxImage captioning · MS-COCO Karpathy testCIDEr: 135.9 -> 141.5Improved
Use HAAV instead of ORTImage captioning · Flickr30K KarpathyCIDEr: 68.8 -> 81.7Improved
Add unlabeled MS-COCO images for semi-supervised trainingHAAV · Flickr30K Karpathy + MS-COCO unlabeledCIDEr: 81.7 -> 85.6Improved
Add contrastive loss to the shared encoder (ablation)HAAV · MS-COCO Karpathy validation, cross-entropy onlyCIDEr: 125.6 -> 127.6Improved
Use unshared rather than shared encoders without contrastive loss (ablation)HAAV · MS-COCO Karpathy validation, cross-entropy onlyCIDEr: 125.6 -> 125.4No clear effect
Use unshared encoders with contrastive loss instead of a shared encoder with contrastive loss (ablation)HAAV · MS-COCO Karpathy validation, cross-entropy onlyCIDEr: 127.6 -> 125.8Hurt
Replace hierarchical decoder with concatenation and one cross-attention (ablation)HAAV · MS-COCO Karpathy validation, cross-entropy onlyCIDEr: 127.6 -> 125.1Hurt
Replace view-level cross-attention with mean pooling (ablation)HAAV · MS-COCO Karpathy validation, cross-entropy onlyCIDEr: 127.6 -> 125.2Hurt
Replace view-level cross-attention with max pooling (ablation)HAAV · MS-COCO Karpathy validation, cross-entropy onlyCIDEr: 127.6 -> 122.9Hurt
Replace view-level cross-attention with sigmoid gating (ablation)HAAV · MS-COCO Karpathy validation, cross-entropy onlyCIDEr: 127.6 -> 125.5Hurt
Replace view-level cross-attention with tanh gating (ablation)HAAV · MS-COCO Karpathy validation, cross-entropy onlyCIDEr: 127.6 -> 124.8Hurt
Use HAAV rather than concatenated views (ablation)Heterogeneous-view captioning · MS-COCO Karpathy validationCIDEr: 122.8 -> 127.6Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy