AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

With a Little Help from Your Own Past: Prototypical Memory Networks for Image Captioning

Manuele Barraco, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

ICCV 2023 · Vision-language and multimodal › Human and embodied modeling · approach: Concept-guided image captioning · 33 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add prototypical memory (prototype vectors from past keys/values) to the decoder (ablation)Transformer captioner · COCO Karpathy test (cross-entropy)CIDEr: 127.8 -> 131.5Improved
add prototypical memory under CIDEr optimization (SCST)Transformer captioner · COCO Karpathy test (CIDEr optimization)CIDEr: 140.3 -> 144.1Improved
add plain learnable memory vectors instead of prototypes (ablation)Transformer captioner decoder · COCO Karpathy test (cross-entropy)CIDEr: 127.8 -> 127.9 (64); 127.6 (1024)No clear effect
increase number of prototypes from 256 to 1024 (ablation)PMA-Net · COCO Karpathy test (cross-entropy)CIDEr: 129.4 -> 131.5Improved
increase memory bank size T from 500 to 1500 (ablation)PMA-Net · COCO Karpathy test (cross-entropy)CIDEr: 128.6 -> 131.5Improved
add segment embeddings to distinguish prototypes from input keys (ablation)PMA-Net · COCO Karpathy test (cross-entropy)CIDEr: 130.1 -> 131.5Improved
add prototype memory in the first decoder layer too (ablation)PMA-Net · COCO Karpathy test (cross-entropy)CIDEr: 129.2 -> 131.5Improved
add prototype memory on robust COCO to reduce hallucinationTransformer captioner · Robust COCO testCHs: 4.6 -> 4.3Improved
add prototype memory for novel object captioningTransformer captioner · nocaps validation out-of-domainCIDEr: 73.9 -> 75.9Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy