AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Engaging Image Captioning via Personality

Kurt Shuster, Samuel Humeau, Hexiang Hu, Antoine Bordes, Jason Weston

CVPR 2019 · Vision-language and multimodal › Human and embodied modeling · approach: Neural image captioning · 168 citations (OpenAlex)

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use personality conditioning (ablation)TransResNet on Personality-Captions · Personality-Captions testR@1: 53.9 -> 77.5Improved
Use ResNeXt-IG-3.5B instead of ResNet152 image features (ablation)TransResNet · Personality-Captions testR@1: 51.7 -> 77.5Improved
Pretrain the full Transformer text encoder instead of training it from scratch (ablation)TransResNet with ResNeXt-IG-3.5B and personality conditioning · Personality-Captions testR@1: 65.9 -> 77.5Improved
Use full Transformer pretraining instead of pretrained word embeddings only (ablation)TransResNet with ResNeXt-IG-3.5B and personality conditioning · Personality-Captions testR@1: 71.7 -> 77.5Improved
Use a Transformer caption encoder rather than bag of words (ablation)TransResNet with ResNeXt-IG-3.5B, word pretraining, and personality conditioning · Personality-Captions testR@1: 66.2 -> 71.7Improved
Add pretrained word embeddings (ablation)UpDown with ResNeXt-IG-3.5B and personality conditioning · Personality-Captions testCIDEr: 16.5 -> 16.3No clear effect
Use ResNeXt-IG-3.5B instead of ResNet152 image features (ablation)TransResNet · Personality-Captions human evaluationengagingness win percentage: 40.9 -> 49.5Improved
Use retrieval rather than generative captioningPersonality-conditioned captions with ResNeXt-IG-3.5B · Personality-Captions human evaluationengagingness win percentage: 20.7% vs. 49.5%Improved
Condition human captions on a personality rather than writing traditional neutral captionsHuman image captions · Personality-Captions human evaluationengagingness win percentage: 64.5% vs. 35.5%Improved
Write engaging captions without a personality cue rather than with personality conditioning (ablation)Human-authored captions overall · Personality-Captions human evaluationengagingness win percentage: 45.5 -> 54.5; positive traits: 51.2 -> 48.8Mixed
Use TransResNet rather than human captionsPersonality-conditioned captions with ResNeXt-IG-3.5B · Personality-Captions human evaluationengagingness win percentage: 49.5%No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy