Research map / Vision-language and multimodal / Human and embodied modeling
Engaging Image Captioning via Personality
Kurt Shuster, Samuel Humeau, Hexiang Hu, Antoine Bordes, Jason Weston
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use personality conditioning (ablation) | TransResNet on Personality-Captions · Personality-Captions test | R@1: 53.9 -> 77.5 | Improved |
| Use ResNeXt-IG-3.5B instead of ResNet152 image features (ablation) | TransResNet · Personality-Captions test | R@1: 51.7 -> 77.5 | Improved |
| Pretrain the full Transformer text encoder instead of training it from scratch (ablation) | TransResNet with ResNeXt-IG-3.5B and personality conditioning · Personality-Captions test | R@1: 65.9 -> 77.5 | Improved |
| Use full Transformer pretraining instead of pretrained word embeddings only (ablation) | TransResNet with ResNeXt-IG-3.5B and personality conditioning · Personality-Captions test | R@1: 71.7 -> 77.5 | Improved |
| Use a Transformer caption encoder rather than bag of words (ablation) | TransResNet with ResNeXt-IG-3.5B, word pretraining, and personality conditioning · Personality-Captions test | R@1: 66.2 -> 71.7 | Improved |
| Add pretrained word embeddings (ablation) | UpDown with ResNeXt-IG-3.5B and personality conditioning · Personality-Captions test | CIDEr: 16.5 -> 16.3 | No clear effect |
| Use ResNeXt-IG-3.5B instead of ResNet152 image features (ablation) | TransResNet · Personality-Captions human evaluation | engagingness win percentage: 40.9 -> 49.5 | Improved |
| Use retrieval rather than generative captioning | Personality-conditioned captions with ResNeXt-IG-3.5B · Personality-Captions human evaluation | engagingness win percentage: 20.7% vs. 49.5% | Improved |
| Condition human captions on a personality rather than writing traditional neutral captions | Human image captions · Personality-Captions human evaluation | engagingness win percentage: 64.5% vs. 35.5% | Improved |
| Write engaging captions without a personality cue rather than with personality conditioning (ablation) | Human-authored captions overall · Personality-Captions human evaluation | engagingness win percentage: 45.5 -> 54.5; positive traits: 51.2 -> 48.8 | Mixed |
| Use TransResNet rather than human captions | Personality-conditioned captions with ResNeXt-IG-3.5B · Personality-Captions human evaluation | engagingness win percentage: 49.5% | No clear effect |
Closest papers by meaning
- Cross-Modality Personalization for Retrieval (CVPR 2019 · similarity 0.86)
- Human-Like Controllable Image Captioning With Verb-Specific Semantic Roles (CVPR 2021 · similarity 0.86)
- Zero-Shot Image Personalization from Personas (ECCV 2026 · similarity 0.85)
- Learning Long- and Short-Term User Literal-Preference with Multimodal Hierarchical Transformer Network for Personalized Image Caption (AAAI 2020 · similarity 0.85)
- CaptionQA: Is Your Caption as Useful as the Image Itself? (CVPR 2026 · similarity 0.85)
- Enhancing Descriptive Captions with Visual Attributes for Multimodal Perception (CVPR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
