Research map / Vision-language and multimodal / CLIP and zero-shot
Towards Unsupervised Image Captioning With Shared Multimodal Embeddings
Iro Laina, Christian Rupprecht, Nassir Navab
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add robust min-distance alignment loss instead of L2 (ablation) | Joint captioning model · COCO test (unpaired, 80 COCO categories) | CIDEr: 53.8 -> 62.4 | Improved |
| add naive L2 alignment to MLE training (ablation) | MLE-only model · COCO test (unpaired, 80 COCO categories) | CIDEr: 57.9 -> 53.8 | No clear effect |
| add triplet loss to the language model (ablation) | Joint robust model · COCO test (unpaired, 80 COCO categories) | CIDEr: 55.6 -> 62.4 | Improved |
| add adversarial training conditioned on visual concepts (ablation) | Joint robust model · COCO test (unpaired, 80 COCO categories) | CIDEr: 62.4 -> 63.6 | Improved |
| add adversarial training conditioned on visual concepts (ablation) | Joint robust model · COCO test (unpaired, 80 COCO categories) | BLEU-4: 18.8 -> 19.3 | Improved |
| add adversarial training (ablation) | Unsupervised model · Flickr30k images / COCO captions (unsupervised) | CIDEr: 8.2 -> 9.9 | Improved |
| add adversarial training (ablation) | Unsupervised model · COCO images / Conceptual Captions (unsupervised) | CIDEr: 20.8 -> 22.7 | Improved |
| add adversarial training (ablation) | Unsupervised model · Flickr30k images / COCO captions (unsupervised) | BLEU-4: 5.9 -> 7.9 | Improved |
| add adversarial training (ablation) | Unsupervised model · COCO images / Conceptual Captions (unsupervised) | BLEU-4: 5.5 -> 6.5 | Improved |
| jointly finetune decoder instead of freezing it (ablation) | Alignment-only model · COCO test (unpaired, 80 COCO categories) | CIDEr: 29.4 -> 57.9 | Improved |
Closest papers by meaning
- Unsupervised Image Captioning (CVPR 2019 · similarity 0.91)
- Text as Partial Constraint: Core–Residual Alignment for Robust Vision–Language Learning (NeurIPS 2026 · similarity 0.89)
- Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Training (AAAI 2024 · similarity 0.88)
- Aligning Forest and Trees in Images & Long Captions for Visually Grounded Understanding (NeurIPS 2026 · similarity 0.88)
- Bridging Vision and Language Spaces with Assignment Prediction (ICLR 2024 · similarity 0.88)
- Relational Distant Supervision for Image Captioning without Image-Text Pairs (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
