Research map / Vision-language and multimodal / Human and embodied modeling
Unpaired Image Captioning by Language Pivoting
Jiuxiang Gu, Shafiq Joty, Jianfei Cai, Gang Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add pivot-embedding connection term (R i->x,x->y) to the pipeline (ablation) | pivot-based captioner · MSCOCO 5K | CIDEr: 12.3 -> 17.3 | Improved |
| add target-language connection term via caption autoencoder (ablation) | pivot-based captioner · MSCOCO 5K | CIDEr: 17.3 -> 17.7 | Improved |
| add pivot connection term (ablation) | pivot-based captioner · Flickr30K 1K | CIDEr: 13.9 -> 14.9 | Improved |
| add target-language connection term (ablation) | pivot-based captioner · Flickr30K 1K | CIDEr: 14.9 -> 16.2 | Improved |
| add both connection terms (ablation) | pivot-based captioner · MSCOCO 5K | BLEU-4: 3.9 -> 5.4 | Improved |
| add both connection terms (ablation) | pivot-based captioner · Flickr30K 1K | BLEU-4: 6.9 -> 7.9 | Improved |
| replace online Google Translation with jointly trained NMT with pivot connection term | pivot-based captioner · MSCOCO 5K | METEOR: 14.5 -> 13.1 | Hurt |
| replace online Google Translation with jointly trained NMT with both terms | pivot-based captioner · MSCOCO 5K | CIDEr: 17.0 -> 17.7 | Improved |
| add both connection terms (ablation) | pivot-based captioner · MSCOCO 5K | Self-BLEU-5: 87.6 -> 80.2 | Improved |
| use paired English captions (upper bound) instead of unpaired pivot | captioner · MSCOCO 5K | Self-BLEU-5: 80.2 -> 90.7 | Hurt |
| use ResNet-101 captioner instead of AIC baseline Inception-v3 captioner | Chinese captioner · AIC-ICC val | CIDEr: 142.5 -> 144.2 | Improved |
Closest papers by meaning
- UNISON: Unpaired Cross-Lingual Image Captioning (AAAI 2022 · similarity 0.91)
- Aligning Source Visual and Target Language Domains for Unpaired Video Captioning (TPAMI 2021 · similarity 0.90)
- Unpaired Image Captioning via Scene Graph Alignments (ICCV 2019 · similarity 0.88)
- Semantic and Expressive Variations in Image Captions Across Languages (CVPR 2025 · similarity 0.86)
- Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Training (AAAI 2024 · similarity 0.86)
- Text as Partial Constraint: Core–Residual Alignment for Robust Vision–Language Learning (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
