Research map / Detection and segmentation / Open-set and few-shot detection
Object-Centric Unsupervised Image Captioning
Zihang Meng, David Yang, Xuefei Cao, Ashish Shah, Ser-Nam Lim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| mine objects from different images instead of pseudo image-sentence pairs (ablation) | unsupervised captioner · COCO + SS | CIDEr: 0.021 -> 0.039 | Improved |
| mine objects from different images instead of pseudo image-sentence pairs (ablation) | unsupervised captioner · COCO + GCC | CIDEr: 0.025 -> 0.032 | Improved |
| raise object coverage from 50.0% to 100% (ablation) | object-mining captioner · COCO + SS | CIDEr: 0.036 -> 0.039 | Improved |
| raise object coverage from 50.0% to 100% (ablation) | object-mining captioner · COCO + GCC | CIDEr: 0.025 -> 0.032 | Improved |
| raise overlap criterion from >= 0 to >= 2 (coverage 4.2% to 47.5%) (ablation) | prior pair-mining method · COCO + SS | CIDEr: 0.015 -> 0.026 | Improved |
| add attribute detector (ablation) | object-mining captioner · COCO + LN-OpenImages | CIDEr: 0.109 -> 0.124 | Improved |
| add relationship detector (ablation) | object-mining captioner · COCO + LN-OpenImages | CIDEr: 0.109 -> 0.120 | Improved |
| add both relationship and attribute detectors (ablation) | object-mining captioner · COCO + LN-OpenImages | CIDEr: 0.109 -> 0.138 | Improved |
| replace objects with same-category objects from other images (ablation) | supervised transformer captioner · COCO + LN-COCO | CIDEr: 0.263 -> 0.227 | Hurt |
| use original locations of substitute objects instead of replaced locations (ablation) | supervised transformer with replaced objects · COCO + LN-COCO | CIDEr: 0.227 -> 0.239 | No clear effect |
| train directly in target language instead of translating English captions (ablation) | object-mining captioner · COCO-CN Chinese | CIDEr: 0.059 -> 0.349 | Improved |
| train directly in target language instead of translating English captions (ablation) | object-mining captioner · Multi30k French | CIDEr: 0.099 -> 0.120 | Improved |
Closest papers by meaning
- Relational Distant Supervision for Image Captioning without Image-Text Pairs (AAAI 2024 · similarity 0.92)
- Unsupervised Image Captioning (CVPR 2019 · similarity 0.91)
- nocaps: novel object captioning at scale (ICCV 2019 · similarity 0.90)
- Towards Unsupervised Image Captioning With Shared Multimodal Embeddings (ICCV 2019 · similarity 0.90)
- Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioning (CVPR 2022 · similarity 0.90)
- Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Training (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
