Research map / Vision-language and multimodal / Human and embodied modeling
Diverse Image Captioning with Context-Object Split Latent Spaces
Shweta Mahajan, Stefan Roth
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add context-based pseudo supervision from neighboring contextual descriptions (ablation) | COS-CVAE (paired) · COCO oracle, 100 samples | CIDEr: 1.812 -> 1.893 | Improved |
| add context-based pseudo supervision (ablation) | COS-CVAE (paired) · COCO oracle, 100 samples | BLEU-4: 0.593 -> 0.633 | Improved |
| add context-based pseudo supervision (ablation) | COS-CVAE (paired) · COCO consensus re-ranking | CIDEr: 1.144 -> 1.120 | No clear effect |
| add attention LSTM decoder to Seq-CVAE (ablation) | Seq-CVAE · COCO oracle, 100 samples | CIDEr: 1.695 -> 1.808 | Improved |
| add attention LSTM decoder to Seq-CVAE (ablation) | Seq-CVAE · COCO diversity | uniqueness / m-BLEU-4 | Hurt |
| use COS factorization instead of Seq-CVAE latent space with attention decoder (ablation) | Seq-CVAE (attn) · COCO oracle, 100 samples | CIDEr: 1.808 -> 1.812 | No clear effect |
| add constrained beam search to COS-CVAE | COS-CVAE · held-out COCO | average F1: 65.0 -> 68.1 | Improved |
| use COS-CVAE with pseudo supervision instead of Seq-CVAE (attn, CBS) | novel object captioning · held-out COCO | average F1: 53.0 -> 65.0 | Improved |
Closest papers by meaning
- On Diversity in Image Captioning: Metrics and Methods (TPAMI 2020 · similarity 0.90)
- Sequential Latent Spaces for Modeling the Intention During Diverse Image Captioning (ICCV 2019 · similarity 0.90)
- Image Captioning with Multi-Context Synthetic Data (AAAI 2024 · similarity 0.87)
- Sieve: Multimodal Dataset Pruning using Image Captioning Models (CVPR 2024 · similarity 0.87)
- Modeling Caption Diversity in Contrastive Vision-Language Pretraining (ICML 2024 · similarity 0.87)
- A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
