Research map / Detection and segmentation / Open-set and few-shot detection
Neural Baby Talk
Jiasen Lu, Jianwei Yang, Dhruv Batra, Devi Parikh
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Generate a hybrid caption template with region-grounded slots, then fill slots using detector concepts | Image captioning · Flickr30k Entities test | CIDEr: 53.1 -> 57.5 | Improved |
| Generate a hybrid caption template with region-grounded slots, then fill slots using detector concepts | Image captioning · COCO test | BLEU4: 33.4 -> 34.7 | Improved |
| Generate a hybrid caption template with region-grounded slots, then fill slots using detector concepts | Image captioning · COCO test | CIDEr: 120.1 -> 107.2 | Hurt |
| Use ground-truth object regions and categories at test time (ablation) | NBT · Robust COCO test | Accuracy: 42.4 -> 45.7 | Improved |
| Use ground-truth object regions and categories at test time (ablation) | NBT · COCO test | BLEU4: 34.7 -> 34.9 | No clear effect |
| Use constrained beam search with 2 top detected concepts instead of greedy decoding (ablation) | NBT with ResNet-101 · Novel object captioning out-of-domain test | Average F1: 53.2 -> 70.3 | Improved |
| Use constrained beam search with 1 top detected concept instead of greedy decoding (ablation) | NBT with ResNet-101 · Novel object captioning out-of-domain test | Average F1: 53.2 -> 57.3 | Improved |
| Use 2 rather than 1 top detected concept in constrained beam search (ablation) | NBT with ResNet-101 · Novel object captioning out-of-domain test | Average F1: 57.3 -> 70.3 | Improved |
| Use ResNet-101 instead of VGG-16 with greedy decoding (ablation) | NBT · Novel object captioning out-of-domain test | Average F1: 48.5 -> 53.2 | Improved |
Closest papers by meaning
- Switchable Novel Object Captioner (TPAMI 2022 · similarity 0.90)
- From Show to Tell: A Survey on Deep Learning-Based Image Captioning (TPAMI 2022 · similarity 0.89)
- Partially-Supervised Image Captioning (NeurIPS 2018 · similarity 0.89)
- A Neural Compositional Paradigm for Image Captioning (NeurIPS 2018 · similarity 0.88)
- Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioning (CVPR 2022 · similarity 0.88)
- Deep Compositional Captioning: Describing Novel Object Categories Without Paired Training Data (CVPR 2016 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
