Research map / Vision-language and multimodal / Human and embodied modeling
Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioning
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add retrieved text descriptions and image conditioning | M²† · MS-COCO Karpathy test | CIDEr: 128.7 -> 135.9 | Helped |
| Add retrieved text descriptions and image conditioning | M²† · MS-COCO Karpathy test | BLEU-4: 38.4 -> 39.7 | Helped |
| Add retrieved text descriptions and image conditioning | M²† with VinVL detector · MS-COCO Karpathy test | CIDEr: 135.9 -> 139.9 | Helped |
| Add Ours to OSCAR | OSCAR · MS-COCO Karpathy test | CIDEr: 137.6 -> 142.2 | Helped |
| Add image conditioning (ablation) | Object-only M² · MS-COCO Karpathy split | CIDEr: 112.39 -> 116.84 | Helped |
| Add retrieved text descriptions (ablation) | Object-only M² · MS-COCO Karpathy split | CIDEr: 112.39 -> 116.99 | Helped |
| Combine retrieved text descriptions with image conditioning (ablation) | Object-only M² · MS-COCO Karpathy split | CIDEr: 112.39 -> 118.87 | Helped |
| Retrieve descriptions directly with CLIP's cross-modal embedding instead of visual similarity (ablation) | M² with retrieved text · MS-COCO Karpathy split | CIDEr: 113.66 -> 116.99 | Helped |
| Use COCO or Conceptual Captions rather than Visual Genome for the description database (ablation) | M² with retrieved descriptions · MS-COCO Karpathy split | CIDEr: 116.99; 117.17; 117.13 | No clear effect |
| Use five-crop retrieval instead of whole-image retrieval (ablation) | M² with retrieved descriptions · MS-COCO Karpathy split | CIDEr: 115.98 -> 116.73 | Helped |
| Fuse image and object features with an FC layer rather than adding image tokens (ablation) | M² image conditioning · MS-COCO Karpathy split | CIDEr: 116.22 -> 116.84 | Helped |
| Use CLIP-I rather than an image-only encoder for image conditioning (ablation) | M² image conditioning · MS-COCO Karpathy split | CIDEr: 114.00 -> 116.84 | Helped |
Closest papers by meaning
- BACON: Improving Clarity of Image Captions via Bag-of-Concept Graphs (CVPR 2025 · similarity 0.90)
- Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Training (AAAI 2024 · similarity 0.89)
- VisMin: Visual Minimal-Change Understanding (NeurIPS 2024 · similarity 0.89)
- DetCLIPv3: Towards Versatile Generative Open-vocabulary Object Detection (CVPR 2024 · similarity 0.89)
- SmartCLIP: Modular Vision-language Alignment with Identification Guarantees (CVPR 2025 · similarity 0.89)
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.