Research map / Vision-language and multimodal / CLIP and zero-shot
SmallCap: Lightweight Image Captioning Prompted With Retrieval Augmentation
Rita Ramos, Bruno Martins, Desmond Elliott, Yova Kementchedjhieva
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add retrieved-caption prompting (ablation) | SmallCap · COCO validation | CIDEr: 113.4 -> 117.3 | Improved |
| Condition on visual features rather than zeroing them (ablation) | SmallCap · COCO validation | CIDEr: 90.1 | Improved |
| Add web and human-labeled data to the datastore (ablation) | SmallCap · nocaps test Out | CIDEr: 65.0 -> 84.4 | Improved |
| Replace COCO captions with target-domain captions in the datastore (ablation) | SmallCap · VizWiz validation | CIDEr: 34.5 -> 47.7 | Improved |
| Add web data to the in-domain datastore (ablation) | SmallCap · Flickr30K validation | CIDEr: 55.4 -> 58.6 | Improved |
| Add human-labeled data to the in-domain datastore (ablation) | SmallCap · MSR-VTT validation | CIDEr: 29.2 -> 30.9 | Improved |
| Combine web and human-labeled data on top of in-domain data (ablation) | SmallCap · Flickr30K, VizWiz, and MSR-VTT validation | CIDEr: 57.9 vs 58.6; 48.0 vs 48.0; 30.7 vs 30.9 | Hurt |
| Add human-labeled data to the in-domain datastore (ablation) | SmallCap · VizWiz validation | CIDEr: 47.7 -> 47.5 | No clear effect |
| Use video-caption data in the datastore (ablation) | SmallCap · MSR-VTT validation | CIDEr: 29.2 -> 31.1 | Improved |
| Reduce the cross-attention dimension from 16 to 8 (ablation) | SmallCap · COCO test | CIDEr: 120.7 -> 119.1 | Hurt |
| Use GPT-2 Large instead of GPT-2 Base (ablation) | SmallCap · COCO test | CIDEr: 119.7 -> 121.8 | Improved |
| Use CLIP ResNet-50x64 rather than ViT-B/32 as the retrieval encoder (ablation) | SmallCap · COCO validation | CIDEr: 109.5 -> 117.9 | Improved |
Later papers that compared with this method
4 later accepted papers report SmallCap as a baseline in their results tables; 3 of them report a better number on most of the results they share with it.
- mVLM: A Vision Language Model for mNPUs (CVPR 2026 · better on 0 of 1 shared result)
- MM-SeR: Multimodal Self-Refinement for Lightweight Image Captioning (CVPR 2026 · better on 1 of 1 shared result)
- ANNEXE: Unified Analyzing, Answering, and Pixel Grounding for Egocentric Interaction (CVPR 2025 · better on 3 of 3 shared results)
- ViPCap: Retrieval Text-Based Visual Prompts for Lightweight Image Captioning (AAAI 2025 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- EVCap: Retrieval-Augmented Image Captioning with External Visual-Name Memory for Open-World Comprehension (CVPR 2024 · similarity 0.90)
- ViPCap: Retrieval Text-Based Visual Prompts for Lightweight Image Captioning (AAAI 2025 · similarity 0.90)
- MM-SeR: Multimodal Self-Refinement for Lightweight Image Captioning (CVPR 2026 · similarity 0.88)
- Modeling Caption Diversity in Contrastive Vision-Language Pretraining (ICML 2024 · similarity 0.88)
- CapsFusion: Rethinking Image-Text Data at Scale (CVPR 2024 · similarity 0.87)
- FlexCap: Describe Anything in Images in Controllable Detail (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
