Research map / Vision-language and multimodal / Human and embodied modeling
Controllable Image Captioning via Prompting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use separately learned prompts for domains in joint training instead of a shared manual prompt (ablation) | ConCap · COCO Karpathy test | CIDEr: 132.2 -> 133.7 | Helped |
| Use separately learned prompts for domains in joint training instead of a shared manual prompt (ablation) | ConCap · TextCaps validation | CIDEr: 110.4 -> 116.7 | Helped |
| Separate domains with distinct manual prompts during joint training instead of using one shared prompt (ablation) | ConCap · COCO Karpathy test | CIDEr: 132.2 -> 132.8 | Helped |
| Separate domains with distinct manual prompts during joint training instead of using one shared prompt (ablation) | ConCap · TextCaps validation | CIDEr: 110.4 -> 113.5 | Helped |
| Jointly train diverse datasets with one shared prompt instead of training each dataset individually (ablation) | ConCap · COCO Karpathy test | CIDEr: 132.6 -> 132.2 | Hurt |
| Jointly train diverse datasets with one shared prompt instead of training each dataset individually (ablation) | ConCap · TextCaps validation | CIDEr: 111.2 -> 110.4 | Hurt |
| Replace a shared manual prompt with learned prompt embeddings (ablation) | Frozen pretrained captioner · COCO Karpathy test | CIDEr: 83.8 -> 125.1 | Helped |
| Replace a shared manual prompt with learned prompt embeddings (ablation) | Frozen pretrained captioner · TextCaps validation | CIDEr: 38.4 -> 56.7 | Helped |
| Use the manual prompt “a picture of” instead of no prompt after fine-tuning (ablation) | ConCap · COCO Karpathy test | CIDEr: 132.4 -> 132.6 | No clear effect |
| Use the manual prompt “a picture of” instead of no prompt after fine-tuning (ablation) | ConCap · TextCaps validation | CIDEr: 113.4 -> 111.2 | Hurt |
| Increase learned prompt length from N=1 to N=16 (ablation) | ConCap · TextCaps validation | CIDEr: 114.5 -> 116.7 | Helped |
| Increase learned prompt length from N=16 to N=24 (ablation) | ConCap · TextCaps validation | CIDEr: 116.7 -> 115.9 | Hurt |
Closest papers by meaning
- Show, Control and Tell: A Framework for Generating Controllable and Grounded Captions (CVPR 2019 · similarity 0.89)
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (ICLR 2026 · similarity 0.86)
- AP-Adapter: Improving Generalization of Automatic Prompts on Unseen Text-to-Image Diffusion Models (NeurIPS 2024 · similarity 0.86)
- Learning to Prompt with Text Only Supervision for Vision-Language Models (AAAI 2025 · similarity 0.86)
- Dynamic Prompt Optimizing for Text-to-Image Generation (CVPR 2024 · similarity 0.86)
- PromptKD: Unsupervised Prompt Distillation for Vision-Language Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.