Research map / Image generation / Text-to-image diffusion
Image Captions are Natural Prompts for Training Data Synthesis
Shiye Lei, Hao Chen, Sen Zhang, Bo Zhao, Dacheng Tao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace basic prompts with real-image captions (CiP) | Stable Diffusion synthetic data · ImageNet-1K val | top-1 accuracy: 45.23 -> 54.06 | Improved |
| generate 10x more synthetic data with CiP | CiP · ImageNet-1K val | top-1 accuracy: 54.06 -> 62.14 | Improved |
| use BLIP-2 captions instead of ViT-GPT2 (ablation) | CiP · ImageNette | top-1 accuracy: 77.0 -> 79.0 | Improved |
| use real-image captions instead of zero-shot captions of synthetic images (ablation) | CiP · ImageNette (gs=1.5) | top-1 accuracy: 67.8 -> 79.0 | Improved |
| zero-shot CiP at large guidance scale (ablation) | Basic prompts · ImageNette (gs=7.5) | top-1 accuracy: 45.8 -> 49.2 | Improved |
| polish captions with Vicuna-13B (ablation) | CiP (BLIP-2) · ImageNette (gs=1.5) | top-1 accuracy: 79.0 -> 66.8 | Hurt |
| use CiP instead of LLM-summarized captions | LLM-Sum · ImageNet-1K val | top-1 accuracy: 53.27 -> 54.06 | Improved |
| LLM-generated captions instead of basic prompts | Basic prompts · ImageNet-1K val | top-1 accuracy: 45.23 -> 43.13 | Hurt |
| use CiP instead of basic prompts with ViT-B/16 | ViT-B/16 from scratch · ImageNet-1K val | top-1 accuracy: 44.00 -> 53.04 | Improved |
| scale CiP synthetic data 10x for OoD robustness | CiP · ImageNet-A (200 classes) | top-1 accuracy: 5.19 -> 10.44 | Improved |
| add CiP synthetic data to real data | ResNet-50 on ImageNette · ImageNette | top-1 accuracy: +3.0% | Improved |
| add CiP synthetic data to real data | ResNet-50 on ImageNet-100 · ImageNet-100 | top-1 accuracy: +1.18% | Improved |
Closest papers by meaning
- Image Captioning with Multi-Context Synthetic Data (AAAI 2024 · similarity 0.88)
- The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better (NeurIPS 2024 · similarity 0.88)
- Real-Fake: Effective Training Data Synthesis Through Distribution Matching (ICLR 2024 · similarity 0.87)
- A User-Friendly Framework for Generating Model-Preferred Prompts in Text-to-Image Synthesis (AAAI 2024 · similarity 0.87)
- CapsFusion: Rethinking Image-Text Data at Scale (CVPR 2024 · similarity 0.87)
- Salient Concept-Aware Generative Data Augmentation (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
