Research map / Image generation / Text-to-image diffusion
Scaling Laws of Synthetic Images for Model Training ... for Now
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Empirically studies scaling laws for training supervised classifiers and CLIP on synthetic images from Stable Diffusion, Imagen, and Muse, finding synthetic data scales worse than real data for ImageNet classification but is competitive for CLIP and OOD robustness.
Problem
Whether synthetic images from text-to-image models follow scaling laws like real data when used at scale to train supervised classifiers and CLIP-style models, and which factors govern that scaling.
Approach
Fits power-law scaling curves (loss vs. dataset size) to synthetic ImageNet and LAION-scale synthetic data, and defines scaling ability (slope) and performance at 1.3M as metrics; shows that CFG scale, prompt design, and T2I model choice strongly change scaling, and diagnoses poor scaling via classes the generators cannot render.
Key results
- Synthetic ImageNet scaling is power-law from 0.125M to about 4M images, but has smaller scaling ability than real ImageNet; default prompts/CFG give flat slopes while tuned prompts and CFG improve scaling.
- On ImageNet-R and ImageNet-Sketch, synthetic data (Muse at large scale) can outperform real ImageNet training images, while in-domain ImageNet-v2 real data scales better.
- In CLIP training on LAION-400M subsets, synthetic+real combination boosts zero-shot performance by up to 5% when fewer than 10M pairs are used.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine synthetic images with real images for CLIP training | CLIP ViT-B/16 · 15 downstream datasets, 1M | zero-shot accuracy (average): 9.9 -> 16.5 | Helped |
| Combine synthetic images with real images for CLIP training | CLIP ViT-B/16 · 15 downstream datasets, 4M | zero-shot accuracy (average): 31.4 -> 36.9 | Helped |
| Combine synthetic images with real images for CLIP training | CLIP ViT-B/16 · 15 downstream datasets, 16M | zero-shot accuracy (average): 49.8 -> 51.9 | Helped |
| Combine synthetic images with real images for CLIP training | CLIP ViT-B/16 · 15 downstream datasets, 128M | zero-shot accuracy (average): 62.3 -> 63.4 | Helped |
| Combine synthetic images with real images for CLIP training | CLIP ViT-B/16 · 15 downstream datasets, 371M | zero-shot accuracy (average): 65.7 -> 66.1 | Helped |
| Train CLIP with synthetic images instead of real images | CLIP ViT-B/16 · 15 downstream datasets, 1M | zero-shot accuracy (average): 9.9 -> 7.9 | Hurt |
| Train CLIP with synthetic images instead of real images | CLIP ViT-B/16 · 15 downstream datasets, 4M | zero-shot accuracy (average): 31.4 -> 21.0 | Hurt |
| Train CLIP with synthetic images instead of real images | CLIP ViT-B/16 · 15 downstream datasets, 16M | zero-shot accuracy (average): 49.8 -> 33.9 | Hurt |
| Train CLIP with synthetic images instead of real images | CLIP ViT-B/16 · 15 downstream datasets, 128M | zero-shot accuracy (average): 62.3 -> 42.9 | Hurt |
| Train CLIP with synthetic images instead of real images | CLIP ViT-B/16 · 15 downstream datasets, 371M | zero-shot accuracy (average): 65.7 -> 48.8 | Hurt |
Surprising findings
- Supervised scaling on synthetic ImageNet saturates beyond about 4M images with ViT-B, requiring larger backbones to continue the power law.
- Some classes (e.g., 'Scaling' classes) scale better with synthetic images than real images, while 'Poor' classes such as vine snake and tiger cat scale poorly due to T2I concept failures.
Limitations
- Imagen and Muse are not publicly available, so results rely on internal versions.
- Synthetic supervised scaling saturates around 4M images with ViT-B backbone, limiting the scaling-law range studied.
- Off-the-shelf text-to-image models fail to generate certain ImageNet concepts, capping supervised performance.
Experimental setup
Datasets: ImageNet LAION-400M ImageNet-A ImageNet-R ImageNet-Sketch ImageNet-v2 Food-101 CIFAR-10 CIFAR-100 SUN397 Stanford Cars FGVC Aircraft
Benchmarks: ImageNet ImageNet-A ImageNet-R ImageNet-Sketch ImageNet-v2 Food-101 CIFAR-10 CIFAR-100 SUN397 Stanford Cars FGVC Aircraft DTD
Metrics: top-1 accuracy cross-entropy loss zero-shot accuracy scaling ability (power-law slope k) performance at 1.3M
Baselines: real ImageNet training data real LAION-400M image-text pairs Stable Diffusion CLIP-template prompts
Builds on: neural scaling laws (Kaplan et al.) classifier-free guidance CLIP Synthetic data from text-to-image models (Fan et al. 2023)
Closest papers by meaning
- The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better (NeurIPS 2024 · similarity 0.90)
- Improving the Scaling Laws of Synthetic Data with Deliberate Practice (ICML 2025 · similarity 0.88)
- Provably Improving Generalization of Few-shot models with Synthetic Data (ICML 2025 · similarity 0.88)
- Scaling Laws for Robust Comparison of Open Foundation Language-Vision Models and Datasets (NeurIPS 2025 · similarity 0.88)
- Real-Fake: Effective Training Data Synthesis Through Distribution Matching (ICLR 2024 · similarity 0.88)
- ImagineFSL: Self-Supervised Pretraining Matters on Imagined Base Set for VLM-based Few-shot Learning (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.