Research map / Label-efficient and robust learning / Recognition and long-tail

Real-Fake: Effective Training Data Synthesis Through Distribution Matching

ICLR 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Data augmentation and generalization bias · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a distribution-matching framework for training data synthesis that fine-tunes Stable Diffusion 1.5 with LoRA using an MMD-augmented diffusion loss, visual guidance, and VAE latent prior initialization, achieving 70.9% ImageNet-1K top-1 with synthetic-only training.

Problem

Synthetic training data from generative models trails real data when used alone to train classifiers, largely due to distribution misalignment between synthetic and real data, and lacks a principled theory explaining its efficacy.

Approach

Recasts training data synthesis as distribution matching (data distribution plus conditional class likelihood) and implements it in Stable Diffusion via three components: an MMD-based augmentation of the diffusion loss, intra-class CLIP visual guidance in the text condition, and VAE-encoded real latents as the initial prior.

Key results

• Training solely on synthetic data equal to 1x ImageNet-1K size yields 70.9% top-1 on ImageNet-1K, rising to 76.0% with 10x synthetic data.
• On IN-10 and IN-100, the full method reaches 90.5% and 80.0% top-1 with synthetic-only training, outperforming CiP (79.4%, 62.4%) and FakeIt baselines.
• Augmenting real data with synthetic data improves IN-10 by 2.1% and IN-100 by 1.9% top-1.

Measured effects reported in the paper

Surprising findings

• Scaling synthetic data alone (about 2.5x for IN-10 and 6.5x for IN-100) surpasses training on real data, and synthetic OOD generalization exceeds real-data training on some OOD sets before in-distribution parity.
• Membership inference with LiRA yields a TPR of 0.001% for synthetic-data-trained models versus 0.01% for real-data-trained ones at 0.1% FPR.

Limitations

• Gap to real data remains about 3.6% top-1 on ImageNet-1K at 10x synthetic scale and up to 10% at 1x.
• Comparison with non-Stable Diffusion baselines (BigGAN, VQ-VAE-2, CDM, Imagen) is acknowledged as potentially unfair due to differing generation resolutions.
• Application to individual human face recognition is left as future work; face attribute experiments show a larger gap for subtle attributes like Attractive.

Experimental setup

Datasets: ImageNette ImageNet100 ImageNet1K CUB Stanford Cars Oxford-IIIT Pet EuroSAT CelebA DISC

Benchmarks: ImageNette ImageNet100 ImageNet1K CUB Stanford Cars Oxford-IIIT Pet EuroSAT ImageNet-v2 ImageNet-Sketch ImageNet-R ImageNet-A CelebA

Metrics: top-1 accuracy top-5 accuracy TPR at low FPR (LiRA membership inference)

Baselines: BigGAN VQ-VAE-2 CDM Imagen FakeIt CiP

Builds on: Stable Diffusion LoRA dataset distillation distribution matching CiP FakeIt classifier-free guidance

Closest papers by meaning

• Provably Improving Generalization of Few-shot models with Synthetic Data (ICML 2025 · similarity 0.89)
• The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better (NeurIPS 2024 · similarity 0.89)
• Efficient Dataset Distillation for Pre-Trained Self-Supervised Models via Statistical Flow Matching (NeurIPS 2026 · similarity 0.89)
• Improving the Scaling Laws of Synthetic Data with Deliberate Practice (ICML 2025 · similarity 0.88)
• AugGen: Synthetic Augmentation using Diffusion Models Can Improve Recognition (NeurIPS 2025 · similarity 0.88)
• Scaling Laws of Synthetic Images for Model Training ... for Now (CVPR 2024 · similarity 0.88)