Research map / Image generation / Autoregressive and latent generation
Do Generated Data Always Help Contrastive Learning?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that inflating contrastive learning data with diffusion-generated images can hurt performance, derives generalization bounds for inflated CL, and proposes AdaInf (real-data reweighting plus weaker augmentation), reaching 94.70% CIFAR-10 linear accuracy with SimCLR.
Problem
Data inflation (adding diffusion-generated images to unlabeled real data) is assumed to help contrastive learning, but the paper finds that vanilla inflation can degrade linear probing accuracy, and lacks theory for this setting.
Approach
Real and generated data should be mixed with real data upweighted (e.g., 10:1), and stronger inflation should be paired with weaker augmentation; theory via augmentation graphs shows inflation improves graph connectivity while augmentation trades connectivity against labeling error, giving complementary roles.
Key results
- AdaInf improves SimCLR linear probing on CIFAR-10 from 91.56% (no inflation) and 91.38% (vanilla inflation) to 93.42%.
- AdaInf reaches 94.70% linear accuracy on CIFAR-10 with SimCLR trained for 500k steps, without external data.
- On the data-scarce 5,000-image CIFAR-10 subset, AdaInf reaches 79.15% vs 74.83% for no inflation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AdaInf instead of no inflation | SimCLR · CIFAR-10 | linear accuracy: 91.56±0.29 -> 93.42±0.20 | Helped |
| Use AdaInf instead of no inflation | SimCLR · CIFAR-100 | linear accuracy: 66.81±0.36 -> 69.6±0.21 | Helped |
| Use AdaInf instead of no inflation | SimCLR · Tiny ImageNet | linear accuracy: 47.21±0.86 -> 48.36±0.46 | Helped |
| Use vanilla inflation instead of no inflation | SimCLR · CIFAR-10 | linear accuracy: 91.56±0.29 -> 91.38±0.11 | Hurt |
| Use AdaInf instead of vanilla inflation | SimCLR · CIFAR-10 | linear accuracy: 91.38±0.11 -> 93.42±0.20 | Helped |
| Use AdaInf instead of vanilla inflation | SimCLR · CIFAR-100 | linear accuracy: 65.52±0.73 -> 69.6±0.21 | Helped |
| Use AdaInf instead of vanilla inflation | SimCLR · Tiny ImageNet | linear accuracy: 41.03±0.39 -> 48.36±0.46 | Helped |
| Use AdaInf instead of no inflation | SimCLR · CIFAR-10, 5,000-image subset | linear accuracy: +4.32% accuracy | Helped |
| Add generated data without reweighting or weak augmentation (ablation) | SimCLR AdaInf · CIFAR-10 | linear accuracy: 91.33 -> 91.35 | No clear effect |
| Add weak augmentation with generated data, without data reweighting (ablation) | SimCLR AdaInf · CIFAR-10 | linear accuracy: 91.35 -> 91.92 | Helped |
| Add data reweighting with generated data, without weak augmentation (ablation) | SimCLR AdaInf · CIFAR-10 | linear accuracy: 91.35 -> 93.21 | Helped |
| Add weak augmentation to generated data and data reweighting (ablation) | SimCLR AdaInf · CIFAR-10 | linear accuracy: 93.21 -> 93.57 | Helped |
Surprising findings
- Vanilla inflation with 1M DDPM/STF generated images can yield worse linear accuracy than training on real data alone (e.g., MoCo V2 drops from 92.75% to 92.51%).
- Weak augmentation, which hurts standard contrastive learning, is the largest contributor to gains when data is inflated.
Limitations
- Theory relies on a simplifying assumption that real and generated distributions are roughly equal (P_d ≈ P_g) and on a toy Gaussian setting for verification.
- Better generative models require more compute and slower sampling, and gains from better FID are marginal.
- Hyperparameters (replication ratio, augmentation strength) are chosen per dataset and generator; the default 10:1 mixing is not universally optimal.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny ImageNet ImageNet100
Benchmarks: CIFAR-10 CIFAR-100 Tiny ImageNet ImageNet100
Metrics: linear probing accuracy FID
Baselines: SimCLR MoCo V2 BYOL Barlow Twins vanilla inflation StyleGAN2-ADA
Builds on: SimCLR InfoNCE solo-learn augmentation graph framework DDPM
Closest papers by meaning
- Synthetic Data Can Also Teach: Synthesizing Effective Data for Unsupervised Visual Representation Learning (AAAI 2023 · similarity 0.88)
- A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics (NeurIPS 2025 · similarity 0.87)
- DiffAug: Enhance Unsupervised Contrastive Learning with Domain-Knowledge-Free Diffusion-based Data Augmentation (ICML 2024 · similarity 0.86)
- Diff-CA: Separating Common and Salient Factors with Diffusion Models (NeurIPS 2026 · similarity 0.85)
- Instruct Where the Model Fails: Generative Data Augmentation via Guided Self-contrastive Fine-tuning (AAAI 2025 · similarity 0.85)
- $\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.