Research map / Image generation / Fast diffusion sampling
Effective Data Augmentation With Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DA-Fusion, which adapts a pretrained Stable Diffusion model with Textual Inversion pseudo-tokens and SDEdit-style real-image splicing with randomized insertion strength to generate semantically diverse augmentations for few-shot image classification.
Problem
Standard augmentations (flips, rotations, color changes) cannot alter high-level semantic attributes, so they lack semantic diversity; few-shot classifiers, including on concepts unseen by the generative model, need richer augmentation.
Approach
Learn one pseudo-token per class via Textual Inversion on a few labelled images, then edit real images with SDEdit-style partial reverse diffusion from a pretrained Stable Diffusion model, randomizing the insertion timestep t0 to vary augmentation intensity, and balance real and synthetic samples during training.
Key results
- DA-Fusion gains of roughly +5 to +15 accuracy points over standard augmentation across seven few-shot domains.
- Improvement of 12.8% for common concepts, 24.2% for fine-grain concepts, and 20.8% for unseen concepts, versus Real Guidance-style baselines as reported in the paper.
- On the full Leafy Spurge dataset, DA-Fusion Pooled gains 1.0% and DA-Fusion Specific gains 1.2% over baseline augmentation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DA-Fusion instead of standard data augmentation | few-shot image classification · Seven tested datasets | accuracy: +5 to +15 accuracy points | Helped |
| Use DA-Fusion instead of Real Guidance | few-shot image classification · Common, fine-grain, and unseen concepts | normalized score: 12.8%; 24.2%; 20.8% | Helped |
| Use DA-Fusion instead of standard data augmentation | few-shot image classification with model-centric leakage prevention · Pascal and COCO | validation accuracy: +5 percentage points | Helped |
| Use DA-Fusion instead of standard data augmentation | few-shot image classification with data-centric leakage prevention · Pascal and COCO | validation accuracy: +10 percentage points | Helped |
| Randomize insertion timestep t0 rather than fixing it at 0.5 (ablation) | DA-Fusion · Few-shot classification, all domains tested in the intensity ablation | average classification accuracy: 51% | Helped |
| Increase synthetic-image sampling probability from alpha = 0.5 to alpha = 0.7 (ablation) | DA-Fusion · Few-shot classification, data-balance ablation | accuracy | Helped |
| Increase synthetic images per real image from M = 10 to M = 20 (ablation) | DA-Fusion · Few-shot classification, data-balance ablation | accuracy | Helped |
| Use DA-Fusion Pooled instead of baseline augmentation | Leafy Spurge classification · Leafy Spurge | cross-validated accuracy: 1.0% | Helped |
| Use DA-Fusion Specific instead of baseline augmentation | Leafy Spurge classification · Leafy Spurge | cross-validated accuracy: 1.2% | Helped |
| Generate image-specific tokens instead of pooled class tokens (ablation) | DA-Fusion on Leafy Spurge · Leafy Spurge | cross-validated accuracy: 1.2% vs 1.0% | Helped |
Surprising findings
- Real Guidance with a descriptive prompt performs no better than traditional augmentation on fine-grain domains such as Flowers102.
- Gains of DA-Fusion are larger when the leakage defense is weaker (data-centric) than when class knowledge is erased from model weights.
Limitations
- Does not explicitly control how objects are modified within an image; attribute-level edits are not localized.
- Temporal consistency is a challenge for extending to decision-making settings.
- Cannot defend against biased content that reinforces stereotypes, per the authors.
Experimental setup
Datasets: Caltech101 Flowers102 FGVC Aircraft Stanford Cars COCO PASCAL VOC Leafy Spurge
Benchmarks: Caltech101 Flowers102 FGVC Aircraft Stanford Cars COCO PASCAL VOC Leafy Spurge
Metrics: validation accuracy normalized score accuracy improvement over baseline
Baselines: RandAugment Real Guidance CutMix standard flips and rotations augmentation
Builds on: Textual Inversion SDEdit Real Guidance RandAugment ESD
Closest papers by meaning
- Semantic-Guided Generative Image Augmentation Method with Diffusion Models for Image Classification (AAAI 2024 · similarity 0.89)
- Data Augmentation with Diffusion for Open-Set Semi-Supervised Learning (NeurIPS 2024 · similarity 0.88)
- Generating Images of Rare Concepts Using Pre-trained Diffusion Models (AAAI 2024 · similarity 0.87)
- Inversion Circle Interpolation: Diffusion-based Image Augmentation for Data-scarce Classification (CVPR 2025 · similarity 0.87)
- DiffuseMix: Label-Preserving Data Augmentation with Diffusion Models (CVPR 2024 · similarity 0.87)
- Enhance Image Classification via Inter-Class Image Mixup with Diffusion Model (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.