Research map / Label-efficient and robust learning / Semi-supervised learning
Data Augmentation with Diffusion for Open-Set Semi-Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Discriminator-Weighted Diffusion (DWD), a class-conditional latent diffusion model trained on labeled and unlabeled data with PU-learned discriminator weights, used to generate labeled or transformed unlabeled data to counter class distribution mismatch in semi-supervised learning.
Problem
Semi-supervised learning degrades when unlabeled data contains many out-of-distribution instances that belong to classes absent from the labeled set; existing methods mostly discard such instances, losing useful diversity.
Approach
Train a class-conditional diffusion model on labeled data (conditionally) and unlabeled data (unconditionally), weighting unlabeled samples with a PU-learning discriminator's importance weights, then seed generation from noised unlabeled images with class conditions so OOD instances are transformed into in-distribution samples.
Key results
- DWD-SL reaches 85.86% on SixAnimal (zeta=75%) versus 85.34% for the best baseline Fix-A-Step, and 82.81% on ImageNet-100 versus 69.30% for MixMatch.
- DWD-UT improves MPL on SixAnimal (zeta=75%) from 65.62% to 83.88%, and improves OpenMatch on CIFAR-10/100 from 70.15% to 80.99%.
- On ImageNet-30, DWD-SL achieves 82.20% versus 78.28% for DPT and 75.26% for DA-Fusion under matched implementation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the original unlabeled dataset with DWD-UT transformed samples | MixMatch · SixAnimal (ζ = 75%) | accuracy: 80.77±0.11 -> 84.72±0.22 | Helped |
| replace the original unlabeled dataset with DWD-UT transformed samples | FixMatch · CIFAR-10/100 | accuracy: 78.91±0.15 -> 83.80±0.25 | Helped |
| replace the original unlabeled dataset with DWD-UT transformed samples | MPL · ImageNet-30 | accuracy: 72.65±0.70 -> 90.20±0.23 | Helped |
| replace the original unlabeled dataset with DWD-UT transformed samples | MixMatch · ImageNet-100 | accuracy: 69.30±0.41 -> 81.62±0.36 | Helped |
| add discriminator filtering of unlabeled data (ablation) | MPL · ImageNet-30 | accuracy: 72.65 -> 83.60 | Helped |
| add diffusion augmentation trained on labeled data only (ablation) | MPL · ImageNet-30 | accuracy: 72.65 -> 87.44 | Helped |
| also train the diffusion model on unlabeled data (ablation) | MPL + labeled-data diffusion augmentation · ImageNet-30 | accuracy: 87.44 -> 88.09 | Helped |
| also apply discriminator weighting during diffusion training (ablation) | MPL + diffusion trained on labeled and unlabeled data · ImageNet-30 | accuracy: 88.09 -> 90.20 | Helped |
| use a moderate seed-image noise level rather than no noise or t = 1000 (ablation) | DWD-UT with MPL · SixAnimal (ζ = 75%) | accuracy: 65.62±0.47 -> 83.88±0.18 | Helped |
| increase generated sample count from 6K to 120K, then to 240K (ablation) | DWD-SL · CIFAR-10/100 | accuracy: 77.64 -> 81.24 -> 81.07 | Mixed |
| vary the unlabeled-data weight α (ablation) | DWD-SL · SixAnimal (ζ = 75%) | accuracy: 84.01; 85.86; 83.83; 83.51 | Mixed |
| vary seed-image noise from t = 600 to t = 800 (ablation) | DWD-UT with MPL · SixAnimal (ζ = 75%) | accuracy: 83.88±0.18 -> 83.83±0.11 | No clear effect |
Surprising findings
- Discriminator-based filtering alone gives only marginal gains over MPL (e.g., 65.62% to 67.19% on SixAnimal), while diffusion-based transformation of unlabeled data gives much larger gains.
Limitations
- Substantial extra computation for diffusion pretraining, finetuning and sampling compared with standard SSL baselines.
- Effectiveness drops when labeled data is extremely scarce, since the diffusion model struggles to model the labeled distribution.
- Fixed noise level t=600 for all samples; the optimal noise level may differ per instance.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-30 ImageNet-100
Benchmarks: SixAnimal (CIFAR-10) CIFAR-10/100 ImageNet-30 ImageNet-100
Metrics: accuracy
Baselines: MixMatch FixMatch Meta Pseudo Labels OpenMatch Fix-A-Step IOMatch DPT DA-Fusion
Builds on: DDPM latent diffusion model PU learning DPM-Solver SDEdit
Closest papers by meaning
- Scaling Up Semi-supervised Learning with Unconstrained Unlabelled Data (AAAI 2024 · similarity 0.88)
- Effective Data Augmentation With Diffusion Models (ICLR 2024 · similarity 0.88)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.87)
- Instruct Where the Model Fails: Generative Data Augmentation via Guided Self-contrastive Fine-tuning (AAAI 2025 · similarity 0.86)
- AugGen: Synthetic Augmentation using Diffusion Models Can Improve Recognition (NeurIPS 2025 · similarity 0.86)
- DiffuseMix: Label-Preserving Data Augmentation with Diffusion Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.