Research map / Label-efficient and robust learning / Domain adaptation and generalization
Domain Gap Embeddings for Generative Dataset Augmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DoGE, an inference-only few-shot cross-distribution data augmentation framework that adds CLIP-space domain gap embeddings to source image embeddings and generates target-distribution images with Stable UnCLIP.
Problem
Traditional augmentation and prior generative methods fail to align synthetic data with shifted test distributions, and existing cross-distribution generation methods require fine-tuning or complex setups, especially in few-shot settings.
Approach
Extract the distribution gap as the difference of mean CLIP image embeddings between a source dataset and a few target samples, then add it (scaled by an edit strength) to source embeddings and decode with Stable UnCLIP, optionally with ControlNet structural guidance and confidence-based cleaning, with no fine-tuning.
Key results
- On the constructed CelebA imbalanced task, DoGE reaches 67.16% test accuracy versus 62.40% for RandAugment and 59.72% for DA-Fusion (base 38.00%).
- On DomainNet Real to Painting, DoGE with ControlNet and cleaning reaches 41.30% vs 32.86% base in the ablation; the main table reports 44.00% vs 39.57% for DA-Fusion.
- On GTA5 to Cityscapes, DAFormer + DoGE reaches 57.3% mIoU-style test accuracy versus 56.4% for DAFormer + DATUM and 48.2% for DAFormer alone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| augment the CelebA training set with DoGE synthetic minority-subpopulation data | ResNet50 classifier · CelebA imbalanced classification | Test Accuracy (%): 38.00 -> 67.16 | Helped |
| add DoGE to LoRA-generated data | LoRA generator and ResNet50 classifier · CelebA imbalanced classification | Test Accuracy (%): 56.05 -> 74.28 | Helped |
| augment with DoGE synthetic data | ResNet50 classifier · DomainNet Real→Painting, Infograph, Clipart, and Sketch | Accuracy (%): 34.64 -> 44.00±.0; 14.48 -> 18.71±.0; 39.06 -> 45.61±.3; 24.70 -> 34.96±.3 | Helped |
| augment with DoGE synthetic data | ResNet50 classifier · FMoW Asia, Americas, and Oceania (2002-12→2016-17) | Accuracy (%): 66.27 -> 72.62±.1; 64.65 -> 78.94±.2; 74.42 -> 78.14±.1 | Helped |
| add ControlNet structure guidance (ablation) | DoGE on DomainNet Real→Painting · DomainNet Real→Painting | Accuracy (%): 38.64 -> 40.29 | Helped |
| add confidence-based generation cleaning (ablation) | DoGE + ControlNet on DomainNet Real→Painting · DomainNet Real→Painting | Accuracy (%): 40.29 -> 41.30 | Helped |
| add DoGE synthetic data to UDA methods | BSP, DANN, CDAN, MCD, MCC, and MemSAC · DomainNet Real→Painting | Test Acc (%): 46.76 -> 47.34; 47.01 -> 49.68; 51.66 -> 52.11; 50.88 -> 52.14; 50.08 -> 52.95; 52.27 -> 54.16 | Helped |
| augment DAFormer with DoGE-generated target data | DAFormer · GTA5→Cityscapes segmentation | Test Accuracy (%): 48.2 -> 57.3 | Helped |
| use ControlNet during DoGE generation (ablation) | DoGE generator · DomainNet Painting images | FID score (#): 24.86 -> 18.25 | Helped |
| increase the edit strength C (ablation) | DoGE generation · face eyeglasses augmentation and Real→Sketch | distribution edit extent | Mixed |
Surprising findings
- Simple mean-difference gap embeddings outperformed PCA-based gap extraction in generation quality.
- DATUM's generated images had a much higher FID (219.00) than the source data (30.98), while DoGE achieved 18.25 with ControlNet.
Limitations
- CLIP latent space may be insufficiently expressive for domain gaps it has not seen.
- Stable UnCLIP may struggle out of domain, such as medical X-ray imagery.
- Edit strength must be tuned per task, and over-large values cause over-augmentation.
Experimental setup
Datasets: CelebA DomainNet FMoW-WILDS GTA5 Cityscapes
Benchmarks: CelebA imbalanced classification DomainNet FMoW-WILDS GTA5 to Cityscapes segmentation
Metrics: test accuracy FID t-SNE distribution alignment
Baselines: RandAugment DA-Fusion DATUM Oversampling DAFormer BSP DANN CDAN MCD MCC MemSAC
Builds on: Stable UnCLIP ControlNet DA-Fusion DATUM CLIP
Closest papers by meaning
- AdvST: Revisiting Data Augmentations for Single Domain Generalization (AAAI 2024 · similarity 0.85)
- Data Augmentation with Diffusion for Open-Set Semi-Supervised Learning (NeurIPS 2024 · similarity 0.85)
- Effective Data Augmentation With Diffusion Models (ICLR 2024 · similarity 0.85)
- UtilGen: Utility-Centric Generative Data Augmentation with Dual-Level Task Adaptation (NeurIPS 2025 · similarity 0.85)
- Towards Theoretical Understandings of Self-Consuming Generative Models (ICML 2024 · similarity 0.84)
- SF(DA)$^2$: Source-free Domain Adaptation Through the Lens of Data Augmentation (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.