Research map / Image generation / Text-to-image diffusion
Adversarial Domain Prompt Tuning and Generation for Single Domain Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PAPT, which learns abstract category and domain prompts for a pre-trained latent diffusion model via adversarial prompt tuning and progressive memory-bank training to generate diverse out-of-domain images for single domain generalization.
Problem
Single domain generalization (SDG) must generalize from one labeled source domain to many unseen target domains; manually written textual domain prompts cannot describe abstract domain styles needed for diverse OOD augmentation.
Approach
Learn category prompts per class and then adversarially learn domain prompts that are stylistically distinct from a growing memory bank of previously learned domain prompts, while keeping class semantics consistent, to drive a frozen-style T2I diffusion model for OOD image generation.
Key results
- On PACS single-domain generalization, PAPT reaches 72.72% average accuracy, versus 68.10% for the RSC+ASR baseline (reported as best prior in the text).
- On VLCS single-domain generalization, PAPT reaches 73.37% average accuracy, versus 63.82% for MAD.
- In multi-source DG with ResNet-50, PAPT reaches 69.64% average, 1.34 points above SMOS (68.30%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PAPT instead of Prompt-Driven | SDG on PACS · PACS (SDG, Avg.) | accuracy: 71.12 -> 72.72 | Helped |
| Use PAPT instead of MAD | SDG on VLCS · VLCS (SDG, Avg.) | accuracy: 63.82 -> 73.37 | Helped |
| Use PAPT instead of UDIM | SDG on OfficeHome · OfficeHome (SDG, Avg.) | accuracy: 58.30 -> 62.03 | Helped |
| Use PAPT instead of SMOS | multi-source DG on PACS · PACS (multi-source DG) | accuracy: 89.40\pm 0.30 -> 91.69\pm 0.44 | Helped |
| Use PAPT instead of SMOS | multi-source DG on VLCS · VLCS (multi-source DG) | accuracy: 79.80\pm 0.10 -> 80.84\pm 0.32 | Helped |
| Use PAPT instead of SMOS | multi-source DG on OfficeHome · OfficeHome (multi-source DG) | accuracy: 71.60\pm 0.10 -> 71.21\pm 0.53 | Hurt |
| Use PAPT instead of SMOS | multi-source DG on TerraIncognita · TerraIncognita (multi-source DG) | accuracy: 55.40\pm 0.40 -> 54.27\pm 1.01 | Hurt |
| Use PAPT instead of SMOS | multi-source DG on DomainNet · DomainNet (multi-source DG) | accuracy: 45.30\pm 0.00 -> 50.18\pm 0.47 | Helped |
| Add T2I-generated images with a simple class prompt (ablation) | PAPT on PACS · PACS (SDG, Avg.) | accuracy: 46.60 -> 58.78 | Helped |
| Add category prompt tuning (ablation) | PAPT on PACS · PACS (SDG, Avg.) | accuracy: 58.78 -> 66.34 | Helped |
| Add adversarial domain prompt tuning (ablation) | PAPT on PACS · PACS (SDG, Avg.) | accuracy: 66.34 -> 70.22 | Helped |
| Add the progressive training scheme (ablation) | PAPT on PACS · PACS (SDG, Avg.) | accuracy: 70.22 -> 72.72 | Helped |
Surprising findings
- Manually designed textual domain prompts (e.g., watercolor, oil painting) give limited gains compared with learned abstract domain prompts, per the ablation.
- Data augmentation baselines like Mixstyle perform poorly on TerraIncognita and VLCS in the multi-source setting, which the authors attribute to viewpoint and background shifts.
Limitations
- Gains on OfficeHome and some PACS target domains are not uniform; PAPT is below some baselines on individual splits (e.g., Photo on PACS).
- Domain prompt count N and lambda are hand-tuned hyperparameters; results sensitive to N in the multi-source setting.
Experimental setup
Datasets: PACS VLCS OfficeHome TerraIncognita DomainNet
Benchmarks: PACS VLCS OfficeHome TerraIncognita DomainNet
Metrics: accuracy (%) average accuracy
Baselines: ERM Augmix RSC L2D ASR-Norm (RSC+ASR) Mixstyle DSU EFDMix MAD P-RC Meta-Casual ITTA
Builds on: latent diffusion models prompt tuning (textual inversion-style) adversarial domain augmentation ASR-Norm
Closest papers by meaning
- Disentangled Prompt Representation for Domain Generalization (CVPR 2024 · similarity 0.88)
- AdvST: Revisiting Data Augmentations for Single Domain Generalization (AAAI 2024 · similarity 0.87)
- Domain-Controlled Prompt Learning (AAAI 2024 · similarity 0.86)
- Domain Generalization in CLIP via Learning with Diverse Text Prompts (CVPR 2025 · similarity 0.86)
- On Discrete Prompt Optimization for Diffusion Models (ICML 2024 · similarity 0.86)
- AP-Adapter: Improving Generalization of Automatic Prompts on Unseen Text-to-Image Diffusion Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.