Research map / Label-efficient and robust learning / Domain adaptation and generalization
Disentangled Prompt Representation for Domain Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DPR, a CLIP prompt-tuning framework for domain generalization that uses GPT-3-generated domain-invariant and domain-specific text descriptions to guide visual prompt learning, plus domain-specific prototype learning at inference.
Problem
Domain generalization with vision foundation models: prompt tuning tends to learn domain- and task-specific features rather than domain-invariant ones that transfer to unseen domains.
Approach
Use an LLM (GPT-3) to produce fine-grained class/domain descriptions and then summarize them into domain-invariant and domain-specific text descriptions; learn disentangled text prompts with CLIP's text encoder, use them to guide domain-invariant and domain-specific visual prompts (VPT), and combine the invariant prediction with a cache-style domain-specific prototype predictor.
Key results
- DPR reaches 97.45% (PACS), 86.43% (VLCS), 86.13% (OfficeHome), 62.05% (DomainNet), 57.10% (TerraInc), average 77.83%, the highest among compared methods.
- Compared with the CLIP baseline (average 70.16%), DPR improves average accuracy by about 7.7 points.
- Text disentanglement alone improves the CoOp-style baseline by +1.4, +1.3, +1.7 points on VLCS, OfficeHome, DomainNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add GPT-Assist descriptions to CoOp prompts (ablation) | Text disentanglement baseline · VLCS | accuracy: 77.6 -> 79.0 | Helped |
| add GPT-Assist descriptions to CoOp prompts (ablation) | Text disentanglement baseline · OfficeHome | accuracy: 83.9 -> 85.2 | Helped |
| add GPT-Assist descriptions to CoOp prompts (ablation) | Text disentanglement baseline · DomainNet | accuracy: 59.8 -> 61.5 | Helped |
| add domain confusion loss (ablation) | Image disentanglement with VPT · VLCS | accuracy: 84.9 -> 85.1 | Helped |
| add domain confusion loss (ablation) | Image disentanglement with VPT · OfficeHome | accuracy: 85.2 -> 85.4 | Helped |
| add domain confusion loss (ablation) | Image disentanglement with VPT · DomainNet | accuracy: 59.8 -> 60.1 | Helped |
| add domain-specific prototype learning (ablation) | Image disentanglement with VPT · VLCS | accuracy: 84.9 -> 85.5 | Helped |
| add domain-specific prototype learning (ablation) | Image disentanglement with VPT · OfficeHome | accuracy: 85.2 -> 85.5 | Helped |
| add domain-specific prototype learning (ablation) | Image disentanglement with VPT · DomainNet | accuracy: 59.8 -> 60.5 | Helped |
| add domain-specific prototype learning (ablation) | Text and image disentanglement · VLCS | accuracy: 85.3 -> 86.0 | Helped |
| add domain-specific prototype learning (ablation) | Text and image disentanglement · OfficeHome | accuracy: 85.5 -> 85.9 | Helped |
| add domain-specific prototype learning (ablation) | Text and image disentanglement · DomainNet | accuracy: 61.5 -> 62.0 | Helped |
Surprising findings
- Domain-invariant GPT-generated descriptions outperform hand-crafted and fine-grained descriptions as text guidance.
- Training the prototype memory bank (DSPL training) gives only small gains (about +0.1 to +0.6 points) over the training-free variant.
Limitations
- Relies on GPT-3 generated descriptions whose quality and cost are not analyzed in depth.
- Evaluated only on image classification benchmarks with a single ViT-B/16 backbone.
Experimental setup
Datasets: PACS VLCS OfficeHome DomainNet TerraIncognita
Benchmarks: PACS VLCS OfficeHome DomainNet TerraInc
Metrics: accuracy
Baselines: CLIP CoOp MIRO SWAD DUPRG GESTUR DPL SEDGE ERM DADA
Builds on: CoOp VPT CLIP Tip-Adapter
Closest papers by meaning
- Domain Generalization in CLIP via Learning with Diverse Text Prompts (CVPR 2025 · similarity 0.90)
- OSLoPrompt: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIP (CVPR 2025 · similarity 0.88)
- Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation (CVPR 2024 · similarity 0.88)
- Compound Text-Guided Prompt Tuning via Image-Adaptive Cues (AAAI 2024 · similarity 0.88)
- Unknown Prompt the only Lacuna: Unveiling CLIP's Potential for Open Domain Generalization (CVPR 2024 · similarity 0.88)
- Concept-Guided Prompt Learning for Generalization in Vision-Language Models (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.