Research map / Label-efficient and robust learning / Domain adaptation and generalization
Preserving Clusters in Prompt Learning for Unsupervised Domain Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CRPL, a CLIP-based prompt-learning UDA method that enhances target pseudo-labels with source prompts and enforces text-visual clustering via a Wasserstein (optimal transport) loss.
Problem
Unsupervised domain adaptation with CLIP prompt learning relies on noisy CLIP zero-shot pseudo-labels under covariate shift, and target text prompts fail to represent whole visual clusters.
Approach
Use source-domain prompts (weighted by distance between source class centroids and target visual embeddings) to form enhanced pseudo-labels averaged with the base prompt, and add a Wasserstein distance term that pulls target text embeddings to the centroids of target visual clusters, exploiting the observed visual-text clustering behavior of CLIP.
Key results
- CRPL reaches 77.6% average accuracy on Office-Home in the source-combined setting, versus 74.5% for PGA.
- M-CRPL reaches 78.9% average on Office-Home in the multi-source setting, outperforming MPGA (75.5%) by 3.4 points.
- On DomainNet, CRPL averages 55.8% source-combined, versus 55.4% for PGA and 50.5% for zero-shot CLIP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CLIP zero-shot hard-label guidance with source-enhanced pseudo-labels (ablation) | target prompt training · Office-Home target Ar | accuracy: 47.6 -> 75.8 | Helped |
| replace CLIP zero-shot hard-label guidance with source-enhanced pseudo-labels (ablation) | target prompt training · Office-Home target Cl | accuracy: 29.0 -> 62.9 | Helped |
| replace CLIP zero-shot hard-label guidance with source-enhanced pseudo-labels (ablation) | target prompt training · Office-Home target Pr | accuracy: 53.5 -> 86.6 | Helped |
| replace CLIP zero-shot hard-label guidance with source-enhanced pseudo-labels (ablation) | target prompt training · Office-Home target Rw | accuracy: 63.5 -> 87.2 | Helped |
| add optimal-transport clustering to CLIP zero-shot hard-label training (ablation) | target prompt training · Office-Home targets Ar and Cl versus Pr and Rw | accuracy: 47.6 -> 7.9; 29.0 -> 4.7; 53.5 -> 80.4; 63.5 -> 82.3 | Mixed |
| add optimal-transport clustering to source-enhanced pseudo-label training (ablation) | target prompt training · Office-Home target Ar | accuracy: 75.8 -> 76.8 | Helped |
| add optimal-transport clustering to source-enhanced pseudo-label training (ablation) | target prompt training · Office-Home target Cl | accuracy: 62.9 -> 63.5 | Helped |
| add optimal-transport clustering to source-enhanced pseudo-label training (ablation) | target prompt training · Office-Home target Pr | accuracy: 86.6 -> 87.5 | Helped |
| add optimal-transport clustering to source-enhanced pseudo-label training (ablation) | target prompt training · Office-Home target Rw | accuracy: 87.2 -> 87.6 | Helped |
| use source-target prompt alignment (ablation) | PGA · Office-Home targets Ar, Cl, Pr, and Rw | accuracy: 72.3, 52.8, 83.7, 82.4 (τT); 75.6, 57.0, 86.0, 86.6 (τavg) | Mixed |
| use source-enhanced pseudo-labels and clustering reinforcement (ablation) | M-CRPL target prompt · Office-Home targets Ar, Cl, Pr, and Rw | accuracy: 76.8, 63.5, 87.5, 87.6 (τT); 75.8, 58.6, 85.2, 85.3 (τavg) | Helped |
| use CRPL instead of PGA | source-combined UDA · Office-Home | average accuracy: 74.5 -> 77.6 | Helped |
Surprising findings
- Applying the Wasserstein clustering loss to the CLIP-pseudo-label baseline collapses accuracy on hard domains (e.g., Ar to 7.9%, Cl to 4.7%), showing dependence on initial predictions.
Limitations
- Clustering loss is unstable when initial pseudo-label accuracy is low, as seen with CPL on Ar and Cl.
- Relies on the base prompt and source prompts; the paper notes difficulty deciding when base prompts are useful since base-training data is unavailable.
- Gains are smaller on DomainNet's distinct domains (e.g., Qdr, Rel) where transfer from sources is weaker.
Experimental setup
Datasets: ImageCLEF Office-Home DomainNet
Benchmarks: ImageCLEF Office-Home DomainNet
Metrics: accuracy
Baselines: CLIP zero-shot DAPL Simple Prompt PGA MPA MPGA DAN DANN CDAN SRDC
Builds on: DAPL MPA PGA CLIP
Closest papers by meaning
- Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation (CVPR 2024 · similarity 0.92)
- Prompt-Based Distribution Alignment for Unsupervised Domain Adaptation (AAAI 2024 · similarity 0.91)
- Enhancing Domain Adaptation through Prompt Gradient Alignment (NeurIPS 2024 · similarity 0.90)
- Target Semantics Clustering via Text Representations for Robust Universal Domain Adaptation (AAAI 2025 · similarity 0.90)
- PromptKD: Unsupervised Prompt Distillation for Vision-Language Models (CVPR 2024 · similarity 0.90)
- Domain Generalization in CLIP via Learning with Diverse Text Prompts (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.