Research map / Label-efficient and robust learning / Domain adaptation and generalization
Enhancing Domain Adaptation through Prompt Gradient Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Casts unsupervised domain adaptation with CLIP prompt learning as multi-objective optimization, aligning per-domain prompt gradients and penalizing their norms (PGA/MPGA).
Problem
Unsupervised domain adaptation with pre-trained vision-language models: existing prompt-based UDA methods enforce invariance constraints on representation, output, or prompt space, and may not exploit the discriminative power of domain-specific features and gradient conflicts between domain objectiv
Approach
Treat UDA as multi-objective optimization over per-domain losses (source domains plus pseudo-labeled target), and add a gradient-cosine alignment term between source and target gradients of the shared prompt plus gradient-norm penalties, implemented without Hessian computation via a first-order approximation.
Key results
- PGA reaches 92.4% average on ImageCLEF and 76.8% on Office-Home (source combined), vs. MPA 91.3% on ImageCLEF multi-source.
- MPGA reaches 79.4% average on Office-Home, outperforming MPA (75.4%) by about 4 points.
- On DomainNet, PGA averages 55.4% (source combined) and MPGA 56.2% (multi-source), vs. MPA 54.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train prompts using target pseudo-label loss only instead of zero-shot CLIP (ablation) | PGA · ImageCLEF | accuracy: 88.1 -> 90.1 | Helped |
| Add source loss to target-loss-only training (ablation) | PGA · ImageCLEF | accuracy: 90.1 -> 90.6 | Helped |
| Add gradient-norm penalization to target and source losses (ablation) | PGA · ImageCLEF | accuracy: 90.6 -> 90.9 | Helped |
| Add gradient alignment to target loss, source loss, and gradient-norm penalization (ablation) | PGA · ImageCLEF | accuracy: 90.9 -> 92.4 | Helped |
Surprising findings
- Self-training on pseudo-labeled target data alone (no source data) raises CLIP zero-shot average on ImageCLEF from 88.1% to 90.1%, a strong baseline close to MPA.
Limitations
- Generalization bound can grow with training iterations unless gradient norms are very small at the end.
- Does not explicitly reduce the source-target distribution gap, which the bound's second term suggests could matter.
- Performs poorly relative to non-prompt methods on the QuickDraw domain of DomainNet.
Experimental setup
Datasets: ImageCLEF Office-Home DomainNet S2RDA-49 S2RDA-MS-39
Benchmarks: ImageCLEF Office-Home DomainNet S2RDA-49 S2RDA-MS-39
Metrics: top-1 accuracy average accuracy
Baselines: Zero-shot CLIP Simple Prompt DAPL MPA DAN DANN D-CORAL MFSAN DAMP DomainNet MCD
Builds on: CoOp DAPL MPA CLIP
Closest papers by meaning
- Prompt-Based Distribution Alignment for Unsupervised Domain Adaptation (AAAI 2024 · similarity 0.95)
- Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation (CVPR 2024 · similarity 0.92)
- Preserving Clusters in Prompt Learning for Unsupervised Domain Adaptation (CVPR 2025 · similarity 0.90)
- Domain-Controlled Prompt Learning (AAAI 2024 · similarity 0.90)
- Exploring Sparse Visual Prompt for Domain Adaptive Dense Prediction (AAAI 2024 · similarity 0.89)
- PromptKD: Unsupervised Prompt Distillation for Vision-Language Models (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.