Research map / Federated learning and privacy / Personalized and heterogeneous FL
DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiPrompT, a federated prompt-tuning method for domain generalization with latent (unlabeled) domains, using a global prompt, per-domain prompts, and a learned query prompt that infers domain labels, built on CLIP.
Problem
Federated domain generalization typically requires domain labels and assumes one-to-one mapping between clients and source domains; this paper targets settings where clients outnumber domains and domain labels are unavailable.
Approach
Disentangles generic knowledge (a global prompt shared across all clients) from domain-specific knowledge (one prompt per latent domain, aggregated domain-wise), uses a learnable Q-prompt with two-step class-then-domain text-image matching to infer latent domain labels, and combines prompts at inference with feature-distance-based weights.
Key results
- DiPrompT averages 93.88% on PACS, 76.18% on Officehome, and 83.70% on VLCS, beating FedCLIP (92.37, 73.76, 81.73) and PromptFL in the same CLIP/ResNet50 setup.
- DiPrompT outperforms the centralized methods with domain labels (SWAD, PCL, I2ADR, Fishr, ITL-Net, VAUE) on the reported averages.
- Communication cost per round is reduced by up to 830x versus FedADG and 625x versus FedGMA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DiPrompT instead of PromptFL | federated domain generalization · PACS average | accuracy: 92.13 -> 93.88 | Helped |
| Use DiPrompT instead of PromptFL | federated domain generalization · Officehome average | accuracy: 73.99 -> 76.18 | Helped |
| Use DiPrompT instead of FedCLIP | federated domain generalization · VLCS average | accuracy: 81.73 -> 83.70 | Helped |
| Add G-Prompt (ablation) | DiPrompT · PACS average | accuracy: 88.50 -> 93.88 | Helped |
| Add D-Prompts (ablation) | DiPrompT · PACS average | accuracy: 92.13 -> 93.88 | Helped |
| Add contrastive loss to D-Prompt optimization (ablation) | DiPrompT · PACS average | accuracy: 93.49 -> 93.88 | Helped |
| Use learnable Q-Prompt instead of static prompts (ablation) | DiPrompT · PACS average | accuracy: 92.40 -> 93.88 | Helped |
| Use collaborative ensemble across prompts instead of selecting only the highest-weight domain prompt (ablation) | DiPrompT · PACS average | accuracy: 93.00 -> 93.88 | Helped |
| Add KL loss to Q-Prompt optimization (ablation) | DiPrompT · PACS average | accuracy: 92.54 -> 93.88 | Helped |
| Add MSE loss to Q-Prompt optimization (ablation) | DiPrompT · PACS average | accuracy: 92.23 -> 93.88 | Helped |
| Provide domain labels during local training (ablation) | DiPrompT · PACS average | accuracy: 93.88 -> 94.26 | Helped |
Surprising findings
- Previous federated DG methods (FedADG, FedSR) often underperform plain FedAvg and FedProx when clients outnumber domains.
Limitations
- Reliance on a single fixed number of predefined latent domains (M) must be set in advance.
- Relies on handcrafted domain name text for query initialization and domain prompts.
Experimental setup
Datasets: PACS Officehome VLCS
Benchmarks: PACS Officehome VLCS
Metrics: accuracy
Baselines: SWAD I2ADR PCL Fishr ITL-Net VAUE FedAvg FedProx FedADG FedSR FedGMA PromptFL
Builds on: CoOp CLIP CLIPood FedAvg PromptFL
Closest papers by meaning
- Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learning (AAAI 2024 · similarity 0.89)
- FedPHA: Federated Prompt Learning for Heterogeneous Client Adaptation (ICML 2025 · similarity 0.89)
- Federated Domain Generalization with Data-free On-server Matching Gradient (ICLR 2025 · similarity 0.88)
- Harmonizing Generalization and Personalization in Federated Prompt Learning (ICML 2024 · similarity 0.88)
- Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Data (NeurIPS 2024 · similarity 0.87)
- Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.