Research map / Federated learning and privacy / Personalized and heterogeneous FL
Global and Local Prompts Cooperation via Optimal Transport for Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedOTP, a federated prompt learning method where each client keeps a local prompt and shares a global prompt, aligned to visual patch features via unbalanced optimal transport.
Problem
Federated prompt learning on pretrained vision-language models lacks mechanisms to handle severe data heterogeneity involving both label shift and feature shift.
Approach
Each client learns a shared global prompt (aggregated on the server) and a private local prompt, and aligns both textual prompt features with image patch features using unbalanced optimal transport with one relaxed marginal constraint, so prompts attend only to the most relevant patches.
Key results
- On the pathological non-IID CLIP datasets with 10 clients (ResNet50), FedOTP reaches 98.82% on OxfordPets and 96.23% on Flowers102, versus 91.84% and 86.46% for pFedPrompt.
- On DomainNet with feature and label shifts, FedOTP achieves 48.08% average accuracy versus 10.29% for PromptFL.
- On CIFAR-100 with 100 clients under Dirichlet partition, FedOTP reaches 78.03% versus 73.67% for PromptFL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedOTP instead of pFedPrompt | Federated prompt learning · Pathological non-IID; Food101, DTD, Caltech101, Flowers102, OxfordPets | accuracy: 92.26±1.34 -> 92.73±0.15; 77.14±0.09 -> 87.67±0.70; 96.54±1.31 -> 97.02±0.36; 86.46±0.15 -> 96.23±0.44; 91.84±0.41 -> 98 | Helped |
| Use FedOTP instead of CoOp | Federated prompt learning · DomainNet (average) | accuracy: 44.32±16.74 -> 48.08±15.21 | Helped |
| Use FedOTP instead of CoOp | Federated prompt learning · CIFAR-10 | accuracy: 93.11±0.39 -> 96.05±0.12 | Helped |
| Use FedOTP instead of CoOp | Federated prompt learning · CIFAR-100 | accuracy: 74.83±0.45 -> 78.03±0.08 | Helped |
| Replace unbalanced OT with similarity averaging (ablation) | FedOTP · Food101, DTD, Caltech101, Flowers102, OxfordPets (2 shots) | accuracy: 89.12±0.28 -> 83.38±0.54; 85.50±0.35 -> 81.01±0.23; 95.05±0.49 -> 92.68±0.44; 93.96±0.48 -> 91.73±0.68; 97.73±0.57 -> 96 | Hurt |
| Replace unbalanced OT with classical OT (ablation) | FedOTP · Food101, DTD, Caltech101, Flowers102, OxfordPets (2 shots) | accuracy: 89.12±0.28 -> 88.07±0.63; 85.50±0.35 -> 81.42±0.99; 95.05±0.49 -> 93.17±0.68; 93.96±0.48 -> 92.84±1.34; 97.73±0.57 -> 96 | Hurt |
| Increase the number of training shots (ablation) | FedOTP · Food101, DTD, Caltech101, Flowers102, OxfordPets | accuracy | Helped |
Surprising findings
- Standard federated prompt baselines such as PromptFL fall well below local CoOp training on DomainNet (10.29% vs 44.32% average), indicating that naive prompt aggregation can hurt under heterogeneity.
Limitations
- Generalization to novel unseen clients is not evaluated and is left to future work.
- Reported results rely on few-shot CLIP datasets and small client counts in many settings.
Experimental setup
Datasets: Food101 DTD Caltech101 Flowers102 OxfordPets CIFAR-10 CIFAR-100 DomainNet Office-Caltech10
Benchmarks: Food101 DTD Caltech101 Flowers102 OxfordPets CIFAR-10 CIFAR-100 DomainNet
Metrics: average test accuracy
Baselines: Zero-Shot CLIP CoOp PromptFL pFedPrompt PromptFL+FT PromptFL+FedProx PromptFL+FedPer PromptFL+FedAMP
Builds on: CoOp PromptFL PLOT Sinkhorn algorithm FedPer
Closest papers by meaning
- Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning (CVPR 2024 · similarity 0.91)
- Harmonizing Generalization and Personalization in Federated Prompt Learning (ICML 2024 · similarity 0.90)
- Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Method (NeurIPS 2024 · similarity 0.90)
- FedPHA: Federated Prompt Learning for Heterogeneous Client Adaptation (ICML 2025 · similarity 0.90)
- Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Models (ICLR 2025 · similarity 0.90)
- Global Prompt Refinement with Non-Interfering Attention Masking for One-Shot Federated Learning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.