Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedPHA: Federated Prompt Learning for Heterogeneous Client Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedPHA, a federated prompt learning method for CLIP where each client has a fixed-length shared global prompt plus a variable-length local prompt, with SVD-based null-space projection and bidirectional pull/push alignment to reduce global-local conflicts.
Problem
Federated prompt learning for vision-language models typically requires uniform prompt lengths for aggregation, and shared global prompts can conflict with client-specific knowledge under data heterogeneity.
Approach
A G-L architecture with a fixed-length global prompt (aggregated) and variable-length local prompts (kept local), coupled via shared frozen prefix/suffix tokens and padding; local prompts are projected onto the null space of the global prompt's SVD, with a pull loss keeping them close to their projection and a margin push loss separating them from the global prompt.
Key results
- On single-domain CLIP datasets with 10 clients (16-shot), FedPHA reaches 99.05% on Caltech101, 96.42% on Food101 and 91.79% on DTD, outperforming FedOTP (98.11, 92.96, 89.97).
- On Office31 FedPHA averages 94.74% vs. 91.58% for FedPGP; on CIFAR-100 (100 clients) it reaches 75.92% vs. 75.07% for FedOTP.
- SVD projection and prompt decomposition add under 1% of per-round client time (4.2 ms and 2.8 ms vs. 4536.8 ms training).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedPHA instead of FedOTP | federated prompt learning · Food101 | accuracy: 92.96±0.16 -> 96.42±0.05 | Helped |
| Use FedPHA instead of FedOTP | federated prompt learning · Caltech101, Flowers102, OxfordPets, and DTD | accuracy: 98.11±0.04 -> 99.05±0.08; 98.47±0.07 -> 99.25±0.05; 98.60±0.11 -> 99.25±0.06; 89.97±0.17 -> 91.79±0.17 | Helped |
| Use FedPHA instead of FedPGP | federated prompt learning · Office31 A, D, W, and Avg | accuracy: 89.55 -> 90.44; 90.70 -> 95.80; 94.50 -> 97.98; 91.58 -> 94.74 | Helped |
| Use FedPHA instead of FedPGP | federated prompt learning · OfficeHome A, C, P, and Avg | accuracy: 88.34 -> 88.70; 78.09 -> 79.59; 95.49 -> 95.93; 88.95 -> 89.51 | Helped |
| Use FedPHA instead of FedPGP | federated prompt learning · OfficeHome R | accuracy: 93.86 -> 93.83 | Hurt |
| Use FedPHA with random client prompt lengths (4–32) instead of fixed length 16 (ablation) | FedPHA · CIFAR-10 | accuracy: 94.11±0.14 -> 93.80±0.17 | Hurt |
| Use FedPHA with random client prompt lengths (4–32) instead of fixed length 16 (ablation) | FedPHA · CIFAR-100 | accuracy: 75.92±0.13 -> 75.63±0.17 | Hurt |
| Allow different clients to use heterogeneous prompt lengths instead of requiring a common length (ablation) | FedPHA · Office31 | accuracy: 95.45% | Helped |
| Increase prompt length from 4 to 32 (ablation) | FedPHA, Client 1 and Client 2 · Office31 | accuracy | Helped |
| Use prompt length 16 rather than 32 (ablation) | FedPHA, Client 3 · Office31 | accuracy | Helped |
Surprising findings
- A fixed-length (16-token) FedPHA variant slightly outperforms the random-length (4-32 token) variant on CIFAR-10/100 (94.11 vs 93.80; 75.92 vs 75.63), which the authors attribute to random lengths not matching client distributions.
Limitations
- Gains over strongest baselines are modest on some datasets (e.g., Office31 and CIFAR-100 margins of a few tenths of a percent to ~3 points).
- Random prompt-length heterogeneity does not outperform fixed length, so the benefit of length heterogeneity is not clearly demonstrated in the robustness experiments.
- Hardware and compute details are not stated in the text provided.
Experimental setup
Datasets: Food101 DTD Caltech101 Flowers102 OxfordPets Office31 OfficeHome CIFAR-10 CIFAR-100
Benchmarks: Caltech101 Food101 Flowers102 OxfordPets DTD Office31 OfficeHome CIFAR-10 CIFAR-100
Metrics: test accuracy
Baselines: Zero-shot CLIP PromptFL PromptFL+Prox FedOTP FedPGP
Builds on: CoOp PromptFL FedOTP FedPGP FedProx
Closest papers by meaning
- Harmonizing Generalization and Personalization in Federated Prompt Learning (ICML 2024 · similarity 0.93)
- Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning (CVPR 2024 · similarity 0.91)
- Global and Local Prompts Cooperation via Optimal Transport for Federated Learning (CVPR 2024 · similarity 0.90)
- Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation (NeurIPS 2025 · similarity 0.90)
- Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Models (ICLR 2025 · similarity 0.90)
- Global Prompt Refinement with Non-Interfering Attention Masking for One-Shot Federated Learning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.