Research map / Federated learning and privacy / Personalized and heterogeneous FL
Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PFPT, a probabilistic federated prompt-tuning method that aligns client prompt sets into global summarizing prompts via a Bernoulli point process and bipartite matching, outperforming FL baselines on heterogeneous and imbalanced vision data.
Problem
Federated fine-tuning of pre-trained vision transformers under non-IID and imbalanced client data suffers from solution drift and unaligned prompt sets, so full-model and naive federated prompt-tuning degrade badly.
Approach
Treats server aggregation as probabilistic set modeling: each client's prompts are samples from a Gaussian centered on global summarizing prompts selected by Bernoulli trials; the prompt-to-summary association is inferred by iterative weighted bipartite matching (Hungarian), which is linear in the assignment variables.
Key results
- On the extremely imbalanced 4-dataset, PFPT reaches 62.23% accuracy versus 39-44% for FedAvg-PT, FedProx-PT, FedOpt-PT and GMM-PT.
- On TinyImageNet (imbalance setting), PFPT achieves 78.21% vs 75.21% for FedAvg-PT and 11.92% for PFedPG.
- On CIFAR-100-LT (IF=10), PFPT achieves 71.66% vs 41.93% for FEDIC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use probabilistic prompt aggregation and alignment (PFPT) instead of the strongest listed prompt-tuning baseline | federated prompt-tuning on CIFAR-10 · CIFAR-10, α=0.5 | accuracy (%): 94.26 ± 0.34 -> 94.39 ± 0.51 | Helped |
| Use probabilistic prompt aggregation and alignment (PFPT) instead of FedAvg-PT | federated prompt-tuning on CIFAR-100 · CIFAR-100, α=0.1 | accuracy (%): 74.50 ± 0.46 -> 75.08 ± 0.51 | Helped |
| Use probabilistic prompt aggregation and alignment (PFPT) instead of the strongest listed baseline | federated prompt-tuning on TinyImageNet · TinyImageNet, α=0.1 | accuracy (%): 79.19 ± 0.28 -> 82.31 ± 0.26 | Helped |
| Use probabilistic prompt aggregation and alignment (PFPT) instead of the strongest listed baseline | federated prompt-tuning on the synthetic 4-dataset · 4-dataset, α=0.1 | accuracy (%): 48.74 ± 0.63 -> 70.29 ± 0.32 | Helped |
| Use PFPT instead of FEDIC | federated long-tailed classification · CIFAR-100-LT, IF=100, α=0.1 | accuracy (%): 33.67 -> 60.74 | Helped |
| Add client-specific prompt selection to FedAvg-PT (ablation) | FedAvg-PT on the synthetic 4-dataset · 4-dataset, α=0.5 | accuracy (%): 59.48 ± 0.59 -> 75.92 | Helped |
| Add client-specific prompt selection to FedProx-PT (ablation) | FedProx-PT on the synthetic 4-dataset · 4-dataset, α=0.5 | accuracy (%): 59.50 ± 0.70 -> 76.85 | Helped |
Surprising findings
- PFedPG performs far worse than other baselines (e.g., 11.57% on CIFAR-100 imbalance) when evaluated on a global test set.
- Simple FedAvg-based prompt tuning is competitive with more specialized FL methods like Scaffold and FedOpt on CIFAR-10.
Limitations
- Prompts are appended at the top layer, so prompt tuning still requires computing all intermediate gradients, raising memory use.
- Performance may degrade when the pre-trained and downstream domains differ significantly, possibly requiring more prompts.
- PFPT tunes about twice as many parameters as the other baselines.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 TinyImageNet MNIST-M Fashion-MNIST CINIC-10 MMAFEDB ImageNet CIFAR-100-LT ImageNet-LT
Benchmarks: CIFAR-10 CIFAR-100 TinyImageNet synthetic 4-dataset CIFAR-100-LT ImageNet-LT
Metrics: test accuracy (%)
Baselines: FedAvg-PT FedProx-PT Scaffold-PT FedOpt-PT PFedPG GMM-PT FEDIC CReFF
Builds on: FedAvg FedProx visual prompt tuning Scaffold FedOpt
Closest papers by meaning
- Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning (CVPR 2024 · similarity 0.90)
- Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learning (AAAI 2024 · similarity 0.89)
- FedPHA: Federated Prompt Learning for Heterogeneous Client Adaptation (ICML 2025 · similarity 0.89)
- Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Method (NeurIPS 2024 · similarity 0.88)
- Harmonizing Generalization and Personalization in Federated Prompt Learning (ICML 2024 · similarity 0.88)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.