Research map / Federated learning and privacy / Personalized and heterogeneous FL
Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedAPT, a federated adaptive prompt tuning method that tunes a shared meta prompt and an adaptive key-selection network on frozen CLIP so each test image gets a domain-personalized prompt.
Problem
Federated learning across multiple visual domains with non-IID clients requires a single global model that performs well in all domains; full-parameter federated training is costly and a single global prompt does not adapt to domain shift.
Approach
Keep a shared meta prompt plus randomly generated, frozen per-domain keys; a federated adaptive network predicts domain-membership weights from image features, which modulate the meta prompt per test sample, so no extra per-domain prompt parameters are needed.
Key results
- On DomainNet supervised with domain differences, FedAPT reaches 67.39% average accuracy versus 63.39% for PromptFL and 55.23% for fully-trained ResNet-full, with about 2.83M communicated parameters versus 24.21M for ResNet-full.
- On Office-Caltech10 supervised, FedAPT averages 98.07% versus 97.94% for ResNet-full, with 2.829M parameters.
- In the unsupervised setting on DomainNet with domain differences, FedAPT averages 57.49% versus 56.78% for PromptFL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive per-sample key selection instead of the meta prompt alone (ablation) | FedAPT · DomainNet supervised, domain differences | accuracy (average): 60.39 -> 67.39 | Helped |
| Use adaptive key selection instead of a fixed domain key (ablation) | FedAPT · DomainNet supervised, domain differences | accuracy (average): 59.08 -> 67.39; 58.70 -> 67.39; 58.59 -> 67.39; 58.87 -> 67.39; 59.90 -> 67.39; 59.08 -> 67.39 | Helped |
| Add keys during local training (ablation) | FedAPT · Local client domains | accuracy: 78.30 vs 78.03; 51.48 vs 51.59; 73.99 vs 74.00; 49.05 vs 49.35; 85.35 vs 85.67; 69.72 vs 69.73 | No clear effect |
| Use a single selected key (τ=0.01) rather than soft key membership (τ=1) (ablation) | FedAPT adaptive network · DomainNet seen domains and unseen domain c | accuracy: 52.23 vs 52.27; 72.76 vs 72.14; 51.25 vs 51.05; 84.67 vs 84.18; 69.35 vs 68.77; 68.26 vs 70.50 | Mixed |
| Increase prompt length from 8 to 24 (ablation) | FedAPT · DomainNet supervised, domain differences | accuracy (average): 67.10 -> 67.70 | Helped |
| Use FedAPT instead of PromptFL | CLIP ViT-B/32 · DomainNet supervised, domain differences | accuracy (average): 63.39 -> 67.39 | Helped |
| Use FedAPT instead of ResNet-full | DomainNet supervised, domain differences · DomainNet supervised, domain differences | accuracy (average): 55.23 -> 67.39 | Helped |
| Use FedAPT instead of ResNet-full | Office-Caltech10 · Office-Caltech10 supervised, domain differences | accuracy (average): 97.94 -> 98.07 | Helped |
| Use FedAPT instead of PromptFL | CLIP ViT-B/32 · DomainNet unsupervised, domain differences | accuracy (average): 56.78 -> 57.49 | Helped |
| Precompute text encodings for keys (ablation) | FedAPT inference · DomainNet inference | inference time: original (min) 19; 39; 31; 30; 60; 30 -> Improved (sec) 19.5; 26; 30; 65; 67; 32 | Mixed |
Surprising findings
- Fully-trained ResNet50 federated training is outperformed on DomainNet by a CLIP prompt-tuning method with far fewer communicated parameters.
- Directly selecting a single key (sharp softmax temperature) improves client-domain accuracy but hurts unseen-domain generalization; a softer temperature keeps unseen-domain accuracy nearly unchanged.
Limitations
- Gains over PromptFL and CLIP-FC are modest in several settings, especially with extreme category imbalance where all CLIP tuning methods perform similarly.
- Evaluated only on two image classification datasets with a single CLIP ViT-B/32 backbone.
- Per-sample prompts require running the text encoder per key, which increases inference cost without the precomputation trick.
Experimental setup
Datasets: Office-Caltech10 DomainNet
Benchmarks: Office-Caltech10 DomainNet
Metrics: classification accuracy
Baselines: CLIP-zeroshot ResNet-full ViT-full ResNet-tuning ViT-tuning CLIP-FC FedCLIP PromptFL
Builds on: CoOp CoCoOp PromptFL FedAvg SimCLR
Closest papers by meaning
- Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning (CVPR 2024 · similarity 0.90)
- Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Models (ICLR 2025 · similarity 0.89)
- Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Data (NeurIPS 2024 · similarity 0.89)
- DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning (CVPR 2024 · similarity 0.89)
- FedPHA: Federated Prompt Learning for Heterogeneous Client Adaptation (ICML 2025 · similarity 0.89)
- Federated Disentangled Tuning with Textual Prior Decoupling and Visual Dynamic Adaptation (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.