Research map / Federated learning and privacy / Personalized and heterogeneous FL
Federated prompt and LoRA tuning
Adapt foundation models federatedly with prompts and low-rank adapters.
91 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2020–2026) · an approach in Personalized and heterogeneous FL, keyword cluster “domain · label · multi”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
17 papers in 2023–24 and 69 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 6.8% to 17.7%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2020 · Unsupervised Model Personalization While Preserving Privacy and Scalability: An Open Problem (CVPR 2020 · 28 citations)
- 2021 · Federated Continual Learning with Weighted Inter-client Transfer (ICML 2021 · 63 citations)
- 2022 · Cross-Modal Federated Human Activity Recognition via Modality-Agnostic and Modality-Specific Representation Learning (AAAI 2022 · 34 citations)
- 2023 · On the Importance and Applicability of Pre-Training for Federated Learning (ICLR 2023 · 10 citations)
- 2024 · Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources (NeurIPS 2024 · 60 citations)
- 2025 · FedPIA – Permuting and Integrating Adapters Leveraging Wasserstein Barycenters for Finetuning Foundation Models in Multi-Modal Federated Learning (AAAI 2025 · 8 citations)
- 2026 · FedALT: Federated Fine-Tuning Through Adaptive Local Training with Rest-of-World LoRA (AAAI 2026 · 5 citations)
Earliest papers here
- Unsupervised Model Personalization While Preserving Privacy and Scalability: An Open Problem (CVPR 2020 · 28 citations)
- Federated Continual Learning with Weighted Inter-client Transfer (ICML 2021 · 63 citations)
- Decentralised Learning from Independent Multi-Domain Labels for Person Re-Identification (AAAI 2021 · 31 citations)
What has been tried
40 papers here report 424 measured changes (a component added, removed or swapped, compared with a baseline): helped 71%, no clear effect 6%, hurt 17%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Integrate FlexLoRA with FedIT at homogeneous rank 8 | Test loss convergence, homogeneous rank 8 · test loss | No clear effect | Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources (NeurIPS 2024 · 60 citations) | |
| Replace unbalanced OT with similarity averaging | Food101, DTD, Caltech101, Flowers102, OxfordPets (2 shots) · accuracy | 89.12±0.28 -> 83.38±0.54; 85.50±0.35 -> 81.01±0.23; 95.05±0.49 -> 92.68±0.44; 93.96±0.48 -> 91.73±0.68; 97.73±0.57 -> 96 | Hurt | Global and Local Prompts Cooperation via Optimal Transport for Federated Learning (CVPR 2024 · 37 citations) |
| Vary the scaling factor among 0.01, 0.05, 0.1, and 0.2 | Dolly and Wizard; TinyLlama and Llama · accuracy | Mixed | FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations (NeurIPS 2024 · 36 citations) | |
| Use adaptive per-sample key selection instead of the meta prompt alone | DomainNet supervised, domain differences · accuracy (average) | 60.39 -> 67.39 | Helped | Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learning (AAAI 2024 · 21 citations) |
| Add traceable task learning (TTL) | CIFAR-100, task scale 2 · accuracy | 69.20% -> 71.40% | Helped | Traceable Federated Continual Learning (CVPR 2024 · 19 citations) |
| Add server and client permutation-and-integration (PIA) | Task 1 VQA · accuracy (overall) | 43.04 -> 46.66 | Helped | FedPIA – Permuting and Integrating Adapters Leveraging Wasserstein Barycenters for Finetuning Foundation Models in Multi-Modal Federated Learning (AAAI 2025 · 8 citations) |
| Use LR scaling with α = 2 instead of the α = 8 baseline | QNLI · accuracy (%) | 91.36\% -> 71.88\% | Hurt | Improving LoRA in Privacy-preserving Federated Learning (ICLR 2024 · 6 citations) |
| Inject artificial noise into learned population statistics | CIFAR-10, large batch size · test loss | 0.41 -> 0.32 | Helped | Population Normalization for Federated Learning (CVPR 2025 · 2 citations) |
| Initialize experts from the common expert instead of from scratch | CIFAR100 after 2000 rounds · zero-shot accuracy | 83.27% | Helped | Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings (NeurIPS 2025 · 2 citations) |
| Vary the global-local mixing coefficient θ across its range rather than use either endpoint alone | CIFAR-100, varying data heterogeneity and client number · accuracy | Helped | Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Method (NeurIPS 2024 · 2 citations) | |
| Use CoPreFL rather than CoPreFL-SGD; apply meta-updates to server data | CIFAR-100, scenario II, non-IID · accuracy | 83.63 ± 3.00 -> 86.63 ± 2.93 | Helped | Rethinking the Starting Point: Collaborative Pre-Training for Federated Downstream Tasks (AAAI 2025 · 1 citation) |
| Use the AFM adapter instead of MLP or MoE in FORLA | Mixed-domain Average · mBO | 41.53; 42.29; 43.81 | Helped | FORLA: Federated Object-centric Representation Learning with Slot Attention (NeurIPS 2025 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 38 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-100 · 10 of 38 papers
- CIFAR-10 · 7 of 38 papers
- Tiny-ImageNet · 6 of 38 papers
- DTD · 6 of 38 papers
- Caltech101 · 6 of 38 papers
- Food101 · 5 of 38 papers
- Flowers102 · 5 of 38 papers
- OxfordPets · 5 of 38 papers
Baselines
- PromptFL · 7 of 38 papers
- FedOTP · 5 of 38 papers
- LoRA · 4 of 38 papers
- FedProx · 4 of 38 papers
- FedIT · 4 of 38 papers
- FFA-LoRA · 4 of 38 papers
- FedAvg · 3 of 38 papers
- Scaffold · 3 of 38 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 86 papers; from OpenAlex and the papers' affiliations).
- Nanyang Technological University · 8 papers
- Wuhan University · 6 papers
- Beijing University of Posts and Telecommunications · 4 papers
- Chinese Academy of Sciences · 3 papers
- ShanghaiTech University · 3 papers
- Zhejiang University · 3 papers
- Carnegie Mellon University · 3 papers
- Shanghai Key Laboratory of Intelligent Information Processing · 2 papers
- Fudan University · 2 papers
- University of Oxford · 2 papers
Use it
Download BibTeX of 26 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Non-IID federated learning (198 papers)
- Federated learning with pretrained models (164 papers)
- Federated semi-supervised and graph learning (84 papers)
- Federated medical and multimodal learning (44 papers)
- Vertical federated learning (38 papers)
Elsewhere in the topic
- Personalization with global-local trade-off (48 papers)
- Sparse and collaboration-graph personalization (36 papers)
- Bayesian personalized FL (35 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).