Research map / Federated learning and privacy / Personalized and heterogeneous FL
Federated learning with pretrained models
Handle client imbalance and long-tailed data with pretrained and CLIP models and consistency.
164 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2020–2026) · an approach in Personalized and heterogeneous FL, keyword cluster “domain · label · multi”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
67 papers in 2023–24 and 75 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 26.9% to 19.3%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2020 · Ensemble Distillation for Robust Model Fusion in Federated Learning (NeurIPS 2020 · 504 citations)
- 2021 · FedBN: Federated Learning on Non-IID Features via Local Batch Normalization (ICLR 2021 · 342 citations)
- 2022 · FedProto: Federated Prototype Learning across Heterogeneous Clients (AAAI 2022 · 751 citations)
- 2023 · FairFed: Enabling Group Fairness in Federated Learning (AAAI 2023 · 177 citations)
- 2024 · FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning (AAAI 2024 · 82 citations)
- 2025 · FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors (CVPR 2025 · 28 citations)
- 2026 · Plug-and-Play Parameter-Efficient Tuning of Embeddings for Federated Recommendation (AAAI 2026 · 1 citation)
Earliest papers here
- Ensemble Distillation for Robust Model Fusion in Federated Learning (NeurIPS 2020 · 504 citations)
- Robust Federated Learning: The Case of Affine Distribution Shifts (NeurIPS 2020 · 78 citations)
- Robust Federated Learning via Collaborative Machine Teaching (AAAI 2020 · 33 citations)
What has been tried
109 papers here report 1,091 measured changes (a component added, removed or swapped, compared with a baseline): helped 71%, no clear effect 6%, hurt 19%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add a frozen shared adapter teacher to DAT | Domain, Function, and Scene · accuracy | 58.24{ ±0.98} -> 61.11{ ±2.98}; 50.62{ ±1.45} -> 54.39{ ±2.36}; 33.04{ ±0.65} -> 39.35{ ±1.25} | Helped | FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning (AAAI 2024 · 82 citations) |
| Add FPHL, omitting FAEL | Digits · average accuracy | 76.00 -> 77.05 | Helped | Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversity (CVPR 2024 · 40 citations) |
| Add CLIP knowledge distillation loss KL | CIFAR-10-LT IF=100 · top-1 accuracy (%) | 70.55 -> 72.20 | Helped | CLIP-Guided Federated Learning on Heterogeneity and Long-Tailed Data (AAAI 2024 · 38 citations) |
| Increase mutation range alpha from 1 to 4 | CIFAR-10 ResNet-18 learning curves · Test accuracy (%) | Helped | FedMut: Generalized Federated Learning via Stochastic Mutation (AAAI 2024 · 33 citations) | |
| Add EN-client identification and aggregate only identified mildly noisy clients | CIFAR-10, 20 clients, five 99% noisy clients · accuracy | 74.26 -> 79.95 | Helped | Federated Learning with Extremely Noisy Clients via Negative Distillation (AAAI 2024 · 33 citations) |
| use layer-wise rather than shared aggregation weights | CIFAR-10, CIFAR-100, Tiny-ImageNet · Top-1 test accuracy | 65.13 vs 63.55; 42.52 vs 41.89; 33.42 vs 33.07 | Helped | FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors (CVPR 2025 · 28 citations) |
| Replace direct proximal-gradient projection with FedSOL proximal perturbation | CIFAR-10, LDA α=0.1 · test accuracy | 56.35±2.85 -> 60.31±2.07 | Helped | FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learning (CVPR 2024 · 19 citations) |
| add instance-level personalization to concept-level personalization | DomainNet/PACS/OfficeHome Avg · accuracy | 63.96 -> 67.82; 60.08 -> 62.67; 56.32 -> 61.52 | Helped | FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models (CVPR 2025 · 14 citations) |
| Add CA-MLE to FedC-Tran without ULE | FLAIR coarse-grained · C-AP | 56.00 -> 56.10 | No clear effect | Language-Guided Transformer for Federated Multi-Label Classification (AAAI 2024 · 13 citations) |
| Use loss-based projection-target ordering instead of random ordering | Adult, COMPAS, and Bank · client fairness violation | Adult: .049 -> .036; Compas: .044 -> .022; Bank: .047 -> .028 | Helped | Multi-Dimensional Fair Federated Learning (AAAI 2024 · 13 citations) |
| Add consensus-maximization aggregation | DomainNet, Office-Caltech10, and PACS, ALL · accuracy (%) | 75.66 -> 76.77; 84.44 -> 87.15; 82.92 -> 83.81 | Helped | Federated Learning with Domain Shift Eraser (CVPR 2025 · 12 citations) |
| Add G-Prompt | PACS average · accuracy | 88.50 -> 93.88 | Helped | DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning (CVPR 2024 · 12 citations) |
What reviewers will expect
Datasets and baselines named in the 39 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 19 of 39 papers
- CIFAR-100 · 18 of 39 papers
- Tiny-ImageNet · 8 of 39 papers
- MNIST · 6 of 39 papers
- Fashion-MNIST · 6 of 39 papers
- PACS · 5 of 39 papers
- OfficeHome · 5 of 39 papers
- DomainNet · 5 of 39 papers
Baselines
- FedAvg · 20 of 39 papers
- FedProx · 11 of 39 papers
- Ensemble · 3 of 39 papers
- DENSE · 3 of 39 papers
- FedProto · 3 of 39 papers
- Ditto · 3 of 39 papers
- Local · 3 of 39 papers
- Scaffold · 3 of 39 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 142 papers; from OpenAlex and the papers' affiliations).
- Zhejiang University · 8 papers
- Nanyang Technological University · 7 papers
- National University of Singapore · 6 papers
- Tsinghua University · 5 papers
- National Taiwan University · 4 papers
- The Chinese University of Hong Kong · 4 papers
- IBM · 4 papers
- Xiamen University · 3 papers
- Vector Institute · 3 papers
- Beihang University · 3 papers
Use it
Download BibTeX of 36 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Non-IID federated learning (198 papers)
- Federated semi-supervised and graph learning (84 papers)
- Federated prompt and LoRA tuning (91 papers)
- Federated medical and multimodal learning (44 papers)
- Vertical federated learning (38 papers)
Elsewhere in the topic
- Personalization with global-local trade-off (48 papers)
- Sparse and collaboration-graph personalization (36 papers)
- Bayesian personalized FL (35 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).