Research map / Federated learning and privacy / Personalized and heterogeneous FL
Non-IID federated learning
Counter data heterogeneity with drift decoupling, distillation and re-labeling.
198 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Personalized and heterogeneous FL, keyword cluster “domain · label · multi”
State History Tried Baselines Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
66 papers in 2023–24 and 108 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 26.5% to 27.8%.
Open questions the data points to
- Papers report FedProx on TinyImageNet test (top-1 accuracy) anywhere from 23.2 to 50.63: comparisons against it depend on whose number you take.
- Papers report SCAFFOLD on Fashion-MNIST NIID-1 (accuracy (%)) anywhere from 77.02 to 89.65: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Learning privately from multiparty data (ICML 2016 · 56 citations)
- 2019 · Bayesian Nonparametric Federated Learning of Neural Networks (ICML 2019 · 144 citations)
- 2021 · Model-Contrastive Federated Learning (CVPR 2021 · 1,518 citations)
- 2022 · Fine-Tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning (CVPR 2022 · 370 citations)
- 2023 · Rethinking Federated Learning With Domain Shift: A Prototype View (CVPR 2023 · 181 citations)
- 2024 · FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning (AAAI 2024 · 114 citations)
- 2025 · FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning (AAAI 2025 · 18 citations)
- 2026 · FedLAGC: Towards High Performance System-Heterogeneous Federated Learning via Layer-Adaptive Submodel Extraction and Gradient Correction (AAAI 2026 · 1 citation)
Earliest papers here
- Learning privately from multiparty data (ICML 2016 · 56 citations)
- Variations on the Hotelling-Downs Model (AAAI 2016 · 25 citations)
- Bayesian Nonparametric Federated Learning of Neural Networks (ICML 2019 · 144 citations)
What has been tried
117 papers here report 1,164 measured changes (a component added, removed or swapped, compared with a baseline): helped 71%, no clear effect 5%, hurt 20%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| replace adaptive-margin contrastive learning with standard contrastive loss | Cifar100, practical · accuracy | 46.94 -> 40.11 | Hurt | FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning (AAAI 2024 · 114 citations) |
| Add collaborative data condensation (CDC, SWD regularization) | CIFAR10 α=0.02 · accuracy | +0.99 (65.15 vs 64.16) | Helped | An Aggregation-Free Federated Learning for Tackling Data Heterogeneity (CVPR 2024 · 61 citations) |
| replace relaxed contrastive loss with supervised contrastive loss | CIFAR-100, α=0.05, 1000R · accuracy | 54.63 -> 42.93 | Hurt | Relaxed Contrastive Learning for Federated Learning (CVPR 2024 · 43 citations) |
| add the additional local knowledge-transfer loss | Tiny-ImageNet, practical · test accuracy | 24.39 -> 28.17 | Helped | An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learning (CVPR 2024 · 30 citations) |
| add classifier variance regularization | STL-10 (α=0.01) · accuracy | 24.1±1.3 -> 30.4±1.4 | Helped | FedUV: Uniformity and Variance for Heterogeneous Federated Learning (CVPR 2024 · 23 citations) |
| use balanced classifiers rather than local classifiers as clustering input | CIFAR-10, sudden drift, Dir(0.5), 100 clients · generalized accuracy | 68.39±0.71 -> 73.21±0.82 | Helped | Classifier Clustering and Feature Alignment for Federated Learning under Distributed Concept Drift (NeurIPS 2024 · 21 citations) |
| Add gradient alignment to agent valuation | PROTEINS · model gradient fairness | 0.736±0.145 -> 0.787±0.052 | Helped | Towards Fair Graph Federated Learning via Incentive Mechanisms (AAAI 2024 · 20 citations) |
| Add the global acyclicity constraint | Sachs · FDR | 69.23 -> 52.63 | Helped | Federated Causality Learning with Explainable Adaptive Optimization (AAAI 2024 · 19 citations) |
| Add embedding projection | Cifar100, HtFE_8, cross-device · test accuracy (%) | 29.54 -> 30.80 (+1.26) | Helped | FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning (AAAI 2025 · 18 citations) |
| Add TERL | CIFAR10, CIFAR100 · Top-1 accuracy | 70.85 -> 72.06; 68.24 -> 70.01; 60.67 -> 62.53; 57.58 -> 59.46 | Helped | Cross-Silo Feature Space Alignment for Federated Learning on Clients with Imbalanced Data (AAAI 2025 · 15 citations) |
| omit DAT and use only CKF | CIFAR-10, β=0.2 · local accuracy | 74.34% -> 73.74% | Hurt | FedGMKD: An Efficient Prototype Federated Learning Framework through Knowledge Distillation and Discrepancy-Aware Aggregation (NeurIPS 2024 · 15 citations) |
| Add the stage-one auxiliary loss | MERGE-FA · accuracy | 88.44 ± 0.36 -> 91.51 ± 0.36 | Helped | On Disentanglement of Asymmetrical Knowledge Transfer for Modality-Task Agnostic Federated Learning (AAAI 2024 · 14 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| FedProx | TinyImageNet test | top-1 accuracy | 23.2 – 50.63 | 2 papers, e.g. CVPR 2021 and AAAI 2023 |
| SCAFFOLD | Fashion-MNIST NIID-1 | accuracy (%) | 77.02 – 89.65 | 2 papers, e.g. ICLR 2024 and ICML 2023 |
| FedAvg | CIFAR100 α=∞ | test accuracy (%) | 67.2 – 74.54 | 2 papers, e.g. ICLR 2025 and ICLR 2023 |
What reviewers will expect
Datasets and baselines named in the 58 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-100 · 23 of 58 papers
- CIFAR-10 · 21 of 58 papers
- Tiny-ImageNet · 8 of 58 papers
- PACS · 5 of 58 papers
- Fashion-MNIST · 4 of 58 papers
- CINIC-10 · 4 of 58 papers
- MNIST · 4 of 58 papers
- Office-Home · 4 of 58 papers
Baselines
- FedAvg · 22 of 58 papers
- FedProx · 14 of 58 papers
- SCAFFOLD · 12 of 58 papers
- FedProto · 7 of 58 papers
- MOON · 6 of 58 papers
- FedDyn · 6 of 58 papers
- FedSAM · 5 of 58 papers
- FedKD · 4 of 58 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 174 papers; from OpenAlex and the papers' affiliations).
- Shanghai Jiao Tong University · 8 papers
- Wuhan University · 7 papers
- National University of Singapore · 6 papers
- Hong Kong University of Science and Technology · 5 papers
- The University of Sydney · 5 papers
- Zhejiang University · 5 papers
- East China Normal University · 5 papers
- Nanyang Technological University · 5 papers
- Tsinghua University · 4 papers
- Fudan University · 4 papers
Use it
Download BibTeX of 36 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Federated learning with pretrained models (164 papers)
- Federated semi-supervised and graph learning (84 papers)
- Federated prompt and LoRA tuning (91 papers)
- Federated medical and multimodal learning (44 papers)
- Vertical federated learning (38 papers)
Elsewhere in the topic
- Personalization with global-local trade-off (48 papers)
- Sparse and collaboration-graph personalization (36 papers)
- Bayesian personalized FL (35 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).