Research map / Federated learning and privacy / Personalized and heterogeneous FL
Does One-shot Give the Best Shot? Mitigating Model Inconsistency in One-shot Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FAFI, a one-shot federated learning framework that makes client models consistent via self-alignment local training (contrastive feature alignment plus learnable category prototypes) and fuses features with noise-based informativeness weighting at the server.
Problem
One-shot federated learning (OFL) produces inconsistent local models under data heterogeneity (intra-model prediction inconsistency and inter-model parameter/feature divergence), so server-side aggregation yields a degraded global model.
Approach
Shift the fix to the client side: train with contrastive feature alignment and category-wise learnable prototypes instead of a classifier, then at the server fuse per-client features weighted by their dissimilarity to a Gaussian-noise feature and classify by nearest aggregated global prototype.
Key results
- FAFI outperforms 11 OFL baselines across CIFAR-10, CIFAR-100 and Tiny-ImageNet under non-IID settings, with a reported 10.86% average accuracy gain.
- On Tiny-ImageNet at alpha 0.05 and 0.1, the average improvement over the best baseline is 17.71%.
- On CIFAR-10 with alpha=0.05, FAFI reaches 71.84% accuracy versus 54.42% for the strongest baseline FuseFL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FAFI instead of the strongest baseline (FuseFL) | OFL · CIFAR-10, α=0.05 | accuracy: ↑17.42 | Helped |
| Use FAFI instead of the strongest baseline (IntactOFL) | OFL · CIFAR-100, α=0.1 | accuracy: ↑6.33 | Helped |
| Use FAFI instead of the strongest baseline (IntactOFL) | OFL · Tiny-ImageNet, α=0.5 | accuracy: ↑21.39 | Helped |
| Add IFFI to SALT (ablation) | FAFI on CIFAR-10 · CIFAR-10, α=0.1 | accuracy: 55.76 -> 77.83 | Helped |
| Add SALT to IFFI (ablation) | FAFI on CIFAR-10 · CIFAR-10, α=0.1 | accuracy: 63.75 -> 77.83 | Helped |
| Add self-supervised feature alignment (Lssl) (ablation) | SALT on CIFAR-10 · CIFAR-10, α=0.1 | accuracy: 52.34 -> 77.83 | Helped |
| Use category-wise prototypes instead of a classifier (ablation) | SALT on CIFAR-10 · CIFAR-10, α=0.1 | accuracy: 22.34 -> 52.34 | Helped |
| Use Lproto rather than Lce with prototypes and Lssl (ablation) | SALT on CIFAR-10 · CIFAR-10, α=0.1 | accuracy: 52.34 -> 77.83 | Helped |
Surprising findings
- Better local performance coincides with larger parameter discrepancies across one-shot local models, so inter-model inconsistency grows during long local training even near convergence.
- Among baselines, the iterative FuseFL, which shares intermediate features across clients, is the strongest, while many one-shot server-side aggregation methods fall well below multi-round FL.
Limitations
- Authors note limitations under domain shift and multi-task scenarios where static class prototypes cannot adapt to conflicting feature distributions.
- Performance decreases as the number of clients grows (tested up to 100 clients).
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet
Metrics: test accuracy communication cost
Baselines: MA-Echo O-FedAvg FedFisher FedDF F-ADI F-DAFL DENSE Ensemble Co-Boosting IntactOFL FuseFL
Builds on: FedProto FedTGP IntactOFL Co-Boosting FuseFL
Closest papers by meaning
- FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion (NeurIPS 2024 · similarity 0.89)
- One-shot Federated Learning via Synthetic Distiller-Distillate Communication (NeurIPS 2024 · similarity 0.88)
- Revisiting Ensembling in One-Shot Federated Learning (NeurIPS 2024 · similarity 0.87)
- Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boosting (ICLR 2024 · similarity 0.86)
- Capture Global Feature Statistics for One-Shot Federated Learning (AAAI 2025 · similarity 0.86)
- FedBEns: One-Shot Federated Learning based on Bayesian Ensemble (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.