Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedSelect, a personalized federated learning method that grows client-specific subnetworks of parameters for personalization, guided by per-parameter update magnitude (inspired by lottery ticket iterative search), while averaging the rest globally.
Problem
Standard federated learning degrades under heterogeneous client data, and existing personalized FL methods pre-select layers (e.g., classifier heads) for personalization, which may be suboptimal and shared across all clients.
Approach
Iteratively grows a per-client personalized parameter subnetwork by selecting parameters with the largest absolute change during local training (gradient-based lottery ticket hypothesis), keeping those local and averaging the remaining parameters globally with mask-aware aggregation, rather than pruning to zero.
Key results
- FedSelect reaches 82.25% on CIFAR10, 72.05% on CIFAR10-C, 38.69% on Mini-ImageNet and 80.51% on OfficeHome, the best of all compared methods in Table 1.
- The authors report an average improvement of over 2.4% over baselines across the four datasets.
- Ablation: personalizing the parameters with least change, layer-specific personalization, and random partitions all underperform FedSelect (e.g., CIFAR10-C 64.20, 61.12 vs 70.60 at alpha=0.30).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedSelect's customized parameter selection instead of the strongest listed baseline | FedSelect · CIFAR10 | accuracy: 77.65 -> 82.25 | Helped |
| Use FedSelect's customized parameter selection instead of the strongest listed baseline | FedSelect · CIFAR10-C | accuracy: 69.50 -> 72.05 | Helped |
| Use FedSelect's customized parameter selection instead of the strongest listed baseline | FedSelect · Mini-ImageNet | accuracy: 37.72 -> 38.69 | Helped |
| Use FedSelect's customized parameter selection instead of the strongest listed baseline | FedSelect · OfficeHome | accuracy: 80.07 -> 80.51 | Helped |
| Personalize parameters with the least variation rather than selecting high-change parameters (ablation) | FedSelect · CIFAR-10 | accuracy: 79.87 vs 82.25 | Hurt |
| Personalize parameters with the least variation rather than selecting high-change parameters (ablation) | FedSelect · CIFAR10-C | accuracy: 64.20 vs 70.60 | Hurt |
| Personalize parameters with the least variation rather than selecting high-change parameters (ablation) | FedSelect · Mini-ImageNet | accuracy: 35.65 vs 37.84 | Hurt |
| Personalize parameters with the least variation rather than selecting high-change parameters (ablation) | FedSelect · OfficeHome | accuracy: 78.23 vs 80.51 | Hurt |
| Set personalization limit α to 0.05 instead of α = 0.30 (ablation) | FedSelect · CIFAR10 | accuracy: 82.25 -> 80.10 | Hurt |
| Set personalization limit α to 0.05 instead of α = 0.30 (ablation) | FedSelect · CIFAR10-C | accuracy: 70.60 -> 68.30 | Hurt |
| Set personalization limit α to 0.05 instead of α = 0.30 (ablation) | FedSelect · Mini-ImageNet | accuracy: 37.84 -> 37.60 | Hurt |
| Set personalization limit α to 0.05 instead of α = 0.30 (ablation) | FedSelect · OfficeHome | accuracy: 80.51 -> 78.46 | Hurt |
Surprising findings
- FedAvg with fine-tuning performs competitively with other personalized FL baselines, consistent with prior observations.
- FedBABU drops to 6.99% on Mini-ImageNet, indicating severe degradation of head-only personalization under that setting.
Limitations
- Personalization may trade off against generalization under test-time distribution shifts and overfitting.
- Communication overhead and resource demands in decentralized settings remain challenges.
- Vulnerable to adversarial attacks and model poisoning, as noted by the authors.
Experimental setup
Datasets: CIFAR-10 CIFAR10-C Mini-ImageNet OfficeHome
Benchmarks: CIFAR-10 CIFAR10-C Mini-ImageNet OfficeHome
Metrics: personalized accuracy (mean across clients)
Baselines: Local Only FedAvg FedAvg + FT Ditto FedPAC FedRep FedPer FedBABU LG-FedAvg
Builds on: LocalAlt partial model personalization iterative magnitude search lottery ticket FedAvg LotteryFL
Closest papers by meaning
- FedAS: Bridging Inconsistency in Personalized Federated Learning (CVPR 2024 · similarity 0.89)
- Flexible Sharpness-Aware Personalized Federated Learning (AAAI 2025 · similarity 0.89)
- PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees (CVPR 2024 · similarity 0.89)
- PeFLL: Personalized Federated Learning by Learning to Learn (ICLR 2024 · similarity 0.88)
- Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees (ICML 2025 · similarity 0.88)
- Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.