Research map / Federated learning and privacy / Personalized and heterogeneous FL
Flexible Sharpness-Aware Personalized Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedFSA, a personalized federated learning method that applies larger SAM perturbations to the layers with highest estimated sharpness (via a global perturbation sensitivity metric) and evaluates it on image classification with a 5-layer CNN and ResNet18.
Problem
Personalized federated learning under statistical (non-IID) heterogeneity suffers from poor local generalization; existing PFL methods focus on global/local information fusion and neglect local optimization generalization, and prior federated SAM methods do not select which layers to perturb or how
Approach
Estimate layer-wise sharpness cheaply via a perturbation sensitivity metric (squared parameter change over local training, approximating gradient-times-perturbation), then apply a larger SAM perturbation only to the top-C sharpest layers each client selects for the next round, with default perturbation elsewhere and momentum.
Key results
- On CIFAR100 Dir(0.3), FedFSA reaches 60.87% average test accuracy, reported as 8.26% higher than the second-best method.
- FedFSA achieves best accuracy in most settings on CIFAR10, CIFAR100 and Tiny-ImageNet, but not on Fashion-MNIST where MoFedSAM is slightly better.
- Applying FSA to FedSpeed and other FL methods yields up to 4.45% improvement; on ResNet18 FedFSA converges faster and generalizes better than MoFedSAM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedFSA instead of MoFedSAM | 5-layer CNN · FMNIST Pat(5) | test accuracy: 93.79±0.21 -> 93.78±0.11 | Hurt |
| Use FedFSA instead of MoFedSAM | 5-layer CNN · FMNIST Dir(0.5) | test accuracy: 92.94±0.27 -> 92.88±0.35 | Hurt |
| Use FedFSA instead of MoFedSAM | 5-layer CNN · CIFAR10 Pat(5) | test accuracy: 88.33±0.14 -> 88.64±0.09 | Helped |
| Use FedFSA instead of MoFedSAM | 5-layer CNN · CIFAR10 Dir(0.5) | test accuracy: 88.16±0.25 -> 88.37±0.27 | Helped |
| Use FedFSA instead of MoFedSAM | 5-layer CNN · CIFAR100 Pat(15) | test accuracy: 70.33±0.29 -> 72.28±0.40 | Helped |
| Use FedFSA instead of FedSpeed | 5-layer CNN · CIFAR100 Dir(0.3) | test accuracy: 52.61±0.55 -> 60.87±0.61 | Helped |
| Use FedFSA instead of MoFedSAM | 5-layer CNN · TINY Pat(50) | test accuracy: 33.80±0.29 -> 41.39±0.25 | Helped |
| Use FedFSA instead of MoFedSAM | 5-layer CNN · TINY Dir(0.3) | test accuracy: 28.02±0.60 -> 33.76±0.93 | Helped |
| Increase the larger perturbation amplitude ρlarger, then reduce it when perturbation becomes excessive (ablation) | FedFSA on CIFAR100 Dir(0.3) · CIFAR100 Dir(0.3) | test accuracy | Mixed |
| Increase ρlarger or vary TopC (ablation) | FedFSA on FMNIST Dir(0.5) · FMNIST Dir(0.5) | test accuracy | Hurt |
| Select perturbation-insensitive layers instead of the most sensitive layers (ablation) | FedFSA on CIFAR10 and CIFAR100 · CIFAR10 and CIFAR100 Dir(0.3) | test accuracy | Hurt |
| Select the most perturbation-sensitive layers rather than random layers (ablation) | FedFSA with ResNet18 on CIFAR100 Dir(0.3) · CIFAR100 Dir(0.3) | test accuracy | Helped |
Surprising findings
- Reversing the selection (perturbing the least sensitive layers) performs worst, and on ResNet18 CIFAR10 FedFSA's layer selection performs similarly to random selection.
Limitations
- No theoretical analysis of how FedFSA improves generalization.
- The large perturbation amplitude rho_larger is fixed rather than adaptive.
- Evaluated only on image classification with small CNN and ResNet18 models.
Experimental setup
Datasets: Fashion-MNIST CIFAR10 CIFAR100 Tiny-ImageNet
Benchmarks: Fashion-MNIST CIFAR10 CIFAR100 Tiny-ImageNet
Metrics: average test accuracy
Baselines: FedAvg-FT FedCR FedALA FedSAM MoFedSAM FedSMOO FedSpeed
Builds on: Sharpness-Aware Minimization FedSAM MoFedSAM LLMC FedLESAM
Closest papers by meaning
- Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization (ICML 2024 · similarity 0.92)
- One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory (ICML 2025 · similarity 0.92)
- Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning (CVPR 2025 · similarity 0.91)
- Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees (ICML 2025 · similarity 0.90)
- FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning (NeurIPS 2026 · similarity 0.90)
- FedAS: Bridging Inconsistency in Personalized Federated Learning (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.