Research map / Federated learning and privacy / Personalized and heterogeneous FL
Rethinking the Flat Minima Searching in Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies a "flatness discrepancy" between local and global models in federated learning and proposes FedGF, which perturbs local SAM steps with an interpolation of local and global perturbations to find flatter global minima.
Problem
Under heterogeneous (non-IID) client data, SAM-based federated learning methods find flat minima for local objectives but the aggregated global model does not lie in a flat region of the global objective (flatness discrepancy), degrading generalization.
Approach
Defines flatness discrepancy and proposes FedGF, which approximates a global perturbation from the difference between consecutive global models and interpolates it with the local SAM perturbation using an adaptive coefficient c driven by local-global model divergence.
Key results
- FedGF reaches 78.41% on CIFAR-10 non-IID (α=0, 5 clients) versus 73.32% for the best prior baseline FedASAM, a gain of about 5.09 points.
- On CIFAR-100 non-IID (α=0, 20 clients) FedGF attains 47.77% versus the runner-up gains of 6.51% to 10.94% reported across settings.
- FedGF converges markedly faster in non-IID settings; FedSMOO needs about 2.4 to 3.0 times more model transmissions to reach FedAvg's final accuracy on CIFAR-10 and CIFAR-100 non-IID.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FedGF instead of FedASAM | CIFAR-10 federated learning · CIFAR-10 Dir.(α = 0), 5 clients | test accuracy: 73.32 -> 78.41 | Helped |
| use FedGF instead of FedASAM | CIFAR-10 federated learning · CIFAR-10 Dir.(α = 0), 10 clients | test accuracy: 74.5 -> 79.68 | Helped |
| use FedGF instead of MoFedSAM | CIFAR-10 federated learning · CIFAR-10 Dir.(α = 0), 20 clients | test accuracy: 76.66 -> 80.86 | Helped |
| use FedGF instead of FedSMOO | CIFAR-100 federated learning · CIFAR-100 Dir.(α = 0), 5 clients | test accuracy: 35.35 -> 45.37 | Helped |
| use FedGF instead of FedSMOO | CIFAR-100 federated learning · CIFAR-100 Dir.(α = 0), 10 clients | test accuracy: 38.78 -> 46.86 | Helped |
| use FedGF instead of FedASAM | CIFAR-100 federated learning · CIFAR-100 Dir.(α = 0), 20 clients | test accuracy: 38.93 -> 47.77 | Helped |
| use adaptive c rather than a fixed c (ablation) | FedGF · CIFAR-10 non-IID (α = 0) | test accuracy: 68.11 -> 78.41; 71.24 -> 78.41; 78.05 -> 78.41 | Helped |
| use adaptive c rather than a fixed c (ablation) | FedGF · CIFAR-10 IID (α = 10) | test accuracy: 83.78 -> 84.71; 82.95 -> 84.71; 81.94 -> 84.71 | Helped |
| use adaptive c rather than a fixed c (ablation) | FedGF · CIFAR-100 non-IID (α = 0) | test accuracy: 29.43 -> 45.37; 26.64 -> 45.37; 44.39 -> 45.37 | Helped |
| use adaptive c rather than a fixed c (ablation) | FedGF · CIFAR-100 IID (α = 10) | test accuracy: 54.06 -> 54.16; 52.47 -> 54.16; 46.68 -> 54.16 | Helped |
Surprising findings
- Flatter local minima (FedSAM) do not translate into a flat global model; flatness discrepancy grows as heterogeneity increases.
- Several regularization-based methods (SCAFFOLD, FedDyn, FedGAMMA) fail to train under strong non-IID settings on CIFAR-100.
Limitations
- Experiments use only small LeNet-style models on CIFAR-10/100; larger architectures are reported only in the appendix.
- The global perturbation is approximated by the difference of consecutive global models, so the gradient estimate carries an error term.
- The convergence analysis relies on assumptions such as smoothness and bounded heterogeneity.
Experimental setup
Datasets: CIFAR-10 CIFAR-100
Benchmarks: CIFAR-10 CIFAR-100
Metrics: test accuracy LPF flatness lambda max Hessian eigenvalue flatness discrepancy communication cost
Baselines: FedAvg FedAvgM FedProx SCAFFOLD FedDyn FedSAM FedASAM MoFedSAM FedGAMMA FedSMOO
Builds on: FedAvg SAM FedSAM MoFedSAM FedSMOO
Closest papers by meaning
- FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning (NeurIPS 2026 · similarity 0.94)
- One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory (ICML 2025 · similarity 0.92)
- Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning (CVPR 2025 · similarity 0.91)
- Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization (ICML 2024 · similarity 0.91)
- FedWMSAM: Fast and Flat Federated Learning via Weighted Momentum and Sharpness-Aware Minimization (NeurIPS 2025 · similarity 0.89)
- Flexible Sharpness-Aware Personalized Federated Learning (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.