Research map / Federated learning and privacy / Personalized and heterogeneous FL
Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FEDGLOSS, a federated learning method that applies sharpness-aware minimization on the server and approximates the global perturbation with the previous round's pseudo-gradient, avoiding extra communication, with ADMM for local-global consistency.
Problem
Federated learning under data heterogeneity converges to sharp global minima; existing client-side SAM methods (FedSAM) optimize only local sharpness, leaving local and global loss landscapes inconsistent, while global-sharpness alternatives (FedSMOO) double communication.
Approach
Run SAM on the server to minimize global sharpness, approximating the perturbation with the previous round's perturbed global pseudo-gradient so no extra client exchange is needed, and use ADMM to align local and global updates.
Key results
- On CIFAR10 with alpha=0 (CNN), FedGloSS with local SAM reaches 83.9% vs. 81.3% for FedSMOO at half the communication cost.
- On CIFAR100 alpha=0.5 (CNN), FedGloSS with local SAM reaches 53.4% vs. 51.7% for FedSMOO.
- On LANDMARKS-USER-160K (MobileNetv2), FedGloSS reaches 59.7% vs. 59.5% for FedSMOO and 56.3% for FedAvg, with 1x communication cost.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FEDGLOSS with local SAM instead of FEDSMOO | Federated CNN training · CIFAR10 α = 0 and α = 0.05; CIFAR100 α = 0 and α = 0.5 | accuracy: 81.3±0.5 -> 83.9±0.4; 82.8±0.6 -> 84.4±0.5; 47.8±0.5 -> 50.6±0.6; 51.7±0.46 -> 53.4±0.5 | Helped |
| Use FEDGLOSS with local SGD instead of FEDAVG | Federated CNN training · CIFAR10 α = 0 and α = 0.05; CIFAR100 α = 0 and α = 0.5 | accuracy: 59.9±0.4 -> 69.5±0.4; 65.7±1.0 -> 75.5±0.3; 28.6±0.7 -> 42.5±0.6; 38.5±0.5 -> 47.9±0.5 | Helped |
| Use FEDGLOSS with local SAM instead of FEDSMOO | Federated ResNet18 training · CIFAR100 α = 0.5 and CIFAR10 α = 0.05 | accuracy: 44.8±0.5 -> 47.2±0.2; 75.3±0.6 -> 80.0±0.3 | Helped |
| Use FEDGLOSS instead of FEDAVG | MobileNetv2 · LANDMARKS-USER-160K | accuracy: 56.3±0.2 -> 59.7±1.2 | Helped |
| Approximate the perturbation with the previous round's pseudo-gradient instead of computing the exact perturbation (ablation) | FEDGLOSS · CIFAR10 α = 0 and CIFAR100 α = 0 | accuracy @100%: 83.9±0.2 -> 83.9±0.4; 50.8±0.1 -> 50.6±0.6 | No clear effect |
| Add ADMM-based global-local consistency (ablation) | FEDGLOSS with local SAM · CIFAR10 α = 0, CNN | accuracy: 70.2±0.9 -> 83.9±0.4 | Helped |
| Add server-side SAM for global sharpness minimization (ablation) | FEDGLOSS with local SAM · CIFAR10 α = 0, CNN | accuracy: 79.3±3.1 -> 83.9±0.4 | Helped |
| Add server-side SAM for global sharpness minimization (ablation) | FEDGLOSS with local SAM · CIFAR100 α = 0, CNN | accuracy: ✗ -> 50.6±0.6 | Helped |
| Add ADMM-based global-local consistency (ablation) | FEDGLOSS with local SAM · CIFAR100 α = 0, CNN | accuracy: 28.7±0.5 -> 50.6±0.6 | Helped |
Surprising findings
- Using SAM on the server stabilizes ADMM and mitigates the parameter-norm explosion that affects FedDyn under high heterogeneity.
- FedSAM's local models can appear flat locally while lying near saddle points or sharp minima in the global loss landscape, especially on CIFAR100.
Limitations
- Relies on stateful clients because ADMM needs persistent local dual variables.
- Approximating the sharpness direction with the previous round's pseudo-gradient introduces a gap from exact SAM (δ does not vanish under heterogeneity and partial participation).
Experimental setup
Datasets: CIFAR10 CIFAR100 LANDMARKS-USER-160K
Benchmarks: CIFAR10 CIFAR100 LANDMARKS-USER-160K
Metrics: test accuracy maximum Hessian eigenvalue communication cost in bits rounds to reach FedAvg accuracy
Baselines: FedAvg FedProx FedDyn SCAFFOLD FedSAM FedSpeed FedGamma FedSMOO
Builds on: SAM FedSAM FedSMOO FedAvg FedDyn
Closest papers by meaning
- One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory (ICML 2025 · similarity 0.94)
- Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization (ICML 2024 · similarity 0.93)
- FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning (NeurIPS 2026 · similarity 0.92)
- Rethinking the Flat Minima Searching in Federated Learning (ICML 2024 · similarity 0.91)
- Flexible Sharpness-Aware Personalized Federated Learning (AAAI 2025 · similarity 0.91)
- FedWMSAM: Fast and Flat Federated Learning via Weighted Momentum and Sharpness-Aware Minimization (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.