Research map / Federated learning and privacy / Personalized and heterogeneous FL
One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedGMT, a federated learning method that measures global sharpness via an EMA of the global model trajectory (KL-based loss, one backward pass) combined with ADMM-style alignment, with a convergence analysis and experiments on CIFAR-10/100, CINIC-10, and AG News.
Problem
Heterogeneous (non-IID) federated learning suffers from client drift; existing SAM-based FL methods perturb using local sharpness rather than the global objective and need two backward passes per iteration.
Approach
Measure global sharpness using a KL divergence between the current local model and an exponential moving average (EMA) of the global model trajectory, which needs only one extra forward pass and no extra backward pass; ADMM aligns local and global updates.
Key results
- FedGMT reaches 80.48% on CIFAR-10 Dir(1.0) vs 80.38%/80.22% for FedSpeed/FedSMOO-level baselines and 75.73% for FedAvg; on CIFAR-10 Dir(0.01) it improves to 74.67% vs 61.94% FedAvg.
- On CIFAR-10 Dir(0.01), ADMM plus the global trajectory loss improves accuracy by 13.39% over FedAvg-level performance in the ablation (from 61.28% with only the loss to 74.67%).
- FedGMT reaches target accuracy in fewer rounds and less computation time than FedSpeed and FedSMOO (e.g., 392 rounds and 40.38 min vs 688 rounds and 149.30 min on CIFAR-100-Path(10)-level settings in Table 2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedGMT instead of the strongest listed baseline | Federated learning · CIFAR-10 CNN, Path(10), Path(4), and Path(2) | test accuracy@1 (%): 83.49(0.47) -> 83.64(0.11); 80.51(1.19) -> 82.79(0.22); 76.74(1.49) -> 78.04(1.03) | Helped |
| Use FedGMT instead of the strongest listed baseline | Federated learning · CIFAR-10 CNN, Dir(1.0), Dir(0.1), and Dir(0.01) | test accuracy@1 (%): 80.38(0.38) -> 80.48(0.18); 77.08(0.97) -> 79.17(0.49); 72.11(1.79) -> 74.67(0.77) | Helped |
| Use FedGMT instead of the strongest listed baseline | Federated learning · CIFAR-100 ResNet-8, Path(2) and Dir(0.1), participation 20%, 10%, and 5% | test accuracy@1 (%): 47.48(0.87) -> 49.76(0.33); 45.76(1.12) -> 48.19(0.56); 40.06(1.61) -> 46.11(0.67); 48.26(0.42) -> 48.65(0.22); 45.73(0. | Helped |
| Use FedGMT instead of the strongest listed baseline | Federated learning · CINIC-10 ViT, Path(2) and Dir(0.1) | test accuracy@1 (%): 49.27(2.17) -> 52.67(1.13); 49.07(2.29) -> 57.84(0.77) | Helped |
| Use FedGMT instead of the strongest listed baseline | Federated learning · AG News FastText, Path(2) and Dir(0.1) | test accuracy@1 (%): 85.37( 2.57) -> 89.39( 0.62); 85.68(1.62) -> 88.39(0.67) | Helped |
| Add ADMM and global trajectory loss (ablation) | FedGMT · CIFAR-10, Dir(1) | test accuracy (%): 75.73(0.85) -> 80.48(0.18) | Helped |
| Add ADMM to global trajectory loss (ablation) | FedGMT · CIFAR-10, Dir(1) | test accuracy (%): 78.18(0.43) -> 80.48(0.18) | Helped |
| Add global trajectory loss to ADMM (ablation) | FedGMT · CIFAR-10, Dir(1) | test accuracy (%): 78.13(0.34) -> 80.48(0.18) | Helped |
| Add ADMM and global trajectory loss (ablation) | FedGMT · CIFAR-10, Dir(0.1) | test accuracy (%): 70.61(3.51) -> 79.17(0.49) | Helped |
| Add ADMM to global trajectory loss (ablation) | FedGMT · CIFAR-10, Dir(0.1) | test accuracy (%): 72.68(2.19) -> 79.17(0.49) | Helped |
| Add global trajectory loss to ADMM (ablation) | FedGMT · CIFAR-10, Dir(0.1) | test accuracy (%): 75.97(1.09) -> 79.17(0.49) | Helped |
Surprising findings
- Using the global trajectory loss alone (without ADMM) is ineffective under high heterogeneity, e.g. 61.28% vs 61.94% FedAvg on CIFAR-10 Dir(0.01).
- FedSAM performs no better than FedAvg in severely heterogeneous settings, since local sharpness does not reflect global sharpness.
Limitations
- EMA coefficient alpha is sensitive; large alpha hurts under high heterogeneity and the authors leave an adaptive mechanism to future work.
- FedGMT and FedGMT-v2 incur higher communication cost (1.5x) than baselines unless the v2 variant is used, and v2 trades slight accuracy for communication.
- Convergence analysis relies on assumptions such as bounded gradients and softmax outputs bounded away from zero.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 CINIC-10 AG News
Benchmarks: CIFAR-10 CIFAR-100 CINIC-10 AG News
Metrics: test accuracy communication rounds to target accuracy computation time
Baselines: FedAvg FedDyn FedNTD FedSAM FedSpeed FedSMOO FedLESAM-D
Builds on: SAM FedSAM FedSMOO FedSpeed FedLESAM FedAvg
Closest papers by meaning
- Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning (CVPR 2025 · similarity 0.94)
- Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization (ICML 2024 · similarity 0.93)
- FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning (NeurIPS 2026 · similarity 0.93)
- Rethinking the Flat Minima Searching in Federated Learning (ICML 2024 · similarity 0.92)
- Flexible Sharpness-Aware Personalized Federated Learning (AAAI 2025 · similarity 0.92)
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.