Research map / Federated learning and privacy / Personalized and heterogeneous FL

Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization

ICML 2024 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Non-IID federated learning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedLESAM, which estimates the global SAM perturbation locally on each client as the difference between the global models from the previous active and current rounds, removing the extra backpropagation of local SAM while tightening the convergence bound over FedSAM.

Problem

Federated learning with data heterogeneity and multi-step local updates yields sharp global minima; existing federated SAM methods compute perturbations from local client gradients, which do not reflect the global loss landscape's flatness and add computation cost.

Approach

Clients approximate the global perturbation direction by the normalized difference between the global model received in the previous active round and the current global model, so perturbations are consistent across local steps and require only one backpropagation per iteration. Two variants, FedLESAM-S (Scaffold) and FedLESAM-D (FedDyn), add drift correction.

Key results

• On CIFAR10 Dirichlet 0.6 with 100 clients (10% active), FedLESAM-S reaches 84.94% vs 79.52% for FedAvg and 84.55% for FedSMOO.
• FedLESAM-S averages about 6% gain over FedAvg on CIFAR100 and outperforms all baselines in the reported settings.
• FedLESAM-S needs 0.81x (68% target) and 0.53x (74% target) of FedAvg's computation time on CIFAR10, versus 1.96x and 1.56x for FedSAM.

Measured effects reported in the paper

Surprising findings

• Plain FedLESAM (FedAvg-based) is often weaker than FedSAM-free baselines such as Scaffold and FedCM on CIFAR10, so the variance-reduction or dynamic-regularizer variants are needed to get the best results.
• FedDyn and FedSMOO overfit on unseen domains in leave-one-domain-out evaluation on OfficeHome and DomainNet.

Limitations

• Global perturbation is only approximated by the previous-round model difference, and the estimation bias depends on learning rates and heterogeneity; the convergence analysis assumes full participation and one-step local updates for the estimation bound.
• Plain FedLESAM can underperform strong baselines when the global update changes quickly or local models diverge, motivating the S and D variants.
• Sensitivity to the perturbation magnitude rho requires tuning relative to the local learning rate.

Experimental setup

Datasets: CIFAR10 CIFAR100 OfficeHome DomainNet

Benchmarks: CIFAR10 CIFAR100 OfficeHome DomainNet

Metrics: test accuracy communication rounds communication cost computational time global sharpness perturbation drift

Baselines: FedAvg FedSAM MoFedSAM FedGAMMA FedSMOO Scaffold FedDyn FedAdam FedCM

Builds on: SAM FedAvg FedSAM Scaffold FedDyn

Closest papers by meaning

• Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning (CVPR 2025 · similarity 0.93)
• One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory (ICML 2025 · similarity 0.93)
• FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning (NeurIPS 2026 · similarity 0.93)
• Flexible Sharpness-Aware Personalized Federated Learning (AAAI 2025 · similarity 0.92)
• Rethinking the Flat Minima Searching in Federated Learning (ICML 2024 · similarity 0.91)
• FedWMSAM: Fast and Flat Federated Learning via Weighted Momentum and Sharpness-Aware Minimization (NeurIPS 2025 · similarity 0.91)