Research map / Federated learning and privacy / Personalized and heterogeneous FL

FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning

NeurIPS 2026 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Non-IID federated learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedVSSAM, a federated SAM method that uses one PS-maintained variance-suppressed adjusted direction across local flatness search, local update, and global update to mitigate flatness incompatibility under data heterogeneity.

Problem

SAM-based federated learning under data heterogeneity: device-side SAM finds locally flat regions that are incompatible with the globally flat region, hurting global generalization and convergence.

Approach

Define flatness incompatibility (inter-device dispersion of local flatness proxies), attribute it to data heterogeneity and the friendly adversary phenomenon, and anchor SAM perturbation and update directions to a shared server-side EMA direction h^t mixed with local gradients, with no extra uplink vectors.

Key results

• FedVSSAM reaches 83.56% on CIFAR-10 (Dirichlet α=0.5, 100 devices), vs 82.36% for FedGAMMA and 81.49% for FedSMOO; needs 111 rounds to 75%, 64.8% fewer than FedAvg.
• On CIFAR-100 (Dirichlet α=0.5, 100 devices) FedVSSAM reaches 58.78% vs 56.87% for FedSMOO (second best).
• On DBpedia-14 (Dirichlet α=0.1) FedVSSAM reaches 96.90% vs 96.44% for FedACG.

Surprising findings

• FedACG reaches the 90% threshold on DBpedia-14 earlier than FedVSSAM under Dirichlet partition, though FedVSSAM has higher final accuracy.
• Several SAM-based baselines (FedLESAM, FedGAMMA, FedSAM) perform worse than or comparable to FedAvg in some severe heterogeneity settings.

Limitations

• Analysis relies on standard smoothness, bounded variance and bounded heterogeneity assumptions and uniform random device participation.
• Experiments are simulations on vision and text tasks only; no real-world hardware deployment.
• Mixing coefficients (gamma_l, gamma_g) are chosen per heterogeneity level rather than adapted automatically.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 DBpedia-14

Benchmarks: CIFAR-10 CIFAR-100 DBpedia-14

Metrics: test accuracy rounds to target accuracy

Baselines: FedAvg FedProx FedAdam SCAFFOLD FedACG FedCM FedDyn FedSAM MoFedSAM FedGAMMA FedSMOO FedLESAM

Builds on: FedSAM SAM FedAvg Friendly SAM FedCM

Closest papers by meaning

• Rethinking the Flat Minima Searching in Federated Learning (ICML 2024 · similarity 0.94)
• One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory (ICML 2025 · similarity 0.93)
• Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization (ICML 2024 · similarity 0.93)
• Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning (CVPR 2025 · similarity 0.92)
• FedWMSAM: Fast and Flat Federated Learning via Weighted Momentum and Sharpness-Aware Minimization (NeurIPS 2025 · similarity 0.90)
• Flexible Sharpness-Aware Personalized Federated Learning (AAAI 2025 · similarity 0.90)