Research map / Federated learning and privacy / Personalized and heterogeneous FL
Federated Domain Generalization with Data-free On-server Matching Gradient
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedOMG, a communication-free on-server gradient-matching method that finds an invariant gradient direction as a convex combination of client gradients for federated domain generalization and standard FL.
Problem
Federated domain generalization (FDG): training a global model over distributed source domains (clients) that generalizes to unseen target domains without sharing data or adding communication cost.
Approach
Instead of optimizing model parameters for gradient inner product maximization (which needs second-order derivatives), it optimizes convex combination coefficients over local client gradients at the server, with a radius constraint around the FedAvg gradient and a closed-form-like relaxation, yielding an on-server invariant gradient update that is orthogonal to existing FL/FDG methods.
Key results
- On PACS, FedOMG reaches 88.4% average accuracy vs 84.2% for StableFDG under leave-one-domain-out with ResNet-18.
- On VLCS, FedOMG averages 82.0% vs 75.8% for FedSR; on OfficeHome 70.0% vs 65.2% for FedGA.
- On non-IID FL with 100 clients (alpha=0.1, full participation), FedOMG gets 98.75% on MNIST and 98.91% on EMNIST; FedOMG+FedRod gives the best results (e.g., 95.81% CIFAR-10).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedOMG instead of FedRod | federated learning · MNIST, non-IID α=0.1 | accuracy: 98.09 $\pm$ 0.27 -> 98.75 $\pm$ 0.02 | Helped |
| Use FedOMG instead of FedAS | federated learning · CIFAR-10, non-IID α=0.1 | accuracy: 89.15 $\pm$ 0.06 -> 90.40 $\pm$ 0.03 | Helped |
| Use FedOMG instead of FedAS | federated learning · CIFAR-100, non-IID α=0.1 | accuracy: 50.37 $\pm$ 0.18 -> 48.76 $\pm$ 0.15 | Hurt |
| Use FedOMG instead of FedPac | federated learning · EMNIST, non-IID α=0.1 | accuracy: 97.74 $\pm$ 0.32 -> 98.91 $\pm$ 0.02 | Helped |
| Use FedOMG instead of FedRod | federated learning · MNIST, non-IID α=0.5 | accuracy: 96.85 $\pm$ 0.11 -> 98.58 $\pm$ 0.09 | Helped |
| Use FedOMG instead of FedAS | federated learning · CIFAR-10, non-IID α=0.5 | accuracy: 75.75 $\pm$ 0.21 -> 72.68 $\pm$ 0.02 | Hurt |
| Use FedOMG instead of FedAS | federated learning · CIFAR-100, non-IID α=0.5 | accuracy: 32.57 $\pm$ 0.31 -> 30.64 $\pm$ 0.32 | Hurt |
| Use FedOMG instead of FedRod | federated learning · EMNIST, non-IID α=0.5 | accuracy: 96.46 $\pm$ 0.22 -> 98.41 $\pm$ 0.02 | Helped |
| Apply on-server gradient matching with FedOMG | federated domain generalization · VLCS, average | accuracy: 76.6 -> 82.0 | Helped |
| Apply on-server gradient matching with FedOMG | federated domain generalization · PACS, average | accuracy: 84.2 -> 88.4 | Helped |
| Apply on-server gradient matching with FedOMG | federated domain generalization · OfficeHome, average | accuracy: 65.2 -> 70.0 | Helped |
| Add FedOMG to FedRod | FedRod · federated learning, listed datasets and non-IID scenarios | accuracy | Helped |
Surprising findings
- FedOMG is not the top method on CIFAR-100 non-IID (48.76% vs FedAS/FedBabu ~50%), and FedOMG+FedRod is needed to surpass them.
- FedSAM+OMG, not FedOMG alone, gives the best FDG average on PACS and VLCS, indicating the gains overlap with sharpness-aware methods.
Limitations
- Theoretical guarantees rely on strong convexity assumptions that do not match deep networks.
- Performance is sensitive to the searching radius kappa and other hyper-parameters (e.g., best at kappa ~0.5).
- Experiments use a small number of clients and a single backbone per setting; reported comparisons have some inconsistencies across tables.
Experimental setup
Datasets: MNIST EMNIST CIFAR-10 CIFAR-100 PACS VLCS OfficeHome
Benchmarks: MNIST EMNIST CIFAR-10 CIFAR-100 PACS VLCS OfficeHome
Metrics: top-1 accuracy
Baselines: FedAvg PerAvg FedRod FedBABU FedPAC FedAS FedGA FedSAM FedIIR FedSR StableFDG
Builds on: Fish Fishr FedAvg FedRod meta-learning
Closest papers by meaning
- Multi-Source Collaborative Gradient Discrepancy Minimization for Federated Domain Generalization (AAAI 2024 · similarity 0.92)
- Federated Unsupervised Domain Generalization Using Global and Local Alignment of Gradients (AAAI 2025 · similarity 0.92)
- Benchmarking Algorithms for Federated Domain Generalization (ICLR 2024 · similarity 0.92)
- TTA-FedDG: Leveraging Test-Time Adaptation to Address Federated Domain Generalization (AAAI 2025 · similarity 0.90)
- Principled Federated Domain Adaptation: Gradient Projection and Auto-Weighting (ICLR 2024 · similarity 0.90)
- Efficiently Assemble Normalization Layers and Regularization for Federated Domain Generalization (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.