Research map / Federated learning and privacy / Personalized and heterogeneous FL
Federated Unsupervised Domain Generalization Using Global and Local Alignment of Gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedGaLA, an unsupervised federated domain generalization method that combines client-level gradient-alignment filtering of batch gradients with server-level cosine-weighted aggregation on top of federated SimCLR.
Problem
Federated domain generalization when clients hold only unlabeled data from different domains, so the global model must generalize to unseen target domains without labels or data sharing.
Approach
Discards local batch gradients whose cosine similarity with the global update direction is below a threshold (client level) and weights client updates at the server by their cosine alignment with the average update (global level), with a theoretical link between domain shift and gradient covariance.
Key results
- FedGaLA reaches 60.0% average linear-eval accuracy on PACS with 10% labels versus 58.6% for FedSimCLR, the strongest federated baseline.
- On TerraInc with 10% labels FedGaLA averages 56.7% vs 55.1% for FedSimCLR; on Office-Home 23.0% vs 22.0%.
- Ablation on PACS: removing both global and local alignment drops average accuracy from 60.0% to 58.6%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add global gradient alignment (ablation) | FedGaLA · PACS, 10% labeled | accuracy: 58.8 -> 60.0 | Helped |
| add local gradient alignment (ablation) | FedGaLA · PACS, 10% labeled | accuracy: 58.5 -> 60.0 | Helped |
| add global and local gradient alignment (ablation) | FedSimCLR · PACS, 10% labeled | accuracy: 58.6 -> 60.0 | Helped |
| apply FedGaLA instead of FedSimCLR | federated unsupervised domain generalization · PACS, 10% labeled | accuracy: 58.6 -> 60.0 | Helped |
| apply FedGaLA instead of FedSimCLR | federated unsupervised domain generalization · DomainNet, 10% labeled | accuracy: 39.5 -> 41.1 | Helped |
| apply FedGaLA instead of FedSimCLR | federated unsupervised domain generalization · Office-Home, 30% labeled | accuracy: 30.7 -> 30.9 | No clear effect |
| replace SimCLR with MoCo (ablation) | FedGaLA · PACS, 10% labeled | accuracy: 60.0 -> 40.7 | Hurt |
| replace SimCLR with BYOL (ablation) | FedGaLA · PACS, 10% labeled | accuracy: 60.0 -> 44.6 | Hurt |
| increase local epochs per communication round from E=1 to E=9 (ablation) | FedGaLA · PACS | accuracy: 67.8 -> 64.7 | Hurt |
| replace gradient discarding with re-weighting (ablation) | FedGaLA · PACS, 10% labeled | accuracy: 58.1 -> 60.0 | Hurt |
| add L2 regularization λ=0.001 (ablation) | FedGaLA · PACS, 10% labeled | accuracy: 60.0 -> 58.4 | Hurt |
| add FedProx μ=0.001 (ablation) | FedGaLA · TerraInc, 10% labeled | accuracy: 56.7 -> 57.9 | Helped |
Surprising findings
- Performance gains over baselines shrink or vanish at 30% label ratio, where some baselines are competitive on individual domains.
- Removing only global alignment (58.8%) helps less than the reported full method, while the discard-based local alignment outperforms soft re-weighting of unaligned gradients.
Limitations
- Theoretical analysis relies on strong simplifying assumptions (Gaussian, independent features, one-layer sigmoid encoder).
- Only cross-silo setting with a small number of domains/clients is studied.
- Gains over FedSimCLR are often within reported standard deviations.
Experimental setup
Datasets: PACS Office-Home DomainNet TerraInc
Benchmarks: PACS Office-Home DomainNet TerraInc
Metrics: linear evaluation accuracy
Baselines: FedEMA FedBYOL FedMoCo FedSimSiam FedSimCLR SimCLR MoCo BYOL AdCo DARLING
Builds on: SimCLR FedAVG FedProx FedEMA
Closest papers by meaning
- Federated Domain Generalization with Data-free On-server Matching Gradient (ICLR 2025 · similarity 0.92)
- Multi-Source Collaborative Gradient Discrepancy Minimization for Federated Domain Generalization (AAAI 2024 · similarity 0.90)
- Benchmarking Algorithms for Federated Domain Generalization (ICLR 2024 · similarity 0.87)
- Efficiently Assemble Normalization Layers and Regularization for Federated Domain Generalization (CVPR 2024 · similarity 0.87)
- TTA-FedDG: Leveraging Test-Time Adaptation to Address Federated Domain Generalization (AAAI 2025 · similarity 0.87)
- Principled Federated Domain Adaptation: Gradient Projection and Auto-Weighting (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.