Research map / Federated learning and privacy / Personalized and heterogeneous FL
Principled Federated Domain Adaptation: Gradient Projection and Auto-Weighting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a theoretical framework (Delta error, domain distance, target variance) for federated domain adaptation aggregation rules, plus FedGP (projection-filtered source gradients combined convexly with target gradient) and an auto-weighting scheme for beta, evaluated on semi-synthetic and DomainBed datasets.
Problem
Federated domain adaptation where a target client has limited labeled data and suffers from domain shift from source clients; existing personalized FL and UFDA methods assume abundant target data.
Approach
Analyzes server aggregation rules via a functional L^pi gradient error (Delta error) that decomposes into target variance and source-target gradient distance; proposes FedGP, which convexly combines the target gradient with its positive projection onto source gradients, and derives closed-form auto-weights beta minimizing the Delta error.
Key results
- On ColoredMNIST, FedGP_Auto reaches 84.1 average target accuracy versus 52.9 for FedDA (fixed beta) and 49.0 for Source Only.
- On VLCS, FedGP_Auto and FedDA_Auto both reach 83.8 average accuracy, above KD3A (79.0, using 100% unlabeled target data).
- FedGP with fixed beta=0.5 is reported as competitive and robust to beta, while FedDA benefits substantially from auto-weighting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add auto-weighting to FedDA (ablation) | FedDA · ColoredMNIST (0.1%), Avg | accuracy (%): 52.9 -> 82.7 | Helped |
| Add auto-weighting to FedDA (ablation) | FedDA · VLCS (5%), Avg | accuracy (%): 79.5 -> 83.8 | Helped |
| Add auto-weighting to FedDA (ablation) | FedDA · TerraIncognita (5%), Avg | accuracy (%): 64.7 -> 74.6 | Helped |
| Add gradient projection and filtering instead of direct convex gradient combination (ablation) | FedDA · Fashion-MNIST and CIFAR-10 noisy-feature experiments | performance: +15% | Helped |
| Add filtering after gradient projection (ablation) | FedGP · Fashion-MNIST and CIFAR-10 noisy-feature experiments | performance: 1%-2% | Helped |
| Vary the source-target balancing weight β (ablation) | FedDA and FedGP · Fashion-MNIST, CIFAR-10, and ColoredMNIST | accuracy | Mixed |
| Use FedGP rather than FedDA with fixed β=0.5 | FedDA and FedGP · ColoredMNIST (0.1%), Avg | accuracy (%): 52.9 -> 82.4 | Helped |
| Use FedGP rather than FedDA with fixed β=0.5 | FedDA and FedGP · VLCS (5%), Avg | accuracy (%): 79.5 -> 80.7 | Helped |
| Use FedGP rather than FedDA with fixed β=0.5 | FedDA and FedGP · TerraIncognita (5%), Avg | accuracy (%): 64.7 -> 71.2 | Helped |
Surprising findings
- Even noisy gradients computed from very limited target data provide a valuable signal for aggregation.
- Gradient projection with filtering of negative source directions outperforms plain convex combination (FedDA) even at the same beta.
Limitations
- Auto-weighting relies on estimators of domain distance and target variance from intermediate local updates, with approximations (mean-field) in the FedGP Delta error.
- Only one target client and a single target domain are considered; extension to multiple simultaneous target clients is left open.
Experimental setup
Datasets: Fashion-MNIST CIFAR-10 ColoredMNIST VLCS TerraIncognita DomainNet
Benchmarks: ColoredMNIST VLCS TerraIncognita Fashion-MNIST noisy features CIFAR-10 noisy features
Metrics: target domain test accuracy
Baselines: FedAvg Ditto FedRep APFL KNN-per KD3A Best DG Finetune_Offline Source Only Target Only
Builds on: FedAvg Byzantine-robust gradient filtering DomainBed
Closest papers by meaning
- Federated Domain Generalization with Data-free On-server Matching Gradient (ICLR 2025 · similarity 0.90)
- Enhancing Federated Domain Adaptation with Multi-Domain Prototype-Based Federated Fine-Tuning (ICLR 2025 · similarity 0.90)
- FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors (CVPR 2025 · similarity 0.89)
- Adaptive Federated Learning with Auto-Tuned Clients (ICLR 2024 · similarity 0.89)
- Multi-Source Collaborative Gradient Discrepancy Minimization for Federated Domain Generalization (AAAI 2024 · similarity 0.88)
- Benchmarking Algorithms for Federated Domain Generalization (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.