Research map / Federated learning and privacy / Personalized and heterogeneous FL
An Aggregation-Free Federated Learning for Tackling Data Heterogeneity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedAF, an aggregation-free federated learning method where clients learn condensed data with distribution matching plus a sliced-Wasserstein collaborative regularizer, and the server trains the global model on condensed data and averaged soft labels.
Problem
Federated learning under strong cross-client data heterogeneity suffers from client drift and catastrophic forgetting in the aggregate-then-adapt paradigm, and existing aggregation-free methods do not exploit other clients' knowledge to improve condensed data quality or global training.
Approach
Clients condense local data with a distribution-matching loss regularized by a sliced-Wasserstein collaborative term that aligns their local logit knowledge with the global class-wise mean logits; the server trains the global model directly on the received condensed data with a local-global soft-label KL matching loss, avoiding parameter aggregation.
Key results
- On CIFAR10 with alpha=0.02, FedAF reaches 65.15% accuracy vs 39.71% for FedAvg (up to 25.44% gain) and 60.28% for FedDM.
- FedAF reaches 60% accuracy on CIFAR10 (alpha=0.02) in 3 rounds versus 15 for FedDM, reported as an 80% convergence speedup.
- On DomainNet, FedAF averages 54.68% vs 52.62% for FedDM and 47.95% for FedDyn.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add collaborative data condensation (CDC, SWD regularization) (ablation) | FedAF · CIFAR10 α=0.02 | accuracy: +0.99 (65.15 vs 64.16) | Helped |
| Add local-global knowledge matching (LGKM) (ablation) | FedAF · CIFAR10 α=0.02 | accuracy: +1.03 (65.15 vs 64.12) | Helped |
| Add collaborative data condensation (CDC) (ablation) | FedAF · CIFAR10 α=0.05 | accuracy: +1.62 (67.50 vs 65.88) | Helped |
| Add local-global knowledge matching (LGKM) (ablation) | FedAF · CIFAR10 α=0.05 | accuracy: +1.23 (67.50 vs 66.27) | Helped |
| Add collaborative data condensation (CDC) (ablation) | FedAF · CIFAR10 α=0.1 | accuracy: +1.21 (69.11 vs 67.90) | Helped |
| Add local-global knowledge matching (LGKM) (ablation) | FedAF · CIFAR10 α=0.1 | accuracy: +0.97 (69.11 vs 68.14) | Helped |
| Increase IPC from 10 to 50 (and higher IPC) (ablation) | FedAF · CIFAR10 α=0.02 | accuracy: 53.39 at IPC=10 to 65.15 at IPC=50 to 69.14 at IPC=100 | Helped |
| Model re-sampling with gamma=0.9 vs no re-sampling (gamma=1.0) (ablation) | FedAF · CIFAR10 α=0.1 (10 rounds) | accuracy: 66.97 vs 64.92 | Helped |
| Use FedAF (aggregation-free) instead of FedAvg | global model training · CIFAR10 / CIFAR100 / FMNIST, α=0.02 to 0.1 | accuracy: up to 25.44% (CIFAR10), 17.91% (CIFAR100), 31.03% (FMNIST) over FedAvg | Helped |
| Use FedAF instead of FedDM (adds CDC and LGKM) | global model training · CIFAR10 / CIFAR100 / FMNIST, label-skew | accuracy: up to 4.87% (CIFAR10), 4.56% (CIFAR100), 2.17% (FMNIST) | Helped |
| Use FedAF instead of FedDM on DomainNet feature-skew | global model training · DomainNet (6 domains) | average accuracy: 54.68 vs 52.62 | Helped |
| Use FedAF instead of FedDyn (best aggregate-then-adapt baseline) | global model training · CIFAR10 α=0.02 | accuracy: 65.15 vs 45.73 | Helped |
Surprising findings
- Using only the global-model soft-label matching or only the collaborative condensation each already improves over FedDM, and the gains are not strongly tied to heterogeneity level in the IPC ablation.
Limitations
- Communication cost grows roughly linearly with images per class, and privacy depends on compression ratio.
- Accuracy is only reported within a small number of communication rounds (20 for label-skew, 10 for DomainNet).
- Evaluated on small benchmarks with a small number of clients (6 to 10).
Experimental setup
Datasets: FashionMNIST CIFAR10 CIFAR100 DomainNet
Benchmarks: FashionMNIST CIFAR10 CIFAR100 DomainNet
Metrics: global model accuracy convergence speed (communication rounds)
Baselines: FedAvg FedProx FedBN MOON FedDyn FedGen FedDM
Builds on: FedDM Dataset Condensation with Distribution Matching FedAvg FedGen
Closest papers by meaning
- Fake It Till Make It: Federated Learning with Consensus-Oriented Generation (ICLR 2024 · similarity 0.91)
- Federated Learning with Sample-level Client Drift Mitigation (AAAI 2025 · similarity 0.90)
- FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors (CVPR 2025 · similarity 0.90)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.89)
- FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-Aware Model Update (AAAI 2024 · similarity 0.89)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.