Research map / Federated learning and privacy / Communication-efficient FL
Federated Wasserstein Distance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedWaD, a method to estimate the Wasserstein distance between distributions held on separate clients by iteratively exchanging interpolating measures (geodesic points) via a server, and applies it to federated coresets and OTDD-based clustering for FL.
Problem
Computing the Wasserstein distance between two distributions whose samples are stored on different clients without sharing the samples with a central server.
Approach
Uses the triangle-inequality equality along Wasserstein geodesics: the server iteratively estimates an interpolating measure xi* between mu and nu, with clients computing interpolating measures between their private data and the current iterate; a barycentric-map approximation keeps the support size fixed to S.
Key results
- On 2D Gaussians, the approximate FedWaD with support size 10 is at least 10x faster than exact computation for sample sizes above 100 and remains robust at large N, where exact FedWaD and exact WD show numerical errors for N>=1000 and N>=5000 respectively.
- Clustering clients with FedWaD-based OTDD distances improves FedRep by up to 9 points on CIFAR10 and 29 points on MNIST with strong cluster structure (Table: MNIST FedRep 20 clients affinity 99.1 vs vanilla 81.1).
- Federated coreset 1-NN classifier is competitive with and for many clients outperforms personalized FL methods FedRep and FedPer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Cluster clients using FedWaD OTDD distance and Affinity spectral clustering | FedAvg · MNIST, strong structure, 20 clients | accuracy: 26.3 $\pm$ 3.8 -> 99.5 $\pm$ 0.0 | Helped |
| Cluster clients using FedWaD OTDD distance and Affinity spectral clustering | FedAvg · MNIST, no structure, 20 clients | accuracy: 25.1 $\pm$ 6.6 -> 71.3 $\pm$ 7.3 | Helped |
| Cluster clients using FedWaD OTDD distance and Affinity spectral clustering | FedRep · MNIST, strong structure, 20 clients | accuracy: 81.1 $\pm$ 8.1 -> 99.1 $\pm$ 0.0 | Helped |
| Cluster clients using FedWaD OTDD distance and Affinity spectral clustering | FedRep · CIFAR10, no structure, 20 clients | accuracy: 85.3 $\pm$ 2.0 -> 90.7 $\pm$ 2.5 | Helped |
| Use Affinity spectral clustering rather than vanilla federated training | FedPer · MNIST, no structure, 40 clients | accuracy: 92.3 $\pm$ 1.3 -> 90.2 $\pm$ 4.7 | Hurt |
| Use Affinity spectral clustering rather than vanilla federated training | FedPer · MNIST, no structure, 100 clients | accuracy: 96.6 $\pm$ 0.9 -> 96.6 $\pm$ 1.6 | No clear effect |
| Approximate interpolating measures instead of using exact interpolating measures (ablation) | FedWaD · 2D Gaussian Wasserstein distance estimation | running time: at least 10 times faster | Helped |
| Increase the number of epochs (ablation) | FedWaD OTDD · MNIST–KMNIST distance | distance approximation | No clear effect |
| Increase interpolating-measure support points (ablation) | FedWaD OTDD · MNIST–KMNIST distance | distance approximation | Helped |
| Increase interpolating-measure support size (ablation) | Approximate FedWaD · 2D Gaussian Wasserstein distance estimation | relative error | Helped |
| Use approximate rather than exact Wasserstein distance for coreset learning (ablation) | MNIST and FashionMNIST coresets · Coreset loss | loss minimum | Hurt |
| Increase the number of epochs from 20 to 500 (ablation) | FedWaD OTDD · Pairwise distances among MNIST, KMNIST, USPS, and FashionMNIST | OTDD approximation | Helped |
Surprising findings
- Exact FedWaD running time for 1:3 sample ratio exceeds 1:1 due to non-permutation optimal plans enlarging interpolating supports.
- Coreset learning with FedWaD reaches worse minima than exact Wasserstein, and neither recovers all 10 modes when classes are spread heterogeneously across clients.
Limitations
- Convergence proof requires leaking client distributions at the limit (t=0 and t=1), so practical guarantees are weaker.
- Approximation error grows in high dimension (d=50), where FedWaD is not accurate.
- Coreset quality degrades as class diversity across clients increases; approximation yields worse minima than exact WD.
Experimental setup
Datasets: MNIST FashionMNIST KMNIST USPS CIFAR10
Benchmarks: MNIST CIFAR10 FashionMNIST KMNIST USPS
Metrics: relative error of Wasserstein distance running time classification accuracy relative error to OTDD
Baselines: exact Wasserstein distance OTDD FedAvg FedRep FedPer
Builds on: optimal transport dataset distance (OTDD) FedRep FedPer FedAvg
Closest papers by meaning
- Wasserstein Wormhole: Scalable Optimal Transport Distance with Transformer (ICML 2024 · similarity 0.86)
- Scalable Approximation Algorithms for $p$-Wasserstein Distance and Its Variants (ICML 2025 · similarity 0.85)
- Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning (CVPR 2025 · similarity 0.85)
- Data Valuation and Detections in Federated Learning (CVPR 2024 · similarity 0.84)
- A New Robust Partial p-Wasserstein-Based Metric for Comparing Distributions (ICML 2024 · similarity 0.84)
- OmniFC: Rethinking Federated Clustering via Lossless and Secure Distance Reconstruction (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.