Research map / Federated learning and privacy / Differential privacy
Wasserstein Differential Privacy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Wasserstein differential privacy (WDP), a DP variant defined via the Wasserstein distance between output distributions on adjacent datasets, with a Wasserstein accountant for subsampled DP-SGD.
Problem
Existing DP frameworks are divergence-based, so their privacy losses are not metrics (asymmetric, no triangle inequality), which leads to overestimated privacy budgets and composition issues.
Approach
Define the privacy budget as the upper bound of the Wasserstein distance between output distributions of a mechanism on adjacent datasets, yielding a metric-satisfying privacy loss; derive composition, group privacy, conversions to DP/RDP, and a subsampled-Gaussian accountant (Wasserstein accountant) for SGD.
Key results
- On MNIST, WDP (delta=1e-10) reaches epsilon 0.76 versus 2.2 for DP (delta=1e-5) and 0.95 for BDP at the same ~96% accuracy target.
- WDP reaches ~96% MNIST accuracy after about 16 epochs, versus more than 100 epochs for DP and more than 50 for BDP (as reported).
- Privacy budgets under WDP are reported to be less sensitive to the order mu than RDP, and to not explode under low noise.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use WDP instead of DP and BDP | CNN trained with DP-SGD · MNIST | privacy budget: 2.2 (0.898) -> 0.76 (0.681); 0.95 (0.721) -> 0.76 (0.681) | Helped |
| Use WDP instead of DP and BDP | CNN trained with DP-SGD · CIFAR-10 | privacy budget: 8.0 (0.999) -> 0.52 (0.627); 0.76 (0.681) -> 0.52 (0.627) | Helped |
| Use WDP instead of DP and BDP | CNN trained with DP-SGD · SVHN | privacy budget: 5.0 (0.999) -> 0.40 (0.599); 0.87 (0.705) -> 0.40 (0.599) | Helped |
| Use WDP instead of DP and BDP | CNN trained with DP-SGD · Fashion-MNIST | privacy budget: 2.9 (0.623) -> 0.45 (0.611); 0.91 (0.713) -> 0.45 (0.611) | Helped |
| Use Wasserstein accountant instead of moments accountant or Bayesian accountant | Privacy accounting over synthetic gradients without gradient clipping · Synthetic gradients | privacy budget | Helped |
| Use Wasserstein accountant instead of moments accountant or Bayesian accountant | CNN training with DP-SGD · MNIST | epochs to reach 96% accuracy: more than 100 epochs; 50 epochs; 16 epochs | Helped |
| Increase beta | WDP privacy accountant · Privacy accounting experiment | privacy budget | Helped |
| Increase delta | WDP privacy accountant · Privacy accounting experiment | privacy budget | Helped |
Surprising findings
- Reported WDP budgets are lower than DP and BDP even though the WDP framework is an alternative definition, with the authors noting WDP may release much smaller budgets when the dataset is very small.
Limitations
- Authors acknowledge WDP may release much smaller privacy budget when the queried dataset is very small.
- Wasserstein distance is computationally expensive, which the authors argue is avoided but is not directly measured.
- Some proofs appear to rely on loose steps (e.g., Pinsker and total variation bounds, equating TV with Wasserstein with unit cost), and the comparison with DP-SGD depends on noise scales chosen differently per framework.
Experimental setup
Datasets: MNIST CIFAR-10 SVHN Fashion-MNIST synthetic Weibull gradients
Benchmarks: MNIST CIFAR-10 SVHN Fashion-MNIST
Metrics: privacy budget epsilon test accuracy probability of attack success
Baselines: moments accountant Bayesian accountant DP RDP BDP
Builds on: differential privacy Renyi differential privacy Bayesian differential privacy DP-SGD moments accountant
Closest papers by meaning
- Beyond the Calibration Point: Mechanism Comparison in Differential Privacy (ICML 2024 · similarity 0.88)
- 𝑓-Differential Privacy Filters: Validity and Approximate Solutions (NeurIPS 2026 · similarity 0.88)
- Provable Privacy with Non-Private Pre-Processing (ICML 2024 · similarity 0.88)
- The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD (ICLR 2025 · similarity 0.87)
- Shifted Interpolation for Differential Privacy (ICML 2024 · similarity 0.87)
- Auditing $f$-differential privacy in one run (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.