Research map / Federated learning and privacy / Differential privacy
One-shot Empirical Privacy Estimation for Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a one-shot empirical privacy estimator that inserts random unit-sphere canary clients into a single DP-FedAvg training run and estimates epsilon from the cosine between each canary and the final model.
Problem
Empirically auditing user-level differential privacy of federated learning training without retraining many times, without knowledge of task/architecture/data, and under realistic (final-model-only) adversary threat models.
Approach
Insert k random canary clients whose updates are isotropic unit-sphere vectors; in high dimensions the cosine between each canary and the final model is approximately N(0,1/d) if unobserved and a shifted Gaussian if observed, so fitting a Gaussian to the observed cosines and computing epsilon between two Gaussians yields a one-shot estimate. Provably converges to the true Gaussian-mechanism epsilo
Key results
- With final model only and clipping only, epsilon_est on StackOverflow is 4.60 versus analytical infinity; with noise multiplier 0.2317 (analytical epsilon 30), epsilon_est-final is 0.569 while epsilon_est-all is 2.693.
- Gaussian-mechanism one-shot estimates for d=10^4 to 10^7 closely match analytical epsilon (e.g., 10.0 +/- 0.10 at d=10^7 for epsilon=10).
- Adding 1000 canary clients to 341k StackOverflow clients reduces accuracy by at most 0.1 percentage points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add 1k random canary clients | Stackoverflow DP-FedAvg · Stackoverflow word prediction | accuracy: 25.3% -> 25.3%; 24.0% -> 23.9%; 23.1% -> 23.1%; 21.5% -> 21.5%; 20.6% -> 20.5% | No clear effect |
| Restrict the adversary from all model updates to the final model only | One-shot canary estimation on Stackoverflow · Stackoverflow | ε estimate: 45800 -> 4.60; 382 -> 1.97; 89.4 -> 1.18; 2.693 -> 0.569 | Helped |
| Restrict the adversary from all model updates to the final model only | One-shot canary estimation on EMNIST · EMNIST | ε estimate: 48300 -> 5.72; 17.9 -> 1.20 | Helped |
| Increase the number of canary presentations (ablation) | One-shot canary estimation · Stackoverflow and EMNIST | ε estimate | Hurt |
| Use CANIFE instead of one-shot random canaries | Fashion-MNIST privacy estimation · Fashion-MNIST | ε estimate: 0.88 ± 0.12 vs. 6.8 ± 1.1; lower bound 1.82 ± 0.46 | Mixed |
Surprising findings
- Estimates of epsilon from final-model-only observation are far lower than from all intermediate iterates, suggesting a small amount of noise suffices against final-model attacks.
- On a Fashion MNIST comparison, CANIFE's estimate (0.88) falls below the empirical lower bound (1.82), while the proposed method gives 6.8.
Limitations
- Estimate is not a formal upper bound on worst-case privacy leakage; it reflects one specific powerful attack.
- Accuracy of Gaussian approximation and estimate relies on high dimensionality (d around 1M or more); noisier for small models.
- Gaussianity of the joint cosine statistics is proven only asymptotically and checked empirically.
Experimental setup
Datasets: StackOverflow EMNIST Fashion MNIST
Benchmarks: StackOverflow next-word prediction EMNIST character recognition Fashion MNIST
Metrics: epsilon estimate (epsilon_est) 95% high-probability lower bound (epsilon_lo) word prediction accuracy Anderson-Darling test statistic
Baselines: CANIFE Jagielski et al. (2020) lower bound
Builds on: DP-FedAvg Jagielski et al. auditing lower bound privacy loss distributions CANIFE
Closest papers by meaning
- Auditing $f$-differential privacy in one run (ICML 2025 · similarity 0.90)
- Auditing Private Prediction (ICML 2024 · similarity 0.89)
- Mitigating the Privacy–Utility Trade-off in Decentralized Federated Learning via f-Differential Privacy (NeurIPS 2025 · similarity 0.89)
- Towards the Robustness of Differentially Private Federated Learning (AAAI 2024 · similarity 0.89)
- Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning (ICML 2024 · similarity 0.88)
- Nearly Tight Black-Box Auditing of Differentially Private Machine Learning (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.