Research map / Federated learning and privacy / Differential privacy
How Private are DP-SGD Implementations?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that DP-SGD privacy under shuffling batch sampling can be much weaker than the standard Poisson-subsampling accounting, via dominating-pair analysis of the ABLQ mechanism with lower bounds for shuffling.
Problem
Privacy accounting for DP-SGD is typically done assuming Poisson subsampling, but practical implementations use shuffling; the privacy gap between these batch samplers is unclear.
Approach
Compares the privacy loss curves of adaptive batch linear query mechanisms under deterministic, Poisson, and shuffle batch samplers using dominating pairs; shows shuffling always beats deterministic batching but can be far weaker than Poisson subsampling, with explicit lower bounds on shuffle-based privacy loss.
Key results
- For sigma=0.5, T=10,000, delta=1e-6, Poisson accounting gives epsilon < 1.96 (PLD) while the shuffle lower bound exceeds 10.994, close to deterministic 10.997.
- For sigma=0.4, T=10,000, delta_P(4) <= 1.18e-5 whereas delta_S(4) >= 0.226, close to delta_D(4) ~ 0.244.
- Shows delta_D and delta_P are incomparable: delta_P(eps) < delta_D(eps) for small eps but delta_P(eps) >> delta_D(eps) for large eps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Poisson batch sampling with shuffle batch sampling | ABLQ · δ = 10^{-6}, T = 10,000, σ = 0.5 | ε: < 1.96 (Poisson PLD) vs > 10.994 (Shuffle) | Hurt |
| Replace Poisson batch sampling with shuffle batch sampling | ABLQ · δ = 10^{-6}, T = 10,000, σ = 1.3 | ε: < 0.031 (Poisson PLD) vs > 0.26 (Shuffle) | Hurt |
| Replace Poisson batch sampling with shuffle batch sampling | ABLQ · σ = 0.4, T = 10,000, ε = 4 | δ: ≤ 1.18 · 10^{-5} vs ≥ 0.226 | Hurt |
| Replace deterministic batching with shuffle batch sampling | ABLQ · All σ, ε ≥ 0 and T ≥ 1 | δ | Helped |
| Replace deterministic batching with Poisson batch sampling | ABLQ · Small ε and sufficiently large ε, for σ > 0 and T > 1 | δ | Mixed |
| Replace Poisson batch sampling with shuffle batch sampling | ABLQ · δ = 10^{-6}, T = 100,000, σ = 0.4 | ε: ≤ 3 (Poisson PLD) vs ≥ 14.45 (Shuffle) | Hurt |
| Replace Poisson batch sampling with shuffle batch sampling | ABLQ · δ = 10^{-5}, T = 1,000, σ = 0.7 | ε: ≤ 0.61 (Poisson PLD) vs ≥ 6.528 (Shuffle) | Hurt |
| Replace Poisson batch sampling with shuffle batch sampling | ABLQ · σ = 0.8, T = 1,000, ε = 1 | δ: ≤ 9.873 · 10^{-9} (Poisson PLD) vs ≥ 0.018 (Shuffle) | Hurt |
Surprising findings
- Poisson subsampling can give a worse privacy guarantee than deterministic batching at large epsilon.
- Shuffle batch sampling provides very little amplification over deterministic batching for small noise multipliers, so Poisson-based reported epsilon can badly underestimate privacy loss.
Limitations
- Lower bound only for shuffle sampler; conjectured tight dominating pair is unproven.
- Analysis limited to a single epoch / single pass mechanism.
- Model of shuffling is simplified and does not capture practical loaders such as tf.data shuffle buffers.
Experimental setup
Metrics: epsilon delta hockey stick divergence
Baselines: Poisson subsampling accounting (RDP) Poisson subsampling accounting (PLD) deterministic batching
Builds on: DP-SGD Renyi DP privacy loss distributions dominating pairs
Closest papers by meaning
- Scalable DP-SGD: Shuffling vs. Poisson Subsampling (NeurIPS 2024 · similarity 0.97)
- Differentially Private Stochastic Gradient Descent with Fixed-Size Minibatches: Tighter RDP Guarantees with or without Replacement (NeurIPS 2024 · similarity 0.89)
- The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD (ICLR 2025 · similarity 0.88)
- Proactive DP: A Multiple Target Optimization Framework for DP-SGD (ICML 2024 · similarity 0.87)
- Privacy Profiles for Private Selection (ICML 2024 · similarity 0.87)
- A Generalized Shuffle Framework for Privacy Amplification: Strengthening Privacy Guarantees and Enhancing Utility (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.