Research map / Federated learning and privacy / Differential privacy
Unleashing the Power of Randomization in Auditing Differentially Private ML
Krishna Pillutla, Galen Andrew, Peter Kairouz, Hugh Brendan McMahan, Alina Oprea, Sewoong Oh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| audit with K=16 random canaries instead of a single canary (data poisoning) | DP-SGD auditing · FMNIST/Linear, Purchase/MLP, eps 2-16 | sample complexity factor: 2.3x average | Improved |
| audit with K=16 random canaries instead of a single canary (random gradients) | DP-SGD auditing · FMNIST/Linear, FMNIST/MLP, Purchase/MLP | sample complexity factor: 3.0x average | Improved |
| LiDP auditing vs auditing DP across 12 settings per canary type | LiDP auditing · FMNIST, Purchase-100 | sample complexity factor: 11 out of the 12 settings | Mixed |
| inject K canaries with 2nd-order Wilson interval vs single canary | Gaussian mechanism auditing · Gaussian sum query | trials n to reach same eps lower bound: 4x to 16x | Improved |
| reach eps lower bound 0.85 | LiDP + 2nd-order Wilson · Gaussian sum query | required trials: 65536 -> 4096 | Improved |
| reach eps lower bound 0.67 | LiDP + 2nd-order Wilson · Gaussian sum query | required trials: 4096 -> 1024 | Improved |
| increase number of canaries K (ablation) | LiDP auditing with 2nd/4th-order CIs · Gaussian sum query | eps lower bound | Mixed |
| increase K (bias term) (ablation) | LiDP auditing · Gaussian sum query | Delta Bias | Hurt |
| increase K (variance term) (ablation) | LiDP auditing · Gaussian sum query | Delta Var | Improved |
| use 4th-order instead of 2nd-order interval (ablation) | LiDP auditing · Gaussian sum query, small eps | eps lower bound | No clear effect |
| increase dimension d (ablation) | LiDP auditing · Gaussian sum query | eps lower bound | Improved |
| 4th-order vs 2nd-order Wilson, FMNIST/Linear data poisoning eps=2 (ablation) | LiDP auditing · FMNIST/Linear | sample complexity factor: 0.68 -> 0.42 | Hurt |
Closest papers by meaning
- Auditing Private Prediction (ICML 2024 · similarity 0.90)
- How Well Can Differential Privacy Be Audited in One Run? (NeurIPS 2025 · similarity 0.88)
- Differentially Private Model Merging (NeurIPS 2026 · similarity 0.88)
- Auditing $f$-differential privacy in one run (ICML 2025 · similarity 0.88)
- Nearly Tight Black-Box Auditing of Differentially Private Machine Learning (NeurIPS 2024 · similarity 0.88)
- Optimal Differentially Private Model Training with Public Data (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
