Research map / Federated learning and privacy / Differential privacy
Optimizing Canaries for Privacy Auditing with Metagradient Descent
Matteo Boglioni, Terrance Liu, Andrew Ilyas, Steven Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use metagradient canaries instead of random canaries | DP-SGD auditing, procedure B · MNIST DP training, eps=8 | empirical epsilon lower bound: 0.016 -> 0.669 | Improved |
| use metagradient canaries instead of random canaries | DP-SGD auditing, procedure B · Fashion-MNIST DP finetuning, eps=8 | empirical epsilon lower bound: 0.056 -> 1.483 | Improved |
| use metagradient canaries instead of random canaries | DP-SGD auditing, procedure B · CIFAR-10 DP training, eps=8 | empirical epsilon lower bound: 0.405 -> 0.732 | Improved |
| use metagradient canaries instead of random canaries | DP-SGD auditing, procedure B · CIFAR-100 DP finetuning, eps=8 | empirical epsilon lower bound: 0.354 -> 1.286 | Improved |
| use metagradient canaries instead of random canaries | DP-SGD auditing, procedure A · CIFAR-100 DP finetuning, eps=8 | empirical epsilon lower bound: 0.342 -> 0.770 | Improved |
| use metagradient canaries instead of random canaries | DP-SGD auditing, procedure A · CIFAR-10 DP training, eps=8 | empirical epsilon lower bound: 0.285 -> 0.392 | Improved |
| use metagradient canaries instead of random canaries at strict privacy eps=1 | DP-SGD auditing, procedure B · CIFAR-10 DP training, eps=1 | empirical epsilon lower bound: 0.030 -> 0.000 | No clear effect |
| use metagradient canaries instead of random canaries at strict privacy eps=1 | DP-SGD auditing, procedure A · CIFAR-100 DP training, eps=1 | empirical epsilon lower bound: 0.075 -> 0.074 | No clear effect |
| use procedure B instead of procedure A with metagradient canaries | metagradient canaries · CIFAR-10 DP training, eps=8 | empirical epsilon lower bound: 0.392 -> 0.732 | Improved |
| add DP finetuning from pretrained model instead of training from scratch (ablation) | metagradient canaries, procedure B · CIFAR-10, eps=8 | empirical epsilon lower bound: 0.732 -> 1.207 | Improved |
| use ClipBKD canaries instead of metagradient canaries | one-run auditing · CIFAR-10 | empirical epsilon lower bound | Hurt |
| use Gradient Max canaries instead of metagradient canaries | one-run auditing · CIFAR-10 vs MNIST | empirical epsilon lower bound | Mixed |
Closest papers by meaning
- Nearly Tight Black-Box Auditing of Differentially Private Machine Learning (NeurIPS 2024 · similarity 0.91)
- Auditing Private Prediction (ICML 2024 · similarity 0.89)
- Tighter Privacy Auditing of DP-SGD in the Hidden State Threat Model (ICLR 2025 · similarity 0.88)
- The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD (ICLR 2025 · similarity 0.88)
- Differentially Private Stochastic Gradient Descent with Fixed-Size Minibatches: Tighter RDP Guarantees with or without Replacement (NeurIPS 2024 · similarity 0.88)
- Privacy Auditing of Large Language Models (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
