Research map / Federated learning and privacy / Privacy attacks and secure learning
Attack-Aware Noise Calibration for Differential Privacy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes calibrating DP-SGD noise directly to operational membership-inference risk (advantage or FPR/FNR sensitivity-specificity) via an f-DP accountant based on discrete PLRVs, instead of going through (epsilon, delta).
Problem
Standard DP noise calibration chooses a privacy budget epsilon and then maps it to attack risk, which is overly conservative and lowers utility; the paper asks how to calibrate noise directly to operational attack risk.
Approach
Use f-DP (trade-off curves) and discrete privacy loss random variables from dominating pairs to tightly compute attack advantage and FPR/FNR trade-off for composed mechanisms like DP-SGD, then binary-search the noise scale to meet a target advantage or minimum sensitivity at a given low FPR.
Key results
- Calibrating to FPR/FNR reduces required noise scale up to 2x relative to standard calibration for the same privacy risk.
- For GPT-2 on SST-2 with DP LoRA, FPR/FNR calibration yields an 18.3 percentage point accuracy gain at the same attack risk level.
- Advantage calibration can reduce noise up to 3.5x but can increase attack sensitivity at low FPR, a pitfall avoided by FPR/FNR calibration.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Calibrate DP-SGD noise directly to FPR/FNR risk instead of interpreting a fixed (ε, δ) guarantee | GPT-2 small with DP-LoRA · SST-2 | accuracy: +18.3 p.p. | Helped |
| Calibrate noise directly to attack advantage rather than using standard calibration | DP-SGD · DP-SGD simulations | noise scale: up to 3.5× reduction | Helped |
| Calibrate to attack advantage instead of standard (ε, δ) guarantees | DP mechanisms · Low-FPR membership inference | attack sensitivity: ≈30 p.p. | Hurt |
| Calibrate directly to an FPR/FNR constraint rather than attack advantage | DP mechanisms · Low-FPR membership inference | attack risk | Helped |
Surprising findings
- Advantage calibration, a common practice, can increase attack true-positive rate in the low-FPR regime compared with standard calibration.
- Standard conversion from (epsilon, delta) to attack risk substantially overestimates attack risk, leading to unnecessarily high noise.
Limitations
- Choice of target FPR and FNR is left open and needs legal/practical guidance.
- Pathological mechanisms can have catastrophic failures at FPR 0, so standard calibration is still advised for them.
- Advantage calibration can increase risk in the low-FPR regime.
Experimental setup
Datasets: SST-2 CIFAR-10
Benchmarks: SST-2 CIFAR-10
Metrics: task accuracy TPR at fixed FPR (attack sensitivity) advantage noise scale
Baselines: standard (epsilon, delta) calibration Gaussian differential privacy (1.1-GDP) approximation
Builds on: f-DP dp-sgd numerical accounting of privacy loss random variables (Google dp_accounting) Gaussian differential privacy
Closest papers by meaning
- Unifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacy (NeurIPS 2025 · similarity 0.90)
- Certification for Differentially Private Prediction in Gradient-Based Training (ICML 2025 · similarity 0.89)
- Noise-Aware Differentially Private Regression via Meta-Learning (NeurIPS 2024 · similarity 0.89)
- Beyond the Calibration Point: Mechanism Comparison in Differential Privacy (ICML 2024 · similarity 0.88)
- Privacy without Noisy Gradients: Slicing Mechanism for Generative Model Training (NeurIPS 2024 · similarity 0.88)
- Setting $\varepsilon$ is not the Issue in Differential Privacy (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.