Research map / Federated learning and privacy / Privacy attacks and secure learning

A Unified Framework for Adversary-Aware Differential Privacy Bounds

NeurIPS 2026 · Federated learning and privacy › Privacy attacks and secure learning · approach: Membership inference and auditing · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a unified, adversary-aware upper bound on attack success against DP mechanisms, depending only on privacy parameters and the adversary's prior success rate, covering multi-target, non-uniform-prior, and approximate-match attacks, and evaluates it on DP-finetuned GPT-2 canary extraction and noisy-marginal tabular reconstruction.

Problem

Worst-case membership inference bounds from DP give little guidance on the actual risk from more complex adversaries (attribute inference, reconstruction, multi-target attacks, non-uniform priors, approximate success metrics), and empirical attacks only give lower bounds.

Approach

A generic attack game with product priors and decomposable success metrics yields a high-probability bound on attack success as a sum of Bernoulli variables with flip probabilities that depend only on the DP parameter and the adversary's prior success probability per target; extended to approximate and f-DP via a linear-program/dual argument and worst-case-prior variants.

Key results

• For uniform 9-digit canaries, the bound aligns almost perfectly with the idealized Gaussian Bayes-optimal attack, and the framework is shown tight for randomized response with priors near uniform.
• At epsilon=1, the advantage bound for the median numerical password is far below 0.05 but reaches 0.24 for the most probable password.
• To keep worst-case canary advantage at or below 0.05, epsilon_protect is 17.8 for uniform 9-digit canaries, 0.25 for numerical passwords, and 2.37 for PIIs.

Surprising findings

• Non-uniform (natural) secrets such as common passwords may require far smaller epsilon (0.25) to protect than uniform canaries (17.8), so large-epsilon guidance from uniform canary experiments underestimates risk.
• The black-box model attack falls well short of the Bayes-optimal idealized attack at medium privacy, suggesting empirical attacks substantially underestimate leakage.

Limitations

• Bounds rely on decomposable success metrics; non-decomposable metrics are left open.
• Assumes the adversary knows the exact prior; robustness to prior approximation is future work.
• Gap between bound and Bayes-optimal attack remains for highly skewed (non-uniform) priors.

Experimental setup

Datasets: Enron emails Secret Sharer canaries (synthetic) numerical password prior (Zipf-based) noisy k-way marginals tabular data

Benchmarks: Secret Sharer canary extraction on DP-finetuned GPT-2 idealized Gaussian Bayes-optimal attack noisy marginals multi-attribute inference

Metrics: attack success probability generalized advantage epsilon_protect at advantage 0.05

Baselines: KGK+ MMC Reconstruction Robustness Model Attack (black-box)

Builds on: Steinke, Nasr, Jagielski (tight privacy auditing bounds) f-DP Secret Sharer KGK+ bounds MMC bound Reconstruction Robustness

Closest papers by meaning

• Unifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacy (NeurIPS 2025 · similarity 0.91)
• Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy (NeurIPS 2026 · similarity 0.89)
• The Limits of Differential Privacy in Online Learning (NeurIPS 2024 · similarity 0.88)
• 𝑓-Differential Privacy Filters: Validity and Approximate Solutions (NeurIPS 2026 · similarity 0.88)
• Tighter Privacy Auditing of DP-SGD in the Hidden State Threat Model (ICLR 2025 · similarity 0.88)
• Assessing Per-Sample Membership Inference Vulnerability without Retraining (NeurIPS 2026 · similarity 0.88)