Research map / Federated learning and privacy / Differential privacy

Privacy Guarantees in Posterior Sampling under Contamination

NeurIPS 2026 · Federated learning and privacy › Differential privacy · approach: Local-DP and private estimation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that replacing a random fraction of data points with heavy-tailed contamination (Huber model) makes Bayesian posterior sampling differentially private without bounded data or parameter spaces, with (epsilon, delta) rates and a simulation-based privacy estimator.

Problem

Differential privacy guarantees for Bayesian posterior sampling typically require bounded observation or parameter spaces; the paper seeks a general mechanism without these restrictions.

Approach

Privatise posterior sampling by pre-contaminating data: each observation is replaced with probability p_n by a draw from a heavy-tailed density, so the contaminated likelihood k_p = (1-p)f + p g bounds likelihood ratios; p_n -> 0 gives asymptotically vanishing epsilon and delta under a central DP view, with a random (high-probability) DP definition.

Key results

• Estimated epsilon for Bayesian contamination mechanism falls from 2.85 at n=100 to 0.18 at n=50,000 (delta_n=1/(10n)).
• For n>=1000 the Bayesian contamination approach achieves MSE comparable to the Clipped Mean Estimator; CoinPress performs best overall.
• Logistic regression contamination reduces estimated epsilon by roughly a factor of four versus no contamination.

Surprising findings

• Cauchy regression without contamination achieves lower estimated epsilon than with contamination, attributed to faster posterior contraction.
• Setting the failure probability p_n=0 does not imply (epsilon, delta)-DP uniformly, motivating the random DP definition.

Limitations

• Bounds are asymptotic and require very large n for constants to be realistic.
• Privacy level is not a closed-form function of contamination rate; estimated via simulation.
• Epsilon is hard to control for small n; higher-dimensional parameters need larger n.

Experimental setup

Datasets: simulated truncated normal data for private mean estimation simulated linear, logistic and Cauchy regression data

Benchmarks: simulated private mean estimation (theta*=30, sigma=8) simulated regression privacy estimates

Metrics: epsilon estimate MSE bias variance

Baselines: CoinPress Clipped Mean Estimator with private quantile estimation Laplace/Gaussian noise injection

Builds on: Huber's contamination model random differential privacy (Hall et al. style posterior DP) Hellinger bracketing entropy (Ghosal-van der Vaart posterior contraction) CoinPress

Closest papers by meaning

• On Inherent Privacy of Posterior Sampling: A Unified R\'enyi-Divergence Framework (NeurIPS 2026 · similarity 0.94)
• Tractable MCMC for Private Learning with Pure and Gaussian Differential Privacy (ICLR 2024 · similarity 0.90)
• Exactly Minimax-Optimal Locally Differentially Private Sampling (NeurIPS 2024 · similarity 0.89)
• Locally Optimal Private Sampling: Beyond the Global Minimax (NeurIPS 2025 · similarity 0.89)
• Prior-itizing Privacy: A Bayesian Approach to Setting the Privacy Budget in Differential Privacy (NeurIPS 2024 · similarity 0.88)
• Differential Privacy Guarantees of Markov Chain Monte Carlo Algorithms (ICML 2025 · similarity 0.88)