Research map / Federated learning and privacy / Differential privacy

On Inherent Privacy of Posterior Sampling: A Unified R\'enyi-Divergence Framework

NeurIPS 2026 · Federated learning and privacy › Differential privacy · approach: Local-DP and private estimation

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Derives an exact Rényi-divergence characterization of the inherent privacy of One-Posterior-Sampling (OPS) and explicit RDP bounds under bounded, sub-Gaussian, and sub-exponential log-likelihood ratio regimes across categorical, GLM, and Gaussian models.

Problem

Quantifying the differential privacy guarantees that come "for free" from drawing a single sample from a Bayesian posterior (One-Posterior-Sampling), under weaker assumptions than prior uniform-boundedness analyses.

Approach

Shows the Rényi divergence between neighboring posteriors is exactly determined by two exponential moments of the pointwise log-likelihood ratio under the posterior (Theorem 3), then bounds these moments under bounded, sub-Gaussian, or sub-exponential tail conditions, and applies this to unbounded and bounded neighboring relations.

Key results

• For categorical likelihood with eta=0.1 bounded neighboring, Corollary 5 yields (2, ~3.033)-RDP versus pure DP ~4.394.
• For GLMs with Gaussian prior, OPS satisfies (alpha, 2 alpha L^2 tau^2)-RDP, with bounded neighboring worst-case 4x that of unbounded.
• Under unbounded neighboring for Gaussian-Gaussian conjugate pair, the reverse Rényi divergence is infinite for alpha >= omega = n + sigma^2/tau^2, so OPS is not pure DP.

Surprising findings

• Under unbounded neighboring, the two directions of the Rényi divergence have different finite ranges of orders alpha.
• Label replacement in logistic regression yields a 75% smaller bound than covariate replacement because log-partition terms cancel.

Limitations

• Bounds for GLMs and linear regression do not depend explicitly on sample size n due to uniform worst-case score bounds.
• Sub-exponential parameters derived for specific models are loose, e.g. for linear regression with covariate replacement.

Experimental setup

Metrics: Rényi divergence bound pure DP epsilon

Baselines: Dimitrakakis et al. 2014 Dimitrakakis et al. 2017 Wang et al. 2015 Geumlek et al. 2017

Builds on: Rényi differential privacy Geumlek et al. 2017 OPS Rényi analysis Dimitrakakis et al. 2014 posterior sampling DP Gopi et al. 2022 strongly log-concave Lipschitz perturbation

Closest papers by meaning

• Privacy Guarantees in Posterior Sampling under Contamination (NeurIPS 2026 · similarity 0.94)
• Tractable MCMC for Private Learning with Pure and Gaussian Differential Privacy (ICLR 2024 · similarity 0.90)
• Locally Optimal Private Sampling: Beyond the Global Minimax (NeurIPS 2025 · similarity 0.89)
• Exactly Minimax-Optimal Locally Differentially Private Sampling (NeurIPS 2024 · similarity 0.89)
• Private Statistical Estimation via Truncation (NeurIPS 2025 · similarity 0.88)
• On the Computational Complexity of Private High-dimensional Model Selection (NeurIPS 2024 · similarity 0.88)