Research map / Federated learning and privacy / Differential privacy
Provable Privacy with Non-Private Pre-Processing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a framework bounding the extra privacy cost of non-private data-dependent pre-processing (deduplication, quantization, imputation, PCA, scaling) in DP pipelines, using a new Smooth RDP notion and pre-processing sensitivity, plus a PTR-based unconditional variant.
Problem
Non-private data-dependent pre-processing (e.g., deduplication, imputation, PCA) introduces dependencies between records that break the independence assumption of DP, so the privacy cost of pre-processed DP pipelines is often overlooked.
Approach
Introduces Smooth RDP, a variant of RDP that bounds divergence between datasets with bounded L12 distance, and combines it with the L2 and L-infinity sensitivity of pre-processing functions to bound the overall RDP of a pre-processed DP pipeline, avoiding the polynomial blow-up of group privacy; a Propose-Test-Release wrapper gives unconditional guarantees.
Key results
- Pre-processing (deduplication, quantization, mean imputation, PCA, standard scaling) incurs only a constant-order privacy cost under SRDP analysis when sensitivities are small, versus polynomial growth with group privacy.
- For PCA rank reduction with DP-GD, the SRDP-based bound stays constant in n while the group-privacy bound grows with dataset size.
- On a synthetic low-rank dataset, non-private PCA followed by DP-SGD outperforms logistic regression without pre-processing at every epsilon, while DP-PCA performs worst.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use non-private PCA before DP-SGD instead of no pre-processing | Private logistic regression on the synthetic approximately low-rank dataset · Synthetic approximately low-rank dataset | excess empirical loss | Helped |
| Use DP-PCA instead of non-private PCA | Private logistic regression on the synthetic approximately low-rank dataset · Synthetic approximately low-rank dataset | excess empirical loss | Hurt |
| Use non-private PCA before DP-SGD instead of DP-PCA | Private logistic regression on the synthetic approximately low-rank dataset · Synthetic approximately low-rank dataset | excess empirical loss | Helped |
| Use non-private PCA before DP-SGD instead of no pre-processing | Private logistic regression on the synthetic approximately low-rank dataset · Synthetic approximately low-rank dataset | privacy cost | Hurt |
| Use SRDP-based analysis instead of group privacy analysis | Pre-processed Gaussian mechanism with PCA rank reduction · PCA rank reduction | privacy parameter | Helped |
Surprising findings
- Deduplication can have unbounded L-infinity sensitivity (equal to n) on pathological datasets, matching group privacy, even though it is typically cheap.
- Private PCA (DP-PCA) performs worst among the compared pipelines on the high-dimensional synthetic experiment, despite the non-private PCA route incurring a privacy cost.
Limitations
- Guarantees are vacuous or degrade to group privacy on pathological datasets (e.g., imputation with p near n, PCA with tiny eigengap, deduplication with a reference point).
- Conditional guarantees require knowledge of the dataset collection; the unconditional PTR version needs a lower bound on the eigengap and may refuse outputs.
- Analysis covers only a limited set of pre-processing methods and DP mechanisms, not pre-trained deep feature extractors or private data synthesis.
Experimental setup
Datasets: synthetic approximately low-rank 2-class dataset (1000 points, dim 6000)
Benchmarks: synthetic low-rank logistic regression
Metrics: excess empirical loss privacy parameter epsilon (RDP)
Baselines: group privacy analysis dependent differential privacy (DDP) DP-PCA
Builds on: Rényi differential privacy Propose-Test-Release DP-SGD group privacy
Closest papers by meaning
- Certification for Differentially Private Prediction in Gradient-Based Training (ICML 2025 · similarity 0.89)
- Purifying Approximate Differential Privacy with Randomized Post-processing (NeurIPS 2025 · similarity 0.89)
- Auditing Private Prediction (ICML 2024 · similarity 0.89)
- Differentially Private Model Merging (NeurIPS 2026 · similarity 0.88)
- The Limits of Differential Privacy in Online Learning (NeurIPS 2024 · similarity 0.88)
- Mitigating the Privacy–Utility Trade-off in Decentralized Federated Learning via f-Differential Privacy (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.