Research map / Federated learning and privacy / Differential privacy
Privacy without Noisy Gradients: Slicing Mechanism for Generative Model Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a slicing privacy mechanism that adds noise to random k-dimensional projections of private data, plus a smoothed-sliced f-divergence with a kernel moment-matching estimator, to train DP generative models without noisy gradients or adversarial training.
Problem
Training differentially private generative models typically requires noisy gradient updates or adapted discriminators, which make hyper-parameter tuning, epoch selection and convergence difficult.
Approach
Decouple training into a one-time noisy random-projection release (k-slicing privacy mechanism, with the projection matrix released) and non-adversarial generator training that matches noisy slices via a smoothed-sliced f-divergence estimated with a kernel density-ratio moment-matching estimator; post-processing lets any architecture, epochs or restarts be used without extra privacy cost. Gives an
Key results
- At epsilon=5.1 on Income, the proposed method scores 0.70 TVComplement and 0.96 CorrSim versus SliceWass 0.61 and 0.93, and DP-SGD 0.38 and 0.75.
- On Coverage (epsilon=5.1) the proposed method reaches 0.74 KSComplement and 0.87 TVComplement, outperforming all baselines.
- On MNIST, MERF does better at low privacy budgets but the slicing-based methods outperform MERF at higher budgets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Algorithm instead of SliceWass (higher-dimensional slices rather than 1-dimensional slices) | Privacy-preserving synthetic tabular data generation · Income, Coverage, Mobility, Employment, and TravelTime (ε=5.1) | Synthetic data quality metrics | Mixed |
| Use Algorithm instead of DP-SGD, PATE, or MERF | Privacy-preserving synthetic tabular data generation · US Census ACS tasks (ε=5.1) | Synthetic data quality metrics | Helped |
| Increase projection dimension from k=1 to k=2 or 3 | Slicing-based generative models · Synthetic data generation | Synthetic data quality | Helped |
| Use Algorithm instead of SliceWass with 100 slices | Domain adaptation classifier · MNIST↔USPS | Domain adaptation performance | Helped |
| Use slicing-based Algorithm or SliceWass rather than MERF | DP synthetic image generation · MNIST, higher privacy budgets | Downstream classifier accuracy | Helped |
| Use MERF instead of slicing-based algorithms | DP synthetic image generation · MNIST, lower privacy budgets | Downstream classifier accuracy | Hurt |
Surprising findings
- MERF outperforms the slicing-based methods at low privacy budgets on MNIST.
- Using k=1 slices (SliceWass) underperforms the proposed higher-dimensional slicing in most settings, so k>1 helps despite 1D sufficing theoretically.
Limitations
- Privacy guarantees depend on subsampling amplification and the bound is vacuous without it in the asymptotic consistency regime.
- Only 1-dimensional-slice analysis is compared against SliceWass; hyperparameters were not extensively tuned.
- Sample-complexity bounds for the smoothed-sliced f-divergence estimator are not derived.
Experimental setup
Datasets: US Census ACS PUMS 2018 California (Folktables: Income, Coverage, Mobility, Employment, TravelTime) MNIST USPS
Benchmarks: Folktables US Census tasks (SDMetrics-style metrics) MNIST downstream classification MNIST to USPS domain adaptation
Metrics: KSComplement TVComplement ContingencySimilarity CorrelationSimilarity BinaryLogisticRegression F1 downstream accuracy
Baselines: DP-SGD PATE MERF SliceWass
Builds on: sliced Wasserstein privacy (SliceWass) Rényi differential privacy kernel mean matching MERF
Closest papers by meaning
- Privacy Amplification Persists under Unlimited Synthetic Data Release (NeurIPS 2026 · similarity 0.89)
- Exactly Minimax-Optimal Locally Differentially Private Sampling (NeurIPS 2024 · similarity 0.89)
- Towards hyperparameter-free optimization with differential privacy (ICLR 2025 · similarity 0.89)
- Certification for Differentially Private Prediction in Gradient-Based Training (ICML 2025 · similarity 0.89)
- Privacy Amplification Through Synthetic Data: Insights from Linear Regression (ICML 2025 · similarity 0.88)
- Shifted Interpolation for Differential Privacy (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.