Research map / Federated learning and privacy / Differential privacy
Effectively Using Public Data in Privacy Preserving Machine Learning
Milad Nasr, Saeed Mahloujifar, Xinyu Tang, Prateek Mittal, Amir Houmansadr
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pretrain on public data (warm) instead of cold start (ablation) | DP-SGD on CIFAR10 WRN16-4 · CIFAR10 eps=2 | Test Acc (%): +3.5 points (64.9 to 68.1) | Improved |
| pretrain on generated (DDPM-augmented) public data (ablation) | DP-SGD warm pipeline on CIFAR10 · CIFAR10 eps=2 | Test Acc (%): +3.9 points (68.1 to 72.0) | Improved |
| include generated/public data in private training set (extended) (ablation) | DP-SGD warm-aug on CIFAR10 · CIFAR10 eps=2 | Test Acc (%): +1.7 points (72.0 to 73.7) | Improved |
| use DOPE-SGD adaptive origin clipping with public-data gradient estimate (ablation) | extended training on CIFAR10 · CIFAR10 eps=2 | Test Acc (%): +1.1 points (73.7 to 74.8) | Improved |
| ensemble of intermediate models (majority vote / parameter averaging) (ablation) | DOPE-SGD on CIFAR10 · CIFAR10 eps=2 | Test Acc (%): +0.3 points (74.8 to 75.1) | Improved |
| DOPE-SGD vs DP-SGD with public data pretraining across privacy budgets | CIFAR10 WRN16-4 · CIFAR10 eps=1 to 6 | Test Acc (%): gap shrinks as eps grows: e.g., +12.0 at eps=1 (60.1 to 72.1), +2.2 at eps=6 (77.8 vs 80.0 not stated; eps=6 warm 77.1 t | Improved |
| DOPE-SGD vs mirror gradient descent and gradient scaling baselines with public data | CIFAR10 eps=2 with 4% public data · CIFAR10 eps=2 | Test Acc (%): 75.1 vs 70.5 (MGD augmented) and 69.1 (GS augmented) | Improved |
| DOPE-SGD vs DP-SGD fine-tuning pretrained GPT-2 on public data | GPT-2 fine-tuning on WikiText-2 · WikiText-2 eps=1 | Test Ppl (lower better): 21.23 to 19.09 | Improved |
| leveraging non-private users (DOPE-SGD) vs DP-SGD on all users in federated user-level DP | EMNIST federated learning · EMNIST user-level eps=1 | Test Acc (%): 85.2 vs 83.4 (DP-SGD all) and 84.6 (DP-SGD only-private) | Improved |
| increase augmented data size from 5K to 40K generated samples (ablation) | DOPE-SGD on CIFAR10 from 2000 public images · CIFAR10 eps=2 | Test Acc (%): 69.5% to 75.1% | Improved |
| increase public data from 50 to 2000 images (ablation) | DOPE-SGD on CIFAR10 · CIFAR10 eps=2 | Test Acc (%): 66.6% to 75.1% | Improved |
| use of Gradient Scaling (Li et al.) with augmentation when public data is large | CIFAR10 eps=2 · CIFAR10 eps=2 | Test Acc (%): Gradient Scaling reaches 69.1% vs MGD 70.5% with augmentation | Hurt |
Closest papers by meaning
- Optimal Differentially Private Model Training with Public Data (ICML 2024 · similarity 0.91)
- Differentially Private Prototypes for Imbalanced Transfer Learning (AAAI 2025 · similarity 0.89)
- Noise-Aware Differentially Private Regression via Meta-Learning (NeurIPS 2024 · similarity 0.88)
- Nearly Tight Black-Box Auditing of Differentially Private Machine Learning (NeurIPS 2024 · similarity 0.88)
- Differentially Private Model Merging (NeurIPS 2026 · similarity 0.87)
- Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
