AI Paper Atlas

Research map / Federated learning and privacy / Differential privacy

Effectively Using Public Data in Privacy Preserving Machine Learning

Milad Nasr, Saeed Mahloujifar, Xinyu Tang, Prateek Mittal, Amir Houmansadr

ICML 2023 · Federated learning and privacy › Differential privacy · approach: Private training with public data

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
pretrain on public data (warm) instead of cold start (ablation)DP-SGD on CIFAR10 WRN16-4 · CIFAR10 eps=2Test Acc (%): +3.5 points (64.9 to 68.1)Improved
pretrain on generated (DDPM-augmented) public data (ablation)DP-SGD warm pipeline on CIFAR10 · CIFAR10 eps=2Test Acc (%): +3.9 points (68.1 to 72.0)Improved
include generated/public data in private training set (extended) (ablation)DP-SGD warm-aug on CIFAR10 · CIFAR10 eps=2Test Acc (%): +1.7 points (72.0 to 73.7)Improved
use DOPE-SGD adaptive origin clipping with public-data gradient estimate (ablation)extended training on CIFAR10 · CIFAR10 eps=2Test Acc (%): +1.1 points (73.7 to 74.8)Improved
ensemble of intermediate models (majority vote / parameter averaging) (ablation)DOPE-SGD on CIFAR10 · CIFAR10 eps=2Test Acc (%): +0.3 points (74.8 to 75.1)Improved
DOPE-SGD vs DP-SGD with public data pretraining across privacy budgetsCIFAR10 WRN16-4 · CIFAR10 eps=1 to 6Test Acc (%): gap shrinks as eps grows: e.g., +12.0 at eps=1 (60.1 to 72.1), +2.2 at eps=6 (77.8 vs 80.0 not stated; eps=6 warm 77.1 tImproved
DOPE-SGD vs mirror gradient descent and gradient scaling baselines with public dataCIFAR10 eps=2 with 4% public data · CIFAR10 eps=2Test Acc (%): 75.1 vs 70.5 (MGD augmented) and 69.1 (GS augmented)Improved
DOPE-SGD vs DP-SGD fine-tuning pretrained GPT-2 on public dataGPT-2 fine-tuning on WikiText-2 · WikiText-2 eps=1Test Ppl (lower better): 21.23 to 19.09Improved
leveraging non-private users (DOPE-SGD) vs DP-SGD on all users in federated user-level DPEMNIST federated learning · EMNIST user-level eps=1Test Acc (%): 85.2 vs 83.4 (DP-SGD all) and 84.6 (DP-SGD only-private)Improved
increase augmented data size from 5K to 40K generated samples (ablation)DOPE-SGD on CIFAR10 from 2000 public images · CIFAR10 eps=2Test Acc (%): 69.5% to 75.1%Improved
increase public data from 50 to 2000 images (ablation)DOPE-SGD on CIFAR10 · CIFAR10 eps=2Test Acc (%): 66.6% to 75.1%Improved
use of Gradient Scaling (Li et al.) with augmentation when public data is largeCIFAR10 eps=2 · CIFAR10 eps=2Test Acc (%): Gradient Scaling reaches 69.1% vs MGD 70.5% with augmentationHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy