AI Paper Atlas

Research map / Federated learning and privacy / Differential privacy

Differentially Private Sharpness-Aware Training

Jinseong Park, Hoki Kim, Yujin Choi, Jaewook Lee

ICML 2023 · Federated learning and privacy › Differential privacy · approach: Private training with public data · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Replace DP-SGD update with DP-SAM (two-step, perturbed-gradient ascent with separate privatized gradient)DP-SGD on from-scratch classification · MNIST, FashionMNIST, CIFAR-10, SVHNaccuracy: lower than DP-SGD in all reported cases (e.g., 92.50 vs 95.15 on MNIST GNResNet-10 eps=1)Hurt
Reuse previous step's privatized gradient as ascent direction (DP-SAT)DP-SGD on from-scratch classification · MNIST, FashionMNIST, CIFAR-10, SVHNaccuracy: e.g., +0.85 on MNIST GNResNet-10 eps=1 (96.00 vs 95.15); ERR up to 20.18%Improved
Reuse previous step's privatized gradient as ascent direction (DP-SAT)DP-SGD fine-tuning of ViT/CrossViT/DeiT · CIFAR-10 and CIFAR-100 fine-tuningaccuracy: improves in most cases (e.g., +3.9 on CIFAR-100 ViT_small eps=0.5) but DeiT_base CIFAR-10 eps=0.5 is -0.06 (92.15 vs 92.Mixed
Reuse previous privatized gradient for ascent step (DP-SAT) vs two-step DP-SAMtraining cost · CIFAR-10 training speedtraining speed ratio vs DP-SGD: DP-SAT 101.4% vs DP-SAM 181.9% of DP-SGD timeImproved
Add DP-SAT momentum variant (ascent direction from momentum of all previous privatized gradients) (ablation)DP-SAT · MNIST, FashionMNIST, CIFAR-10 (DPNAS)accuracy: differences within about ±0.5, mixed sign (e.g., 98.83 vs 98.71 at eps=2 MNIST; 69.63 vs 69.86 at eps=3 CIFAR-10)No clear effect
Fix radius rho to 0.03 (0.01 for DPNAS-CIFAR10) for all settings instead of tuning per setting (ablation)DP-SAT · MNIST, FashionMNIST, CIFAR-10, SVHNaccuracy: DP-SAT still beats DP-SGD in all listed cases, e.g., 96.00 vs 95.15 on GNResNet-10 MNIST eps=1No clear effect
Clipping-induced flat minimum effect: flatter minima reduce clipped gradient proportion (theoretical argument)DP-SGD · theory and MNIST gradient-norm analysisproportion clippedNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy