Research map / Federated learning and privacy / Differential privacy
Differentially Private Sharpness-Aware Training
Jinseong Park, Hoki Kim, Yujin Choi, Jaewook Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace DP-SGD update with DP-SAM (two-step, perturbed-gradient ascent with separate privatized gradient) | DP-SGD on from-scratch classification · MNIST, FashionMNIST, CIFAR-10, SVHN | accuracy: lower than DP-SGD in all reported cases (e.g., 92.50 vs 95.15 on MNIST GNResNet-10 eps=1) | Hurt |
| Reuse previous step's privatized gradient as ascent direction (DP-SAT) | DP-SGD on from-scratch classification · MNIST, FashionMNIST, CIFAR-10, SVHN | accuracy: e.g., +0.85 on MNIST GNResNet-10 eps=1 (96.00 vs 95.15); ERR up to 20.18% | Improved |
| Reuse previous step's privatized gradient as ascent direction (DP-SAT) | DP-SGD fine-tuning of ViT/CrossViT/DeiT · CIFAR-10 and CIFAR-100 fine-tuning | accuracy: improves in most cases (e.g., +3.9 on CIFAR-100 ViT_small eps=0.5) but DeiT_base CIFAR-10 eps=0.5 is -0.06 (92.15 vs 92. | Mixed |
| Reuse previous privatized gradient for ascent step (DP-SAT) vs two-step DP-SAM | training cost · CIFAR-10 training speed | training speed ratio vs DP-SGD: DP-SAT 101.4% vs DP-SAM 181.9% of DP-SGD time | Improved |
| Add DP-SAT momentum variant (ascent direction from momentum of all previous privatized gradients) (ablation) | DP-SAT · MNIST, FashionMNIST, CIFAR-10 (DPNAS) | accuracy: differences within about ±0.5, mixed sign (e.g., 98.83 vs 98.71 at eps=2 MNIST; 69.63 vs 69.86 at eps=3 CIFAR-10) | No clear effect |
| Fix radius rho to 0.03 (0.01 for DPNAS-CIFAR10) for all settings instead of tuning per setting (ablation) | DP-SAT · MNIST, FashionMNIST, CIFAR-10, SVHN | accuracy: DP-SAT still beats DP-SGD in all listed cases, e.g., 96.00 vs 95.15 on GNResNet-10 MNIST eps=1 | No clear effect |
| Clipping-induced flat minimum effect: flatter minima reduce clipped gradient proportion (theoretical argument) | DP-SGD · theory and MNIST gradient-norm analysis | proportion clipped | No clear effect |
Closest papers by meaning
- Towards hyperparameter-free optimization with differential privacy (ICLR 2025 · similarity 0.89)
- Delving into Differentially Private Transformer (ICML 2024 · similarity 0.89)
- Privacy without Noisy Gradients: Slicing Mechanism for Generative Model Training (NeurIPS 2024 · similarity 0.89)
- Certification for Differentially Private Prediction in Gradient-Based Training (ICML 2025 · similarity 0.88)
- A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization (ICML 2024 · similarity 0.88)
- Online Sensitivity Optimization in Differentially Private Learning (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
