Research map / Federated learning and privacy / Differential privacy
Enhancing DPSGD via Per-Sample Momentum and Low-Pass Filtering
Xincheng Xu, Thilina Ranbaduge, Qing Wang, Thierry Rakotoarivelo, David Smith
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add per-sample momentum and low-pass filtering over DPSGD | DPSGD, ViT from scratch · CIFAR-100 (eps=1) | test accuracy (%): 7.52 -> 11.40 | Improved |
| add per-sample momentum and low-pass filtering over DPSGD | DPSGD, ViT from scratch · CIFAR-10 (eps=1) | test accuracy (%): 35.74 -> 40.96 | Improved |
| add per-sample momentum and low-pass filtering over DPSGD | RoBERTa-base fine-tuning · MNLI (eps=1) | test accuracy (%): 51.36 -> 56.81 | Improved |
| add per-sample momentum and low-pass filtering over DPSGD | RoBERTa-base fine-tuning · SST-2 (eps=8) | test accuracy (%): 90.83 -> 90.39 | No clear effect |
| add low-pass filter alone (LP-DPSGD) to DPSGD | DPSGD, 5-layer CNN on CIFAR-10 · CIFAR-10 (eps=8) | test accuracy (%) | Hurt |
| add inner-outer momentum (InnerOuter) to DPSGD | DPSGD, ViT from scratch · CIFAR-10 (eps=1) | test accuracy (%): 35.74 -> 11.55 | Hurt |
| add per-sample momentum to DP-PMLF (ablation) | DP-PMLF · MNIST and CIFAR-10 (CNN-5, ResNet-18), eps 1-8 | test accuracy (%) | Improved |
| add low-pass filter to DP-PMLF (ablation) | DP-PMLF · MNIST (CNN-5, ResNet-18) | test accuracy (%) | Improved |
| add low-pass filter to DP-PMLF at large eps (ablation) | DP-PMLF · CIFAR-10 (eps > 6) | test accuracy (%): 0.5-0.7% lower than w/o LF | Hurt |
| add low-pass filter to DP-PMLF at small eps (ablation) | DP-PMLF · CIFAR-10 (eps <= 6) | test accuracy (%) | Improved |
Closest papers by meaning
- Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach (ICLR 2024 · similarity 0.92)
- DOPPLER: Differentially Private Optimizers with Low-pass Filter for Privacy Noise Reduction (NeurIPS 2024 · similarity 0.92)
- Towards hyperparameter-free optimization with differential privacy (ICLR 2025 · similarity 0.89)
- The importance of feature preprocessing for differentially private linear optimization (ICLR 2024 · similarity 0.89)
- DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction (ICLR 2025 · similarity 0.88)
- Differentially Private Stochastic Gradient Descent with Fixed-Size Minibatches: Tighter RDP Guarantees with or without Replacement (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
