Research map / Federated learning and privacy / Differential privacy
The Role of Adaptive Optimizers for Honest Private Hyperparameter Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DPAdam with the default Adam settings and tune clipping rather than searching a wide learning-rate and clipping grid | Private optimizer selection · Adult, ENRON, Gisette, and MNIST (logistic regression) | epsilon at comparable best accuracy: DPSGD: 5.01, 30.86, 26.40, 3.01; DPAdam: 1.91, 12.80, 10.76, 1.14 | Helped |
| Use Moments Accountant composition instead of the LT selection method when the candidate pool is small | Honest private hyperparameter selection · DP hyperparameter tuning | number of candidates at the same privacy cost: 14 candidates for n=5000; up to 175 candidates when n=100000 | Helped |
| Increase the number of candidates selected by LT | LT private selection · DP hyperparameter tuning | privacy cost: 1 candidate: 33; 1,000 candidates: 39 (n=5,000) | No clear effect |
| Tune learning rate and clipping threshold jointly rather than fixing one | DPSGD · Synthetic dataset and ENRON | training loss | Helped |
| Use DPAdam's adaptive learning rate with default α, β1, and β2 | DPAdam · MNIST, Gisette, Adult, and ENRON; logistic regression and TLNN | testing accuracy | Helped |
| Use the default momentum setting instead of searching momentum values | DPMomentum · MNIST, Gisette, Adult, and ENRON | testing accuracy | Hurt |
| Replace DPAdam's second-moment updates with its converged effective step size (ablation) | DPAdam, yielding DPAdam Without Second Moments · MNIST, Gisette, Adult, and ENRON; logistic regression and TLNN | testing accuracy | No clear effect |
| Use ADADP instead of DPAdam | Adaptive private optimizers · MNIST, Gisette, Adult, and ENRON; logistic regression and TLNN | testing accuracy | Mixed |
| Use the converged effective step size from the first iteration (ablation) | DPAdam Without Second Moments · MNIST logistic regression and TLNN | median testing accuracy | Helped |
Closest papers by meaning
- Towards hyperparameter-free optimization with differential privacy (ICLR 2025 · similarity 0.92)
- Online Sensitivity Optimization in Differentially Private Learning (AAAI 2024 · similarity 0.92)
- A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization (ICML 2024 · similarity 0.91)
- The Limits of Differential Privacy in Online Learning (NeurIPS 2024 · similarity 0.87)
- DOPPLER: Differentially Private Optimizers with Low-pass Filter for Privacy Noise Reduction (NeurIPS 2024 · similarity 0.87)
- DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.