Research map / Federated learning and privacy / Differential privacy
Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks
Jiayuan Ye, Zhenyu Zhu, Fanghui Liu, Reza Shokri, Volkan Cevher
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase width | Fully connected ReLU networks trained with noisy gradient descent · Privacy vs width, depth 10, 50 epochs | KL privacy loss | Hurt |
| Increase depth with LeCun initialization | Fully connected ReLU networks trained with noisy gradient descent · Privacy vs depth, width 1024, 20 epochs | KL privacy loss | Mixed |
| Increase depth with Xavier initialization | Fully connected ReLU networks trained with noisy gradient descent · Privacy vs depth, width 1024, 20 epochs | KL privacy loss | Mixed |
| Increase depth with He initialization | Fully connected ReLU networks trained with noisy gradient descent · Privacy vs depth, width 1024, 20 epochs | KL privacy loss | Hurt |
| Increase depth | Fully connected ReLU networks trained with noisy gradient descent · Privacy vs depth, width 1024, 50 epochs | KL privacy loss | Hurt |
| Use LeCun initialization rather than an initialization with larger per-layer variance | Linearized fully connected ReLU network · KL privacy bound | gradient norm constant B | Improved |
| Increase depth under LeCun initialization | Linearized fully connected ReLU network · KL privacy bound | KL privacy | Improved |
| Increase depth under Xavier initialization | Linearized fully connected ReLU network · KL privacy bound | KL privacy | Improved |
| Increase depth under He initialization | Linearized fully connected ReLU network · KL privacy bound | KL privacy | Hurt |
| Increase depth under NTK initialization | Linearized fully connected ReLU network · KL privacy bound | KL privacy | Hurt |
| Increase depth under LeCun initialization | Linearized fully connected ReLU network · Excess empirical risk under fixed KL privacy | excess empirical risk | Improved |
| Increase depth under Xavier initialization | Linearized fully connected ReLU network · Excess empirical risk under fixed KL privacy | excess empirical risk | Improved |
Closest papers by meaning
- Certification for Differentially Private Prediction in Gradient-Based Training (ICML 2025 · similarity 0.88)
- Optimization, Generalization and Differential Privacy Bounds for Gradient Descent on Kolmogorov–Arnold Networks (ICML 2026 · similarity 0.87)
- Parameters or Privacy: A Provable Tradeoff Between Overparameterization and Membership Inference (NeurIPS 2022 · similarity 0.87)
- Privacy without Noisy Gradients: Slicing Mechanism for Generative Model Training (NeurIPS 2024 · similarity 0.86)
- Shifted Interpolation for Differential Privacy (ICML 2024 · similarity 0.86)
- Online Sensitivity Optimization in Differentially Private Learning (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
