Research map / Federated learning and privacy / Differential privacy
Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use mixed ghost clipping instead of Opacus | DP training of VGG19 · CIFAR10 | maximum batch size: 18× bigger | Helped |
| Use mixed ghost clipping instead of Opacus | DP training of VGG19 · CIFAR10 | throughput: 3× faster | Helped |
| Use mixed ghost clipping instead of FastGradClip | DP training of VGG19 · CIFAR10 | throughput: 1.7× faster | Helped |
| Use mixed ghost clipping instead of Opacus | DP training of ResNet18 · CIFAR10 | time per epoch: 45 -> 37 | Helped |
| Use mixed ghost clipping instead of Opacus | DP training of ResNet50 · ImageNet | maximum physical batch size: 55 -> 129 | Helped |
| Use BEiT large instead of the prior reported DP result | DP image classification · CIFAR10, ε=1 | accuracy: 94.8% -> 96.7% | Helped |
| Use BEiT large instead of the prior reported DP result | DP image classification · CIFAR100, ε=1 | accuracy: 67.4% -> 83.0% | Helped |
| Use BEiT large instead of CrossViT base | DP fine-tuning · CIFAR100, ε=1 | accuracy: 71.9% -> 83.0% | Helped |
| Use ghost clipping alone instead of mixed ghost clipping | DP training of convolutional models · ImageNet | memory feasibility | Hurt |
| Select ghost norm or gradient instantiation layerwise using their memory costs | mixed ghost clipping · VGG-11 | space complexity: 5.34×10^9; 1.33×10^8; 3.40×10^6 | Helped |
| Train with larger pretrained BEiT rather than smaller CrossViT | DP fine-tuning · CIFAR10, ε=1 | accuracy: 95.5% -> 96.7% | Helped |
Closest papers by meaning
- Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger (NeurIPS 2023 · similarity 0.91)
- Delving into Differentially Private Transformer (ICML 2024 · similarity 0.87)
- DP-SGD Without Clipping: The Lipschitz Neural Network Way (ICLR 2024 · similarity 0.86)
- Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach (ICLR 2024 · similarity 0.85)
- ViP: A Differentially Private Foundation Model for Computer Vision (ICML 2024 · similarity 0.85)
- Towards hyperparameter-free optimization with differential privacy (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.