Research map / Federated learning and privacy / Differential privacy
Tempered Sigmoid Activations for Deep Learning with Differential Privacy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace ReLU with tanh activations | DP-SGD model · MNIST | test accuracy: 96.6% -> 98.1% | Helped |
| Replace ReLU with tanh activations | DP-SGD model · FashionMNIST | test accuracy: 81.9% -> 86.1% | Helped |
| Replace ReLU with tanh activations | DP-SGD model · CIFAR10 | test accuracy: 61.6% -> 66.2% | Helped |
| Use a tuned tempered sigmoid instead of ReLU | DP-SGD model · MNIST | test accuracy: 96.6% -> 98.1% | Helped |
| Use a tuned tempered sigmoid instead of ReLU | DP-SGD model · FashionMNIST | test accuracy: 81.9% -> 86.0% | Helped |
| Use a tuned tempered sigmoid instead of ReLU | DP-SGD model · CIFAR10 | test accuracy: 61.6% -> 66.0% | Helped |
| Replace ReLU with tanh activations | DP-SGD model · MNIST | activation norm: 3 | Helped |
| Use layer-wise tempered sigmoid parameters instead of shared parameters (ablation) | Tempered sigmoid model · MNIST, FashionMNIST, and CIFAR10 | test accuracy | No clear effect |
| Use tempered loss (ablation) | DP-SGD training · Experiments reported in the paper | test performance | No clear effect |
| Use SGD rather than Adam (ablation) | DP-SGD with tanh · FashionMNIST | test accuracy: 86.0% -> 86.1% | No clear effect |
Closest papers by meaning
- Online Sensitivity Optimization in Differentially Private Learning (AAAI 2024 · similarity 0.90)
- Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach (ICLR 2024 · similarity 0.89)
- Privacy without Noisy Gradients: Slicing Mechanism for Generative Model Training (NeurIPS 2024 · similarity 0.88)
- Certification for Differentially Private Prediction in Gradient-Based Training (ICML 2025 · similarity 0.87)
- Differentially private optimization for non-decomposable objective functions (ICLR 2025 · similarity 0.87)
- Shifted Interpolation for Differential Privacy (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.