Research map / Federated learning and privacy / Differential privacy
Delving into Differentially Private Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Re-Attention Mechanism and Phantom Clipping to make DP-SGD training of Transformers accurate and efficient, evaluated with small Transformers on MovieLens and Amazon Games recommendation.
Problem
Training Transformer models with differential privacy suffers from attention distraction (noise-induced variance inflates attention scores of rarely seen tokens) and from incompatibility of efficient per-sample gradient clipping with embedding sharing.
Approach
Reduce DP Transformer training to DP vanilla neural net training by (1) correcting attention scores using tracked variance of embedding/key distributions via Bayesian-style error propagation (Re-Attention), and (2) extending Ghost Clipping to shared input/output embeddings with a closed-form per-sample norm (Phantom Clipping).
Key results
- Phantom Clipping allows up to 450x larger batch size than Ghost Clipping on Amazon and up to 100x training speedup, reaching 0.68x the speed of non-private training (single V100).
- Phantom Clipping is reported to be 10-400x more memory efficient than Ghost Clipping on small models.
- Re-Attention consistently improves training stability and convergence of DP-trained small Transformers on MovieLens and Amazon, with larger gains on the more imbalanced Amazon data and at epsilon=5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Re-Attention Mechanism to correct attention distraction | Differentially private Transformer · MovieLens and Amazon | training stability | Helped |
| Apply Re-Attention Mechanism to correct attention distraction | Differentially private Transformer · Amazon, especially at ε=5 | model accuracy | Helped |
| Use shared rather than unshared embedding parameters (ablation) | Differentially private Transformer · MovieLens, ε=5 | NDCG@10 | Helped |
| Use Phantom Clipping instead of Ghost Clipping | Small Transformer models · Real-world recommendation datasets | training speed: 4-100$\times$ faster | Helped |
| Use Phantom Clipping instead of Ghost Clipping | Small Transformer models · Real-world recommendation datasets | memory efficiency: 10-400$\times$ more memory efficient | Helped |
Surprising findings
- Parameter sharing of the embedding layer gives consistent and significant NDCG gains over non-shared embeddings under DP-SGD.
Limitations
- Only small Transformer models were evaluated; scaling to large models is untested.
- Open question whether the modular reduction captures all hardness of DP Transformer training.
- Re-Attention relies on approximations (moment matching, one-sample parameter estimate) rather than exact variance tracking.
Experimental setup
Datasets: MovieLens-1M Amazon Games
Benchmarks: MovieLens-1M Amazon Games
Metrics: NDCG@10 HIT@10
Baselines: vanilla Transformer Transformer without parameter sharing GRU LSTM Ghost Clipping
Builds on: DP-SGD Ghost Clipping fastDP Bayesian deep learning error propagation
Closest papers by meaning
- Towards hyperparameter-free optimization with differential privacy (ICLR 2025 · similarity 0.90)
- DOPPLER: Differentially Private Optimizers with Low-pass Filter for Privacy Noise Reduction (NeurIPS 2024 · similarity 0.88)
- Differentially Private Prototypes for Imbalanced Transfer Learning (AAAI 2025 · similarity 0.87)
- A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization (ICML 2024 · similarity 0.87)
- DP-SGD Without Clipping: The Lipschitz Neural Network Way (ICLR 2024 · similarity 0.87)
- Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.