Research map / Federated learning and privacy / Differential privacy
Scaling Laws for Differentially Private Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fits semi-parametric scaling laws for DP-Adam training of BERT models (4M to 778M params) that predict loss under compute, privacy and data budgets, yielding compute-optimal configurations.
Problem
Scaling laws for LLM training do not account for differential privacy; DP training dynamics (noise, large batch needs, privacy budget, finite data) make the compute-optimal allocation of model size, batch size, and iterations unknown.
Approach
Decouple noise calibration by fixing a physical batch size and varying the noise-batch ratio post hoc via privacy accounting; fit a loss surface over model size, iterations, and noise-batch ratio, then query it under fixed compute, privacy, and data budgets to find compute-optimal DP configurations.
Key results
- Compute-optimal DP configurations save roughly 5x to 100x compute versus baseline configurations at comparable privacy and utility.
- Optimal model size under DP is at least an order of magnitude smaller than non-private; ~1e8 parameters compute-optimal at 1e22 FLOPs versus ~1e10 non-privately.
- Token-to-model ratio under DP for privacy budget 1-10 is roughly 1000 to 100000, versus about 20x in non-private training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Allocate compute using the DP compute-optimal configuration rather than a fixed baseline configuration | DP-Adam training · Data budget N=10^7, reasonable privacy budgets | compute for similar utility: 2 × 10^18 FLOPs vs. 5× the FLOPs | Helped |
| Use a compute-optimal configuration instead of BertLarge-based non-private scaling allocation | DP-Adam training · DP privacy budgets | utility | Helped |
| Increase the privacy budget | Compute-optimal DP-Adam training · Fixed data and compute budgets | cross entropy | Helped |
| Increase compute beyond the critical compute budget without increasing privacy budget or data budget | Compute-optimal DP-Adam training · Fixed privacy and data budgets | cross entropy | No clear effect |
| Use physical batch size 128 rather than larger physical batch sizes (ablation) | DP-Adam at noise-batch ratio 0.5^15 · BertTiny training set | cross entropy: 3.586 vs 3.971, 4.01, and 4.057 | Helped |
| Use physical batch size 128 rather than larger physical batch sizes (ablation) | DP-Adam at noise-batch ratio 0.5^15 · BertTiny training set | accuracy: 43.59% vs 37.27%, 37.55%, and 36.73% | Helped |
| Increase physical batch size (ablation) | DP-Adam at the smallest tested noise-batch ratio · BertTiny, 128K iterations | loss | Helped |
| Increase physical batch size (ablation) | DP-Adam at medium and larger tested noise-batch ratios · BertTiny, 128K iterations | loss | Hurt |
| Tune learning rate across 0.5^6, 0.5^7, 0.5^8, and 0.5^9 instead of using a fixed learning rate (ablation) | DP-Adam at noise-batch ratio 0.5^15 · Physical batch size comparison | cross entropy | No clear effect |
| Increase model size from BertTiny to BertLarge (ablation) | DP-Adam physical batch size ablation · Noise-batch ratios 0.5^20, 0.5^15, and 0.5^10 | cross entropy | Mixed |
| Use a separate NanoDO training pipeline instead of the main pipeline (ablation) | DP-Adam, 30 million parameter decoder-only transformer · 32K iterations | cross entropy | No clear effect |
Surprising findings
- Smaller physical batch sizes can achieve lower loss than larger ones at larger noise-batch ratios, contrary to the hypothesis underlying the methodology.
- Even with infinite compute, increasing compute yields little benefit under modest privacy and data budgets; the saturating compute budget is an inflection point.
Limitations
- Fixed physical batch size of 1024 is assumed to approximate larger hypothetical batch sizes; ablations show physical batch size has a nuanced effect.
- Only BERT-style masked LM models up to 778M params, fixed sequence length 512, pretraining from scratch; results may not transfer to finetuning or other mechanisms.
- Assumes DP-Adam with a fixed clipping and learning rate protocol; better DP optimizers could shift optimal points.
Experimental setup
Datasets: BERT pretraining dataset (~3.3B words)
Metrics: validation cross-entropy loss training loss compute budget FLOPs noise-batch ratio
Baselines: BertLarge fixed configuration BertMedium fixed configuration BertTiny fixed configuration non-private scaling laws (Chinchilla)
Builds on: Chinchilla scaling laws DP-SGD Adam dp_accounting
Closest papers by meaning
- Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives (NeurIPS 2024 · similarity 0.90)
- Privacy Auditing of Large Language Models (ICLR 2025 · similarity 0.90)
- DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer (ICLR 2024 · similarity 0.90)
- Privacy-Preserving Instructions for Aligning Large Language Models (ICML 2024 · similarity 0.89)
- DP-MemArc: Differential Privacy Transfer Learning for Memory Efficient Language Models (AAAI 2025 · similarity 0.89)
- Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.