Research map / Federated learning and privacy / Differential privacy
Private online and user-level learning
Analyze private learning in online, shuffle and user-level settings.
116 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Differential privacy, keyword cluster “estimation · error · algorithms”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
33 papers in 2023–24 and 30 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 15.3% to 11.3%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Differential Privacy Preservation for Deep Auto-Encoders: an Application of Human Behavior Prediction (AAAI 2016 · 220 citations)
- 2017 · Cleaning the Null Space: A Privacy Mechanism for Predictors (AAAI 2017 · 19 citations)
- 2018 · Model-Agnostic Private Learning (NeurIPS 2018 · 21 citations)
- 2019 · Privacy-Preserving Q-Learning with Functional Noise in Continuous Spaces (NeurIPS 2019 · 34 citations)
- 2020 · Differentially-Private Federated Linear Bandits (NeurIPS 2020 · 40 citations)
- 2021 · Remember What You Want to Forget: Algorithms for Machine Unlearning (NeurIPS 2021 · 64 citations)
- 2022 · The Fundamental Price of Secure Aggregation in Differentially Private Federated Learning (ICML 2022 · 11 citations)
- 2023 · Bayesian Estimation of Differential Privacy (ICML 2023 · 9 citations)
- 2024 · On Differentially Private Federated Linear Contextual Bandits (ICLR 2024 · 3 citations)
- 2025 · Certification for Differentially Private Prediction in Gradient-Based Training (ICML 2025 · 1 citation)
- 2026 · Back to Square Roots: An Optimal Bound on the Matrix Factorization Error for Multi-Epoch Differentially Private SGD (ICLR 2026 · 1 citation)
Earliest papers here
- Differential Privacy Preservation for Deep Auto-Encoders: an Application of Human Behavior Prediction (AAAI 2016 · 220 citations)
- Efficient Private Empirical Risk Minimization for High-dimensional Learning (ICML 2016 · 34 citations)
- Online and Differentially-Private Tensor Decomposition (NeurIPS 2016 · 23 citations)
What has been tried
89 papers here report 546 measured changes (a component added, removed or swapped, compared with a baseline): helped 60%, no clear effect 10%, hurt 25%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Increase from 1 to 8 elements per coordinate | DP-SGD, 1000 dimensions, 100 steps · mean bound on ε | 0.49 (± 0.01) -> 0.62 (± 0.01) | Helped | How Well Can Differential Privacy Be Audited in One Run? (NeurIPS 2025 · 1 citation) |
| Use k=1 histogram bin instead of a larger bin count | Law School · KS distance (ΔSP) | Helped | Differentially Private Post-Processing for Fair Regression (ICML 2024 · 1 citation) | |
| Use disjoint batches for local-vector updates and embedding gradients instead of reusing the same batches | Linear regression · gradient norm bound | Helped | Private Model Personalization Revisited (ICML 2025 · 0 citations) | |
| Increase the interpolation parameter ρ from the bag-level loss toward the instance-level loss | Linear regression, proportional regime · bias | Hurt | Learning from Aggregate responses: Instance Level versus Bag Level Loss Functions (ICLR 2024 · 0 citations) | |
| Replace Gaussian-process perturbations and averaging with regularization for convex classes | Semi-private learning, convex function classes · private labeled sample complexity dependence on accuracy | O\left( \mathsf{vc}(\cF) \cdot \varepsilon^{-3} \cdot \alpha^{-14} \right) -> (\sup_m \overline{\cG}_m(\cF))^2 \cdot \va | Helped | Oracle-Efficient Differentially Private Learning with Public Data (NeurIPS 2024 · 0 citations) |
| vary the truncation constant c | Setting 2 dataset · coverage | Mixed | Online Differentially Private Conformal Prediction for Uncertainty Quantification (ICML 2025) |
What reviewers will expect
Datasets and baselines named in the 23 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 3 of 23 papers
Baselines
- DPO · 2 of 23 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 63 papers; from OpenAlex and the papers' affiliations).
- Google · 10 papers
- Wayne State University · 6 papers
- Microsoft · 3 papers
- KAUST · 3 papers
- Johns Hopkins University · 3 papers
- University of Waterloo · 3 papers
- Northeastern University · 2 papers
- Amazon · 2 papers
- University of California, San Diego · 2 papers
- UC Santa Barbara · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Privacy accounting and mechanisms (127 papers)
- Local-DP and private estimation (113 papers)
- Private data analysis frameworks (105 papers)
- Differentially private clustering (41 papers)
Elsewhere in the topic
- Private training with public data (96 papers)
- Differentially private synthetic data (81 papers)
- DP-SGD refinements (60 papers)
- Differentially private federated learning (35 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).