Research map / Federated learning and privacy / Differential privacy
Private Model Personalization Revisited
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a user-level differentially private federated variant of FedRep for noisy linear regression (sub-Gaussian data, disjoint batches, private SVD initialization) and a JL-based margin-loss information-theoretic classification construction with dimension-independent risk.
Problem
Private model personalization under user-level differential privacy in the shared low-dimensional representation framework, with statistically heterogeneous users and noisy labels.
Approach
Modify FedRep so each user computes local vectors and a clipped gradient on disjoint batches, and the server aggregates noisy clipped gradients for a private federated shared-embedding update; for classification, combine Johnson-Lindenstrauss dimension reduction with the exponential mechanism to get margin-based bounds independent of d.
Key results
- Private FedRep's privacy error term scales as d^2k/(n^2 eps^2) versus d^3k^2/(n^2 eps^2) for Priv-AltMin, an improvement of roughly dk in natural parameter regimes.
- Analysis holds for sub-Gaussian features and noisy labels, whereas prior work assumed Gaussian features.
- Classification construction yields a margin-based excess risk bound independent of the input dimension d for bounded features.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace centralized Priv-AltMin with federated Private FedRep | Private model personalization for linear regression · Linear regression | privacy error term in excess population risk: improve the privacy error term in [cite] by a factor of $\widetilde{O}(dk)$ | Helped |
| Use disjoint batches for local-vector updates and embedding gradients instead of reusing the same batches (ablation) | Private FedRep · Linear regression | gradient norm bound | Helped |
| Use disjoint batches for local-vector updates and embedding gradients instead of reusing the same batches (ablation) | Private FedRep · Linear regression | privacy noise required | Helped |
| Extend the data assumptions from Gaussian features to sub-Gaussian features | Private FedRep · Linear regression | applicability to feature distributions | Helped |
| Add label noise to the realizable setting | Private FedRep · Linear regression | applicability to label settings | Helped |
| Use the proposed private initialization with top-k SVD and added Gaussian noise | Private FedRep · Initialization under sub-Gaussian feature vectors | applicability to feature distributions | Helped |
Limitations
- Theory assumes sub-Gaussian isotropic features and a known condition-number bound; requires n large relative to dk and m.
- Experiments are limited to a single synthetic setting; the classification part is information-theoretic without an efficient algorithm.
Experimental setup
Datasets: synthetic Gaussian data
Benchmarks: synthetic linear regression data
Metrics: population MSE
Baselines: Priv-AltMin local GD non-private FedRep
Builds on: FedRep Priv-AltMin exponential mechanism Johnson-Lindenstrauss transform
Closest papers by meaning
- Differentially Private Model Personalization (NeurIPS 2021 · similarity 0.91)
- One-shot Empirical Privacy Estimation for Federated Learning (ICLR 2024 · similarity 0.87)
- Clients Collaborate: Flexible Differentially Private Federated Learning with Guaranteed Improvement of Utility-Privacy Trade-off (ICML 2025 · similarity 0.87)
- Efficiently Computing Similarities to Private Datasets (ICLR 2024 · similarity 0.86)
- It's My Data Too: Private ML for Datasets with Multi-User Training Examples (ICML 2025 · similarity 0.86)
- Learning from End User Data with Shuffled Differential Privacy over Kernel Densities (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.