Research map / Federated learning and privacy / Differential privacy
A Huber Loss Minimization Approach to Mean Estimation under User-level Differential Privacy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Huber-loss-minimization mean estimator with smooth-sensitivity Gaussian noise for user-level DP, avoiding clipping bias and adapting to imbalanced user sample sizes.
Problem
Mean estimation under user-level differential privacy where heavy-tailed sample distributions and imbalanced per-user sample sizes make the two-stage winsorized clipping approach biased and high-sensitivity.
Approach
Replace the two-stage clipped mean with the minimizer of a weighted sum of user-wise Huber losses, with per-user weights and connecting points set by user sample size; sensitivity is bounded via case analysis (no outliers, few outliers, other) and privatized with smooth sensitivity and Gaussian noise.
Key results
- On heavy-tailed Lomax data the Huber method converges significantly faster than the two-stage WME, while performing nearly the same on uniform and Gaussian data.
- Under increasing user imbalance gamma, WME degrades while the Huber loss minimization approach remains stable.
- Theory: MSE bound R^2/(mn) + dR^2/(mn^2 eps^2) ln(mnd) ln(1/delta) for bounded support, matching the two-stage method up to log factors.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Huber loss minimization instead of the Winsorized Mean Estimator (WME) | User-level private mean estimation · Uniform and Gaussian distributions, balanced users | mean squared error | No clear effect |
| Use Huber loss minimization instead of WME | User-level private mean estimation · Lomax and IPUMS data, balanced users | mean squared error | Helped |
| Use Huber loss minimization instead of WME | User-level private mean estimation · Higher-dimensional distributions, balanced users | mean squared error | Helped |
| Use adaptive user weights and connecting points in Huber loss | Huber loss minimization · Imbalanced users; Uniform, Gaussian, Exponential, and IPUMS total income | mean squared error | Helped |
| Use Huber loss minimization instead of WME | User-level private mean estimation · Bounded-support distributions, balanced users | mean squared error | No clear effect |
| Use Huber loss minimization instead of the two-stage approach | User-level private mean estimation · Heavy-tailed distributions | mean squared error | Helped |
Limitations
- Requires a minimum number of users n for the guarantees to hold.
- Per-user sample sizes m_i are assumed public; the case where they are private is not analyzed.
- Imbalance analysis relies on Assumption that at least half of samples belong to users with not-very-large sample sizes.
Experimental setup
Datasets: IPUMS (total income and salary) synthetic uniform, Gaussian, Lomax, exponential distributions
Benchmarks: IPUMS total income IPUMS salary
Metrics: mean squared error
Baselines: Winsorized Mean Estimator (WME)
Builds on: Winsorized Mean Estimator smooth sensitivity framework Weiszfeld algorithm
Closest papers by meaning
- Bridging Central and Local Differential Privacy in Data Acquisition Mechanisms (NeurIPS 2022 · similarity 0.89)
- Improved Communication-Privacy Trade-offs in $L_2$ Mean Estimation under Streaming Differential Privacy (ICML 2024 · similarity 0.88)
- Optimal Survey Design for Private Mean Estimation (ICML 2025 · similarity 0.88)
- Privacy Guarantees in Posterior Sampling under Contamination (NeurIPS 2026 · similarity 0.87)
- Mean Estimation in the Add-Remove Model of Differential Privacy (ICML 2024 · similarity 0.87)
- Universal Exact Compression of Differentially Private Mechanisms (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.