Research map / Federated learning and privacy / Differential privacy
Fairness Issues and Mitigations in (Differentially Private) Socio-Demographic Data Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an optimization-based survey sampling design (two-phase remote and door-to-door) that meets per-group error tolerances at lower cost, and shows that differential privacy noise in population counts positively biases small groups, which can reduce sampling unfairness.
Problem
Survey sampling for socio-demographic statistics (e.g., the ACS) controls error only over the whole population, so sampling errors and their variance differ across demographic groups, creating fairness issues; differential privacy applied to the population counts used for survey design may further a
Approach
Recasts survey allocation as a MILP that minimizes sampling cost subject to per-group error constraints (via a Chebyshev bound and a variance proxy a_i/x), models a two-phase remote plus in-person design, and derives a closed-form positive bias of Laplace-noised, non-negativity-clipped counts that over-allocates to small groups.
Key results
- Phase 1 and 2 optimization is about 86% of the cost of Phase 1 Only while meeting confidence constraints for every group, whereas Standard Allocation leaves minority groups failing the 90% confidence constraints.
- Theorem: the bias of the Laplace-noised, max(0,.)-clipped count is (b/2)exp(-N/b) > 0, so smaller groups are overestimated; in experiments, more noise (smaller epsilon) reduced error variance for minorities under Standard Allocation.
- Heuristic equal per-group allocation at the same sampling rate substantially improves fairness over Standard Allocation, with runtime that is constant in the number of groups.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace proportional allocation with optimized Phase 1-only allocation | Survey sampling · Connecticut ACS 2022, race groups | variance fairness | Helped |
| replace proportional allocation with optimized Phase 1-only allocation | Survey sampling · Connecticut ACS 2022, race groups | group confidence compliance | Helped |
| add targeted Phase 2 geographic sampling to Phase 1 optimization | Optimization: Phase 1 Only · Connecticut ACS 2022, race groups | survey cost: 86% of Phase 1 Only cost | Helped |
| add targeted Phase 2 geographic sampling to Phase 1 optimization | Optimization: Phase 1 Only · Connecticut ACS 2022, race groups | variance fairness | Hurt |
| use DP-noised counts instead of exact counts | Standard Allocation · Connecticut ACS 2022, minority race groups | variance of errors | Helped |
| use DP-noised counts instead of exact counts | Optimization: Phase 1 Only · Connecticut ACS 2022, race groups | variance of errors | No clear effect |
| use DP-noised counts for Phase 2 region selection | Optimization: Phase 1 and 2 · Connecticut ACS 2022, race groups | variance of errors | Mixed |
| allocate an equal number of surveys to each subgroup | Heuristic · Nebraska ACS 2022, race groups | variance fairness | Helped |
| use optimized sampling instead of proportional allocation | Survey sampling · Connecticut ACS 2022, education groups | confidence compliance | Helped |
| increase Phase 1 failure rate F1 (ablation) | Optimization: Phase 1 and 2 · Connecticut, race groups | survey cost | Hurt |
| increase Phase 2 failure rate F2 (ablation) | Optimization: Phase 1 and 2 · Connecticut, race groups | survey cost | Hurt |
| increase Phase 2 per-survey cost c2 (ablation) | Optimization: Phase 1 and 2 · Connecticut, race groups | survey cost | Hurt |
Surprising findings
- Adding more privacy noise (smaller epsilon) can reduce error-variance unfairness for minority groups because the positive bias overestimates small counts, leading to more surveys allocated to them.
- Equal allocation of surveys across subgroups, irrespective of population size, can yield roughly similar relative errors and better fairness than proportional allocation.
Limitations
- Variance proxy functions are fit empirically from prior data and the Chebyshev bound is loose, so the constraint is a stronger approximation of the probability requirement.
- Phase 2 uses simple random sampling within selected regions, which adds uncertainty in successful samples per group and slightly reduces fairness versus Phase 1 Only.
- Results are reported on simulated sampling from IPUMS ACS data with assumed failure rates and costs rather than real survey operations.
Experimental setup
Datasets: ACS 2021 (IPUMS) prior population counts ACS 2022 (IPUMS) ground truth
Benchmarks: ACS 2022 IPUMS microdata for Connecticut, Nebraska, Maine, Nevada
Metrics: survey cost relative to standard allocation fairness of variance (max variance discrepancy across groups confidence compliance at 10% error threshold relative group errors
Baselines: Standard Allocation (proportional stratified random sampling) Optimization: Phase 1 Only Heuristic equal allocation
Builds on: Laplace mechanism Chebyshev inequality proportional stratified random sampling
Closest papers by meaning
- A Simple and Practical Method for Reducing the Disparate Impact of Differential Privacy (AAAI 2024 · similarity 0.90)
- Optimal Survey Design for Private Mean Estimation (ICML 2025 · similarity 0.86)
- Size-adaptive Hypothesis Testing for Fairness (NeurIPS 2025 · similarity 0.83)
- Privacy Guarantees in Posterior Sampling under Contamination (NeurIPS 2026 · similarity 0.83)
- Improving the Variance of Differentially Private Randomized Experiments through Clustering (ICML 2025 · similarity 0.83)
- Mean Estimation in the Add-Remove Model of Differential Privacy (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.