Research map / Federated learning and privacy / Differential privacy
Differential Privacy Under Class Imbalance: Methods and Empirical Insights
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formalizes differential privacy for imbalanced binary classification, showing SMOTE and bagging are unsuitable under DP, and proposes private synthetic data pre-processing and class-weighted private ERM and DP-SGD, evaluated empirically on imblearn datasets.
Problem
Differentially private binary classification under severe class imbalance, where standard imbalanced-learning methods (oversampling, SMOTE, bagging, reweighting) may break or weaken privacy guarantees and minority-class accuracy degrades.
Approach
Shows SMOTE and oversampling greatly increase downstream sensitivity and that bagging's "intrinsic" privacy gives meaningless parameters; instead uses private synthetic data (GEM) to balance classes as pre-processing and adapts objective-perturbation ERM and DP-SGD to inverse-frequency class weights with proven privacy.
Key results
- GEM + NonPriv. XGBoost ranked best overall, first on 7 of 8 metrics on average (average rank 1.45 on AUC, F1, Bal-ACC, Precision).
- Priv. Weighted LogReg achieved the best average Recall among private models (rank 2.11) and beat unweighted Priv. LogReg on 3 of 4 imbalance-focused metrics.
- SMOTE requires epsilon adjustment by a factor of about 213 (desired epsilon 1 needs input epsilon 0.00469 for d=25, k=5), so unadjusted SMOTE yields epsilon of about 213 for input epsilon 1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use private synthetic data preprocessing (GEM) with XGBoost instead of the other private methods | Private imbalanced classification · Imbalanced-learn datasets, all ε settings | Average performance rank across metrics: 1.45 AUC; 1.45 F1; 1.48 Bal-ACC; 1.45 Precision; 2.26 Recall; 1.45 Worst-ACC; 1.48 Macro-Avg-ACC; 1.45 G-Mean | Helped |
| Add inverse-frequency class weights to private logistic regression | Priv. LogReg · Imbalanced-learn datasets, all ε settings | Recall average rank: 2.20 -> 2.11 | Helped |
| Add inverse-frequency class weights to private logistic regression | Priv. LogReg · Imbalanced-learn datasets, all ε settings | AUC average rank: 2.77 -> 3.19 | Hurt |
| Add inverse-frequency class weights to private logistic regression | Priv. LogReg · Imbalanced-learn datasets, all ε settings | F1 average rank: 2.62 -> 2.65 | Mixed |
| Use weighted DP-SGD FTTransformer rather than the other private methods | Private imbalanced classification · Imbalanced-learn datasets, all ε settings | F1 average rank: 3.58 | Hurt |
| Increase the privacy budget ε | GEM + XGBoost · All datasets | Classification performance | Helped |
| Apply SMOTE with the required sensitivity adjustment before DP logistic regression | DP logistic regression · Mammography | Performance | Hurt |
| Use private bagging with DP logistic regression weak learners instead of a single DP logistic regression classifier | DP logistic regression · Mammography subsamples, F1 | F1 | Hurt |
| Use GEM + NonPriv. XGBoost instead of SMOTE + NonPriv. XGBoost | XGBoost on synthetic 2D mixture · Synthetic 2D mixture | AUC: AUC ∈ [0.94,0.97] (non-DP classifiers) | Hurt |
| Use a weighted threshold with γ < 1/2 instead of the standard Bayes classifier threshold | Bayes classifier for the Gaussian mixture · Known Gaussian mixture | Recall | Helped |
Surprising findings
- Private bagging with weak learners is shown to be meaningless: non-private bagging's implicit privacy requires epsilon at most 1/n when delta is n^-c.
- Private weighted FTTransformer trained with DP-SGD consistently underperformed, even though it performs strongly non-privately.
Limitations
- Private synthetic data (GEM) suffers from the curse of dimensionality and is limited to low-dimensional datasets.
- Empirical evaluation limited to eight small low-dimensional tabular datasets with default hyperparameters and no tuning for private models.
- Negative SMOTE and bagging results rely on worst-case sensitivity analysis that may be conservative.
Experimental setup
Datasets: ecoli yeast_me2 solar_flare_m0 abalone car_eval_34 car_eval_4 mammography abalone_19
Benchmarks: imblearn datasets (8 UCI-derived)
Metrics: AUC F1 Balanced Accuracy Precision Recall Worst-class accuracy Macro-average accuracy G-Mean
Baselines: Priv. LogReg SMOTE + NonPriv. XGBoost NonPriv. Weighted XGBoost NonPriv. LogReg NonPriv. FTT Priv. Weighted FTT
Builds on: SMOTE objective perturbation DP-SGD GEM PrivBayes
Closest papers by meaning
- Differentially Private Prototypes for Imbalanced Transfer Learning (AAAI 2025 · similarity 0.89)
- Multi-Class Support Vector Machine with Differential Privacy (NeurIPS 2025 · similarity 0.87)
- Does Training with Synthetic Data Truly Protect Privacy? (ICLR 2025 · similarity 0.87)
- Auditing Private Prediction (ICML 2024 · similarity 0.87)
- Instance-Specific Asymmetric Sensitivity in Differential Privacy (NeurIPS 2024 · similarity 0.87)
- Enhancing Learning with Label Differential Privacy by Vector Approximation (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.