Research map / Optimization / Optimizers
Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Convex-Concave Loss (CCL), which adds a concave term to cross-entropy so that gradient descent increases training-loss variance, reducing membership inference advantage while preserving utility.
Problem
Membership inference attacks exploit gap between member and non-member loss distributions; existing defenses like RelaxLoss use gradient ascent, which can cause oscillation and suboptimal utility.
Approach
Theoretically, convex losses shrink loss variance on training data while concave terms enlarge it under gradient descent; CCL adds a concave term (exponential or quadratic in p_y) to cross-entropy, enabling high training-loss variance with ordinary gradient descent.
Key results
- On CIFAR-10, CCQL reduces loss-metric attack advantage from 29.67% (Dropout) to 18.40% while keeping test accuracy not worse than the vanilla model.
- CCQL and CCEL achieve the highest P1 scores on all five datasets versus undefended models (e.g., CIFAR-100 P1 0.487 vs 0.273 undefended).
- Reports a relative reduction of 62.01% in privacy risk for the loss-metric attack on CIFAR-10.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a concave quadratic term to cross-entropy (CCQL) | Cross-entropy-trained MLP · Texas100 | P1 score: 0.551 -> 0.607 | Helped |
| Add a concave exponential term to cross-entropy (CCEL) | Cross-entropy-trained MLP · Texas100 | P1 score: 0.551 -> 0.608 | Helped |
| Add a concave quadratic term to cross-entropy (CCQL) | Cross-entropy-trained MLP · Purchase100 | P1 score: 0.858 -> 0.868 | Helped |
| Add a concave exponential term to cross-entropy (CCEL) | Cross-entropy-trained MLP · Purchase100 | P1 score: 0.858 -> 0.864 | Helped |
| Add a concave quadratic term to cross-entropy (CCQL) | Cross-entropy-trained DenseNet-121 · ImageNet | P1 score: 0.598 -> 0.610 | Helped |
| Add a concave exponential term to cross-entropy (CCEL) | Cross-entropy-trained DenseNet-121 · ImageNet | P1 score: 0.598 -> 0.609 | Helped |
| Add a concave quadratic term to cross-entropy (CCQL) | Cross-entropy-trained ResNet-34 · CIFAR-10 | P1 score: 0.741 -> 0.769 | Helped |
| Add a concave exponential term to cross-entropy (CCEL) | Cross-entropy-trained ResNet-34 · CIFAR-10 | P1 score: 0.741 -> 0.797 | Helped |
| Add a concave quadratic term to cross-entropy (CCQL) | Cross-entropy-trained DenseNet-121 · CIFAR-100 | P1 score: 0.273 -> 0.487 | Helped |
| Add a concave exponential term to cross-entropy (CCEL) | Cross-entropy-trained DenseNet-121 · CIFAR-100 | P1 score: 0.273 -> 0.480 | Helped |
| Use a smaller alpha, increasing the relative influence of the concave term (ablation) | CCL · CIFAR-10 | loss variance | Helped |
| Use a smaller alpha, increasing the relative influence of the concave term (ablation) | CCL · CIFAR-10 | test accuracy | Hurt |
Surprising findings
- Focal loss, which is more convex than cross-entropy, yields lower loss variance and higher privacy risk, supporting the claim that convexity reduces loss variance.
Limitations
- Cannot break the trade-off between utility and MIA defense.
- Unexplored effectiveness against label-only attacks such as data augmentation attacks.
- Smaller alpha causes underfitting and lower accuracy; early-stage concave gradients are smaller, so CE is used as the base.
Experimental setup
Datasets: Texas100 Purchase100 CIFAR-10 CIFAR-100 ImageNet
Benchmarks: Texas100 Purchase100 CIFAR-10 CIFAR-100 ImageNet
Metrics: attack advantage test accuracy P1 score
Baselines: RelaxLoss Mixup+MMD Adversary regularization Dropout Label Smoothing Confidence Penalty Early Stopping
Builds on: RelaxLoss cross-entropy loss focal loss Mixup+MMD adversarial regularization
Closest papers by meaning
- Assessing Per-Sample Membership Inference Vulnerability without Retraining (NeurIPS 2026 · similarity 0.89)
- Curvature Clues: Decoding Deep Learning Privacy with Input Loss Curvature (NeurIPS 2024 · similarity 0.87)
- Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learning (NeurIPS 2025 · similarity 0.86)
- Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks (AAAI 2024 · similarity 0.86)
- Estimating Model-Level Membership Inference Vulnerability Without Reference Models (NeurIPS 2026 · similarity 0.86)
- Low-Cost High-Power Membership Inference Attacks (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.