Research map / Adversarial robustness and security / Certified and empirical robustness

Curvature Clues: Decoding Deep Learning Privacy with Input Loss Curvature

NeurIPS 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Decision-boundary geometry of robustness · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes black-box membership inference via zero-order estimates of input loss curvature (trace of input Hessian), with a theory bounding train-test curvature KL divergence by privacy and dataset size.

Problem

Membership inference attacks need to distinguish training from test examples; prior input-loss-curvature work only studied train-set curvature, and curvature-based attacks were unavailable in black-box settings.

Approach

Input loss curvature (trace of the input Hessian of loss) is higher on test than train examples; a zero-order finite-difference estimator of this trace enables black-box MIA, and a theory bounds train-test KL divergence as a function of DP epsilon and training set size, predicting when curvature beats probability-based attacks.

Key results

• Curv ZO NLL achieves 77.45 AUROC and 69.16 balanced accuracy on ImageNet, and 93.49 AUROC on CIFAR100, outperforming the listed baselines in the main table.
• Curv ZO LR achieves the highest TPR at 0.1% FPR (21.07%) and 0.01% FPR (15.74%) on CIFAR100 with 1 augmentation, versus 15.52% and 5.02% for the best baseline.
• Theory predicts curvature-based MIA outperforms probability-based attacks beyond a training set size threshold; experiments show Curv ZO LR beats prior works beyond about 30-40% subsets of CIFAR100 and NLL beyond 10%.

Measured effects reported in the paper

Surprising findings

• Curvature-based coresets (lowest-curvature samples) produce models that are more susceptible to MIA than randomly chosen subsets of the same size.
• The NLL curvature test is better in AUROC and balanced accuracy, while the parametric LR test is better at low FPR.

Limitations

• Parametric LR curvature attack underperforms non-parametric NLL below a certain training set size, which the theory does not explain.
• Requires training many shadow models (64 or 52), which is computationally expensive.
• Requires at least 4x more queries than the compared baseline method.

Experimental setup

Datasets: CIFAR10 CIFAR100 ImageNet

Benchmarks: ImageNet CIFAR100 CIFAR10

Metrics: AUROC balanced accuracy TPR at low FPR

Baselines: LiRA RMIA Shokri shadow-model attack loss-threshold attack entropy attack MERLIN Calibrated attack

Builds on: LiRA Hutchinson trace estimator DP-SGD Opacus

Closest papers by meaning

• Unveiling Privacy, Memorization, and Input Curvature Links (ICML 2024 · similarity 0.90)
• Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss (ICML 2024 · similarity 0.87)
• Assessing Per-Sample Membership Inference Vulnerability without Retraining (NeurIPS 2026 · similarity 0.85)
• Global curvature for second-order optimization of neural networks (ICML 2025 · similarity 0.85)
• Memorization Through the Lens of Curvature of Loss Function Around Samples (ICML 2024 · similarity 0.85)
• Hardness of Learning Neural Networks under the Manifold Hypothesis (NeurIPS 2024 · similarity 0.84)