Research map / Federated learning and privacy / Differential privacy
Unveiling Privacy, Memorization, and Input Curvature Links
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives theoretical links between stability-based memorization, input loss curvature (trace of input Hessian), and differential privacy, and validates them empirically on CIFAR100, CIFAR10 and ImageNet.
Problem
Input loss curvature has been shown empirically to proxy stability-based memorization scores at ~3 orders of magnitude less compute, but lacks theoretical grounding; the links between memorization, input curvature and differential privacy are unclear.
Approach
Derives upper bounds showing memorization is bounded by input loss curvature plus a constant (via error stability, generalization, uniform model bias and Hessian Lipschitzness), that curvature is bounded by the DP parameter, and that memorization is bounded by 1-e^{-epsilon}; also tightens privacy-to-stability bound from L(e^eps-1) to L(1-e^{-eps}).
Key results
- Empirical memorization vs. input loss curvature shows a clear linear trend on CIFAR100 (1000 Small Inception models) and ImageNet (100 ResNet50 models), with a sub-population loss-bound correction improving the fit.
- Measured memorization of the top 500 most memorized CIFAR100 samples increases with privacy budget epsilon and lies well below the theoretical bound 1-e^{-epsilon}.
- Proves epsilon-DP implies L(1-e^{-epsilon})-stability, improving the prior L(e^epsilon-1) bound.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase input loss curvature | Small Inception on CIFAR100 and ResNet50 on ImageNet · CIFAR100; ImageNet | memorization score | Mixed |
| Increase the privacy budget ε | DP-SGD ResNet18 · CIFAR100 | memorization score | Helped |
| Increase the privacy budget ε | DP-SGD ResNet18 · CIFAR10; CIFAR100 | input loss curvature | Hurt |
| Model a sub-population-specific loss bound | Curvature–memorization fit · CIFAR100; ImageNet | fit to empirical curvature and memorization | Helped |
| Vary the finite-difference step h from 1×10⁻¹ to 1×10⁻³ | Input curvature estimation · CIFAR100; ImageNet | input loss curvature | No clear effect |
| Vary the number of Rademacher vectors n among 5, 10, and 20 | Input curvature estimation · CIFAR100; ImageNet | input loss curvature | No clear effect |
| Use DP-SGD with stronger privacy | ResNet18 · CIFAR100 top 500 most-memorized examples | memorization score | Hurt |
Surprising findings
- Measured memorization scores are significantly lower than the 1-e^{-epsilon} bound, suggesting DP-SGD may be overly conservative.
- Linearity between memorization and curvature diminishes at extremes of the data range, requiring a sub-population loss-bound correction.
Limitations
- Theorems rely on assumptions (error stability, generalization, uniform model bias, Hessian Lipschitzness) that are only qualitatively justified for deep nets.
- Loss bound L is not constant for cross-entropy; empirical fitting of L is needed to match theory.
- Constant offset c1 and c2 are not tightly estimated; the upper bound is loose in privacy experiments.
Experimental setup
Datasets: CIFAR100 CIFAR10 ImageNet
Benchmarks: CIFAR100 ImageNet CIFAR10
Metrics: memorization score input loss curvature (Hutchinson trace estimate) fitted linear relation loss bound fit
Baselines: stability-based memorization score (Feldman & Zhang) C-score proxy
Builds on: Feldman memorization score Hutchinson trace estimator DP-SGD Lipschitz Hessian stability analysis
Closest papers by meaning
- Curvature Clues: Decoding Deep Learning Privacy with Input Loss Curvature (NeurIPS 2024 · similarity 0.90)
- Memorization Through the Lens of Curvature of Loss Function Around Samples (ICML 2024 · similarity 0.90)
- A Geometric Framework for Understanding Memorization in Generative Models (ICLR 2025 · similarity 0.86)
- Towards Memorization Estimation: Fast, Formal and Free (ICML 2025 · similarity 0.85)
- Classifier-Free Guidance Inside the Attraction Basin May Cause Memorization (CVPR 2025 · similarity 0.84)
- Learning the Signature of Memorization in Autoregressive Language Models (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.