Research map / Federated learning and privacy / Privacy attacks and secure learning

Privacy-Preserving Low-Rank Adaptation Against Membership Inference Attacks for Latent Diffusion Models

AAAI 2025 · Federated learning and privacy › Privacy attacks and secure learning · approach: Membership inference and auditing · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MP-LoRA, a min-max membership-privacy LoRA for latent diffusion models, and its stabilized variant SMP-LoRA that divides the adaptation loss by (1 - λ·MI gain), which constrains local smoothness and improves convergence while defending against loss-based MI attacks.

Problem

LoRA-adapted latent diffusion models leak membership information to membership inference attacks, and a min-max privacy defense (MP-LoRA) optimizes unstably because local smoothness is unconstrained by the gradient norm.

Approach

Replace the additive MI-gain penalty with a ratio: the adaptation loss is divided by (1 - λ·MI gain + δ), so the Hessian norm becomes positively correlated with and bounded by the gradient norm, yielding stable optimization and privacy preservation.

Key results

• On Pokemon, SMP-LoRA reaches FID 0.32 and ASR 51.97% versus LoRA's FID 0.20 and ASR 82.27%, while MP-LoRA reaches FID 2.10 and ASR 51.67%.
• On CelebA_Large_5X, SMP-LoRA lowers ASR from 92.80% (LoRA) to 51.85% with FID 0.59 vs 0.53.
• The Hessian-gradient norm PCC is 0.043 for MP-LoRA (p=0.052) versus 0.761 (p<0.001) for SMP-LoRA.

Measured effects reported in the paper

Surprising findings

• Vanilla MP-LoRA defends privacy well but its FID rises to 2.10 on Pokemon, showing that the additive min-max objective destroys generation quality.
• Lowering lambda below 0.05 raises ASR above 50% on several datasets, so stronger generation quality trades off against protection.

Limitations

• Defense is evaluated mainly against the loss-based white-box MI attack, with ASR reported close to random only for that attack.
• Some settings show a trade-off: lower lambda improves image quality but weakens privacy, and FID is unavailable for small CelebA subsets.
• Hessian norm is estimated by power iteration every 100 steps rather than computed exactly.

Experimental setup

Datasets: Pokemon CelebA AFHQ MS-COCO

Benchmarks: Pokemon CelebA_Small CelebA_Large CelebA_Gender CelebA_Varying CelebA_Large_5X AFHQ_Large_5X MS-COCO_Large_5X

Metrics: FID KID ASR AUC TPR@5%FPR

Baselines: LoRA MP-LoRA black-box MI attack white-box gradient-based MI attack gradient clipping gradient normalization

Builds on: LoRA min-max membership privacy game (L0,L1)-smoothness DreamBooth

Closest papers by meaning

• Membership Inference Attacks on Diffusion Models via Quantile Regression (ICML 2024 · similarity 0.89)
• Improving LoRA in Privacy-preserving Federated Learning (ICLR 2024 · similarity 0.89)
• Differentially Private Federated Low Rank Adaptation Beyond Fixed-Matrix (NeurIPS 2025 · similarity 0.87)
• PID: Prompt-Independent Data Protection Against Latent Diffusion Models (ICML 2024 · similarity 0.87)
• Privacy Attacks on Image AutoRegressive Models (ICML 2025 · similarity 0.86)
• Private Adaptive Covariance Estimation via Gaussian Graphical Models (NeurIPS 2026 · similarity 0.86)