Research map / Image generation / Editing and style transfer
Enhancing Facial Privacy Protection via Weakening Diffusion Purification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes adversarially modifying the latent code of a Stable Diffusion LDM for facial privacy protection, using learned unconditional (null-text) embeddings to weaken diffusion purification and self-attention guidance to preserve structure.
Problem
Protecting facial privacy against unauthorized automatic face recognition by generating protected face images that impersonate a target identity; existing diffusion-based methods suffer from diffusion purification that erases adversarial modifications, yielding low protection success rate.
Approach
Learn per-timestamp unconditional embeddings as null-text guidance in the reverse diffusion of an LDM so that identity-related adversarial modifications persist, and align self-attention maps between original and protected latents to keep structure, with direct unrestricted optimization of the adversarial latent code.
Key results
- Average PSR of 79.17 across four black-box FR models and both datasets, versus 77.88 for DiffAM and 51.05 for DiffProtect.
- FID of 15.32 in Table 2, the best among compared methods, versus 26.10 for DiffAM.
- On the synthesized-target obfuscation/impersonation setting, PSR of 90.2 on IRSE50 and 85.0 on Mobileface (Table 3), compared with CLIP2Protect's 83.4 and 62.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add learned unconditional embeddings as null-text guidance (ablation) | Ours · Facial privacy protection across timestamps | PSR | Helped |
| Add learned unconditional embeddings as null-text guidance (ablation) | Ours · Facial privacy protection across timestamps | FID | Helped |
| Add self-attention guidance (ablation) | Ours · Facial privacy protection across timestamps | PSR | Hurt |
| Add self-attention guidance (ablation) | Ours · Facial privacy protection across timestamps | FID | Helped |
| Use Ours instead of DiffAM | Facial privacy protection · CelebA-HQ and LADN average | PSR: about 1.5% | Helped |
| Use Ours instead of DiffAM | Facial privacy protection · CelebA-HQ and LADN average | FID: 26.1015 -> 15.3212 | Helped |
Surprising findings
- Removing self-attention guidance raises PSR but substantially worsens FID, showing a protection-quality trade-off.
Limitations
- PSR on IR152 and Facenet is lower than DiffAM in some settings (e.g., Facenet 59.53 vs 64.67 on CelebA-HQ), and the average gain over DiffAM is small (about 1.5 points).
- Structure preservation via self-attention lowers SSIM relative to DiffAM and DiffProtect.
- Impersonating real identities raises ethical concerns; obfuscation experiments are only qualitative for one setting.
Experimental setup
Datasets: CelebA-HQ LADN 100k Faces Generated by AI
Benchmarks: CelebA-HQ LADN
Metrics: protection success rate (PSR) at FAR 0.01 FID PSNR SSIM
Baselines: PGD MI-FGSM TI-DIM TIP-IM Adv-Makeup AMT-GAN CLIP2Protect DiffAM DiffProtect
Builds on: DiffProtect Null-text inversion DDIM inversion Stable Diffusion AMT-GAN
Closest papers by meaning
- Transferable Facial Privacy Protection against Blind Face Restoration via Domain-Consistent Adversarial Obfuscation (ICML 2024 · similarity 0.91)
- DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection (CVPR 2024 · similarity 0.90)
- Perturb a Model, Not an Image: Towards Robust Privacy Protection via Anti-Personalized Diffusion Models (NeurIPS 2025 · similarity 0.89)
- SimAC: A Simple Anti-Customization Method for Protecting Face Privacy against Text-to-Image Synthesis of Diffusion Models (CVPR 2024 · similarity 0.88)
- Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model (AAAI 2024 · similarity 0.87)
- PID: Prompt-Independent Data Protection Against Latent Diffusion Models (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.