Research map / Image generation / Editing and style transfer
Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FaceLock, an adversarial perturbation method that protects portraits from malicious diffusion-based editing by erasing biometric identity after editing rather than canceling the edit, and critiques common image-editing metrics.
Problem
Malicious diffusion-based edits to human portraits threaten privacy and identity; existing perturbation defenses that try to cancel edit effects are unstable across diverse editing instructions.
Approach
Optimize adversarial perturbations so that, after editing, the subject's facial biometric information is destroyed or altered: the facial recognition model (CVLFace) and a feature-embedding distance from pretrained CNNs are integrated into the diffusion loop via PGD, with pixel-level penalties found less effective.
Key results
- FaceLock reaches FR similarity 0.315 versus 0.590 for PhotoGuard and 0.641 for EditShield in Table 1 (lower is better).
- FaceLock achieves the highest LPIPS (0.436) among all methods in Table 1, and the lowest FR under Gaussian blur, rotation and JPEG purification in Table 3.
- Perturbation budget 0.02 gives FR 0.314 on CelebA-HQ with InstructPix2Pix; budget 0.05 gives 0.196 but shows visible artifacts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FACELOCK instead of the strongest baseline, PhotoGuard Encoder attack | Portrait protection with InstructPix2Pix · CelebA-HQ, InstructPix2Pix edits | FR: 0.590±0.264 -> 0.315±0.109 | Helped |
| Use FACELOCK instead of the strongest baseline, PhotoGuard Encoder attack | Portrait protection with InstructPix2Pix · CelebA-HQ, InstructPix2Pix edits | LPIPS: 0.403±0.071 -> 0.436±0.065 | Helped |
| Add diffusion-process optimization to facial-recognition perturbation (CVL -> CVL-D) (ablation) | FACELOCK design · InstructPix2Pix edits | FR: 0.667 -> 0.380 | Helped |
| Add diffusion-process optimization to facial-recognition perturbation (CVL -> CVL-D) (ablation) | FACELOCK design · InstructPix2Pix edits | LPIPS: 0.091 -> 0.381 | Helped |
| Add pixel-level penalty to CVL-D (CVL-D -> CVL-DP) (ablation) | FACELOCK design · InstructPix2Pix edits | LPIPS: 0.381 -> 0.381 | No clear effect |
| Add pixel-level penalty to CVL-D (CVL-D -> CVL-DP) (ablation) | FACELOCK design · InstructPix2Pix edits | FR: 0.380 -> 0.573 | Hurt |
| Add feature-embedding disparity loss to CVL-D (ablation) | FACELOCK design · InstructPix2Pix edits | LPIPS: 0.381 -> 0.423 | Helped |
| Add feature-embedding disparity loss to CVL-D (ablation) | FACELOCK design · InstructPix2Pix edits | FR: 0.380 -> 0.377 | Helped |
| Increase perturbation budget from 0.01 to 0.02 (ablation) | FACELOCK · InstructPix2Pix edits | FR: 0.557 -> 0.314 | Helped |
| Increase perturbation budget from 0.02 to 0.03 (ablation) | FACELOCK · InstructPix2Pix edits | FR: 0.314 -> 0.259 | Helped |
| Increase perturbation budget from 0.03 to 0.04 (ablation) | FACELOCK · InstructPix2Pix edits | FR: 0.259 -> 0.216 | Helped |
| Increase perturbation budget from 0.04 to 0.05 (ablation) | FACELOCK · InstructPix2Pix edits | FR: 0.216 -> 0.196 | Helped |
Surprising findings
- CLIP-based scores can rank over-edited images (strong target-element replication with identity loss) higher than visually balanced edits, so they can be gamed.
- SSIM and PSNR computed against the undefended edit can wrongly score a failed defense as better when the edit diverges more from the undefended output.
Limitations
- Tailored to single portraits; extending to images with multiple individuals is unresolved.
- May require adaptation for rectified flow and other emerging generative models.
- Evaluation limited to InstructPix2Pix as the main target model and 25 manually chosen prompts.
Experimental setup
Datasets: CelebA-HQ
Benchmarks: CelebA-HQ
Metrics: CLIP-S PSNR SSIM LPIPS CLIP-I FR similarity
Baselines: PhotoGuard EditShield Untargeted encoder attack CW L2 attack VAE attack
Builds on: PhotoGuard EditShield PGD InstructPix2Pix CVLFace
Closest papers by meaning
- DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing (ICLR 2025 · similarity 0.88)
- Can Protective Perturbation Safeguard Personal Data from Being Exploited by Stable Diffusion? (CVPR 2024 · similarity 0.86)
- Perturb a Model, Not an Image: Towards Robust Privacy Protection via Anti-Personalized Diffusion Models (NeurIPS 2025 · similarity 0.86)
- IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generation (CVPR 2025 · similarity 0.86)
- SimAC: A Simple Anti-Customization Method for Protecting Face Privacy against Text-to-Image Synthesis of Diffusion Models (CVPR 2024 · similarity 0.86)
- MYOPIA: Protecting Face Privacy from Malicious Personalized Text-to-Image Synthesis via Unlearnable Examples (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.