Research map / Image generation / Editing and style transfer
IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IDProtector, a feed-forward ViT-S/8 adversarial noise encoder that adds imperceptible perturbations to portrait photos to disrupt encoder-based ID-preserving generation (InstantID, IP-Adapter, IP-Adapter-Plus, PhotoMaker).
Problem
Encoder-based zero-shot ID-preserving generation (e.g., InstantID, IP-Adapter) threatens facial privacy, and existing protections target finetuning-based customization and are slow per-image optimization methods that don't cover encoder-based pipelines.
Approach
Train a ViT noise encoder that directly predicts bounded adversarial noise for a portrait in one forward pass, with a composite loss that attacks ArcFace and CLIP embeddings of multiple encoder-based pipelines, plus affine-misalignment noise injected into face alignment for robustness.
Key results
- IDProtector reduces InstantID ISM by over 0.4 (ISM 0.775 to 0.231 on CelebA) where baselines reduce it by less than 0.1.
- Protection takes 0.173 s per image versus 23.3 to 323 s for baselines and about 4089 s for PGD-based variant.
- Generalizes to unseen closed-source generators with ISM reduction of 0.10 on Midjourney and 0.54 on Jing Gou.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply IDProtector to the reference image | ID-Adapter · CelebA test | ISM: 0.176 -> 0.060 | Helped |
| apply IDProtector to the reference image | IP-Adapter-Plus · CelebA test | ISM: 0.240 -> 0.114 | Helped |
| apply IDProtector to the reference image | PhotoMaker · CelebA test | ISM: 0.265 -> 0.123 | Helped |
| apply IDProtector to the reference image | InstantID · CelebA test | ISM: 0.775 -> 0.231 | Helped |
| use per-image PGD optimization instead of the noise encoder (ablation) | IDProtector · CelebA test, IP-Adapter | ISM: 0.060 -> 0.024 | Helped |
| use per-image PGD optimization instead of the noise encoder (ablation) | IDProtector · CelebA test, IP-Adapter-Plus | ISM: 0.114 -> 0.072 | Helped |
| use per-image PGD optimization instead of the noise encoder (ablation) | IDProtector · CelebA test, PhotoMaker | ISM: 0.123 -> 0.094 | Helped |
| use per-image PGD optimization instead of the noise encoder (ablation) | IDProtector · CelebA test, InstantID | ISM: 0.231 -> -0.166 | Helped |
| use per-image PGD optimization instead of the noise encoder (ablation) | IDProtector · VGG Face, IP-Adapter | ISM: 0.105 -> 0.043 | Helped |
| use per-image PGD optimization instead of the noise encoder (ablation) | IDProtector · VGG Face, IP-Adapter-Plus | ISM: 0.138 -> 0.202 | Hurt |
| use per-image PGD optimization instead of the noise encoder (ablation) | IDProtector · VGG Face, PhotoMaker | ISM: 0.126 -> 0.228 | Hurt |
| use per-image PGD optimization instead of the noise encoder (ablation) | IDProtector · VGG Face, InstantID | ISM: 0.232 -> 0.707 | Hurt |
Surprising findings
- SER-FIQ quality scores were inconsistent with qualitative assessments of generated image quality, per the authors.
Limitations
- Adversarial noise is not fully invisible; trade-off between attack effectiveness and imperceptibility.
- Image quality of generated outputs is noticeably affected by protection.
Experimental setup
Datasets: CelebA VGG Face
Benchmarks: CelebA test subset VGG Face subset
Metrics: ISM (ArcFace cosine similarity) FDR SER-FIQ PSNR SSIM time per image
Baselines: ACE AdvDM Anti-DreamBooth PhotoGuard SimAC PGD
Builds on: PGD Anti-DreamBooth InstantID IP-Adapter PhotoMaker
Closest papers by meaning
- Adv-CPG: A Customized Portrait Generation Framework with Facial Adversarial Attacks (CVPR 2025 · similarity 0.87)
- Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing (CVPR 2025 · similarity 0.86)
- Recoverable Facial Identity Protection via Adaptive Makeup Transfer Adversarial Attacks (AAAI 2025 · similarity 0.86)
- Enhancing Facial Privacy Protection via Weakening Diffusion Purification (CVPR 2025 · similarity 0.86)
- SimAC: A Simple Anti-Customization Method for Protecting Face Privacy against Text-to-Image Synthesis of Diffusion Models (CVPR 2024 · similarity 0.85)
- PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.