Research map / Adversarial robustness and security / Attacks on text and graphs
Adv-CPG: A Customized Portrait Generation Framework with Facial Adversarial Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adv-CPG builds a tuning-free SDXL-based customized portrait generator whose outputs double as black-box facial adversarial examples that evade face recognition systems, using an ID encryptor and an encryption enhancer.
Problem
Customized portrait generation (CPG) produces high-fidelity portraits from a face image and text, but these portraits can be tracked and misused by malicious face recognition systems; existing adversarial protection methods only protect the original face and do not support personalized portrait gene
Approach
Embed adversarial identity-misleading signals into the generated customized portrait itself: an IP-Adapter-style ID encryptor injects target identity features from CLIP and white-box FR models, an ID-gradient encryption enhancer guides denoising toward the target identity, and an MMIC adds LLaVA-generated fine-grained facial descriptions for fidelity.
Key results
- Average ASR on face verification across four FR models and two datasets is 79.65%, about 28.1% above TIP-IM and 2.86% above DiffAM.
- On LFW face identification, Adv-CPG averages 34.55% Rank-1 targeted and 61.35% Rank-5 targeted ASR, the best among compared methods.
- Ablation on CelebA-HQ: average ASR drops from 79.28% to 56.04% without the ID encryptor and to 71.02% without the encryption enhancer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Adv-CPG instead of TIP-IM | black-box face verification · FFHQ and CelebA-HQ | average ASR (%): 28.1% | Helped |
| Use Adv-CPG instead of DiffAM | black-box face verification · FFHQ and CelebA-HQ | average ASR (%): 2.86% | Helped |
| Use Adv-CPG instead of GIFT | face identification · LFW, Rank-1 targeted | average ASR (%): 32.55 -> 34.55 | Helped |
| Use Adv-CPG instead of GIFT | face identification · LFW, Rank-5 targeted | average ASR (%): 59.95 -> 61.35 | Helped |
| Add the ID encryptor (ablation) | Adv-CPG · CelebA-HQ, face verification | average ASR (%): 56.04 -> 79.28 | Helped |
| Add the encryption enhancer (ablation) | Adv-CPG · CelebA-HQ, face verification | average ASR (%): 71.02 -> 79.28 | Helped |
Surprising findings
- Adv-CPG has the lowest-quality-metric advantage not uniformly: Adv-Makeup scores best on FID/PSNR/SSIM, which the authors attribute to its minimal eye-shadow-only edits, while its ASR is low.
Limitations
- Evaluation limited to black-box face verification/identification on a small set of FR models and two commercial APIs; physical-world attacks not evaluated beyond claims.
- Relies on white-box FR models and an LLM-generated facial description pipeline (LLaVA-1.5) that may add cost and variability.
- Background and scene generation quality and fidelity are assessed only qualitatively.
Experimental setup
Datasets: FGID CelebA SFHQ FFHQ CelebA-HQ LFW
Benchmarks: FFHQ CelebA-HQ LFW Face++ Aliyun
Metrics: attack success rate Rank-1 targeted ASR Rank-5 targeted ASR confidence score FID PSNR SSIM
Baselines: PGD MI-FGSM TIP-IM Clip2Protect GIFT DiffAM SD4Privacy Adv-Diffusion P3-Mask DFPP
Builds on: Stable Diffusion XL IP-Adapter PuLID InstantID Adv-Diffusion FreeTuner
Closest papers by meaning
- IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generation (CVPR 2025 · similarity 0.87)
- Non-Adaptive Adversarial Face Generation (NeurIPS 2025 · similarity 0.85)
- Harnessing Global-Local Collaborative Adversarial Perturbation for Anti-Customization (CVPR 2025 · similarity 0.85)
- Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing (CVPR 2025 · similarity 0.85)
- FaceRSA: RSA-Aware Facial Identity Cryptography Framework (AAAI 2024 · similarity 0.84)
- Prompt-Agnostic Adversarial Perturbation for Customized Diffusion Models (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.