Research map / Adversarial robustness and security / Attacks on text and graphs
Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adv-Diffusion generates imperceptible adversarial face identity perturbations by adding gradient-based identity attack terms inside the reverse process of a pretrained latent inpainting diffusion model, conditioned on identity-agnostic-masked faces.
Problem
Adversarial face images for attacking face recognition systems suffer from low transferability and high detectability; pixel-space L_p perturbations and patch/makeup attacks are either easily detected or non-transferable.
Approach
Inject adversarial gradients (from a face recognition model against the target identity) into the latent reverse diffusion process of a pretrained Stable Diffusion inpainting model, conditioned on the identity-sensitive region, with an adaptive strength weight tied to the predicted variance so perturbations concentrate on surrounding semantic regions (hair, ears, background) while preserving ident
Key results
- On CelebA-HQ, Adv-Diffusion (T=50) reaches 53.33% ASR on IR152 and 84.11% on IRSE50 in black-box setting, above MI-FGSM (44.81%, 77.05%).
- Adv-Diffusion achieves FID 15.51 on CelebA-HQ versus 79.42 for MI-FGSM and 22.57 for AMT-GAN.
- In a user study with 27 participants, 65.6% preferred Adv-Diffusion's adversarial images as best.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Adv-Diffusion instead of the strongest listed baseline | black-box face recognition attacks · CelebA-HQ and FFHQ, four target models | ASR: 53.33 vs 44.81; 84.11 vs 77.05; 36.84 vs 31.80; 73.02 vs 65.30 (CelebA-HQ); 53.42 vs 46.30; 82.91 vs 81.00; 32.21 vs 31. | Helped |
| use Adv-Diffusion instead of AMT-GAN | adversarial face image generation · CelebA-HQ | FID: 22.57 -> 15.51 | Helped |
| use Adv-Diffusion instead of MIFGSM | adversarial face image generation · CelebA-HQ | PSNR: 28.85 -> 29.01 | Helped |
| use Adv-Diffusion instead of PGD | adversarial face image generation · CelebA-HQ | SSIM: 0.85 -> 0.80 | Hurt |
| add adaptive strength (ablation) | Adv-Diffusion · CelebA-HQ, IR152 | ASR: 71.11 -> 53.43 | Hurt |
| add adaptive strength (ablation) | Adv-Diffusion · CelebA-HQ, IR152 | FID: 62.37 -> 15.34 | Helped |
| add adaptive strength (ablation) | Adv-Diffusion · CelebA-HQ, IR152 | PSNR: 12.62 -> 22.01 | Helped |
| add masked image condition (ablation) | Adv-Diffusion · CelebA-HQ, IR152 | ASR: 59.11 -> 53.43 | Hurt |
| add masked image condition (ablation) | Adv-Diffusion · CelebA-HQ, IR152 | FID: 55.23 -> 15.34 | Helped |
| add masked image condition (ablation) | Adv-Diffusion · CelebA-HQ, IR152 | PSNR: 13.22 -> 22.01 | Helped |
| increase T from 50 to 1000 (ablation) | Adv-Diffusion · CelebA-HQ, IR152 | ASR: 53.33 -> 56.71 | Helped |
| increase T from 50 to 1000 (ablation) | Adv-Diffusion · FFHQ | FID: 18.10 -> 45.51 | Hurt |
Surprising findings
- Removing the adaptive strength raises ASR to 71.11 but drops PSNR to 12.62 and raises FID to 62.37, showing a stealthiness-transferability trade-off.
- AMT-GAN attains a very low PSNR (9.31) despite competitive FID, and the paper's SSIM for Adv-Diffusion (0.80) is below PGD (0.85).
Limitations
- Relies on pretrained generative models, producing artifacts and failure cases for complex hairstyles and backgrounds.
- Mainly designed for face images; non-face results are only preliminary.
- Quality and ASR trade off through T and s; higher T gives more deformation and lower image quality (FID 45.51 at T=1000 on FFHQ).
Experimental setup
Datasets: FFHQ CelebA-HQ
Benchmarks: FFHQ CelebA-HQ
Metrics: attack success rate (ASR) FID PSNR SSIM similarity score
Baselines: FGSM PGD MI-FGSM AMT-GAN Adv-Attribute Adv-Hat Gen-AP
Builds on: latent diffusion model Stable Diffusion inpainting DDPM FGSM PGD MI-FGSM AMT-GAN
Closest papers by meaning
- AdvAD: Exploring Non-Parametric Diffusion for Imperceptible Adversarial Attacks (NeurIPS 2024 · similarity 0.90)
- CAT: Contrastive Adversarial Training for Evaluating the Robustness of Protective Perturbations in Latent Diffusion Models (ICML 2025 · similarity 0.90)
- Variance as a Catalyst: Efficient and Transferable Semantic Erasure Adversarial Attack for Customized Diffusion Models (ICML 2025 · similarity 0.89)
- MimicDiffusion: Purifying Adversarial Perturbation via Mimicking Clean Diffusion Model (CVPR 2024 · similarity 0.88)
- AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion Models (ICML 2025 · similarity 0.88)
- Prompt-Agnostic Adversarial Perturbation for Customized Diffusion Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.