Research map / Adversarial robustness and security / Attacks on text and graphs
Transferable Adversarial Face Attack with Text Controlled Attribute
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TCA² to generate photorealistic, text-controlled adversarial impersonation faces on a StyleGAN2 latent space, improving black-box transferability via data augmentation and meta-learning based model augmentation.
Problem
Unrestricted adversarial impersonation attacks on face recognition offer limited control over facial attributes and transfer poorly to unseen black-box FR models.
Approach
Edit StyleGAN2 latent codes with a learned fusion network conditioned on CLIP text embedding, an inverted source latent, and the target's FR softmax vector; use CLIP guidance and LPIPS preservation losses, and meta-learning over FR models (meta-train/meta-test splits) plus random resize/pad input transforms to improve transferability.
Key results
- On CelebA-Identity, TCA² reaches 73.10% ASR on MobileFace and 53.72% on IR152, the best in those columns, versus 72.27% and 52.88% for Adv-Diffusion.
- On KID-F, TCA² achieves 39.64% ASR on FaceNet, above Adv-Diffusion's 35.99%, while Adv-Diffusion is slightly higher on IRSE50 (83.79 vs 82.31).
- Removing model augmentation drops MobileFace ASR on CelebA-Identity from 73.10% to 14.31%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TCA^2 instead of Adv-Diffusion | Black-box face impersonation attack · CelebA-Identity, MobileFace | ASR (%): 72.27 -> 73.10 | Helped |
| Use TCA^2 instead of Adv-Diffusion | Black-box face impersonation attack · CelebA-Identity, IRSE50 | ASR (%): 80.08 -> 78.42 | Hurt |
| Use TCA^2 instead of Adv-Diffusion | Black-box face impersonation attack · CelebA-Identity, IR152 | ASR (%): 52.88 -> 53.72 | Helped |
| Use TCA^2 instead of Adv-Diffusion | Black-box face impersonation attack · CelebA-Identity, FaceNet | ASR (%): 36.12 -> 42.26 | Helped |
| Use TCA^2 instead of Adv-Diffusion | Black-box face impersonation attack · KID-F, MobileFace | ASR (%): 65.55 -> 65.57 | Helped |
| Use TCA^2 instead of Adv-Diffusion | Black-box face impersonation attack · KID-F, IRSE50 | ASR (%): 83.79 -> 82.31 | Hurt |
| Use TCA^2 instead of Adv-Diffusion | Black-box face impersonation attack · KID-F, IR152 | ASR (%): 52.01 -> 53.51 | Helped |
| Use TCA^2 instead of Adv-Diffusion | Black-box face impersonation attack · KID-F, FaceNet | ASR (%): 35.99 -> 39.64 | Helped |
| Add data augmentation (ablation) | TCA^2 · CelebA-Identity and KID-F, MobileFace and FaceNet | ASR (%): 64.31 -> 73.10; 40.55 -> 42.26; 55.50 -> 65.57; 36.77 -> 39.64 | Helped |
| Add model augmentation (ablation) | TCA^2 · CelebA-Identity and KID-F, MobileFace and FaceNet | ASR (%): 14.31 -> 73.10; 20.72 -> 42.26; 15.76 -> 65.57; 10.22 -> 39.64 | Helped |
Surprising findings
- Without model augmentation, transferability collapses to near baseline levels, indicating meta-learning is the dominant transferability factor.
- Norm-based MI-FGSM achieves higher ASR than several unrestricted methods on IRSE50 (70.37%), so TCA² does not uniformly beat all baselines.
Limitations
- Not uniformly best: Adv-Diffusion beats TCA² on IRSE50 in both datasets, and MI-FGSM beats it on some columns
- Relies on StyleGAN2 inversion (BDInvert) and a fixed set of 18 text prompts
- Defense against unrestricted adversarial attacks is left unexplored
Experimental setup
Datasets: CelebA-Identity CelebA-HQ KID-F
Benchmarks: CelebA-Identity KID-F Face++ Aliyun
Metrics: attack success rate FID PSNR SSIM
Baselines: MI-FGSM PGD Adv-Makeup Adv-Hat Adv-Attribute Latent-HSJA Adv-Diffusion
Builds on: StyleGAN2 CLIP DIM Adv-Attribute Adv-Diffusion
Closest papers by meaning
- Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model (AAAI 2024 · similarity 0.86)
- DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection (CVPR 2024 · similarity 0.85)
- Non-Adaptive Adversarial Face Generation (NeurIPS 2025 · similarity 0.85)
- Improving the Transferability of Adversarial Attacks on Face Recognition with Diverse Parameters Augmentation (CVPR 2025 · similarity 0.85)
- Harnessing Global-Local Collaborative Adversarial Perturbation for Anti-Customization (CVPR 2025 · similarity 0.85)
- Recoverable Facial Identity Protection via Adaptive Makeup Transfer Adversarial Attacks (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.