Research map / Adversarial robustness and security / Attacks on text and graphs
Recoverable Facial Identity Protection via Adaptive Makeup Transfer Adversarial Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RMT-GAN, a makeup-transfer GAN that adaptively selects a similar-geometry target face as both makeup reference and attack identity, and adds a cycle-consistency recovery module so authorised FR systems can still recognise protected photos.
Problem
Unauthorised face recognition threatens privacy; existing makeup-transfer adversarial attacks use a fixed, unrelated confusion identity that hurts attack success and visual quality, and protected photos cannot be recognised by authorised FR systems.
Approach
Select the adversarial target adaptively from the top 20% of reference faces closest in landmark geometry to the source (also serving as makeup reference), and train a recovery module with cycle-consistency and identity losses to remove the adversarial makeup and restore recognisable identity.
Key results
- RMT-GAN achieves average black-box ASR of 66.9 on CelebA-HQ and 88.9 on LADN, versus 61.6 and 68.2 for the strongest baseline CLIP2Protect.
- Adaptive target selection raises CelebA-HQ average ASR from fixed-target results to 66.9 (vs 58.4 on IR152 fixed to adaptive 91.4) and improves FID from 24.6 to 21.3.
- In a 25-person user study, RMT-GAN achieves a mean score of 83.3 versus 62.8 for AMT-GAN and 55.1 for Adv-Makeup.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive rather than fixed target selection (ablation) | RMT-GAN · CelebA-HQ, black-box hold-out models | ASR: 42.1 -> 74.0; 18.2 -> 29.4; 29.9 -> 47.8; 51.1 -> 82.4 | Helped |
| Use adaptive rather than fixed target selection (ablation) | RMT-GAN · LADN dataset, black-box hold-out models | ASR: 58.1 -> 74.8; 37.6 -> 70.2; 42.0 -> 75.1; 66.6 -> 87.7 | Helped |
| Use adaptive rather than fixed target selection (ablation) | RMT-GAN · CelebA-HQ | FID: 24.626 -> 21.254 | Helped |
| Use adaptive rather than fixed target selection (ablation) | RMT-GAN · CelebA-HQ | PSNR: 20.573 -> 21.171 | Helped |
| Use adaptive rather than fixed target selection (ablation) | RMT-GAN · LADN dataset | FID: 33.508 -> 29.777 | Helped |
| Use adaptive rather than fixed target selection (ablation) | RMT-GAN · LADN dataset | PSNR: 20.879 -> 21.654 | Helped |
| Use adaptive rather than fixed target selection (ablation) | RMT-GAN · CelebA-HQ | SSIM: 0.810 -> 0.811 | Helped |
| Use adaptive rather than fixed target selection (ablation) | RMT-GAN · LADN dataset | SSIM: 0.797 -> 0.800 | Helped |
| Add adaptive target selection instead of using fixed targets | RMT-GAN · CelebA-HQ and LADN dataset | ASR: 66.9 vs 61.6; 88.9 vs 68.2 | Helped |
| Add the recovery module and cycle-consistency training | RMT-GAN · CelebA-HQ and LADN dataset | recovered-image ASR | Helped |
Surprising findings
- Adv-Makeup achieves the best FID/PSNR/SSIM despite its low ASR, because it only edits eye regions.
Limitations
- Black-box attack evaluated only against a few FR models and two commercial APIs; the authors aim to improve effectiveness on more unauthorised FR systems.
- Current method applies full-face makeup, and local makeup flexibility is future work.
- Recovered images show some deviation from originals, giving slightly higher or lower ASR.
Experimental setup
Datasets: Makeup Transfer (MT) dataset CelebA-HQ LADN dataset
Benchmarks: CelebA-HQ LADN dataset
Metrics: attack success rate (ASR) at FAR@0.01 FID PSNR SSIM user study score
Baselines: FGSM PGD MI-FGSM TI-DIM TIP-IM Adv-Makeup AMT-GAN CLIP2Protect
Builds on: AMT-GAN PSGAN Adv-Makeup CLIP2Protect TIP-IM
Closest papers by meaning
- DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection (CVPR 2024 · similarity 0.89)
- Transferable Facial Privacy Protection against Blind Face Restoration via Domain-Consistent Adversarial Obfuscation (ICML 2024 · similarity 0.86)
- IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generation (CVPR 2025 · similarity 0.86)
- Non-Adaptive Adversarial Face Generation (NeurIPS 2025 · similarity 0.86)
- Enhancing Facial Privacy Protection via Weakening Diffusion Purification (CVPR 2025 · similarity 0.85)
- Transferable Adversarial Face Attack with Text Controlled Attribute (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.