Research map / Adversarial robustness and security / Attacks on text and graphs
Robust Secure Swap: Responsible Face Swap With Persons of Interest Redaction and Provenance Traceability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Secure Swap retrofits GAN-based face swap models with an ID Passport layer that redacts persons of interest (POI) and embeds a detachable, customer-specific invisible watermark into nonPOI outputs for provenance tracing.
Problem
Face swap models can be misused to impersonate celebrities or defraud ordinary people; existing defenses are mostly post-hoc forensic detection or adversarial perturbations applied to images, rather than proactive governance built into the generator itself.
Approach
Insert a learnable ID Passport layer into a copy of a pretrained GAN face swap model, trained with knowledge distillation from the teacher to keep nonPOI swaps faithful, a black-output loss for POI identities, and a watermark encoder/decoder pair whose encoder is detached after training so each released instance carries a unique watermark embedded in its weights.
Key results
- POI redaction success rate SRmask of 1.000 across BlendFace, SimSwap, FaceShifter and MobileFSGAN with 128 to 1024 POI IDs.
- Watermark extraction accuracy AccWM of 0.976 to 1.000 on nonPOI outputs, with TPR of at least 0.999.
- Watermarked BlendFace reaches PSNR 34.321 and SSIM 0.981, above Stable Signature (29.250), PluggableWM (31.284) and WFWM (34.165) in imperceptibility.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the ID Passport layer and watermarking to BlendFace | BlendFace · NonPOI face swaps | PSNR: 34.165 -> 34.321 | Helped |
| Add POI redaction and watermarking | BlendFace, SimSwap, FaceShifter, and MobileFSGAN · POI redaction across the evaluated POI-ID counts | SRmask: 1.000 | Helped |
| Add watermark embedding with the ID Passport layer | Secure Swap · NonPOI outputs across four face swap models | AccWM: 0.976–1.000 | Helped |
| Increase protected identities from 1024 to 4096 (ablation) | Secure Swap · POI redaction and nonPOI image quality | SRmask / LPIPS: 1.000 / 0.018 -> 1.000 / 0.018 | No clear effect |
| Increase training photos per protected identity from 1 to 16 (ablation) | Secure Swap · POI redaction | SRmask: 0.965 -> 1.000 | Helped |
| Train with image-processing attacks for watermark decoding (ablation) | Secure Swap BlendFace · Image-level attacks | AccWM | Helped |
| Increase POI count from 1k to ˜10k (ablation) | Secure Swap · POI redaction | SRmask: 1.000 -> 0.989 | Hurt |
| Increase POI count from 1k to ˜10k (ablation) | Secure Swap · NonPOI outputs | PSNR: 34.321 -> 33.897 | No clear effect |
| Fine-tune Secure Swap rather than train the original model from scratch (ablation) | BlendFace · Complete training runtime | training time: 95h 50m / 7h 52m | Helped |
| Apply JPEG compression (quality factor = 50) | Secure Swap BlendFace · Watermark extraction | AccWM: 76.85 | Hurt |
| Apply a white-box FGSM attack with ϵ = 0.1 | Secure Swap · POI redaction across four face swap models | SRmask: 1.00 | No clear effect |
Surprising findings
- JPEG compression (quality 50) and horizontal flip substantially reduce watermark accuracy (76.85% and 79.38%) before robustness augmentation.
- Training cost for GSS is far lower than full training of the original model (e.g., BlendFace 7h52m vs 95h50m) because it is a fine-tune from the teacher.
Limitations
- Currently applied only to GAN-based face swap models; diffusion-based models are not covered.
- Adding new POI requires retraining the model, though the cost is low.
- JPEG compression, flip, and crop degrade watermark accuracy substantially without augmentation.
Experimental setup
Datasets: CelebA VGGFace2 PubFig83
Benchmarks: CelebA VGGFace2 PubFig83
Metrics: SRmask AccWM TPR MSE PSNR SSIM LPIPS identity similarity attribute similarity FID
Baselines: Arrtributing Stable Signature PluggableWM WFWM FGSM PGD
Builds on: Stable Signature BlendFace SimSwap FaceShifter ArcFace
Closest papers by meaning
- Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing (CVPR 2025 · similarity 0.83)
- Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks (ICLR 2024 · similarity 0.83)
- Disguise without Disruption: Utility-Preserving Face De-identification (AAAI 2024 · similarity 0.82)
- WMCopier: Forging Invisible Watermarks on Arbitrary Images (NeurIPS 2025 · similarity 0.82)
- Steganographic Passport: An Owner and User Verifiable Credential for Deep Model IP Protection Without Retraining (CVPR 2024 · similarity 0.81)
- Transferable Facial Privacy Protection against Blind Face Restoration via Domain-Consistent Adversarial Obfuscation (ICML 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.