Research map / Adversarial robustness and security / Attacks on text and graphs
Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that diffusion purification provably lower-bounds the sum of evasion and spoofing errors for low-perturbation image watermarks, proposes a model-substitution PGD attack that breaks high-perturbation watermarks (TreeRing, StegaStamp), a black-box watermarked-noise spoofing attack, and a robustness-reliability trade-off for classifier-based deepfake detectors.
Problem
AI-generated image detection relies on watermarks and classifier-based deepfake detectors, whose robustness to attacks is poorly understood; the paper asks what fundamental limits and practical attacks constrain them.
Approach
Combines a Wasserstein-distance-based certified bound on diffusion purification (noise then denoise) errors for low-perturbation watermarks, a substitute-classifier PGD transfer attack for high-perturbation watermarks, a blending-based spoofing attack using watermarked noise, and a latent-space noise-robustness bound on AUROC for deepfake detectors.
Key results
- Diffusion purification at t=0.2 reduces AUROC of low-perturbation watermarks (e.g., RivaGAN, DwtDct, DwtDctSvd) to below 0.65.
- Model substitution PGD attack with epsilon=2/255 reduces TreeRing AUROC to 0.14.
- StegaStamp requires epsilon=12/255 before the adversarial attack degrades its AUROC to near random.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply diffusion purification to low-perturbation watermarks | RivaGAN, DwtDct, DwtDctSvd, WatermarkDM, and MBRS · ImageNet | AUROC | Hurt |
| Apply diffusion purification to TreeRing | TreeRing · ImageNet | AUROC | No clear effect |
| Apply model substitution PGD with ε=2/255 | TreeRing · ImageNet watermarked images | AUROC: AUROC ... to 0.14 | Hurt |
| Apply model substitution PGD with ε=12/255 | StegaStamp · ImageNet watermarked images | AUROC: 12/255 | Hurt |
| Apply spoofing by blending clean images with watermarked noise images | Watermarking methods evaluated · ImageNet | AUROC | Hurt |
| Increase diffusion purification strength from t=0.1 to t=0.2 (ablation) | Diffusion purification attack on watermarking methods · ImageNet watermarked images | PSNR: 29.77 -> 26.10 | Hurt |
| Increase diffusion purification strength from t=0.2 to t=0.3 (ablation) | Diffusion purification attack on watermarking methods · ImageNet watermarked images | SSIM: 0.72 -> 0.63 | Hurt |
| Add Gaussian noise to StegaStamp substitute-classifier training data (ablation) | Model substitution attack on StegaStamp · ImageNet watermarked images | attack transferability | Helped |
| Increase training noise σ for deepfake detectors (ablation) | ResNet-18 and VGG-16-BN detectors · FaceForensics++ | AUROC | Hurt |
| Apply Gaussian blur or JPEG compression after diffusion purification | Diffusion purification attack on watermarking methods · ImageNet watermarked images | AUROC | No clear effect |
| Apply DiffPure after model substitution attack | Model substitution attack on TreeRing and StegaStamp · ImageNet watermarked images | AUROC: TreeRing ε=8: 0.002 -> 0.531; StegaStamp ε=8: 0.923 -> 0.703 | Mixed |
Surprising findings
- TreeRing resists diffusion purification even though its high image-space perturbation would be expected to be robust, while a tiny adversarial budget defeats it.
- Post-attack mitigations like Gaussian blur and JPEG fail to restore watermark detection after diffusion purification.
Limitations
- Transferability of the adversarial attack depends on the substitute classifier architecture and training, with only a basic ResNet-18 used.
- Diffusion purification degrades image quality at larger t (e.g., t=0.3), and the theoretical bound relies on Wasserstein distance estimated via l2 surrogate.
- Adversarial perturbations at epsilon=12/255 may be perceptible.
Experimental setup
Datasets: ImageNet LAION-captions FaceForensics++
Benchmarks: ImageNet FaceForensics++
Metrics: AUROC TPR at low FPR PSNR SSIM
Baselines: RivaGAN DwtDct DwtDctSvd WatermarkDM MBRS StegaStamp TreeRing
Builds on: DiffPure PGD Tree-Ring watermark StegaStamp WatermarkDM
Closest papers by meaning
- Invisible Image Watermarks Are Provably Removable Using Generative AI (NeurIPS 2024 · similarity 0.90)
- On the Robustness of Watermarking for Autoregressive Image Generation (NeurIPS 2026 · similarity 0.90)
- Transferable Black-Box One-Shot Forging of Watermarks via Image Preference Models (NeurIPS 2025 · similarity 0.89)
- A Transfer Attack to Image Watermarks (ICLR 2025 · similarity 0.89)
- Leveraging Optimization for Adaptive Attacks on Image Watermarks (ICLR 2024 · similarity 0.88)
- ROBIN: Robust and Invisible Watermarks for Diffusion Models with Adversarial Optimization (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.