Research map / Image generation / Editing and style transfer
Pixel Is Not a Barrier: An Effective Evasion Attack for Pixel-Domain Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AtkPDM, an evasion attack that perturbs images via a UNet mid-block feature Wasserstein loss plus a VAE latent optimization to protect them from SDEdit-based editing by pixel-domain diffusion models.
Problem
Protecting images from SDEdit-based editing by pixel-domain diffusion models (PDMs), which prior adversarial protection methods designed for latent diffusion models (LDMs) fail to attack effectively.
Approach
Attack the denoising UNet's intermediate feature distributions by maximizing a closed-form 2-Wasserstein distance between clean and adversarial noisy features, constrain fidelity with a VGG16 feature Wasserstein loss, and optimize in the latent space of a model-agnostic Stable Diffusion VAE to improve naturalness.
Key results
- On Church, AtkPDM reaches SSIM 0.75 / PSNR 29.61 / LPIPS 0.40 / IA 0.76 under attack, versus AdvDM 0.89 / 31.06 / 0.17 / 0.93, i.e. stronger attack effectiveness on unconditional PDMs.
- AtkPDM+ keeps adversarial image quality high (Church SSIM 0.81, PSNR 28.64, LPIPS 0.13) while still outperforming AdvDM and Diff-Protect on attack metrics.
- Under JPEG compression, crop-and-resize, and LDM purification, the attack effectiveness is maintained or increased relative to no defense.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace semantic loss with feature-space attack loss, using the full AtkPDM method (ablation) | PDM image protection · Church | LPIPS (attack effectiveness): 0.17 ± 0.09 -> 0.40 ± 0.05 | Helped |
| Add latent optimization via VAE to AtkPDM (ablation) | AtkPDM · Church | LPIPS (adversarial image quality): 0.26 ± 0.04 -> 0.13 ± 0.02 | Helped |
| Add latent optimization via VAE to AtkPDM (ablation) | AtkPDM · Church | LPIPS (attack effectiveness): 0.40 ± 0.05 -> 0.33 ± 0.07 | Hurt |
| Add latent optimization via VAE to semantic-loss attack (ablation) | Semantic-loss method · Church | LPIPS (adversarial image quality): 0.73 ± 0.16 -> 0.17 ± 0.03 | Helped |
| Add latent optimization via VAE to semantic-loss attack (ablation) | Semantic-loss method · Church | LPIPS (attack effectiveness): 0.17 ± 0.09 -> 0.15 ± 0.06 | No clear effect |
| Add fidelity loss to semantic-loss attack with VAE (ablation) | Semantic-loss method with VAE · Church | LPIPS (attack effectiveness): 0.15 ± 0.06 -> 0.08 ± 0.03 | Hurt |
| Use feature attack and fidelity loss instead of semantic loss with VAE (ablation) | PDM image protection · Church | LPIPS (attack effectiveness): 0.08 ± 0.03 -> 0.40 ± 0.05 | Helped |
Surprising findings
- Defenses such as crop-and-resize and JPEG compression made the adversarial images more effective rather than less.
- Diff-Protect appears to attack DeepFloyd IF more strongly than AtkPDM, but the authors attribute this to severe corruption of its adversarial image quality.
Limitations
- Visual quality of attacks on PDMs is not directly comparable to results on LDMs.
- Evaluated only in the white-box setting on a few small 256px checkpoints, with black-box transfer shown on a single model pair.
- Experiments use only 30 images per victim model.
Experimental setup
Datasets: Church Cat CelebA-HQ
Benchmarks: Church Cat Face (CelebA-HQ)
Metrics: SSIM PSNR LPIPS Image Alignment (IA)
Baselines: AdvDM Diff-Protect Mist PhotoGuard
Builds on: AdvDM Diff-Protect PhotoGuard Mist MPGD
Closest papers by meaning
- DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing (ICLR 2025 · similarity 0.92)
- Distraction is All You Need: Memory-Efficient Image Immunization against Diffusion-Based Image Editing (CVPR 2024 · similarity 0.89)
- Intriguing Properties of Diffusion Models: An Empirical Study of the Natural Attack Capability in Text-to-Image Generative Models (CVPR 2024 · similarity 0.89)
- Perturbing Attention Gives You More Bang for the Buck: Subtle Imaging Perturbations That Efficiently Fool Customized Diffusion Models (CVPR 2024 · similarity 0.89)
- Targeted Attack Improves Protection against Unauthorized Diffusion Customization (ICLR 2025 · similarity 0.89)
- Anti-Diffusion: Preventing Abuse of Modifications of Diffusion-Based Models (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.