Research map / Image generation / Text-to-image diffusion
AdvPaint: Protecting Images from Inpainting Manipulation via Adversarial Attention Disruption
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdvPaint, an adversarial perturbation method that disrupts self- and cross-attention of a Stable Diffusion inpainting model, using two-stage perturbations split by an enlarged bounding box, to protect images from unauthorized inpainting.
Problem
Protecting published images from misuse by diffusion-based inpainting (foreground replacement or background generation), where prior perturbations targeting whole-image latent objectives fail because perturbations in masked regions are removed.
Approach
Instead of aligning latent representations with target latents, directly maximize the difference in query/key/value of self-attention and query of cross-attention between clean and perturbed images, and optimize separate perturbations inside and outside an enlarged bounding box around the object.
Key results
- On SD inpainter with bounding-box masks for background inpainting, AdvPaint reaches FID 303.90, outperforming the second-best Mist by 148.33 points.
- Foreground inpainting with segmentation masks: AdvPaint FID 347.88 and precision 0.0570 versus Photoguard FID 230.49 and precision 0.5244.
- Two-stage perturbation beats single-stage on all FG/BG settings (e.g., BG bbox FID 303.90 vs 266.00).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use attention-disruption perturbations instead of prior methods | Stable Diffusion Inpainting · Background inpainting, bounding-box mask | FID: 155.57 -> 303.90 | Helped |
| use attention-disruption perturbations instead of prior methods | Stable Diffusion Inpainting · Foreground inpainting, segmentation mask | Prec: 0.3030 -> 0.0570 | Helped |
| use two separate perturbations inside and outside the enlarged bounding box instead of one perturbation (ablation) | AdvPaint · Foreground inpainting, bounding-box mask | FID: 271.73 -> 289.63 | Helped |
| use two separate perturbations inside and outside the enlarged bounding box instead of one perturbation (ablation) | AdvPaint · Background inpainting, segmentation mask | FID: 191.15 -> 219.07 | Helped |
| use two separate perturbations inside and outside the enlarged bounding box instead of one perturbation (ablation) | AdvPaint · Foreground inpainting, segmentation mask | LPIPS: 0.6940 -> 0.6731 | Hurt |
| replace prior latent-space objectives with attention-disruption objectives | Stable Diffusion Inpainting · Foreground inpainting, segmentation mask | FID: 182.12 -> 347.88 | Helped |
Surprising findings
- Prior latent-space objectives (AdvDM, Mist, SDST) lose effectiveness when perturbations are masked out before the inpainting model, so they barely help on inpainting.
- Existing protections are much less effective against DiT-based inpainters (Flux, SD3) and PixArt-delta, which patchify latents.
Limitations
- Computational cost grows with the number of target objects in multi-object images.
- Less effective against DiT-based inpainting models (Flux, SD3, PixArt-delta) that patchify latents.
- Evaluated on a modest in-house set of 100 images.
Experimental setup
Datasets: Pexels images Unsplash images DreamBooth dataset
Benchmarks: in-house 100-image inpainting set (FG/BG, seg and bbox masks)
Metrics: FID Precision LPIPS PSNR
Baselines: PhotoGuard AdvDM Mist CAAT SDST MetaCloak
Builds on: PGD Stable Diffusion inpainting PhotoGuard AdvDM Mist
Closest papers by meaning
- DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing (ICLR 2025 · similarity 0.88)
- Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI (ICLR 2025 · similarity 0.88)
- AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion Models (ICML 2025 · similarity 0.88)
- Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion Models (CVPR 2025 · similarity 0.88)
- Distraction is All You Need: Memory-Efficient Image Immunization against Diffusion-Based Image Editing (CVPR 2024 · similarity 0.87)
- Pixel Is Not a Barrier: An Effective Evasion Attack for Pixel-Domain Diffusion Models (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.