Research map / Adversarial robustness and security / Attacks on text and graphs
Step Vulnerability Guided Mean Fluctuation Adversarial Attack against Conditional Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Mean Fluctuation Attack (MFA) that shifts the mean of predicted noise in conditional diffusion reverse processes via adversarial image conditions, and MFA-MVS, which targets the most vulnerable reverse step modeled by step vulnerability.
Problem
Adversarial attacks against conditional diffusion models (e.g., inpainting and super-resolution) to disrupt generation as a defense against misuse; existing attacks treat the reverse process as a black box and sample attack steps uniformly.
Approach
Attack the reverse process by perturbing image conditions to shift the mean of estimated noise (Mean Fluctuation Attack), and sample attack timesteps according to a modeled step vulnerability (chain amplification, stability, transferability) rather than uniformly, or fix the single most vulnerable step.
Key results
- On Places365 thick-mask inpainting, MFA-MVS reaches FID 52.5 and Delta E 12.77 versus 18.4 and 7.46 for AdvDM (random mask thick setting).
- On ImageNet super-resolution, MFA-MVS lowers PSNR to 16.3 and SSIM to 0.37, compared with 20.4 and 0.56 for no attack.
- Attacking the most vulnerable step (around t=0.8T) also improves AdvDM, indicating step vulnerability transfers across attack methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use most-vulnerable-step targeting instead of uniform step sampling (ablation) | MFA on LDM · Inpainting, Random Mask Thick | FID: 33.4 -> 52.5 | Helped |
| Use most-vulnerable-step targeting instead of uniform step sampling (ablation) | MFA on LDM · Inpainting, Random Mask Medium | FID: 32.2 -> 46.8 | Helped |
| Use most-vulnerable-step targeting instead of uniform step sampling (ablation) | MFA on LDM · Inpainting, Random Mask Thin | FID: 26.3 -> 29.1 | Helped |
| Use most-vulnerable-step targeting instead of uniform step sampling (ablation) | MFA on LDM · Super-resolution, ImageNet | PSNR: 17.4 -> 16.3 | Helped |
| Use step-vulnerability sampling (MFA-VT) instead of uniform step sampling (ablation) | MFA on LDM · Inpainting, Random Mask Thick | FID: 33.4 -> 44.9 | Helped |
| Use step-vulnerability sampling (MFA-VT) instead of uniform step sampling (ablation) | MFA on LDM · Super-resolution, ImageNet | PSNR: 17.4 -> 16.9 | Helped |
| Increase attacking iterations from 10 to 1000 with MFA-VT (ablation) | MFA-VT on LDM · Inpainting | FID: 17.9 -> 63.4 | Helped |
| Increase attacking iterations from 10 to 1000 with MFA-MVS (ablation) | MFA-MVS on LDM · Inpainting | FID: 19.1 -> 59.2 | Helped |
| Use MFA-MVS rather than AdvDM-MVS | LDM · Inpainting, Random Mask Thick | FID: 22.9 -> 52.5 | Helped |
| Use MFA-MVS rather than AdvDM-MVS | LDM · Super-resolution, ImageNet | PSNR: 18.9 -> 16.3 | Helped |
Surprising findings
- A 10% shift in the mean of the initial Gaussian noise can collapse the reverse process into a blank image.
- Attacking only the single most vulnerable step can outperform sampling steps by vulnerability when the iteration budget is small, while the reverse holds for large budgets.
Limitations
- Step vulnerability model is fit empirically to LDM and SR3 with linear and logistic fits, with limited theoretical grounding.
- Experiments are limited to image-conditioned tasks (inpainting and super-resolution); other prompt types are not evaluated.
Experimental setup
Datasets: Places365 ImageNet
Benchmarks: Places365 inpainting ImageNet super-resolution
Metrics: FID Delta E PSNR SSIM
Baselines: Embedding Attack AdvDM AdvDM-MVS
Builds on: AdvDM Latent Diffusion Model SR3
Closest papers by meaning
- Variance as a Catalyst: Efficient and Transferable Semantic Erasure Adversarial Attack for Customized Diffusion Models (ICML 2025 · similarity 0.88)
- Towards Understanding the Robustness of Diffusion-Based Purification: A Stochastic Perspective (ICLR 2025 · similarity 0.88)
- Intriguing Properties of Diffusion Models: An Empirical Study of the Natural Attack Capability in Text-to-Image Generative Models (CVPR 2024 · similarity 0.88)
- LoRID: Low-Rank Iterative Diffusion for Adversarial Purification (AAAI 2025 · similarity 0.88)
- Diffusion Policy Attacker: Crafting Adversarial Attacks for Diffusion-based Policies (NeurIPS 2024 · similarity 0.87)
- AdvAD: Exploring Non-Parametric Diffusion for Imperceptible Adversarial Attacks (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.