Research map / Adversarial robustness and security / Attacks on text and graphs
From Trojan Horses to Castle Walls: Unveiling Bilateral Data Poisoning Effects in Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that BadNets-style data poisoning (trigger plus relabeling, with no change to diffusion training or sampling) degrades text-to-image diffusion models, causing trigger amplification that can be exploited for poisoned-data detection and for robust classifier training.
Problem
Prior data poisoning attacks on diffusion models required altering the training objective, noise distribution, or sampling process, which departs from the BadNets threat model. The paper asks whether plain BadNets-style poisoning of the training set alone can compromise diffusion models, and what de
Approach
Apply an unmodified BadNets-style poisoning (image trigger plus target-label relabeling) to the training set only, then analyze generated images for prompt misalignment and trigger presence. The poisoned DMs amplify triggers in generations, which is reused to improve poisoned-data detection, to build classifiers from generated data, and to link poisoning with data replication.
Key results
- With BadNets-like poisoning at 10% on ImageNette with SD, 54% of generations were prompt-misaligned with the trigger (G1) and 19.4% matched the prompt but contained the trigger (G2), versus 26.1% clean normal generations (G4).
- Training a ResNet-50 on the DM-generated set reduced ASR for BadNets-1 at 1% poisoning on ImageNette from 87.1% to 0.65%, while test accuracy fell from 99.4% to 96.9%.
- Diffusion classifiers on CIFAR10 with poisoning ratio 1% had ASR 62.38% versus 99.40% for ResNet-18, and filtering the top 10% denoising losses lowered ASR to 13.62%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Poison the training data with BadNets-like image triggers and relabeling at p=10% | DDPM on CIFAR10 and Stable Diffusion on ImageNette and Caltech15 · CIFAR10; ImageNette; Caltech15 | FID: 5.868 -> 5.460; 22.912 -> 22.879; 46.489 -> 44.260 | No clear effect |
| Train the image classifier on poisoned-DM-generated data instead of poisoned training data | ResNet-50 · ImageNette, BadNets-1, p=1% | ASR (%): 87.104 -> 0.650 | Helped |
| Train the image classifier on poisoned-DM-generated data instead of poisoned training data | ResNet-50 · ImageNette, BadNets-2, p=1% | ASR (%): 64.621 -> 1.357 | Helped |
| Train the image classifier on poisoned-DM-generated data instead of poisoned training data | ResNet-50 · Caltech15, WaNet, p=1% | ASR (%): 90.000 -> 29.567 | Helped |
| Filter out the top 10% of denoising-loss values (ablation) | Poisoned diffusion classifier on CIFAR10 · CIFAR10, BadNets-1, p=1% | ASR (%): 62.38 -> 13.62 | Helped |
| Poison duplicate images rather than random images (ablation) | BadNets-poisoned Stable Diffusion · ImageNette, p=5% | G1 ratio: 33.8% -> 37.8% | Helped |
| Poison duplicate images rather than random images (ablation) | BadNets-poisoned Stable Diffusion · Caltech15, p=5% | G1 ratio: 52.8% -> 55.1% | Helped |
| Use SDE rather than DDPM sampling (ablation) | BadNets-poisoned diffusion model · CIFAR10 | G1 generations: decrease by 17.1% | Hurt |
| Use clean-label poisoning rather than BadNets-style label corruption (ablation) | Stable Diffusion on ImageNette · ImageNette | prompt-misaligned generations | Hurt |
| Train with relabeling only, omitting image triggers (ablation) | Poisoned Stable Diffusion · ImageNette and Caltech15 | trigger-tainted generations | Hurt |
| Apply detection methods to poisoned-DM-generated images instead of the poisoned training set | CD, STRIP, and FCT detectors · ImageNette, Caltech15, and CIFAR10 | AUROC: +0.006; +0.014; +0.035; +0.028; +0.205; +0.139 | Helped |
| Increase guidance weight (ablation) | Poisoned diffusion model · Trigger amplification | trigger ratio in generations | Hurt |
Surprising findings
- Clean-label poisoning produced trigger-amplified generations but no prompt-misaligned generations, and the authors found it hard to backdoor DMs with clean-label attacks.
- Relabeling-only poisoning yields prompt-misaligned generations but no trigger-tainted generations, so trigger and relabeling effects are coupled in BadNets-style poisoning.
Limitations
- Experiments are limited to small-scale datasets (CIFAR10, ImageNette, Caltech15) and a single SD version fine-tuned only on U-Net.
- Generation-composition analysis depends on externally trained ResNet-50 classifiers used as group assigners.
- Clean-label and BadT2I comparisons are small ablations in the appendix.
Experimental setup
Datasets: CIFAR10 ImageNette Caltech15
Benchmarks: CIFAR10 ImageNette Caltech15
Metrics: FID generation composition (G1-G4 ratios) AUROC attack success rate (ASR) testing accuracy (TA)
Baselines: BadT2I clean-label backdoor attack relabeling-only poisoning ResNet-50 classifier trained on poisoned data
Builds on: BadNets BadDiff BadT2I Stable Diffusion diffusion classifier
Closest papers by meaning
- Intriguing Properties of Diffusion Models: An Empirical Study of the Natural Attack Capability in Text-to-Image Generative Models (CVPR 2024 · similarity 0.89)
- Perturbing Attention Gives You More Bang for the Buck: Subtle Imaging Perturbations That Efficiently Fool Customized Diffusion Models (CVPR 2024 · similarity 0.88)
- The Stronger the Diffusion Model, the Easier the Backdoor: Data Poisoning to Induce Copyright BreachesWithout Adjusting Finetuning Pipeline (ICML 2024 · similarity 0.88)
- Leveraging Catastrophic Forgetting to Develop Safe Diffusion Models against Malicious Finetuning (NeurIPS 2024 · similarity 0.88)
- Slight Corruption in Pre-training Data Makes Better Diffusion Models (NeurIPS 2024 · similarity 0.88)
- Machine Unlearning Fails to Remove Data Poisoning Attacks (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.