Research map / Adversarial robustness and security / Backdoor and poisoning
BackdoorDM: A Comprehensive Benchmark for Backdoor Learning on Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces BackdoorDM, a benchmark for backdoor attacks and defenses on diffusion models that implements nine attacks and four defenses, classifies attack and target types, and proposes MLLM (GPT-4o)-based evaluation of specificity, utility, and efficiency.
Problem
Backdoor learning in diffusion models (unconditional and text-to-image) lacks a comprehensive benchmark, making fair comparison of attacks and defenses difficult, and existing evaluation metrics are inconsistent and miss non-target content preservation.
Approach
A unified formulation of DM backdoor attacks (text/noise triggers, target alignment, utility loss) with a taxonomy of five backdoor target types, plus a unified MLLM-based Q&A evaluation (ACC, ASR, PSR) replacing per-attack classifiers.
Key results
- On ImageFix, VillanDiffusion achieves the best FID (13.50) with the lowest runtime (4017.71s), while VillanCond achieves the lowest MSE (0.0010) at a cost in FID (28.81).
- On ImagePatch, Pixel-Backdoor (BadT2I) outperforms BiBadDiff on all metrics, e.g., ASR_GPT 99.6 vs 34.10 and MSE 0.0087 vs 0.2353.
- On ObjectRep, no method dominates: TPA (RickRolling) reaches ASR_GPT 96.80 but PSR_GPT only 5.50, while EvilEdit has the highest PSR (85.25) and runs in 16.59s with no data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VillanCond instead of the unconditional ImageFix attacks | ImageFix backdoor · ImageFix | MSE ↓: 0.0300 -> 0.0010 | Helped |
| Use VillanCond instead of VillanDiffusion | ImageFix backdoor · ImageFix | FID ↓: 13.50 -> 28.81 | Hurt |
| Use VillanDiffusion instead of InviBackdoor | ImageFix backdoor · ImageFix | FID ↓: 58.19 -> 13.50 | Helped |
| Use Pixel-Backdoor (BadT2I) instead of BiBadDiff | ImagePatch backdoor · ImagePatch | ASR GPT ↑: 34.10 -> 99.6 | Helped |
| Use Pixel-Backdoor (BadT2I) instead of BiBadDiff | ImagePatch backdoor · ImagePatch | ACC GPT ↑: 19.48 -> 84.51 | Helped |
| Use Pixel-Backdoor (BadT2I) instead of BiBadDiff | ImagePatch backdoor · ImagePatch | PSR GPT ↑: 25.72 -> 89.69 | Helped |
| Use TI instead of DB personalization | PaaS ObjectRep attack · ObjectRep | ACC GPT ↑: 70.87 -> 84.27 | Helped |
| Use TPA (RickRolling) instead of EvilEdit | ObjectRep backdoor · ObjectRep | ASR GPT ↑: 61.10 -> 96.80 | Helped |
| Use TPA (RickRolling) instead of EvilEdit | ObjectRep backdoor · ObjectRep | PSR GPT ↑: 85.25 -> 5.50 | Hurt |
| Use EvilEdit instead of TPA (RickRolling) | ObjectRep backdoor · ObjectRep | Runtime ↓: 286.89s -> 16.59s | Helped |
| Use Style-Backdoor (BadT2I) instead of TAA (RickRolling) | StyleAdd backdoor · StyleAdd | PSR GPT ↑: 65.92 -> 90.68 | Helped |
Surprising findings
- The classic BadNets-like poisoning attack (BiBadDiff) performs poorly on DMs compared with BadT2I under practical complex text prompts.
- No single attack performs consistently well across specificity, utility, and efficiency metrics on ObjectRep targets; ViT-based ASR gives lower values than GPT-based evaluation.
Limitations
- Coverage limited to unconditional and text-to-image diffusion models; TTS and text-to-video are not covered.
- SD v3.0 (DiT-based) is excluded because most current attacks are incompatible with it.
- Evaluation relies on MLLM judges whose outputs may vary across models.
Experimental setup
Datasets: CIFAR-10 CelebA-HQ CelebA-HQ-Dialog LAION-Aesthetics v2 5+ MS-COCO 2014 validation
Benchmarks: CIFAR-10 CelebA-HQ CelebA-HQ-Dialog MS-COCO 2014 validation
Metrics: ASR MSE TCS PSR ACC BCS LPIPS FID runtime data usage
Baselines: BadDiffusion TrojDiff InviBackdoor VillanDiffusion VillanCond BiBadDiff Pixel-Backdoor (BadT2I) TPA (RickRolling) Object-Backdoor (BadT2I) TI (PaaS) DB (PaaS) EvilEdit
Builds on: BadDiffusion VillanDiffusion BadT2I RickRolling EvilEdit
Closest papers by meaning
- UFID: A Unified Framework for Black-box Input-level Backdoor Detection on Diffusion Models (AAAI 2025 · similarity 0.90)
- Elijah: Eliminating Backdoors Injected in Diffusion Models via Distribution Shift (AAAI 2024 · similarity 0.90)
- Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness (NeurIPS 2024 · similarity 0.90)
- From Toxic to Trustworthy: Using Self-Distillation and Semi-supervised Methods to Refine Neural Networks (AAAI 2024 · similarity 0.90)
- BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detection (ICLR 2024 · similarity 0.89)
- Demystifying Poisoning Backdoor Attacks from a Statistical Perspective (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.