Research map / Image generation / Text-to-image diffusion
IMMA: Immunizing text-to-image Models against Malicious Adaptation
Amber Yijia Zheng, Raymond A. Yeh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| immunize the ESD-erased model with IMMA before LoRA re-learning | ESD erased SD V1-4 · Object re-learning (10 ImageNet classes) | accuracy of target class (%): 45.5 -> 2.2 | Improved |
| immunize the model with IMMA (effect on other, non-target concepts) | ESD erased SD V1-4 · Object re-learning (10 ImageNet classes) | accuracy of other objects (%): 70.6 -> 49.3 | Hurt |
| apply IMMA against re-learning of artist styles | ESD erased SD V1-4 · Artistic style re-learning (8 artists) | SGR (LPIPS, CLIP, DINO): 21.84%, 5.5%, 23.35% | Improved |
| apply IMMA against Textual Inversion personalization | SD V1-4 · CustomConcept101 (10 concepts) | SGR(D) (%): 36.55 average | Improved |
| apply IMMA against DreamBooth personalization | SD V1-4 · CustomConcept101 (chair, lighthouse) | SGR(C): -8.60 (chair), -1.19 (lighthouse); average 2.27 | Mixed |
| apply IMMA against NSFW nudity re-learning | nudity-erased SD V1-4 · I2P unsafe prompts | change in detected nudity (%): -60% -> -80% | Improved |
| include only upper-level maximization (direct maximization) without lower-level adaptation (ablation) | IMMA · DreamBooth personalization (woodenpot -> purse) | image quality on other concept | Hurt |
| add lower-level re-initialization from previous outer-loop phi (line 5) (ablation) | IMMA · DreamBooth personalization | CLIP similarity to target | Improved |
| update overlapping parameters with adapted values (line 7) (ablation) | IMMA · DreamBooth personalization | CLIP similarity to target | Improved |
| immunize against one adaptation method and adapt with another (crossed adaptation) (ablation) | IMMA · DreamBooth and Textual Inversion | qualitative target concept generation | Improved |
| include JPEG-compressed images in IMMA training data | IMMA vs MIST · Textual Inversion with JPEG compression | qualitative protection | Improved |
| run IMMA sequentially on multiple target concepts | IMMA · castle, chair, guitar with Textual Inversion | protection and other-concept similarity | Mixed |
Later papers that compared with this method
1 later accepted paper reports IMMA as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Towards Resilient Safety-driven Unlearning for Diffusion Models against Downstream Fine-tuning (NeurIPS 2025 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users (NeurIPS 2024 · similarity 0.86)
- Distraction is All You Need: Memory-Efficient Image Immunization against Diffusion-Based Image Editing (CVPR 2024 · similarity 0.84)
- SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation (ICLR 2025 · similarity 0.84)
- The Stronger the Diffusion Model, the Easier the Backdoor: Data Poisoning to Induce Copyright BreachesWithout Adjusting Finetuning Pipeline (ICML 2024 · similarity 0.84)
- Protecting Model Adaptation from Trojans in the Unlabeled Data (AAAI 2025 · similarity 0.84)
- Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
