Research map / Continual learning / Continual fine-tuning of LLMs

Leveraging Catastrophic Forgetting to Develop Safe Diffusion Models against Malicious Finetuning

NeurIPS 2024 · Continual learning › Continual fine-tuning of LLMs · approach: Synthetic data and merging for CLIP · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a contrastive-learning training policy that pushes harmful-concept latents away from clean data so that safety-aligned diffusion models resist malicious fine-tuning via catastrophic forgetting.

Problem

Safety-aligned text-to-image diffusion models (unlearned or filtered) can be re-enabled to generate harmful images by malicious fine-tuning on harmful data.

Approach

Treats harmful data as a new task that the model should forget: contrastive learning widens the latent distance between clean and harmful distributions while clean data is replayed, using either latent transformation or noise-shift guidance on harmful samples, so later harmful fine-tuning is hard to learn.

Key results

• Safety-aligned SD v2.1 with NG reduces NSFW score from 0.6034 to 0.4517 on sexual harmful prompts, and after malicious fine-tuning the score stays low (0.4732 IP-related sexual subset for NG vs 0.5887 baseline IP).
• Safety reinforcement of ESD-Nudity yields NSFW score after malicious fine-tuning of 0.4441 (LT) and 0.4421 (NG) rather than rising as the unlearned base model does.
• Strongly aligned model keeps IP at 3.52% after 200 malicious fine-tuning steps, and reaches 21% vs 35% IP-level comparable to SLD on I2P (Table 5).

Measured effects reported in the paper

Surprising findings

• NSFW score shows a sudden drop around 80 malicious fine-tuning steps, which the authors interpret as abrupt catastrophic forgetting.
• Dynamically changing noise helps in safety alignment but hurts in safety reinforcement.

Limitations

• Model cannot completely prevent harmful image generation; some prompts still produce harmful content.
• Potential attack: first fine-tune to bring clean and harmful latents closer, then perform standard fine-tuning, is not addressed.
• Only evaluated on diffusion models; extension to other networks left to future work.

Experimental setup

Datasets: LAION-5B Mistral-7B-generated harmful prompts DiffusionDB COCO I2P Unsafe prompts COCO-30K

Benchmarks: LAION-5B prompts DiffusionDB COCO I2P Unsafe prompts COCO-30K

Metrics: NSFW Score Inappropriate Rate Human evaluation Aesthetic Score CLIP Score FID

Baselines: SD v1.4 SD v2.1 ESD-Nudity-u1 ESD-Violence-x1 SLD-Medium SLD-Max FMN ESD SLD SD XL

Builds on: Stable Diffusion ESD Selective Amnesia SLD

Closest papers by meaning

• Towards Resilient Safety-driven Unlearning for Diffusion Models against Downstream Fine-tuning (NeurIPS 2025 · similarity 0.90)
• Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Models (ICLR 2025 · similarity 0.90)
• Detect-and-Guide: Self-regulation of Diffusion Models for Safe Text-to-Image Generation via Guideline Token Optimization (CVPR 2025 · similarity 0.89)
• Training-Free Safe Denoisers for Safe Use of Diffusion Models (NeurIPS 2025 · similarity 0.89)
• Growth Inhibitors for Suppressing Inappropriate Image Concepts in Diffusion Models (ICLR 2025 · similarity 0.88)
• Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models (NeurIPS 2024 · similarity 0.88)