Research map / Image generation / Fast diffusion sampling

Training-Free Safe Denoisers for Safe Use of Diffusion Models

NeurIPS 2025 · Image generation › Fast diffusion sampling · approach: Denoising diffusion probabilistic models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a training-free "safe denoiser" that modifies diffusion sampling using a negation set of unsafe images, derived from a theorem relating safe and data denoisers, and combines it with SAFREE/SLD for safer text-to-image generation.

Problem

Diffusion models generate NSFW content, copyrighted material, and memorized private data; existing mitigations rely on text negative prompts or retraining/unlearning, which are limited or costly.

Approach

Derives a closed-form relation between the safe denoiser (data distribution restricted to a safe region) and the data denoiser, E_safe = E_data + beta*(E_data - E_unsafe), approximating E_unsafe as a kernel-weighted mixture of negation images and beta by a kernel mean, applied only at early timesteps.

Key results

• On MMA-Diffusion, SLD + Ours reduces ASR from 0.881 to 0.481 and SAFREE + Ours from 0.601 to 0.469 for SD-v1.4
• On SD-v3 Ring-A-Bell, ASR drops from 0.304 (baseline) to 0.203 with Ours, a 33.2% relative reduction, with FID improving from 23.15 to 22.54
• Average inappropriate probability on CoPro drops from 0.223 to 0.187 for SD-v1.4 + Ours, and from 0.153 to 0.144 for SAFREE + Ours

Measured effects reported in the paper

Surprising findings

• Sparse Repellency on FFHQ lowers female share but degrades FID markedly (130.52 vs 109.07 baseline), while the safe denoiser improves FID to 96.57
• Applying the safe denoiser at the final denoising steps hurts sample quality, so it is restricted to early timesteps

Limitations

• Requires tuning hyperparameters (eta, beta_t, sigma) to balance fidelity and safety
• Needs access to a set of unsafe reference data points; approximation of the normalizing weight is heuristic
• Privacy-sensitive generation lacks standardized quantitative metrics

Experimental setup

Datasets: I2P CoPro Ring-A-Bell UnlearnDiff MMA-Diffusion COCO-2014 FFHQ CelebA-HQ ImageNet

Benchmarks: Ring-A-Bell UnlearnDiff MMA-Diffusion COCO-30K CoPro FFHQ ImageNet

Metrics: ASR TR FID CLIP score inappropriate probability precision recall Top-1 accuracy LAION aesthetic score

Baselines: ESD RECE SLD SAFREE Sparse Repellency SD-v1.4

Builds on: SAFREE Safe Latent Diffusion Sparse Repellency classifier-free guidance

Closest papers by meaning

• Towards Resilient Safety-driven Unlearning for Diffusion Models against Downstream Fine-tuning (NeurIPS 2025 · similarity 0.90)
• Your Diffusion Model is Secretly a Noise Classifier and Benefits from Contrastive Training (NeurIPS 2024 · similarity 0.89)
• Leveraging Catastrophic Forgetting to Develop Safe Diffusion Models against Malicious Finetuning (NeurIPS 2024 · similarity 0.89)
• Positive-Unlabeled Diffusion Models for Preventing Sensitive Data Generation (ICLR 2025 · similarity 0.89)
• Erasing Undesirable Influence in Diffusion Models (CVPR 2025 · similarity 0.88)
• Dynamic Negative Guidance of Diffusion Models (ICLR 2025 · similarity 0.88)