Research map / Image generation / Fast diffusion sampling
Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Score Forgetting Distillation (SFD), a data-free method that distills a diffusion model into a one-step generator while regularizing with a score-based machine-unlearning loss that overrides forget classes/concepts with safe ones.
Problem
Machine unlearning in diffusion models typically requires real data and multi-step sampling is slow; the paper targets fast, data-free forgetting of target classes and concepts in generative diffusion models.
Approach
Combines data-free score distillation (SiD-style model-based Fisher divergence with a fake score network) with a cross-class score-matching forgetting loss that aligns the score of the forget class/concept with that of an override class/concept, yielding a one-step generator that forgets while distilling.
Key results
- On CIFAR-10, SFD reaches UA 99.64% with FID 5.35 in 1 NFE, versus 2000 NFEs for SalUn (UA 99.96, FID 11.25) and 1000 NFEs for Retrain.
- On STL-10, SFD-CFG reaches UA 99.64% and FID 15.32 with 1 NFE, improving over SFD (FID 18.82) and SalUn (FID 20.78).
- For nudity forgetting on I2P, SFD-Two Stage achieves the lowest inappropriate probability (11.03%) vs SD v1.4 (28.54%) and SLD Medium (14.10%); for celebrity forgetting, it reduces Angelina Jolie GCD to 1.06%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SFD instead of the compared methods | DDPM · CIFAR-10 | FID: 7.94 -> 5.35 | Helped |
| Use SFD instead of the compared methods | DDPM · CIFAR-10 | UA: 99.96 -> 99.64 | Mixed |
| Use SFD instead of the compared methods | DDPM · STL-10 | FID: 20.78 -> 18.82 | Helped |
| Add classifier-free guidance during training (ablation) | SFD · STL-10 | FID: 18.82 -> 15.32 | Helped |
| Add classifier-free guidance during training (ablation) | SFD · STL-10 | UA: 99.02% -> 99.64% | Helped |
| Add classifier-free guidance during training (ablation) | SFD · CIFAR-10 | FID: 5.35 -> 7.27 | Hurt |
| Add classifier-free guidance during training (ablation) | SFD · CIFAR-10 | UA: 99.64% -> 99.62% | No clear effect |
| Use the two-stage distillation-then-forgetting schedule instead of joint SFD (ablation) | SFD · CIFAR-10 | FID: null | Hurt |
| Use the two-stage distillation-then-forgetting schedule instead of joint SFD (ablation) | SFD · CIFAR-10 | UA: FID = 5.73, UA = 99.5% | Mixed |
| Use SFD-Two Stage for celebrity forgetting | Stable Diffusion · Brad Pitt forgetting | Prop. w/o Faces: 5.8% -> 1.76% | Helped |
| Use SFD-Two Stage for celebrity forgetting | Stable Diffusion · Angelina Jolie forgetting | GCD: 3.29% -> 1.06% | Helped |
| Use SFD-Two Stage for nudity forgetting | Stable Diffusion · I2P | Inapprop. Prob.: 14.10% -> 11.03% | Helped |
Surprising findings
- CFG during training degraded CIFAR-10 results while improving STL-10, which the authors attribute to similar classes such as automobile and truck.
- The SFD model without AdvUnlearn has Post-ASR of 55.63% under UnlearnDiffAtk, underperforming AdvUnlearn-TE (21.13%).
Limitations
- Forgetting is implemented by overriding a target with a safe class or concept, not by true removal from the distribution.
- Adversarial robustness is weaker than AdvUnlearn-TE on UnlearnDiffAtk (Post-ASR 55.63% vs 21.13%) when used alone.
- CFG benefit is dataset-dependent (helped STL-10, hurt CIFAR-10).
Experimental setup
Datasets: CIFAR-10 STL-10 I2P
Benchmarks: CIFAR-10 STL-10 I2P Giphy celebrity detection
Metrics: Unlearning Accuracy FID Inception Score Precision Recall NFEs GCD score Inappropriate probability Max expected inappropriate probability CLIP score
Baselines: Retrain ESD SA SalUn SLD Medium ESD-x ESD-u SiD-LSG SD v1.4 AdvUnlearn FMN Diff-Instruct
Builds on: Score identity Distillation (SiD) denoising score matching classifier-free guidance ESD SalUn
Closest papers by meaning
- Enabling Preference-driven Unlearning in Few-step Distilled Text-to-Image Diffusion Models (NeurIPS 2026 · similarity 0.90)
- Leveraging Catastrophic Forgetting to Develop Safe Diffusion Models against Malicious Finetuning (NeurIPS 2024 · similarity 0.90)
- UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Models (NeurIPS 2024 · similarity 0.90)
- The Illusion of Unlearning: The Unstable Nature of Machine Unlearning in Text-to-Image Diffusion Models (CVPR 2025 · similarity 0.90)
- One-Step Diffusion Distillation through Score Implicit Matching (NeurIPS 2024 · similarity 0.89)
- Data Unlearning in Diffusion Models (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.