Research map / Image generation / Autoregressive and latent generation

Machine Unlearning for Image-to-Image Generative Models

ICLR 2024 · Image generation › Autoregressive and latent generation · approach: Synthetic training data from generators · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a machine unlearning framework for image-to-image generative models (diffusion, VQ-GAN, MAE) that aligns target encoder outputs with the original encoder on retain samples and with Gaussian noise embeddings on forget samples, evaluated on ImageNet-1K and Places-365.

Problem

Machine unlearning has been studied mainly for classifiers; generative image-to-image models that reconstruct images from partial inputs lack efficient, large-scale unlearning methods that remove forget-set information while preserving retain-set quality.

Approach

Derives a bounded KL-based objective (maximal KL to a zero-mean Gaussian with the same covariance) and, via an InfoNCE/mutual-information bridge, reduces it to an L2 loss on encoder outputs only: retain embeddings are kept close to the original encoder, forget embeddings are pushed toward the original encoder's embedding of Gaussian noise.

Key results

• On MAE with 8x8 center-patch cropping, the method keeps retain FID near original (59.9 vs 56.7) while raising forget FID to 153.0 from 84.1.
• Using a proxy retain set from other classes costs only a slight degradation, e.g., diffusion retain FID 17.9 vs 13.4 with real retain samples.
• An alpha ablation on VQ-GAN and MAE selects alpha=0.25 as balancing retain and forget performance.

Measured effects reported in the paper

Surprising findings

• Random Encoder baseline is competitive on VQ-GAN and MAE, and Retain Label works well on diffusion models but does not generalize to other models.
• The method works with no real retain samples, using images from other classes as a proxy retain set.

Limitations

• Requires access to the original forget samples, which are sometimes unavailable.
• Only evaluated on I2I tasks and mainstream vision datasets, not on text or text-to-image generation or practical harmful-content removal.
• Approximates the covariance matrix as the identity because the exact covariance is too costly at high resolution.

Experimental setup

Datasets: ImageNet-1K Places-365 MNIST CIFAR-10 CIFAR-100

Benchmarks: ImageNet-1K Places-365

Metrics: FID Inception Score CLIP embedding similarity

Baselines: Max Loss Noisy Label Retain Label Random Encoder

Builds on: InfoNCE SISA Palette MAGE

Closest papers by meaning

• Controllable Unlearning for Image-to-Image Generative Models via $\epsilon$-Constrained Optimization (ICLR 2025 · similarity 0.92)
• Boosting Alignment for Post-Unlearning Text-to-Image Generative Models (NeurIPS 2024 · similarity 0.92)
• Generative Unlearning for Any Identity (CVPR 2024 · similarity 0.91)
• Not All Wrong is Bad: Using Adversarial Examples for Unlearning (ICML 2025 · similarity 0.91)
• LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty (CVPR 2025 · similarity 0.91)
• The Illusion of Unlearning: The Unstable Nature of Machine Unlearning in Text-to-Image Diffusion Models (CVPR 2025 · similarity 0.90)