Research map / Image generation / Autoregressive and latent generation
Feature Unlearning for Pre-trained GANs and VAEs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a latent-space feature unlearning framework that fine-tunes pre-trained GANs and VAEs to stop generating a target feature (e.g., hairstyle, glasses), using latent target vectors identified from user-labelled generated images.
Problem
Removing a subtle, local visual feature (e.g., bangs, glasses) from a pre-trained image generative model (GAN or VAE) when the training data is inaccessible and whole-image unlearning would damage other image content.
Approach
Identify a target-feature vector in latent space via mean-difference arithmetic on positive/negative generated samples, then fine-tune the generator so latents containing the feature map to outputs of the original generator from a translated latent (projection shifted below threshold), with reconstruction, L1, and MS-SSIM perceptual losses.
Key results
- On CelebA ProgGAN, 'Bang' target ratio drops from 6.74% to 0.42% (oracle 0.49%) with IS 2.91 vs 2.92 original.
- On FFHQ StyleGAN, 'Glasses' target ratio drops from 6.16% to 0.19% with FID rising from 20.86 to 23.86.
- Under PGD latent attack on ProgGAN, the unlearned model's target ratio is 5.54% vs 14.28% for the original.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply feature unlearning to the listed GAN and VAE models | Pre-trained generative models · MNIST, CelebA, and FFHQ feature-unlearning experiments | target feature ratio (%): 9.47 -> 0.97; 9.22 -> 0.91; 9.10 -> 1.35; 10.92 -> 1.25; 3.36 -> 0.28; 7.22 -> 2.41; 6.74 -> 0.42; 3.02 -> 1.01; 4.48 -> | Helped |
| Apply feature unlearning to StyleGAN | StyleGAN on FFHQ · FFHQ (Glasses) | IS: 3.61 -> 2.37 | Hurt |
| Apply feature unlearning to StyleGAN | StyleGAN on FFHQ · FFHQ (Glasses) | FID: 20.86 -> 23.86 | Hurt |
| Remove reconstruction and perceptual losses (ablation) | Full unlearning objective on ProgGAN · CelebA (Bang) | target feature ratio (%): 0.42 -> 0.08 | Helped |
| Remove reconstruction and perceptual losses (ablation) | Full unlearning objective on ProgGAN · CelebA (Bang) | IS: 2.92 -> 2.89 | Hurt |
| Remove reconstruction and perceptual losses (ablation) | Full unlearning objective on ProgGAN · CelebA (Bang) | FID: 49.82 -> 52.48 | Hurt |
| Remove perceptual loss (ablation) | Full unlearning objective on ProgGAN · CelebA (Bang) | FID: 49.82 -> 50.93 | Hurt |
| Increase α from 1 to 5 (ablation) | Unlearning objective on ProgGAN · CelebA (Bang) | target feature ratio (%): 5.36 -> 0.01 | Helped |
| Increase α from 1 to 5 (ablation) | Unlearning objective on ProgGAN · CelebA (Bang) | FID: 48.41 -> 50.57 | Hurt |
| Apply PGD attack | Original and unlearned ProgGAN · CelebA-HQ (Bang) | target feature ratio (%): Original: 3.54 -> 14.28; Unlearn: 0.96 -> 5.54 | Mixed |
| Collect user-selected positive and negative images to unlearn Glasses | StyleGAN on FFHQ · User study (Glasses) | target feature ratio: 47.92±6.04 -> 3.23±1.92 | Helped |
Surprising findings
- Unlearning 'Glasses' on StyleGAN reduced IS from 3.61 to 2.37 in the table, a notable quality drop not discussed in depth
- Users judged only the target feature was changed in about 45% of pairs; the rest reported one or more extra features changed, attributed to entanglement with 'Young'
Limitations
- Feature entanglement (e.g., Glasses with Young) causes unintended feature changes in user study
- Relies on a classifier or user labels to collect positive/negative samples and the oracle is not ideal since positives carry other features
- Robustness evaluation under a single PGD attack showed both models still fail in some cases
Experimental setup
Datasets: MNIST CelebA CelebA-HQ FFHQ Morpho-MNIST
Benchmarks: MNIST CelebA FFHQ
Metrics: target feature ratio Inception Score Fréchet Inception Distance ROC-AUC user study accuracy
Baselines: Oracle model (retrained without target-feature images) Original pre-trained model
Builds on: Representation detachment Distance supervision PGD MS-SSIM
Closest papers by meaning
- Not All Wrong is Bad: Using Adversarial Examples for Unlearning (ICML 2025 · similarity 0.88)
- Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy (ICLR 2025 · similarity 0.87)
- Machine Unlearning for Image-to-Image Generative Models (ICLR 2024 · similarity 0.87)
- Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models (NeurIPS 2024 · similarity 0.87)
- Generative Unlearning for Any Identity (CVPR 2024 · similarity 0.87)
- Learning to Unlearn: Instance-Wise Unlearning for Pre-trained Classifiers (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.