Research map / Continual learning / Continual fine-tuning of LLMs
Machine Unlearning under Retain–Forget Entanglement
Jingpu Cheng, Ping Liu, Qianxiao Li, CHI ZHANG
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add W2 distance regularization to stage-2 gradient projection (ablation) | two-stage unlearning · CIFAR-100 subclass (ResNet-18) | test accuracy on forget set: 14.33 -> 2.33 | Improved |
| use alpha=0.5 instead of alpha=0 (ablation) | W-PGD · CIFAR-100 bee subclass | test accuracy on forget set: 16.33 -> 6.33 | Improved |
| use alpha=0.5 instead of alpha=1 (full W2) (ablation) | W-PGD · CIFAR-100 bee subclass | test accuracy on forget set: 6.33 -> 6.33 | No clear effect |
| constrain only remote retain set in stage 1 instead of also adjacent set (ablation) | stage 1 augmented Lagrangian · CIFAR-100 subclass | adjacent retain test accuracy: 0.00 -> 72.75 | Improved |
| use two-stage method instead of SalUn | Our method · CIFAR-100 subclass | adjacent retain test accuracy: 34.90 -> 78.17 | Improved |
| use two-stage method instead of fine-tuning | Our method · CIFAR-100 subclass | forget test accuracy: 62.33 -> 2.33 | Improved |
| use two-stage method instead of GDR | Our method · Tiny-ImageNet ViT | forget test accuracy: 24.22 -> 3.11 | Improved |
| use two-stage method instead of Gradient Ascent | Our method · CelebA ViT-B | adjacent retain test accuracy: 0.00 -> 75.05 | Improved |
| use two-stage method vs GDR on ToxiGen | Our method · ToxiGen RoBERTa | forget test accuracy: 19.83 -> 14.29 | Improved |
| use classical PGD with mean-loss preservation instead of W-PGD (ablation) | PGD · CIFAR-100 | forget set accuracy | Hurt |
Closest papers by meaning
- Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance (NeurIPS 2026 · similarity 0.90)
- RA$^2$: Retain-Anchored Attraction Preserves Forget-Adjacent Utility in LLM Unlearning (NeurIPS 2026 · similarity 0.89)
- Fast Machine Unlearning without Retraining through Selective Synaptic Dampening (AAAI 2024 · similarity 0.89)
- When to Forget? Complexity Trade-offs in Machine Unlearning (ICML 2025 · similarity 0.89)
- Contrastive Representation Shaping for LLM Unlearning (NeurIPS 2026 · similarity 0.89)
- Forgetting Has Neighbors: Localized Collateral Forgetting in Machine Unlearning (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
