Research map / Continual learning / Continual fine-tuning of LLMs
The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning
Yiting Hu, Lingjie Duan, Qian Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Hessian-based (G-N) unlearning instead of natural forgetting | CLU · CIFAR-100 Async | approximation error: 0.0835 -> 0.0087 | Improved |
| use diagonal instead of Gauss-Newton Hessian | Hessian-based CLU · CIFAR-100 Async | peak GPU memory: 5678.90 -> 1129.66 | Improved |
| use diagonal instead of Gauss-Newton Hessian | Hessian-based CLU · CIFAR-100 Async | time: 269.11 -> 283.69 | Hurt |
| add forgetting enhancement (discard old Hessian info) to G-N (ablation) | Hessian-based CLU · CIFAR-100 Async | peak GPU memory: 5678.90 -> 2232.40 | Improved |
| add forgetting enhancement to G-N (ablation) | Hessian-based CLU · CIFAR-100 Async | approximation error: 0.0087 -> 0.0463 | Hurt |
| add forgetting enhancement to diagonal Hessian (ablation) | Hessian-based CLU · CIFAR-100 Async | approximation error: 0.0094 -> 0.0451 | Hurt |
| use unlearning instead of full retraining | Hessian-based CLU · CIFAR-100 Async | time: 950.69 -> 269.11 | Improved |
| use well-ordered (Fwd-Sync) instead of Async deletion order | Hessian-based CLU · CIFAR-100 G-N | final approximation error: Async error about twice Fwd-Sync | Improved |
Closest papers by meaning
- When to Forget? Complexity Trade-offs in Machine Unlearning (ICML 2025 · similarity 0.91)
- Model Sensitivity Aware Continual Learning (NeurIPS 2024 · similarity 0.90)
- Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference (NeurIPS 2024 · similarity 0.90)
- Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning (ICLR 2025 · similarity 0.90)
- Forgetting Has Neighbors: Localized Collateral Forgetting in Machine Unlearning (NeurIPS 2026 · similarity 0.90)
- A Unified and General Framework for Continual Learning (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
